February 04, 2008

Financial Times FT MBA Rankings 2008 - Analysis and Review Part I

Summary

This post analyzes the FT MBA
rankings 2008 with focus on the methodology and how certain parameters can unwittingly cause surprising distortions in the ranking. And for those interested in the LBS vs. INSEAD debate, I will be posting a short Part II to analyze why LBS leads INSEAD in the ranking.

Introduction

FT released its Financial Times FT Global MBA Rankings 2008 - one of the most popular MBA rankings, and often debated widely once the ranking is released. Last year, I published the following post, and I still stand behind most of my assertions on the importance of ranking.

This year, my focus is on some aspects of the methodology of ranking and how they affect school rankings. I then look at some surprising entrants/movers this year to see what contributed to that - and how sometimes data and its interpretation can be misleading.

INSEAD moved from 7 to 6 - so there really is not much to talk about. LBS moved to number 2 - a pretty big movement, putting it ahead of HBS and Stanford. How did HBS and Stanford get hurt? Perhaps due to the weight (20%) of the 'weighted average salary' calculation - which this year resulted in some interesting ranks. The strength of the pound/Euro caused some of the shifts as well. In my analysis, I will be focusing on one of the most important metrics in FT's methodology and how it flaws the ranking by not considering some interesting discrepancies of the approach.

Weighted Average Salary (20% weight) - the right metric?

Consider the surprising entrant to the top-20 - India's Indian School of Business (ISB), which is only 6 years old, and debuted at the 20th position ahead of Duke, Ross, Kellogg, Darden, and UNC - all considered among the top schools in the world. There is much celebration (and it does feel good to see an Indian school finally make it to a reputed ranking), but how sustainable is this position? What caused ISB to attain this ranking? How could a 6 year old school with average salaries that are 25% of top international schools, a very low value for money and alumni recommendation, get ahead of Kellogg? By suspending wild celebration for a moment, Let us look at the numbers and their ramifications.

Let me reproduce the comparison between Kellogg - one of the most valued business schools in the world - which now ranks behind ISB. This will be a basis for some further analysis.


Rank '08
Business school name
Country
Weighted sal. ($)
Salary inc. (%)
Value
Aims achieved
Employment
Recommends
Research
11CeibsChina154,144157577692 (87)5866
16Yale School of ManagementU.S.A.136,31814078488 (99)2226
20Indian School of BusinessIndia169,3551298768100 (91)8588
24Northwestern University: KelloggU.S.A.135,64698100597 (96)49

ISB is higher in Weighted Salary, it has a better salary % increase (we'll get to that metric), the employment rates are quite similar, but there is incredible difference between Alumni recommends and Research. Yet ISB ranks ahead. Now let us get into the analysis,

Weighted Average Salary (WAS) - an important parameter in FT MBA ranking, gets 20% in weight.


WAS = average of salary (with some adjustments) x PPP conversion

where PPP conversion index = value provided by world bank.

Using this, ISB's weighted salary shot up to 169,355$ - which makes it look like ISB graduates get the highest salary in the world - and gave a sharp advantage to ISB in its ranking. However, we also see that value for money is 87 - which is strange. Why is a school with the highest weighted salary in the world have such low value for money? Let us look at this closely - the average weighted salary includes a PPP conversion, which means the actual average salary was somewhere around 33,000 USD. This is about 25% of most of the top schools. Now even assuming that WAS is a sensible measure, why is then the value of money so low? The reason has a twist - the fees in ISB, as I last knew, is about 17L- and using the same PPP conversion, turns out to be an average weighted 'fees' of 89,000$ per year ! Which also makes it one of the most expensive schools (if not the most - @178,000$ as fees for 2 years) in the world (based on a 1 year fee). For e.g. INSEAD, which is quite expensive, had an annual fee of 50,000 euros ~65,000$.

The distortion of FT's calculation comes from the following fact - the researcher used the PPP conversion index given by the world bank, which works out to be ~5.1. However, this index considers the country as a whole, and since the average Indian is a lot poorer than an average ISB MBA candidate, the index "inflates" the earnings of the MBA relative to the local population, but turns into an advantage as far as global the rankings go. It also ignores the purchase disparity between a lower middle class Indian and one earning a higher salary - at upper levels, the purchasing cost is not 1/5 th of the US - it is more like 1/2 or 1/3, and in some cases, even lesser! The use of PPP as an index is faulty - plain and simple - and that is what gives a big edge to ISB (and CEIBS as well). A better measure could be a weighted value which is a combination of median salary (or even average salary with removal of outliers) and average weighted salary. This will reduce some of the "distortionary" effects of using the PPP index.

Overall, usage of the average weighted salary and to some extent, the % increase - which actually works to an advantage to schools that bring in younger/lower salary candidates - is what caused surprising distortions in the FT ranking. The 20% weight for a measure that exposes underlying interpretation issues is not the best way to decide a school's rank. Personally, the weightage should be higher for alumni recommendations, research, breadth of alumni, average and median salaries (not weighted by any measure) and actual salary 3 years down the line. If the ranking is global, it needs to use global measure.

So does ISB (and similar schools) merit top 20?

Here it comes - don't get me wrong, ISB is a great school. I've heard a lot of good things about it, very smart people study there, but at this point, my conclusion is that ISB moved to 20 due to a statistical distortion. The fact that the school graduates do earn pretty good salaries in Indian terms gets magnified so heavily by FT's weighted criteria is what caused it to move up to the table. The danger for borderline schools like ISB and a few others that capitalize on the weightage of a specific measurement is that if FT decides to tweak a single parameter, the schools can again drop off to oblivion (Singapore's NUS dropped off the list this year much to their dismay). And for that reason, they need to pay continuous attention to all other parameters and gain strength in them.


The effect of % salary increases

The other heavy weightage is due to the '% salary increase', and in emerging markets schools bring in younger students with lower salaries and when they graduate the new salary is much higher. Even if the absolute salaries are lower, the % increase is higher - why is this so? This is because post MBA most employers pay within a certain band typically irrespective of the incoming salary. This becomes a disadvantage for European schools - especially INSEAD and IMD where the salary of the incoming class is already quite high - higher than most other schools in the world. Therefore, the % increase is low. If this methodology was tweaked to reduce the weightage, INSEAD would shoot up to be within top 5.

update - March 2008: I recently found this statement from Rotman's dean Roger Martin where he delves into some more analysis on his school's ranking position that support my argument above. I feel your pain, Dean Martin!

My personal take on FT Rankings

The nice thing about FT rankings is that the methodology is transparent and available for analysis and critique. I do hope that for the next ranking they provide greater weight to alumni recommendations, breadth, network strength and median salary levels which are greater representations of a school's reputation and strength and reduce weights on parameters that cause wide distortions and even misleading exuberance.

Rankings have been, and will continue to be important. I disagree with blanket statements that "one should choose schools by filtering what is important to them." If the overall ranking methodology is absurd - like EIU's for example - which places Wharton at 21 below HKUST and puts Harvard at 13 - this assertion is plain ridiculous.

It simply implies that the parameters (or interpretation) are chosen by the ranking journalists who have little clue about what is important for rankings so they reflect a modicum of public perception. If you were to apply based on EIU's rankings, you would drop Harvard to go to Cranfield ; drop Wharton to go to Hong Kong university and drop Yale or Darden and go to
Vlerick Leuven Gent Management School. Even my otherwise gentle 89 year old grandmother wouldn't agree with that.

Conclusion - Part I


This concludes Part I of ranking analysis. I will continue this with an analysis of LBS vs. INSEAD and why Harvard is not Number 1. I can live with FT rankings to some extent simply because it is reasonably accurate with respect to applicant preferences, brand recollect, overlap with US News for US schools. The specific order within the 1-10 do not matter so much because no matter what you tweak, minor differences will upset ranking orders.

For those who wish to critique my analysis or want to rant on my ISB assertion, you can do so as long as your post is objective and provides a clear reposte as to why my analysis is wrong. I would be happy to be proved wrong.

My Personal "Necromonger's MBA Ranking 2008"

And for the curious, what is my personal perception of the world's top 10 schools? This is my list in alphabetical order -
  • Chicago
  • Columbia
  • Duke
  • Harvard
  • INSEAD
  • LBS
  • Sloan
  • Stanford
  • Tuck
  • Wharton.

And 10-20 (in no particular order): Kellogg, Ross, UNC, Darden, Haas, NYU, Oxford, IESE, IMD, Yale.

Coming next (in a few days)

  • LBS vs. INSEAD - what made LBS gain ground?
  • What pushed Harvard to No. 5?

20 comments:

The Teacher said...

Excellent analysis. I do agree with you that due to the extra weightage (close to 40%) attached to the salary factor, ISB's ranking has shot up. I attribute the high %increase to the ever increasing exposure that ISB is getting, considering it is expanding fast and is in its earlier stages.

By the way, in your busy schedule, how do you get the time to do such a thorough analysis and write all this? Hats off dude!! :)

Anonymous said...

Surprised that you put Duke over Kellogg; otherwise I agree with your top 10.

Good critique of the FT rankings.

necromonger said...

Teacher - Thanks for the comment. As they say - where there is a will... :) It's more about managing time I guess, because the actual writing does not take a lot of time.

Anonymous - I've been reading a lot of good things about Duke. To be honest, I don't have a super strong reason why Duke should be above Kellogg except that it's a personal perception.

Rahul said...

Good Analysis. Completely agree with your analysis and mostly with your rankins (barring UNC in top 20). I think ISB has still a long way to go in terms of being a truly international school - it needs more international participants. Hopefully, this ranking will help it attract more international participants.

tandoor said...

How come you haven't considered any of the Canadian schools in your top 20 list? If you go by the BW rankings for top 10 non-US schools, Queens, Ivey and the rising star Rotman, are ranked higher than INSEAD & LBS..FT seems to be more Euro biased with unknown schools in top 30..also Canadian schools are much more known in the US market than most Euro schools - so thats pretty important too

tandoor said...

OK I read your previous year's comments after posting my comment earlier..to your point about INSEAD being global, well sorry to say that, after having lived in US for 10+ years, the Canadian schools are more known than INSEAD, Oxford or Cambridge.. exception is LBS. So your point about Canadian schools not being global doesn't hold good since your view is based on your experience out in Europe and you live there. All the major American firms have their Canadian branches in Toronto, and Toronto is the 3rd biggest financial city in North America, so yes although BW can be North American biased, Canadian schools are worth their worth!

necromonger said...

tandoor,

You're making many assumptions - I've never lived in Europe. I've lived in India and the US. I'm sure the Canadian schools are great schools, but "better known in US" is not an automatic qualification. Let me throw the question back at you - if you were to re-do top 20 which schools would you throw out and replace with canadian schools and still satisfy all the 3 rankings?

tandoor said...

Easily a bunch of UK schools, along with CEIBS & ISB...same with a bunch of unknown UK schools in the top 30; but I wouldn't replace any top 20 schools with Canadian schools..they should definitely be there in top 30 though especially with Rotman in top25. Ivey & Rotman are very well known in north-east and midwest US, especially Ivey which is one of the oldest business schools in NA..Rotman is well known in the NYC area and more so with its #6 rank in Finance. My interactions with recruiters and finance alumni vouches for this. As far the BW rankings,I would bring LBS & INSEAD over the top 3 Canadian schools definitely, but they deserve to be in the top 10 (non-US schools).

necromonger said...

tandoor,

I guess we just read each others messages wrong. I have no dispute with your assertions for top 30. The Canadian schools are all reputed and I agree that they could potentially replace some others in the 30. My analysis was limited to the 20 - I did not look beyond that.

Anonymous said...

Nice analysis putting the spotlight on a well-known weakness.
Do people actually know why FT introduced PPP? Well, rumours here in Asia say that FT wanted to publish their paper in China, hence you better get on good terms with the government :)

Two more points:
Hong Kong University is not HKUST. Hong Kong University is HKU, and HKUST is Hong Kong University of Science & Technology.

Lastly, I studied at HKUST and did exchange at CBS. Well if you ask me, not much difference in terms of quality of the program (students, courses, etc.), except - and that's a big one - that CBS profits from a brand built over decades, which in turn helps the grads to get better jobs, etc.

Anonymous said...

Not much difference in quality? That's ridiculous. How many classes did you take, how valid are your personal comparisons, given your bias towards promoting your alma mater, and how many other measures- nobel laureates at each school; number of papers published at top journals by professors; average GPA and GMAT of fellow students, did you take into consideration?

habeliraucher said...

Well, I don't know how valid my personal comparison is when you generalize them to compare whole schools. I simply comment based on the time and classes I took at CBS.

To answer your points more specifically:
I did one semester at CBS (electives only) vs two semesters at HKUST (one core, one electives). Each semester contained approx. 16 credits. The classes I took at CBS were from world class (Top Management Process) to very mediocre (Tax Strategy) to weird but interesting (Napoleon's Glance).
The take away point here is that some classes are top while some are complete crap and most are average. One particular issue is that many classes at CBS tend to be very US centric and do not bring in a global perspective.
At HKUST you also have some top classes (Justin Lin's Development in China or Milind Rao Macroeconomics). In addition you have about the same percentage of crap and average classes like at CBS.
CBS has certainly more nobel laureates but the professors also tend to be older ;-) This might possibly change once HKUST as a school gets older (HKUST is only 13 years old after all)

Re research/papers: CBS is ranked 11th, HKUST 20th so that is not far off.

Lastly on academics, the professors that teach the MBA program at HKUST are the same that teach the Kellogg-HKUST EMBA program, ranked No 1 worldwide.

Re student quality. I was the top student in my class at HKUST and I had the top grades in my classes at CBS, regardless whether it was Top Management Process or Tax Strategy.
Of course CBS has more good students by quantity as one intake is roughly 500-700 students. If 10% are top, that means 70 students. HKUST also has roughly 10% top students but that means a much smaller number as the intake is smaller. What I observed however is that CBS and HKUST have about the same percentage of top vs average vs weak students. Once again, personal observations only.

More general, I have to say that my CBS experience was a bit so-so. I came in with very high expectations (due to ranking, reputation, etc.) that the school was not always able to live up to. Maybe this expectation gap is my real bias ;-)
Also I should say that I have friends who did semesters at Kellogg which they really considered to be a class above HKUST. Maybe the discussion here should be Kellogg vs CBS rather than HKUST vs anyone.

A last point: I think people need to say good bye to the thought of US and some selected European schools being top alone. I am a European with working experience in Europe, US and Asia and I can share that Asia in general is moving so much faster that it's a question of when and not whether Asia will break into the top league soon - whether that's MBA schools, Fortune500 companies, or the Olympics gold medal list ;-)

necromonger said...

habeliraucher,

Thanks for the detailed post in response to "Anonymous." I'm not jumping in the fray with comments but responses like yours are very useful for other readers and can be useful for some who may be considering, say, HKUST as a possible option.

Kunal said...

What aabout IE Business School, Spain. It also has been ranked pretty high in most of the rankings..

Arun said...

Excellent articles,arguments and counter arguments. I would like to thank all of you for the knowledge you have shared on this forum. kindred the interest in aspiring managers such as myself

Ken said...

Excellent analysis all round. My question is this- why are UK schools apart from LBS left out in general rankings i.e. they are not considered worthy? Does Lancaster business school in the UK really merit its ranking? 22nd globally ahead of CASS and Manchester Business School? what are your opinions?

Moderator said...

Excellent analysis.Could u please throw some light on how well is an Indiann MBA from a reputed b-school(top 10 b-schools)other than IIM, recognised in the US?Is it possible to get a job there in the US-at least in Indian companies in managerial positions?If not, what are the alternatives?R there any courses or certifications available to gt this program recognised?Please help me out with this information

Arvind Elayappan :-) said...

This blog is a nice piece of work .,... why is this dead now...

gps said...

To reiterate Kunal's question, what about IE business school? The rankings seem to differ on IE vs IESE - does IE suffer from some similar distortion as ISB? Many rankings put IESE and IE close together but you haven't even included IE in your top 20. Any thoughts much appreciated, as am trying to decide whether to apply to IE/IESE (or other european rolling admits school)...

balleballeh said...

However, American MBA takes two years whereas MBA from ISB takes only one year. It is more like EMBA from US which is much costlier.
You can earn around 20 lakh in that one year and decrease your total cost of MBA.
Hence, MBA from ISB makes sense.