Summary
This post analyzes the FT MBA rankings 2008 with focus on the methodology and how certain parameters can unwittingly cause surprising distortions in the ranking. And for those interested in the LBS vs. INSEAD debate, I will be posting a short Part II to analyze why LBS leads INSEAD in the ranking.
Introduction
FT released its Financial Times FT Global MBA Rankings 2008 - one of the most popular MBA rankings, and often debated widely once the ranking is released. Last year, I published the following post, and I still stand behind most of my assertions on the importance of ranking.
This year, my focus is on some aspects of the methodology of ranking and how they affect school rankings. I then look at some surprising entrants/movers this year to see what contributed to that - and how sometimes data and its interpretation can be misleading.
INSEAD moved from 7 to 6 - so there really is not much to talk about. LBS moved to number 2 - a pretty big movement, putting it ahead of HBS and Stanford. How did HBS and Stanford get hurt? Perhaps due to the weight (20%) of the 'weighted average salary' calculation - which this year resulted in some interesting ranks. The strength of the pound/Euro caused some of the shifts as well. In my analysis, I will be focusing on one of the most important metrics in FT's methodology and how it flaws the ranking by not considering some interesting discrepancies of the approach.
Weighted Average Salary (20% weight) - the right metric?
Consider the surprising entrant to the top-20 - India's Indian School of Business (ISB), which is only 6 years old, and debuted at the 20th position ahead of Duke, Ross, Kellogg, Darden, and UNC - all considered among the top schools in the world. There is much celebration (and it does feel good to see an Indian school finally make it to a reputed ranking), but how sustainable is this position? What caused ISB to attain this ranking? How could a 6 year old school with average salaries that are 25% of top international schools, a very low value for money and alumni recommendation, get ahead of Kellogg? By suspending wild celebration for a moment, Let us look at the numbers and their ramifications.
Let me reproduce the comparison between Kellogg - one of the most valued business schools in the world - which now ranks behind ISB. This will be a basis for some further analysis.
ISB is higher in Weighted Salary, it has a better salary % increase (we'll get to that metric), the employment rates are quite similar, but there is incredible difference between Alumni recommends and Research. Yet ISB ranks ahead. Now let us get into the analysis,
Weighted Average Salary (WAS) - an important parameter in FT MBA ranking, gets 20% in weight.
WAS = average of salary (with some adjustments) x PPP conversion
where PPP conversion index = value provided by world bank.
Using this, ISB's weighted salary shot up to 169,355$ - which makes it look like ISB graduates get the highest salary in the world - and gave a sharp advantage to ISB in its ranking. However, we also see that value for money is 87 - which is strange. Why is a school with the highest weighted salary in the world have such low value for money? Let us look at this closely - the average weighted salary includes a PPP conversion, which means the actual average salary was somewhere around 33,000 USD. This is about 25% of most of the top schools. Now even assuming that WAS is a sensible measure, why is then the value of money so low? The reason has a twist - the fees in ISB, as I last knew, is about 17L- and using the same PPP conversion, turns out to be an average weighted 'fees' of 89,000$ per year ! Which also makes it one of the most expensive schools (if not the most - @178,000$ as fees for 2 years) in the world (based on a 1 year fee). For e.g. INSEAD, which is quite expensive, had an annual fee of 50,000 euros ~65,000$.
The distortion of FT's calculation comes from the following fact - the researcher used the PPP conversion index given by the world bank, which works out to be ~5.1. However, this index considers the country as a whole, and since the average Indian is a lot poorer than an average ISB MBA candidate, the index "inflates" the earnings of the MBA relative to the local population, but turns into an advantage as far as global the rankings go. It also ignores the purchase disparity between a lower middle class Indian and one earning a higher salary - at upper levels, the purchasing cost is not 1/5 th of the US - it is more like 1/2 or 1/3, and in some cases, even lesser! The use of PPP as an index is faulty - plain and simple - and that is what gives a big edge to ISB (and CEIBS as well). A better measure could be a weighted value which is a combination of median salary (or even average salary with removal of outliers) and average weighted salary. This will reduce some of the "distortionary" effects of using the PPP index.
Overall, usage of the average weighted salary and to some extent, the % increase - which actually works to an advantage to schools that bring in younger/lower salary candidates - is what caused surprising distortions in the FT ranking. The 20% weight for a measure that exposes underlying interpretation issues is not the best way to decide a school's rank. Personally, the weightage should be higher for alumni recommendations, research, breadth of alumni, average and median salaries (not weighted by any measure) and actual salary 3 years down the line. If the ranking is global, it needs to use global measure.
So does ISB (and similar schools) merit top 20?
Here it comes - don't get me wrong, ISB is a great school. I've heard a lot of good things about it, very smart people study there, but at this point, my conclusion is that ISB moved to 20 due to a statistical distortion. The fact that the school graduates do earn pretty good salaries in Indian terms gets magnified so heavily by FT's weighted criteria is what caused it to move up to the table. The danger for borderline schools like ISB and a few others that capitalize on the weightage of a specific measurement is that if FT decides to tweak a single parameter, the schools can again drop off to oblivion (Singapore's NUS dropped off the list this year much to their dismay). And for that reason, they need to pay continuous attention to all other parameters and gain strength in them.
The effect of % salary increases
The other heavy weightage is due to the '% salary increase', and in emerging markets schools bring in younger students with lower salaries and when they graduate the new salary is much higher. Even if the absolute salaries are lower, the % increase is higher - why is this so? This is because post MBA most employers pay within a certain band typically irrespective of the incoming salary. This becomes a disadvantage for European schools - especially INSEAD and IMD where the salary of the incoming class is already quite high - higher than most other schools in the world. Therefore, the % increase is low. If this methodology was tweaked to reduce the weightage, INSEAD would shoot up to be within top 5.
update - March 2008: I recently found this statement from Rotman's dean Roger Martin where he delves into some more analysis on his school's ranking position that support my argument above. I feel your pain, Dean Martin!
My personal take on FT Rankings
The nice thing about FT rankings is that the methodology is transparent and available for analysis and critique. I do hope that for the next ranking they provide greater weight to alumni recommendations, breadth, network strength and median salary levels which are greater representations of a school's reputation and strength and reduce weights on parameters that cause wide distortions and even misleading exuberance.
Rankings have been, and will continue to be important. I disagree with blanket statements that "one should choose schools by filtering what is important to them." If the overall ranking methodology is absurd - like EIU's for example - which places Wharton at 21 below HKUST and puts Harvard at 13 - this assertion is plain ridiculous.
It simply implies that the parameters (or interpretation) are chosen by the ranking journalists who have little clue about what is important for rankings so they reflect a modicum of public perception. If you were to apply based on EIU's rankings, you would drop Harvard to go to Cranfield ; drop Wharton to go to Hong Kong university and drop Yale or Darden and go to Vlerick Leuven Gent Management School. Even my otherwise gentle 89 year old grandmother wouldn't agree with that.
Conclusion - Part I
This concludes Part I of ranking analysis. I will continue this with an analysis of LBS vs. INSEAD and why Harvard is not Number 1. I can live with FT rankings to some extent simply because it is reasonably accurate with respect to applicant preferences, brand recollect, overlap with US News for US schools. The specific order within the 1-10 do not matter so much because no matter what you tweak, minor differences will upset ranking orders.
For those who wish to critique my analysis or want to rant on my ISB assertion, you can do so as long as your post is objective and provides a clear reposte as to why my analysis is wrong. I would be happy to be proved wrong.
My Personal "Necromonger's MBA Ranking 2008"
And for the curious, what is my personal perception of the world's top 10 schools? This is my list in alphabetical order -
- Chicago
- Columbia
- Duke
- Harvard
- INSEAD
- LBS
- Sloan
- Stanford
- Tuck
- Wharton.
And 10-20 (in no particular order): Kellogg, Ross, UNC, Darden, Haas, NYU, Oxford, IESE, IMD, Yale.
Coming next (in a few days)
- LBS vs. INSEAD - what made LBS gain ground?
- What pushed Harvard to No. 5?
