Top Schools Resign from Ratings Game
by Danielle Tomassini

The Facts

On April 6th, a battle began.

Harvard Business School and the University of Pennsylvania’s Wharton School stated that they would not be providing their MBA students’ email addresses to Business Week in order for the magazine to conduct its survey. This survey allows Business Week to publish MBA program rankings based on customer satisfaction responses. In sum, Harvard and Wharton cited two reasons for the basis of their decision. One, they wanted to insure privacy of their students (as they do alumni & faculty); and two, Business Week's methodology for its rankings do not provide a wholistic picture of a particular school.

Business Week has responded by trumpeting Harvard and Wharton's decision in their own magazine, and has also directly requested interested students from these schools to contact Business Week. Business Week has reported that many students will still be participating in the survey regardless of Harvard and Wharton’s decision.

Wrong?

Are Harvard and Wharton justified in their decision? Most editorials have attacked Harvard and Wharton, stating that they are not transparent and are simply bitter because Wharton has fallen to number 5 from a number 1 ranking, and Harvard has maintained a number 3 ranking – not yet reaching the number 1 spot. While objective rankings that are primarily based on aspects like standard scores are useful, there is something to be said for hearing customers' viewpoints. Although school names like Wharton and Harvard carry a lot of weight, prospective students want to make sure their education is more than just a brand. The school will "sell" you on its premium- reasoning it's worth shelling out $120,000, but the feedback from current students and alumni will reveal if it is truly worth the money.

Rankings also help insure that the school is paying attention to their consumers – students and recruiters. Because part of the rankings are determined by current students, the schools need to make sure their students are satisfied by providing good professors, courses, and resources. Surveys serve as checks and balances on the institutions; schools will want to serve their customers well in order to maintain a high rank.

Furthermore, rankings are something that prospective students find useful. The difference between a 1st spot and a 2nd spot in a ranking may not matter that much. But rankings are useful in determining the difference between number 5 and number 25. Information provided from current students through surveys helps prospective students understand the culture and customer service of the school. Although a prospective student could speak with a current student, the survey data provides information on how the general student population feels, which may be different from the feedback a prospective may glean from one or two people.

Right?

While Wharton and Harvard might be wrong in declaring that third party rankings are not useful, they do have some valid points about the current rankings system. One thing I find interesting is that many people are placing the blame solely on Harvard and Wharton, but not looking at what they are opposing. It does not seem to me that these schools do not value surveys; instead, they do not think that the current system is a valid and rational one.

When Business Week initially launched their rankings back in 1988, the rankings system was probably purer and truer than it is today. Now, however, I think the rankings system has become corrupt and is ripe for change. The problem is three-fold. One, the schools put too much weight on the rankings. From personal experience, students at my school (University of Michigan) will be careful what issues they publicize to prevent a drop in the rankings. Two, if the school falls in rankings, students worry that their education will be less than present value. Furthermore, administration worries that they will not be able to attract the highest caliber of students if they are not ranked in the top 10 or top 5 schools.

Many schools conduct internal surveys of their students. Perhaps it is possible for these internal surveys to be more vital, as students can be even more honest than they would be in an external survey. There have also been suggestions that schools will pander to rankings by playing games. For example, it has been said that some schools waitlist a huge number of people, and will then selectively admit those that they believe will matriculate. This improves their yield, and thus, their rankings. Schools should pay more attention to details other than what will get them ranked higher.

The second part of this problem is what do these rankings really tell us? It is hard to say. Current students may not always be honest, and the formulae that companies like Business Week use to rank schools is subjective. The criteria that Business Week views as most important in a school may not be the same criteria that I would use to choose a school. Surveys and rankings should only be taken as one factor in deciding on a school. Speaking to actual students and alumni will provide greater information than a survey would to determine your personal fit within a school's culture. However, rankings and surveys can at least point you in a general direction.

Solutions?

Linda Abraham, Founder and President of Accepted.com, describes Wharton and Harvard’s decision as “throwing out the baby with the bathwater. It is valuable to see the results from surveys of business school deans, current students and recent alumni, recruiters etc. Those elements are the baby.” Linda goes on to describe the fallacy of the rankings as the bathwater. “There is no one number that can represent the appropriateness of any one school to all prospective students. That term is a fallacy, a misnomer, and an enormous disservice to the applicant community.” External surveys are useful information, but they are not comprehensive and should not be used as the sole decision-making criteria in choosing a school. So how do we make it better?

The MBA program rankings clearly need to change. No one wins by Wharton and Harvard not participating in these surveys; Business Week sales may decrease, prospective students do not receive the entire scope of information, and current students lose bragging rights on their place in a publication's latest rankings.

Popular ratings do not equal valuable ratings. Surveys can provide some perspective on an MBA program, but there may be better ways than how it is currently done. Wharton and Harvard may be wrong to not participate in an external survey, but the system is outdated. One alternative is to do away with the numbers, and use tiers instead. So simply show the top 5, top 10, top 20, etc. schools but don’t assign the schools in an order. That way schools can say if they are in the top 10, but don’t fret if they fall from a 7 to a 9. However, there will still be an air of competition for schools to jump tiers.

Another option is to have a ranking system, but allow individuals to weight the deciding criteria on their own. For example, there could be a website where on the first page there are 5 criteria. There could then be pull-down tabs of 1 to 5, and the individual could assign one of the values to each criteria. The individual could then hit “compute” and the program would figure out how schools would be ranked depending on the weighted criteria. So for my preferences, Michigan could be number 1 and Wharton number 3, but for somebody else it could be Fuqua as number 1 and Columbia as number 3. This allows for the subjectivity to come from the individual and not from Business Week.

Maintaining perspective is important. Wharton and Harvard are not wrong to want to change the system, but they should not do away with surveys. Third party surveys, such as Business Week’s, can provide potential applicants with useful information, but surveys should not define an institution.

Supplemental Article- A Wharton Alum Speaks Out


TrackBack

Comments