Rife with Flaws, are College Rankings Even Useful at This Point?

Rife with Flaws, are College Rankings Even Useful at This Point?

Every year, U.S. News and World Report, Princeton Review, Money and others publish new rankings of the best universities and colleges in the nation. Some institutions — Harvard, Princeton — top every list, every year. Others — Yale, Stanford — fluctuate within and between publications. So what are prospective students and their parents to make of these rankings? More importantly, what are they to make of the apparently mercurial quality of these institutions? Whether or not you're the first in your family to attend college, these publications tend to complicate rather than clarify.

The variance in rankings of these institutions has little to do with changes within the institution from one year to the next. Instead, know that each publication uses it own particular methods for picking and choosing.

First, metrics like acceptance rate and retention rate are measured and categorized. Then, categories like affordability and reputation are weighted based on the publication's discretion to determine a college's score. Finally, institutions are ranked and results published.

click to enlarge Rife with Flaws, are College Rankings Even Useful at This Point?
Photo by Vaughn Gurganian

If it sounds a bit superficial, it is.

Data gathered from those chores can be easily categorized two ways: qualitatively and quantitatively.

For example, Princeton Review surveys thousands of students at colleges and universities around the country. Up until a few years ago, Princeton Review would spend days on campuses handing out paper surveys to students. Needless to say, this method was much less effective than their modern, digital survey, which reaches far more students.

U.S. News and World Report takes this a step further: 22.5 percent ("significant weight") of an institution's rating is based on the "the opinions of those in a position to judge a school's undergraduate academic excellence," including academics, administrative faculty and guidance counselors from public and private high schools. Who, now? It's not very clear; thousands of professionals nationwide are surveyed and/or consulted, and results are truncated at the high and low end to eliminate outliers.

Still with us?

All publications rely on qualitative data to some degree, and all qualitative data is subjective and flawed to some degree.

The National Association for College Admissions Counselling (NACAC) has criticized U.S. News and World Report for allotting such significant weight to reputational surveys. Such surveys are not mandatory, based on a simple one-to-five scale, and respondents are allowed to skip questions. For years NACAC has repeated this critique, yet the heavily weighted qualitative category remains.

While the qualitative value of institutions should be taken with a grain of salt, the quantitative character of institutions cannot be overlooked.

Money's methodology uses surveys to determine the weight of the given category. In the 2016 rankings, weight is divided equally between quality of education, affordability and outcome. Within those categories, metrics like net price of a degree, graduation rate and return on investment (ROI) are weighted. The end result: a scorecard largely based on quantitative data. Every publication accounts for these metrics; but, again, they are weighted based on the publication's discretion.

Not calculated by Money or many publications is the cost of living in the town or city where the college is located. This is a big deal. For example, students can certainly expect to spend much more money on living expenses in the San Francisco Bay Area, where two Top 10 schools (Stanford and University of California-Berkeley) are located, than they would at University of Michigan in Ann Arbor or University of Virginia in Charlottesville, two other Top 10 schools. Moreover, though both Stanford and UC Berkeley offer a higher percentage of students' need-based aid, students at UV and Michigan still pay less toward tuition annually.

Weighing the importance of categories and metrics is itself suspect. For first-generation college students and those going into less lucrative fields, price may outweigh prestige, while students from wealthy families may be able to prioritize prestige.

Long story short, all of that is to say this: Annually published rankings are not a substitute for determining what college is right for you. When deciding what school is right for you, NACAC advises you ask yourself: "Do these rankings reflect my own interests?" (Scene advises you concurrently to ask: "Which bars have the best deals on wings and Jameson shots?") Annual rankings are, however, pricelessly valuable resources for the prospective college student.

Despite flaws in methodology, U.S. News and World Reports, Princeton Review, and Money offer a multitude of rankings from "Best Universities," to "Best Graduate Schools," to "Best Value" and even rankings by degree.

It's up to you to place the context.

Metrics like percentage of students who receive need- or merit-based aid, student-teacher ratio, retention and ROI are difficult if not impossible to come by on your own. However, a visit to the institution's website is necessary to determine the catalog of majors, quality of departments and scholarship opportunities. For many, a visit to the campus and conversation with financial aid officers and academic advisors is necessary to seal the deal.

With the average debt per student breaking the previous year's record year after year, more and more is on the line for ambitious high school students anxious to continue their education and find a lucrative and fulfilling career. The greatest test they'll face in high school is not an exam or standardized test, it is how they determine their trajectory for the years following high school. That trajectory starts with how they approach the college application.

Like this story?
SCENE Supporters make it possible to tell the Cleveland stories you won’t find elsewhere.
Become a supporter today.
Scroll to read more Cleveland News articles

Join Cleveland Scene Newsletters

Subscribe now to get the latest news delivered right to your inbox.