It’s that time of year again, when many high school seniors — and their parents — are stressed out about college applications. They pore over college brochures and websites, talk to friends and spend sleepless nights eyeing rankings in an exhaustive exercise to choose the best school.

But what makes a good choice? Are “top-ranked” and “right-fit” synonymous, as widely believed?

A new paper by scholars at Stanford Graduate School of Education wades through the research to find out. The report, released Oct. 2 by GSE-affiliated Challenge Success, summarizes the most reliable studies of the college experience and its benefits, real or perceived, and finds that college rankings rely on deeply flawed measures that say little about how well undergraduates do in school and later in life. 

Indeed, when it comes to student learning, well-being, job satisfaction and future income — four key outcomes that students and their families value highly — studies suggest that students themselves determine their destinies, not their schools.

"Research tells us that the most successful students, both in college and beyond, are the ones who engage in the undergraduate experience regardless of how selective a school may be," says the paper’s co-author Denise Pope, a senior lecturer at the GSE and co-founder of Challenge Success, which seeks to redefine success in student learning and achievement. "This is almost always the case whether a student attends the top-ranked or 200th-ranked college."

A closer look at methodology

In their report, Pope and her collaborators reviewed dozens of seminal studies on U.S. undergraduate education in recent years in the hopes of shedding light on three critical areas: the methodology behind college rankings; what, if anything, they say about student success; and, the meaning of "right fit" when choosing a school.

Like many critics, Pope says the core problem with college rankings lies in their methodology, which often changes from one year to the next. "The numbers behind them tend to be easy to get, imprecise and arbitrary," says Pope.

Consider two measures used by U.S. News & World Report to rank schools that together account for almost half of a school's score: previous and projected graduation rates, and reputation.

But research suggests that graduation rates have little to do with the institution; they're based more on individual circumstances, like family income. Reputation is also misleading, says Pope: while it's intended to reflect the caliber of teaching, the likelihood that the college administrators and high school counselors surveyed each year track shifts in quality among individual schools is low. Rankings help guide them, which makes reputation "something of a self-fulfilling metric."

The report analyzes other problematic measures, like class size and education levels of faculty. "When it comes to college rankings," says Pope, "there isn’t an agreed-upon set of metrics used or a scientific way of weighting them."

What the research says

Most research on college outcomes looks at income potential, including job opportunities and long-term earnings, according to the new review. The results are mixed, in part because researchers don't have a common definition for what they call "selectivity" and how to measure it. For some, selective includes schools that accept both high and low percentages of applicants.

The review highlights two key results from leading studies on income: While data suggest there are modest financial benefits to attending a "top-ranked" school, the biggest disparities in earnings are seen among graduates from the same institution, whether it's considered selective or not.

There's one exception when it comes to income. Research suggests that first-generation students and low-income minorities who attend "top-tier" schools have significantly higher incomes than those who graduate from other colleges.

The report notes, however, that college isn't all about income potential. Students say they attend college for other reasons as well — namely, to advance their learning, improve their general well-being and find satisfying careers. Here, studies show that the college they attend matters far less than the extent to which they engage in the undergraduate experience. Students who, for example, study hard, form strong relationships with professors and participate in the college community tend to thrive during and after school whether they attended a "top-ranked" institution or not.

"Everyone knows the person who slacked off and went nowhere, and the one who made it big," says Pope. "Is that all due to the college? No. Even from a logical standpoint, it makes sense that individual characteristics make a big difference."

The report ends with two calls to action. One is for students to look past rankings to find programs that are the "right fit" for them — maybe because of academics, financial aid, location or extracurricular opportunities. The other is the need for even more research that looks at other commonly-held assumptions about college, including the belief that attending a "top-tier" school is mostly about the "network effect" of gaining access to elites.

"It's really hard to find an accurate, rigorous way to account for things we assume make a difference for college kids," says Pope.

It's a problem both the research and rankings share.

Denise Pope and GSE Dean Dan Schwartz discussed the research on an episode of "School's In," a half-hour radio show that airs weekly on SiriusXM Insight Channel 121. Listen from the link below, and find more episodes of "School's In" at the Stanford Radio main page.