There’s no shortage of guidebooks, websites, and ranking systems out there to help college-bound students figure out which school is best for them. The Obama administration believes there’s room for one more: The U.S. Department of Education is undertaking a project to rate the nation’s 7,000-plus institutions of higher education, in part to help families make decisions but also to set standards that will determine how much federal aid schools will receive.
The effort is just getting under way, so the agency hasn’t yet decided precisely how schools will be judged. Education Secretary Arne Duncan has said the ratings will take into account whether a college welcomes needy students, helps students graduate on time, and prepares them for good-paying jobs. As it is now, the government doesn’t have a way to measure whether the $150 billion in college grants and loans it doles out to students each year is efficiently spent. With tuition rising and student loans surpassing $1.2 trillion, the administration will try to help answer an elusive question: What is a college education worth?
Universities and some academics are wary. “The public does deserve, and should be asking, what’s the return on investment,” says Davis Jenkins, a senior research associate at Teachers College, Columbia University. “The pitfall—and this is particularly true in the president’s proposal—is that it’s very difficult, one might say impossible, to reduce college to a small set of numbers … that you can post funding to.” Janet Napolitano, who resigned as secretary of the U.S. Department of Homeland Security in July to lead the University of California system, told the Washington Post in December that she is “deeply skeptical” the department can create a meaningful system. Choosing a college is not like “buying a car or a boat,” she said. A December poll by Gallup and Inside Higher Ed found only 13 percent of college presidents and administrators think Obama’s plan is a “good idea.”
Duncan is urging calm. “To be absolutely clear, we have not even begun to develop the college rating system yet,” he said at a conference of college presidents in September. “I’m not sure how this is possible, but a few critics have already gone on the attack against a rating system they don’t like but that doesn’t exist.”
In speeches, the education secretary has tried to reassure faculty and administrators that they’ll be in on the planning. And he’s careful to explain that the goal is to “rate” colleges by grouping them according to performance, not “rank” them best to worst like the ubiquitous U.S. News & World Report guide, which he’s criticized for highlighting “trivial differences between elite institutions.” He’s promised the program will provide apples-to-apples comparisons of similar institutions, so Boston’s Bunker Hill Community College won’t be stacked up against Harvard University.
The administration says it will make decisions about what to measure by the end of 2014. Among the many questions it must sort out: If college administrators know their federal funding depends partly on how quickly students graduate, will they be less likely to admit disadvantaged students who need some remedial classes? And if schools are rewarded for high alumni salaries, what incentive do they have to encourage students to become teachers or public servants? “Believe me,” Duncan said at the college conference, “we are all aware that if this is not done well, a college rating system could create unintended consequences and allow gaming of the system.”
The administration is looking to a handful of states that already tie school funding to performance. Obama has pointed to Tennessee as a model—but its decades-long experiment with school ratings shows how difficult it can be to get it right. In the 1970s a young state researcher named E. Grady Bogue spent five years creating a set of standards to rate Tennessee’s colleges and universities. It was the first in the country and at the time considered so radical that “most of my friends told me it cannot be done, and it should not be done,” Bogue recalled in a September interview before his death in late October. Under his plan, most of a school’s state funding was based on the traditional measure of how many students it enrolled. Yet up to 5 percent of Tennessee’s higher ed budget rewarded performance based on measures such as student satisfaction and the number of accredited majors a college offered.
Almost two dozen states copied Tennessee to one degree or another, but the programs fell out of favor over time. Tight budgets have sparked new interest in making schools more accountable. In 2011, Tennessee became the first to award all college funding based on a range of performance measures. Some of Bogue’s squishier concepts like student satisfaction are now out, replaced with data-driven scores for how long it takes to earn a degree and the number of students who graduate. Schools are also rewarded for teaching adult and low-income students. Russ Deaton, a Bogue protégé who works for the state higher education commission, asked business leaders, politicians, and college presidents what was important to them. “We were the sounding board,” says Tennessee Technological University President Phil Oldham.
Every university in the state is judged by the same 10 criteria, but the measures are weighted differently from school to school. Oldham’s TTU is graded heavily on how many bachelor’s degrees it awards each year, while the University of Tennessee at Knoxville, the state’s flagship institution, is expected to attract more research grants. Oldham says the formula is probably overly complex and has “too many moving parts.” Yet he says that may be necessary to allow “some flexibility for individual campuses to define their niche.”
Tennessee only had to rate 22 colleges and universities. The Obama administration is trying to do it for thousands of schools across 50 states and get it rolling in a year, a goal Deaton calls “probably ambitious.” In the fall, Education Department officials held four sessions with university administrators and have met with associations that represent public schools and historically black colleges, asking for their ideas. On Dec. 17, Duncan issued an official request for suggestions from the public.
The Education Department has already received useful ideas, says Deputy Under Secretary Jamienne Studley, who’s intrigued by the way Cook’s Illustrated lays out “the key characteristics of blenders and canned tomatoes” and summarizes the results as “highly recommended,” “recommended,” “recommended with reservations,” and “not recommended.” “That’s the kind of level of information that we talk about when we talk about it informally,” she says.
It’s not clear whether all this effort will make schools better. Columbia’s Jenkins says studies of states that tied funding to performance didn’t find a big effect on education, in part because the amount of money at stake was small. Oldham says Tennessee’s ratings have made schools’ spending decisions more rigorous. “Efficiency is valued, as opposed to exploration,” he says.
That might not have pleased the grandfather of school ratings. Shortly before he died, Bogue bemoaned the new focus on hard data as too “businesslike,” believing it undervalues the “magical” moments that define the best college experiences. Deaton concedes his mentor may be proven right. “Our approach doesn’t try to solve every problem,” he says. “It’s making a different set of trade-offs.”