Methodology used to rate colleges
Saturday, 25 August 2007
Rebecca Knight
MID-August in the US has long been given over to an annual ritual of anxiety. High-school seniors, panicked parents, college presidents and trustees are joined in fretting about the guide to America's Best Colleges published by the US News & World Report, a weekly news magazine.
The rankings, which started in 1983 and released to subscribers the other day, have long had an impact on college campuses and beyond. However, this August the issue - one of the best-selling covers the magazine puts out every year - will hit newsstands amid a swirl of controversy.
In June the Annapolis Group, a cluster of university presidents representing 115 liberal arts colleges, condemned the methodology the magazine uses to compile the rankings and announced plans to create an alternative to the commercial college ratings.
The group agreed to develop a database, available on the internet, that would provide families and prospective students with "accessible, comprehensive, and quantifiable data" on US colleges and universities. Many presidents also vowed not to participate in the magazine's rankings in future.
"This is not an attack on US News. It's simply a statement that [parents and students] should take the rankings with a grain of salt," said Kate Will, president of Gettysburg College and chair of the Annapolis Group. "This is something they do to sell magazines."
Ms Will said that the news weekly's methodology was "flawed". She specifically pointed to the rating formula for the so-called "reputational score" collected from a survey that asks college presidents, provosts and admissions deans to rank schools' academic programmes on a scale of one to five. "It's unrealistic to expect academic officials to know enough about hundreds of institutions fairly to evaluate the quality of their programmes," she said.
The group's announcement came on the heels of a scathing letter by 12 college presidents that criticised the US News rankings for relying too heavily on measurements that are determined by institutional wealth. The letter now has 61 signatures.
Lloyd Thacker, executive director of the Education Conservancy and the man behind the letter, said the rankings represented the "commoditisation of college admissions".
"Do we treat choosing a college the same way we treat choosing a car, or is education something different?" he asked.
The magazine says its yearly evaluations provide a service to parents and students. For instance, the rankings consider factors such as the proportion of faculty with a doctorate, which the magazine sees as a measure of the quality of the professors; standardised test scores of the incoming class, which is a gauge of the quality of the students; and alumni giving, which is used as a proxy of student satisfaction with the school.
Brian Kelly, editor of US News, said one of the reasons the rankings have become so influential over the years is due to the rising cost of college. Tuition, room and board at a leading private college in the US is pushing $45,000 a year, and parents want some assurance that they are getting their money's worth.
"College has become so expensive that it's a luxury good," he said. "People are trying very hard [to get as much information as possible], as they would with any financial decision. We disagree with colleges when they say this is not a consumer good or a product."
But Mr Thacker claims that the rankings have had a harmful effect on educational priorities, encouraging gamesmanship among colleges eager to move up the list. "It's made colleges compete with each other on differences that do not have much of an educational consequence, such as alumni giving," he said.
Mr Kelly said he doubted that the college's lack of participation would have an impact on future issues. "We are still going to rank these schools, with or without their co-operation. We will get the data from other sources.
"This is a journalistic enterprise. Just because people don't talk to us doesn't mean we're not going to do the story," he said.
..........................................
FT Syndication Service
MID-August in the US has long been given over to an annual ritual of anxiety. High-school seniors, panicked parents, college presidents and trustees are joined in fretting about the guide to America's Best Colleges published by the US News & World Report, a weekly news magazine.
The rankings, which started in 1983 and released to subscribers the other day, have long had an impact on college campuses and beyond. However, this August the issue - one of the best-selling covers the magazine puts out every year - will hit newsstands amid a swirl of controversy.
In June the Annapolis Group, a cluster of university presidents representing 115 liberal arts colleges, condemned the methodology the magazine uses to compile the rankings and announced plans to create an alternative to the commercial college ratings.
The group agreed to develop a database, available on the internet, that would provide families and prospective students with "accessible, comprehensive, and quantifiable data" on US colleges and universities. Many presidents also vowed not to participate in the magazine's rankings in future.
"This is not an attack on US News. It's simply a statement that [parents and students] should take the rankings with a grain of salt," said Kate Will, president of Gettysburg College and chair of the Annapolis Group. "This is something they do to sell magazines."
Ms Will said that the news weekly's methodology was "flawed". She specifically pointed to the rating formula for the so-called "reputational score" collected from a survey that asks college presidents, provosts and admissions deans to rank schools' academic programmes on a scale of one to five. "It's unrealistic to expect academic officials to know enough about hundreds of institutions fairly to evaluate the quality of their programmes," she said.
The group's announcement came on the heels of a scathing letter by 12 college presidents that criticised the US News rankings for relying too heavily on measurements that are determined by institutional wealth. The letter now has 61 signatures.
Lloyd Thacker, executive director of the Education Conservancy and the man behind the letter, said the rankings represented the "commoditisation of college admissions".
"Do we treat choosing a college the same way we treat choosing a car, or is education something different?" he asked.
The magazine says its yearly evaluations provide a service to parents and students. For instance, the rankings consider factors such as the proportion of faculty with a doctorate, which the magazine sees as a measure of the quality of the professors; standardised test scores of the incoming class, which is a gauge of the quality of the students; and alumni giving, which is used as a proxy of student satisfaction with the school.
Brian Kelly, editor of US News, said one of the reasons the rankings have become so influential over the years is due to the rising cost of college. Tuition, room and board at a leading private college in the US is pushing $45,000 a year, and parents want some assurance that they are getting their money's worth.
"College has become so expensive that it's a luxury good," he said. "People are trying very hard [to get as much information as possible], as they would with any financial decision. We disagree with colleges when they say this is not a consumer good or a product."
But Mr Thacker claims that the rankings have had a harmful effect on educational priorities, encouraging gamesmanship among colleges eager to move up the list. "It's made colleges compete with each other on differences that do not have much of an educational consequence, such as alumni giving," he said.
Mr Kelly said he doubted that the college's lack of participation would have an impact on future issues. "We are still going to rank these schools, with or without their co-operation. We will get the data from other sources.
"This is a journalistic enterprise. Just because people don't talk to us doesn't mean we're not going to do the story," he said.
..........................................
FT Syndication Service