|
|
 |
 |
 |
 |
   |
HOW WE KEPT THE DATA UNSULLIEDWe're glad the rankings are taken so seriously. But this year, some people took them too seriously. After an investigation lasting several months, BW determined that some students at five schools tried to ''game'' the system by inflating their responses on the student portion of the BUSINESS WEEK survey. They urged their fellow classmates to answer the survey questions positively in hopes of moving up in the rankings. The five schools: Dartmouth, Duke, Purdue, the University of Texas at Austin, and Washington University. There is no evidence that school administrations or faculty had anything to do with the students' attempts. The deans at four of those schools argue that if any discussion of the surveys did occur, it merely reflected the enthusiasm students feel about improvements at their schools. Says Ramesh K.S. Rao, director of the MBA program at Texas: ''I hate to see [our students'] uniformity and beliefs about the direction we're going penalize us.'' Purdue's dean, Dennis J. Weidenaar, says his own investigation uncovered no proof of any discussions or transgressions at all. Nonetheless, we have strong evidence indicating that some of the graduate surveys at those schools were tainted. As a result, we adjusted the grads' scores at those schools, giving the 1998 data a weight smaller than its usual 25%. We increased the relative weighting of two previous student surveys from 1996 and 1994. We kept the weight of the recruiter survey intact at 50%. The student efforts typically took the form of boosterism. At Washington, students distributed ''mock surveys'' with BUSINESS WEEK's logo, which showed how the rankings were calculated, and reminded students that the BUSINESS WEEK survey was not the right forum for criticism. At Texas, several student government leaders circulated a memo reminding students how important it was to keep the school competitive in the rankings. ''Let's ensure that we place the University of Texas MBA program on a level playing field,'' it stated. And at Dartmouth, students told each other that the survey's 1-to-10 scale was irrelevant; they suggested giving the school ''9''s if they hated the school, ''10''s if they loved it. To maintain the integrity of our data, we checked out every rumor or suggestion of misconduct we heard. We followed a three-step quality-control process. First, we called students, faculty, and deans to determine what happened. We learned about some events from students, who wrote saying they felt their school had been misrepresented. Next, we brought in statistical consultants--David M. Rindskopf and Alan L. Gross, professors of educational psychology at the City University of New York Graduate School & University Center. They pored over the student surveys, searching for aberrant data or odd patterns that failed key variance tests both within one school and relative to others. They advised BUSINESS WEEK to remove anomalous surveys, as well as those of anyone we felt answered inaccurately. CONFIDENCE. As a cross-check, we consulted with David K. Krane and Humphrey Taylor, executiVe vice-president and chairman, respectively, at pollster Louis Harris & Associates Inc. They helped us develop a new survey, asking respondents if they were aware of any attempts to influence the survey and whether, if so, they felt that the efforts had succeeded. Applying all three methods, we concluded that some data from the five schools was suspect. The BUSINESS WEEK survey process differs from other rankings that rely in large part on one source--material provided by the schools. Our rankings, by contrast, combine the views of thousands of people. With such a volume of data, we belieVe our results continue to offer the truest reflection of customer sentiment. And the checking procedures we've put in place give us more confidence than ever in our data. BUSINESS WEEK will continue to do what it takes to maintain the integrity of the B-school surveys. If necessary, we will even remove schools from our rankings. We will also issue guidelines to students and schools to further clarify that the survey is meant to measure individuals' satisfaction level with their schools. It's an opportunity that MBA students didn't get just a few years ago.
RELATED ITEMS
|
Updated Oct. 8, 1998 by bwwebmaster
Copyright 1998, by The McGraw-Hill Companies Inc. All rights reserved.
Terms of Use