|
It's as if the givers of the GMAT -- the test prospective students have to take to get into business school -- are trying to prepare MBA candidates for the rapid changes of the corporate world. Just a year or so ago, would be leaders of industry were forced to adapt to the digital GMAT CAT (computer-adaptive test) -- the formerly hand-written but newly computerized exam more than 1,500 B-schools require before they'll accept a student. On Feb. 10, the other shoe will drop: Starting that day, the essay section of the GMAT test -- the essay section, mind you -- will be graded by a computer. Actually, one of the two readers of each such essay -- which is called the Analytical Writing Assessment (AWA) part of the test -- will be a computer. The other will still be a human, whose days -- who knows? -- could be numbered. It will be the first time a major admissions test will use a computer to evaluate an applicant's literary competence, in this instance using a piece of software called "e-rater." For the 210,000 aspiring MBAs who each year take the test from the Educational Testing Service (ETS) and the Graduate Management Admission Council (GMAC), the e-rater will be a source of some concern, at least initially. GMAT scores have a major bearing on who gets into the best schools -- which often determines who makes the most money. Will the e-rater skew the results? And if so, in whose favor? "One of the problems is that there aren't just hundreds of ways to write an essay, there are infinite ways to approach it," says Jason LaBelle, a B-school applicant and an assistant account executive at DraftWorld, a direct marketing firm. "I'd hate to be the person who writes an excellent essay that doesn't quite fit the [software's] criteria." Frederick McHale, the vice-president for assessment and research at GMAC who spearheaded the GMAT's electronic conversion, doesn't see much danger of that. In fact, he says, it's precisely the focus on logic and reasoning that the AWA imposes that allows e-rater to accurately assess an essay writer's performance. "We're looking for analytical writing ... for this holistic view of whether you were able to express your ideas in writing -- not for creative writing, but for specific thoughts," says McHale, expressing a thought that might give the e-rater a workout. The AWA itself is a recent addition to the GMAT, having first appeared in 1994 as the result of lobbying by schools for a test measuring applicants' communications and writing skills. Ever since, test takers have been required to answer two essay questions selected randomly from a pool of 175 (all 175 may be viewed for free on the GMAC Web site: www.gmat.org). One question asks test takers to analyze a specific issue and develop a reasoned argument. The other asks for a critique of a point-of-view argument. The time limit for each essay is 30 minutes. Graded separately from the verbal and math sections of the GMAT -- which account for most of the test's score -- the AWA is measured on a 6-point scale, with six being the highest mark. Since its inception, each applicant's AWA essays have been reviewed by two college professors, with a third helping out if the initial two readers are more than a point apart in their appraisal. E-rater will displace one human, though the other two will be around in case the computer gets out of line. GMAC and ETS officials are confident that Hal -- or rather, e-rater -- will do just fine, in part because they've tested it for five years using thousands of previously-graded essays. Now it knows the difference, they say, between golden prose and drivel. In fact, e-rater agreed up to 94% of the time with two professors on a test involving thousands of 1997 essays. For GMAC, e-rater makes it easier to turn around test scores faster -- and to save money. Though McHale is uncertain how much time e-rater will save, he's looking for "a significant reduction in the total cost of the test over time." For the immediate future, though, he sees the cost of taking the GMAT not dropping from the current $125. B-school admissions officials, meanwhile, welcome the faster turnaround time and are largely unconcerned by e-rater. That's because many consider the AWA a secondary measurement. Says Linda Baldwin, director of admissions at UCLA's Anderson School: "Many support the concept of the AWA and put it among the many verbal indicators used to judge an applicant's competence, but it's not the verbal indicator. It doesn't make or break the candidate." Setting that aside from a moment, "when you think about it," says GMAC's McHale, "[human] readers look for very specific features in the essays they evaluate, creating rubrics and scoring systems. If you can set rules down, you can program them into a computer." Let's see: Fourscore and seven years ago.... Nah. A computer would never go for that. |
Copyright 1999, by The McGraw-Hill Companies Inc. All rights reserved.