What type of reception has the GMAT CAT received? Is the computerized version prone to glitches? Are test takers satisfied? On Apr. 20, Business Week Online's Nadav Enbar caught up with GMAC President David Wilson to find out the latest. Here is an edited transcript of their conversation:

BW: We have just passed the six-month mark since the introduction of the computer adaptive GMAT (called the GMAT CAT, which was released on Oct. 11, 1997). How's it going? What has worked, what hasn't, and what needs improvement?

Wilson: It's going extremely well. We moved from testing four times a year at 970 locations to 250 days a year at those locations. We originally expected about 5% of our test takers -- or 10,000 people -- would still be taking the paper-based test. (Editor's note: Currently, the test sites in about 30 countries [see below*] have not yet been converted to the computerized version.) The reality, however, is that only 1% of the test-taking population is still using the paper-based version. That's mainly because we were able to install the CAT in mainland China (on Jan. 17).

Our original thought was that problems for the CAT would be driven by the lack of reliable power sources -- because if you get surges, bursts, or brownouts, the system would crash in the middle of a test. And so we thought we would have to rule out test sites that didn't have reliable power sources. To our surprise, however, we've found pretty stable power supplies everywhere. It's only in countries where either there aren't enough test takers to warrant setting up a center -- like Andorra, where there are just six test takers a year -- or places where there is military unrest that we chose not to install the CAT.

BW: Has the amount of people testing per week remained constant? Is there much fluctuation happening?

Wilson: We're testing 5,000 people a week pretty consistently. And, of course, there are surges around the admission deadlines -- a number of schools are still batch processing, while others are going to continuous processing. One always touches wood and hopes that nothing more significant than the one hiccup on Dec. 11th will ever happen again. (Editor's note: On that date, 800 of 1,100 test takers were unable to launch their GMAT CAT because of a malfunction in a security file. For the full story, go to BW's Dec. 17, 1997, news item.)

BW: Has such a widespread shutdown recurred?

Wilson: No. In hindsight, I feel that the December event was a good news/bad news situation. The bad news was that it happened at all. But the good news was that in refusing to boot, what the CAT was doing was triggering its fail-safe mechanism. It was unfortunate that it happened for the 1,100 people, but within less than 24 hours, fewer than 100 people still needed to be rescheduled. Everybody (the test sites) reopened on Sunday, extended their hours, and we moved quickly to make people as comfortable as possible. And we personally called admissions directors for any test takers who had difficult deadlines to meet. You have to put these types of events into perspective, though. If, for example, you go back to October of 1996, we had to eliminate the entire administration in China -- 1,300 test takers were bam!, gone. And they didn't get rescheduled for four months! So, it's hard to argue that the situation hasn't improved.

BW: Are you seeing more test takers in general? Several admissions directors I've spoken to noted that there was an initial boost, but that it has since leveled off.

Wilson: There was a huge surge in paper administrations last March and June. Huge. Like a 35,000 test-taker increase. And that's not uncommon. Typically, the year preceding a new introduction is going to have a surge, while the first year into the new product is going to dip a bit. If we aggregate the number of test takers plus the appointments made, however, I think we're going to be running right on target -- with about 227,000 (last year it was at 231,000).

There are two numbers you have to focus One is the test registrants, and the other is the test takers. On the paper-based test, 18% of the test registrants didn't show up. I think that's partially because they had to book their test a minimum of six weeks in advance. MBAs are working people; they work on Wall Street, in banks, in law firms, in accounting firms, in consulting practices, and in business. Anyone who's been there knows -- things come up, and you have to prioritize. So oftentimes, people would have to cancel their GMAT appointment.

The CAT allows you to -- a) reschedule, and b) make your appointment when you're ready. I don't have specific data, but the number of "no shows" has really come down since we've implemented the CAT.

BW: How close to the test date can a test taker register?

Wilson: It depends on your risk profile. If you're high-risk, you can register on the same day or day before. Obviously, if you try registering for a Saturday test the day beforehand, then don't bank on getting in.

BW: Dave, I've heard several people criticize the CAT's scoring methodology. Some have even gone so far as to say that the CAT is too easy.

One of the things that administrators weren't aware of until now is that historically during the paper-based test (when the test was only administered on one day during October, January, March, and June), scores coming in during October and January achieved a higher mean average than those calculated in March and June. Students who tend to take the GMAT in the fall, tend to be applying to highly competitive, full-time programs. So, you could argue that for the CAT, more competitive test takers are taking the CAT earlier than later. It's too early to judge whether the mean CAT score is higher than the paper test because we haven't completed a full-year's cycle.

But, the answer is yes. The mean score is slightly higher for the CAT at this point, than it was for the paper-based test (the mean is 507, while the standard deviation is 107). But let me remind you that we still don't have a good standard of comparison, because the CAT has only been available for six months. A year ago, we had over 3,000 people take both the paper version and the CAT to gauge whether there was any difference in result. And our study concluded that there was, in fact, no difference.

BW: It has been suggested that the CAT's scoring methodology completely changes the manner in which a test taker should prepare for the test. That if you do well on the first 10-20 questions -- which determine the level of difficulty and point allotment per question -- that you're home free.

Wilson: That's dead wrong. I've heard that same claim being made by both Kaplan and Princeton Review (two prominent GMAT preparatory services), and their material is simply wrong. The logic that an individual who answers the first 20 questions correctly doesn't have to worry about the rest of the questions in the test is absurd. For example, a particular test taker did well for most of the test, but then guessed on the last 10 questions because he figured he had it made. But that just wasn't the case: He went from a projected 640 to a 420.

We now have the capability to save each stroke you make on your CAT keyboard, as well as measure how much time elapsed between each of your answer selections. That allows us to go back and evaluate each individual test. I know, if you took an average of 107 seconds for the first 15 questions, and 3 seconds for the last 15, that you guessed for your last 15 questions. It's unfortunate that people follow the "first 10 strategy" because that just isn't the way the algorithm works.

BW: How does the algorithm work?

Wilson: The first questions do in fact signal whether you go on to more or less difficult questions. That's one of the functions of the adaptive test: It responds to your response. However, that function doesn't stop after a given amount of questions. It continually adapts questions and point allotments based on your response. The bad information being put out there is that once you get to a certain level, after a specific number of questions, that your score isn't going to fluctuate too much. That's just wrong. I'm sorry that's the advice that's being put out there. And we actually advised to the contrary in November of 1996, in a letter that we sent to all of the test prep organizations.

BW: Do you have plans to work with Kaplan or Princeton Review to correct the situation?

Actually, within the next week (4/27), we'll be sending out a letter to all schools and test prep organizations explaining what we feel is misinformation.

BW: Are there any changes in the GMAT CAT that you're thinking of implementing?

Wilson: We are not going to implement changes into the test yet, but you raise a really good question. There were two primary purposes for creating the computer adaptive test. No. 1 was customer service. You can't offer the test four times a year and provide customer service. Prospective MBAs need to be able to schedule the exam when they're ready. There are a variety of programs now that are out there -- weekend MBAs, global MBAs, part-time MBAs, full-time MBAs, evening MBAs, morning MBAs..... Those options are being created because there's a market demand for it. And so, we had to move to it, and that was the first rationale.

But the second rationale was just as important: added flexibility. When we used the paper-based system, it was only delivered four times a year, leaving us no flexibility to add new assessments. We have now created a distribution system for the CAT, which adds to the test's flexibility.

BW: What's the GMAT CAT's price tag?

Wilson: The test is $125 domestic, $160 international.

BW: Has the cost increased since going to the CAT?

Wilson: Yes, it has gone up by about 11% -- from $112 to $125 (domestic) -- since going to the CAT. What's interesting, though, is when people looked at the cost of the paper-based test, they only accounted for the registration fee. Nobody realized that the average fee was about $30 higher, however. That's because if people didn't show up, there were late charges, stand-by charges, you name it. Offering the test 250 days a year instead of four requires a small bump up in price. But the ends, I think, justify the means.

BW: Is the increase in price largely attached to maintaining security?

Wilson: Yes. The security is very expensive -- it always has been. The CAT terminals, in fact, are dumb; it's the server that has the content. We've actually had some break-ins where several of our dumb terminals have been stolen, but nothing of serious input has ever been taken. If you're going provide the schools with the comfort level they need to count on the GMAT score as a thread in the fabric of admissions, it has got to be a reliable score. And so, it's expensive...

BW: Let's return to the new GMAT assessments you are thinking about implementing. What are they?

Wilson: Some of our schools that are more technically focused, or more quantitative, have suggested that our quant score really doesn't differentiate for them. All of their candidates are at the very highest score. What they would like to do is have a second test on differential equations, because they say if a candidate doesn't know "diffeqs" when they show up on campus, they're never going to catch up. The differential equations test would be taken for purposes of diagnosis as opposed to assessment, and would not change your admission or any scholarships or funding that you have lined up. Admissions directors would use the test results to balance their school's team-based learning, for example. The test could help to diversify first-year teams.

The other things we've been doing is trying to build a more robust Web site (www.gmat.org). And it will be up and running at the end of this month.

BW: I've visited the MBA Explorer site. Is that what you're talking about?

Wilson: Yes. It'll be the Explorer, but it won't look the same. There's quite a bit of substance there right now, but unfortunately, it's a great 1996 site. We're taking about 1.5 million hits a month now, and it has stabilized there. We're going to be adding a "GMAC Question of the Day," GMAT registration information, GMAT testing information, school links, and a much more robust search engine for prospective students to investigate different schools. A search will provide the user with a list of 500+ schools with links directly to their home pages. If schools don't have a home page, we're going to create what I call a "Fake-O-Page," which will provide more information on the school.

BW: GMAC is best known for administering the GMAT. The group also puts out CD-ROMs to help prospective students brush up on their quant skills. Have these been popular?

Wilson: If you've ever been in an MBA class, that first month is extremely difficult for the students who are colloquially referred to as poets. The purpose of the CD-ROMs is to really take the Russian history majors and give them an opportunity to raise the floor a bit before that first day in class. While they won't have the same background as the engineers or the business undergrads, they also won't be left way behind, which is very difficult to overcome during part of that first year. The CD-ROMs have been around for a while. Accounting's been out for about a year. Finance has been out for about 8-9 months, and the quant has been out since August, 1997. And there will be version 2.0 that will come out in June, 1998. So far, we've been pleased with the number of people using them.

BW: Does GMAC have its hand in any scholarship funding?

Wilson: Indirectly we do. One of our major initiatives is to support diversity. And one of the ways we've done that is initially through the Minority Summer Institute, which is heavily scholarship-based. GMAC has contributed a total of $1 million to that since 1992. Recently, the Minority Summer Institute has been replaced by the PhD Project, which is co-sponsored by a number of companies. The premise of the PhD Project is that one way to attract minorities into business is to put role models in the classroom. The PhD Project is a two-day program that essentially disseminates information about B-school programs to prospective, minority applicants. It puts together a critical mass of people to learn about graduate study -- faculty from a number of programs come together and speak with about 350 minorities thinking about going to B-school.

This program tries to give anyone who is contemplating or thinking about management education, a better idea of what it is all about. Right now there are something like 320 candidates in doctoral programs who have come through the PhD Project. And there are as many in the pipeline as there are in the classroom today. We're the second-largest benefactor of that, behind KPMG Peat Marwick. We give $150,000 a year, on a revenue base of $30 million (KPMG gives $250,000 a year on a revenue base of $8 billion).

BW: Dave, thanks for your time.

Wilson: My pleasure!

----------------------------------------------------

*The following countries have not yet had their GMAT sites converted to the CAT: Antigua and Barbuda, Azerbaijan, Bangladesh, Barbados, Benin, Brunei, Dominica, Estonia, Gambia, the Israeli/Palestinian West Bank, Georgia, Guyana, Kazakstan, Kyrgystan, Latvia, Malawi, Mariana islands, Moldova, Monaco, Mongolia, Nepal, Sierra Leone, Slovak Republic, Sri Lanka, St. Lucia, Swaziland, Tajikistan, Turkmenistan, Uzbekistan, West Indies Assoc. States, and Zare.