Posted by: Louis Lavelle on September 9, 2008
Today, GMAC officially announced the end of its investigation into testing violations by subscribers to Scoretop.com, a now-shuttered test prep Web site. There will be a complete story at businessweek.com/bschools in a little while, but the upshot: 84 Scoretop subscribers had their GMAT scores canceled, with the vast majority—72—being permitted to retake the exam immediately. GMAC prepared an FAQ on this latest development. It’s attached after the jump.
FAQs about the Scoretop Web Site
The Graduate Management Admission Council (GMAC) has completed its initial investigation of GMAT test takers using Scoretop.com. Earlier this year, GMAC won a $2.35 million default judgment against the operator of the Web site, who sold access to “real, live” GMAT questions and encouraged test takers to post questions after taking the test. We have identified and canceled the scores of those test takers who either posted live GMAT questions on the Scoretop Web site or wrote a message on Scoretop confirming that they saw items from the Web site when they took the GMAT exam. In addition, those who posted live questions from their GMAT exam on Scoretop will be prevented from testing again for at least three years. We have also sent notification of the cancellations to the schools to which these test takers sent score reports.
Q. How many test takers had their scores canceled?
A. GMAC canceled the scores of 84 test takers. Twelve test takers had their scores canceled and will be prevented from retaking the test for at least three years because they posted live GMAT questions on the Web site, and 72 test takers had their scores canceled because they wrote a message on Scoretop confirming that they saw items from the Web site on their GMAT exam.
Q. Which schools were affected?
A. A number of schools all over the world received score reports from test takers whose scores we have subsequently canceled. But just because someone sent scores to a school doesn’t mean the test taker applied to or would have been accepted by the school.
Q: What gives GMAC the authority to cancel GMAT scores?
A: As the sponsor and owner of the GMAT exam, we own all test content, including GMAT exam questions. In addition, we have established terms and conditions, including testing policies and procedures, to enhance the testing environment and the validity and security of the GMAT exam. Every test taker must agree to abide by these policies before taking the test. These policies include agreeing not to reproduce, disclose or obtain unauthorized access to test content by any means before, during or after testing.
Q. Is the GMAT exam compromised because you had to cancel so many scores?
A. The GMAT exam is computer adaptive and has a bank of thousands of questions, so most of these candidates saw one or two questions at most on their GMAT exam. Having access to live questions had a minimal impact on their scores. The reliability of the test taker’s score is less in question than the ethical behavior of those trying to “game” the system.
Q. How was GMAC able to confirm that test takers posted questions they actually saw on the GMAT exam, or that those who claimed to have seen questions from Scoretop on their GMAT exam actually did see live questions?
A. GMAC drew from multiple sources, from a test taker’s posting on Scoretop to his or her unique testing experience, to identify those who posted live test questions or confirmed that they had seen a question on their GMAT exam that had been posted previously on the Scoretop site. GMAC canceled the scores of test takers falling into each of these two categories. The following representative examples are provided for clarification.
Category 1: Those who posted live questions
An extensive analysis of the Scoretop Web site allowed GMAC to identify GMAT test takers who posted GMAT questions. GMAC was able to match the Scoretop.com posting to the question that was included in these test takers' unique, computer adaptive GMAT exams.
For example, Test Taker A posted the following question on Scoretop.com:
Before the elections, the ratio of Republicans to Democrats was 3 to 5. If after the elections it was known that 600 more Republicans and 500 more Democrats voted, the ratio was 4 to 5. How more Democrats were there than Republicans? Well, the substance is there.... Among the answers were 100, 1000, 2500 and two others.
GMAC was able to determine that Test Taker A encountered the following GMAT question on his exam just prior to the Scoretop posting:
In a certain district, the ratio of the number of registered Republicans to the number of registered Democrats was 3/5. After 600 additional Republicans and 500 additional Democrats registered, the ratio was 4/5. After these registrations, there were how many more voters in the district registered as Democrats than as Republicans?
This example shows that the test taker electronically disclosed test content after testing, a violation of GMAC policy.
Category 2: Those who confirmed that they had seen one of the posted questions from Scoretop (a “JJ”) when they took the GMAT.
The term "JJ" stands for "Jungle Juice" or "Jijings" ─ slang words for "the real thing." On Scoretop.com, math JJs were called MJJs, and verbal JJs were called VJJs. GMAC traced a test taker’s confirmation of having seen JJs on the Scoretop Web site to his or her actual GMAT exam.
For example, Test Taker B posted the following on Scoretop.com:
Do the JJs thoroughly. They really will help. I got a lot of MJJs. So you can see for yourself that I could not have made it to 700 without MJJs.
GMAC identified two questions that Test Taker B encountered while taking the GMAT exam that were among the “JJs” that GMAC identified on the Scoretop Web site. This confirmation that the test taker electronically obtained unauthorized access to test content before testing represents a violation of GMAC policy.
GMAC has taken no action against those who simply visited the site and did not perform either of the activities cited above.
Q. Are most of these test takers from one geographic area?
A. No. Test takers around the world are affected.
Q. The bans are for a minimum of three years. Will some be longer, and how will that be decided?
A. All those who posted questions will be prevented from retaking the test for at least three years. After three years, these test takers can request reconsideration to test again, and GMAC will evaluate on a case-by-case basis depending on the severity of the offense.
Q. I have information about a student posting questions on Scoretop, but I don’t see that name in the list of test takers who sent scores to my school and whose scores have been canceled. What action will GMAC take against such test takers?
A. While our investigation of the Web site was detailed, it is possible that there are additional test takers whose postings on the Scoretop Web site were not identified. If school representatives would like to share any information about test taker behavior on Scoretop with us, we will investigate further.
Q. Does this mean that GMAC has completed its investigation, and no more scores will be canceled?
A. We have completed and suspended our initial investigation and are not actively expanding its scope. However, our security efforts to protect the integrity of the GMAT exam are ongoing. If we receive any additional information about a test taker’s activities on Scoretop, we reserve the right to investigate such information and take appropriate action based on the results of such investigations.
Q. Why did GMAC wait so long to take action?
A. We worked with the FBI for several years documenting the evidence necessary to take action. During that time, Scoretop operator Lei Shi was warned repeatedly to stop posting live questions to the site. Ultimately, while the FBI continued its investigation, we decided to try to shut down the site via a civil lawsuit, which was filed in June 2007. We received a favorable ruling and worked to gain control of the Scoretop.com domain and hard drive. Once we received the hard drive, we investigated the postings on the Scoretop Web site and took appropriate action based on the results of this investigation.
Q. Does GMAC have any advice about how to deal with the media in this regard?
A. If your school would like assistance in handling queries from the press, please contact Judy Phair, GMAC vice president of communications, at email@example.com.
Q. If I have any additional questions about this investigation, whom should I contact?
A. Please contact Ashok Sarathy, director of the GMAT Program, at firstname.lastname@example.org.
Q. Whom at GMAC should students talk to?
A. Affected test takers can send an e-mail to email@example.com
Q. How likely is it that there will be another Scoretop?
A. There will always be a few test takers who seek an unfair advantage, and the reach of the Internet facilitates this type of activity. However, GMAC actively monitors Web sites for violations of GMAT Policies. We take our ethical responsibility to schools very seriously and will actively pursue such Web sites. Additionally, we will send targeted messages to test takers to advise them about such activity.
Q. Does GMAC want us to notify the affected test takers of the score cancellation if they applied to our school?
A. Schools should not have to notify the test taker about the cancellation. We have sent all the affected test takers a separate letter informing them of our decision. Other interactions with the test taker are left to the school’s discretion.
Q. When will the test takers get the letter?
A. The letters to the test takers were sent on Friday, September 5, 2008.