It didn't take long for the test center proctor to realize something was amiss. One group of people clearly stood out from the rest of the candidates taking a popular IT certification exam. They sat rigidly in their chairs, hardly moving at all, and they proceeded through the questions at a pace of six items per minute, well above the norm of one to two questions per minute. All scored well above the minimum needed to pass the test.
After the testing concluded, the test center called in Caveon LLC, a consultancy that specializes in test security, including data forensics, to review the situation. "At first blush it looks like by using a Bluetooth speaker and a video camera they were collaborating with a subject-matter expert offsite," says Caveon's vice president Steve Addicott.
[ Also on InfoWorld: Hotshot young techies, take heed of these 10 hard-earned lessons of a lifetime in IT. | Get a digest of the day's top tech stories in the InfoWorld Daily newsletter. ]
Such equipment is readily available online at sites like the aptly named spycheatstuff.com. Aspiring cheaters can buy wireless speakers that fit deep inside the ear canal, where they can't easily be seen, as well as tiny cameras that are simple to hide. The suspected cheaters in this case were most likely sitting still to give their hidden cameras a clear video image of the screen, Addicott says. The review of that particular case is still ongoing.
Cheating is trending
IT certifications have become a primary route to both salary premiums and career advancement, according to a recent Foote Partners report. So it's no surprise that, as the popularity of certifications has grown, so has cheating. "Jobs and careers are at stake here, so people will attempt all sorts of things," says Matthew Poyiadgi, vice president of Pearson Education Inc.'s Pearson VUE business unit, which manages 5,100 test centers worldwide and counts the IT certification program manager CompTIA among its clients.
And while CompTIA estimates that the level of cheating on IT certification exams is less than 5%, industry insiders say the problem is growing and that keeping up with the cheats requires constant vigilance.
So far, cheating doesn't appear to have devalued most IT certifications in the eyes of hiring managers. For the 309 IT certifications that Foote Partners tracks, the average pay premium across 2,600 surveyed companies has gone up for the last four consecutive quarters, says CEO David Foote.
While there's no way to definitively know if a prospective hire has cheated to obtain an IT certification, employers can and should check with the certification body to make sure the person actually attained it. "Trust, but verify," says Addicott.
For the most part, he adds, hiring managers can trust that verified IT certifications were legitimately earned."Just a few rotten apples have cast doubt on the qualifications of individuals in the IT profession," he says. But, he adds, it is possible that a few individuals have benefitted from the live exam content available online and used that to gain a higher score on an exam. So an IT certification should only be one part of the hiring decision.
Other steps include checking references, reviewing employment history and asking a few carefully crafted questions designed to gauge whether the candidate really knows his or her stuff.
Where the cert developers fit in
Developers of IT certification programs, such as Microsoft and CompTIA, contract with Prometric, Pearson VUE and other independent test centers that administer and proctor tests worldwide on their behalf. These businesses also provide training services, and so must have a secure firewall between the testing and training sides of the business.
IT certification bodies and test center operators are engaged in an arms race with pirates who steal test questions and answers, and with cheaters who buy that information, share answers in chat rooms, pay "proxies" (people who will to take tests for them) and bring a range of technologies and techniques into test centers to gain an edge. IT certification organizations, worried about degradation of their credentials, are striking back by turning to more sophisticated methods to catch cheaters and mitigate piracy. And cheaters who get caught increasingly face more than just a slap on the wrist.
Even people who cheat and don't get caught during the exam still have reason to worry. Pearson VUE records every session to digital video and reviews it after the fact. Recently, scrutiny of unusual head movements tipped off the team that one test taker had an embedded camera in his glasses. "The way people are cheating is changing. They're using technology more," Poyiadgi says.
But the most common ways people try to cheat aren't always the most high-tech, says Shelby Grieve, Microsoft's director of professional certifications including the Microsoft Certified Solutions Expert and Microsoft Technology Associate. "The trend has moved from taking exam answers into a testing center to more passive methods of cheating, such as using 'brain-dump' sites and proxy testing services," she says.
Grieve says Microsoft has caught candidates who colluded online through question and answer sharing, as well as people who used low-tech approaches such as copying off other peoples' exams, texting answers and even modifying someone else's printed score report.
Brain-dump sites don't just provide a place where users can share answers, says Caveon's Addicott. "These websites aggressively sell pirated test content and package it as test prep materials -- and they guarantee that you'll pass. It's a real problem with IT certifications," he says. Most of these sites are based in Asia, where it's more difficult to shut down the sites and prosecute the offenders. Overseas test center franchises with lax controls have been a source for test theft and cheating because tests and answer keys are typically always downloaded and stored at each location, giving cheaters easier access, he adds.
"The single biggest factor in how much cheating you have is if you test internationally, and IT certification programs are virtually all international programs," says John Fremer, president of Caveon's Consulting Services group.
Rise of the hired gun
Proxy test-taking is growing concern for Bryan Kainrath, vice president for certification operations at CompTIA, which owns the A+, Network+ and other popular IT certifications. "We're seeing more proxy testing than we have in the past. Most proxy scams involve hiring someone in China to take a test for someone in the U.S. That happens all the time," he says.
A few years ago, a large IT certification provider engaged Caveon to hire a proxy and attempt to pass the test without being caught. "The certification program paid us, we paid a proxy service and one of my colleagues earned this prestigious certification even though he had no background," says Addicott. The price to cheat: A $1,000 check wired through Western Union. The terms were 50% down, with the balance paid after the job was completed.
Proxy test-taking services are big business overseas, in part because what Americans consider cheating is culturally more acceptable in some other locations, Caveon's Fremer says. The buyer signs up and the proxy goes to a test center and takes the test. It's good money, says Fremer. "In some parts of the world you can earn six months' salary with one proxy test-taking event."
A sample letter from Caveon LLC's interaction with a proxy website. By paying the site to hire a proxy to take the test in his place, a Caveon staff person "earned" a prestigious IT certification for which he had no background. Caveon removed the name of the test to protect the client. Source: Caveon LLC.
In some cases, proxies have been able to skirt security protocols by visiting corrupt testing facilities overseas that operate both a legitimate "front room" test area and a fraudulent "back room" operation. "Those stringent protocols aren't followed when the test center runs its own proxy ring," which can be very lucrative, Addicott says.
To address proxy test-taking, test centers typically require candidates to present a photo ID, and a few centers, including those directly managed by Pearson VUE, have added biometric identification and digital signatures, as well as taking the candidate's photo. Once a person has registered under one identity, he can't act as a proxy for someone else. What's more, the person who hired the proxy will be caught if she tries to take another test, since her photo and biometric data won't match.
Test centers might also record the test subjects on digital video, and put the test taker's photo right on the certification report. "Proxy testing used to be a big thing," says Pearson's Poyiadgi. "But once we required digital photos and digital signatures it disappeared."
But while the "gold standard" of testing security applies to the 500 testing centers that Pearson VUE owns, that can vary at the other 4,600 sites owned by Pearson's partners, including IT training organizations and colleges and universities that test students at the end of a training program.
Den of thieves
Pirates use a variety of techniques to steal entire tests and answer keys. These include sending people into test centers to remember or photograph sets of questions. (This type of "item harvesting" might require sending as few as 10 people into a test center to memorize all of the questions on a given test.)
It can also involve outright theft of test data from corrupt or lax test centers. "Because the whole test and answer key is downloaded to servers at each location the entire item bank and answer key are available to be hacked. It's really problematic," Caveon's Addicott says -- and it's leading some certification and testing organizations to move to a SaaS-based test delivery model.
When test takers try to cheat using brain-dump sites, however, they sometimes end up getting cheated themselves. In some cases the sites deliver fraudulent or obsolete content to unsuspecting buyers, says Dave Meissner, chief operating officer at Kryterion Inc., a provider of online IT certification testing services. "If people spent the same energy and creativity to study as they do to cheat they would be far better off."
In response, IT certification bodies have staged coordinated attacks on brain-dump sites where the pirates attempt to sell the looted data, including the use of cease and desist orders and raids, says Kainrath. "We'll meet with Cisco, Microsoft, VMware and try to figure out the best approach to mitigate these sites," he says.
"If we find out that a test center has been colluding in any way, that center is shut down by our security team," says Poyiadgi. Pearson VUE, he adds, has only experienced "a handful of cases."
For the industry as a whole, however, combating intellectual property theft has been an uphill battle. "You can shut the sites down but it's like pulling the top off a weed. It just pops up somewhere else," Kainrath adds.
"It's not mom and pop" thieves, says Fremer. "Organized sophisticated stealers can make millions -- or tens of millions -- from just one certification program."
So, test sites and certification programs try to react quickly to minimize the damage. CompTIA monitors online brain-dump sites and chat rooms for stolen test items, and uses analytics to determine whether any given question's effectiveness in measuring competency might have been compromised. "As soon as there's been any degradation we pull the item," Kainrath says. "We have huge item banks in reserve and can move questions in and out quickly."
That process can present an expensive challenge, however, because organized theft rings can compromise entire tests within three to five weeks of when they're first released, while most IT certification exams are refreshed every 12 to 15 months, Addicott says.
Kainraith admits that's a problem, but he thinks that questions take a bit longer to appear on brain-dump sites, and says CompTIA replaces tests at a rapid pace. "We're able to churn our items a lot faster than 12 to 15 months," he says, although he declined to say how fast.
While CompTIA has the scale and resources to turn over its test questions more quickly, smaller IT certification programs are more limited because the cost of building and maintaining tests ranges from hundreds of dollars per question to thousands of dollars per test item, according to Caveon.
Countermeasures: Tripping up the cheats
Catching cheaters has become its own science. "More candidates are sharing knowledge than we've seen in the past," says Kainrath. But both test centers and IT certification owners have ways of figuring out who's using stolen and shared test data, as well as who might be coming in to steal it.
In addition to using live proctors, Microsoft and others are moving toward online proctoring, which combines the use of a video camera with a live feed of the test taker's screen. While an online proctor is limited by what he can see on a video camera, it's easier to take immediate action against cheaters, Grieve says. Because they can look for suspicious activity at the question level, online proctors can identify cheating sooner and end the test before the candidate can see -- and possibly compromise -- the rest of the exam content.