The Prevention Lockbox
Jamie Mulkey, Ed.D.
Of Wildfires & Surveillance Cameras:
Prevention is Still the Key
Dr. Jamie Mulkey has a distinguished history; holding major positions in testing organizations, certification task forces, and testing-related non-profit associations. As a Caveon founder, Jamie was a major contributor to shaping the testing industry’s discussion around test security and bringing test security services to market. Dr. Mulkey believes that sound, secure test security planning and practices provide the foundation for quality testing and trustworthy results. Her current focus is working with clients to design pre-emptive policies and processes that protect their most important assets—their tests. Dr. Mulkey is also the Executive Director of the of the Institute for Exam Security (i4ES), whose mission it is to broaden the understanding and importance of test security.
In March 2018, I wrote an
discussing how you can determine your organization’s level of test security risk. I recommended identifying the most salient risks in order to develop a plan that would help you understand where to focus your priorities and resources for test security. The next step in this process is to develop strategies and plans to implement a holistic test security program. Breaches are going to happen. When they do, it’s important to be prepared to execute tactics that prevent, deter, detect, and re(act) to test security incidents.


When we talk about prevention, it means we are trying to stop something bad from occurring. For example, this year, wildfires have been at the top of our minds, particularly in California where the environmental conditions continue to dramatically change. Fires are scary. They destroy homes, habitats, and lives, and they do it swiftly with reckless abandon.  
Did you know that there are things we can do to prevent wildfires from creating such dramatic devastation? A recent news piece on
suggested that we can diminish fire destruction by designing more fire-resistant homes and communities. For example, creating 100-foot fire zones, placing rock beds around the house, and providing enough space between trees so flames can’t jump from one to the next as quickly. From a test security perspective, prevention means we are trying to prevent cheating or theft. We have designed test security protocols into our testing process from design, to development, to administration, to scoring and evaluation. Here are a few examples of test security prevention measures:
      • A test security plan
        is in place that specifies the processes the testing organization will follow to prevent test theft.
      • A candidate agreement is created, letting test takers know what is/is not allowed in terms of test preparation materials, materials used during test administration, appropriate clothing to be worn, prohibited electronic devices, the sharing of test content, and the potential sanctions, should any of these rules be broken.
      • Test Development protocols are established. Access rights to test content are spelled out, the confidentiality agreements are signed by Subject Matter Experts (SMEs), and the service level agreements between the testing organization and the test administration provider are decided.
      • A Security Incident Response Plan is developed. It specifies the actions to be taken when a breach occurs. This includes who will be involved in managing and mitigating the investigative process, what the outcomes will be to those who committed the breach, and how the results will be communicated to stakeholders.

These prevention strategies defined by your testing organization flow nicely into the next set of strategies: deterrence. Think of deterrence as a communication plan, letting would-be test cheats and thieves know that there are consequences to test maleficence. A good analogy for thinking about deterrence is the surveillance camera system that is used in retail stores. Through signage, establishments communicate their use of surveillance cameras to deter would-be thieves from stealing their merchandise. The same is true of communicating test security policies to test takers. By communicating the consequences of cheating, stealing, sharing, or selling test items, it is hoped that the consequences will be a big enough warning to deter individuals from committing fraudulent testing activities. Another outcome of deterrence is that it lets individuals know what is and is not acceptable in terms of preparing to take a test, providing assistance to test takers, or the information about the test that can/cannot be shared after the testing event. Examples of this include:
      • Communicating to test takers
        that they must only use sanctioned or authorized test preparation materials from the testing organization.
      • Communicating to
        teachers they must cover all bulletin boards with butcher paper so that students are not advantaged by information during a scheduled test administration.
      • Communicating to
        candidates what is and is not allowed to be shared through social media chat rooms after they have taken a test.
By effectively communicating these rules and policies, you can deter fraudulent activities.
You might also like...
More Reads
In ancient days, the cornerstone was the sturdiest, most important element of a building's construction, and was paramount for a steady foundation. Much like the ancient cornerstone, "Prevention" is the foundation in which a secure testing programs are built today. Here's how:
Read more →
This dear friend has inspired nefarious creativity among us for all 105 years of its life. But due to unforeseeable advances in stratagem and technology, its effective time here on Earth has drawn to a close.
Read more →

At this point, you’ve done your best to prevent test theft and cheating from occurring. However, how will you know when a breach has occurred? Perhaps you suspect your test items are being shared on the Internet because your exam’s pass rate has seen a dramatic increase over the last month. You may receive an anonymous tip that a test preparation provider is asking their students to remember test content, and bringing it back to them so they can teach other students. Maybe an alert from your search engine has flagged a suspicious photo of a test item through a social media platform. This is where detection comes in. When you need to know what’s going on, employ detection tools to verify maleficence so that you can take action. Detection activities can include web and media monitoring on a continuous basis or during stated testing window time frames. Either way, it’s important to know what’s out there. Determine who’s talking about your test and what they are communicating. This will help you understand the scope of any unauthorized materials or discussions taking place. Let’s say you find your exam content online. Now what? The question you want to answer is: what impact does this content have on my exam’s performance? Are there any test takers that have gained an unfair advantage? Can my test takers’ scores be trusted? To answer these questions, statistical analysis known as data forensics should be used to determine if there are irregularities that are indicative of test cheating or theft. Examples of data forensic analyses can detect such anomalies as:
      • Similarity
        between 2 or more individuals.
        These test takers not only get the right answers the same, they
        get the incorrect responses the same as well.
      • Speedy test takers. Someone who successfully completes a test in 15 minutes when the average time for a 100-item test is 1.5 hours.
      • Pacing the test.
        When a classroom of students shows similar timing for each item response and the class is getting the items correct. This may be a result of a teacher who is pacing her students; students complete one item at a time while the teacher walks the classroom to check students’ answers.
Once you have the results of the data analyzed, you can use the policies you’ve established in your Test Security Plan to make decisions about the actions to take. Once those actions are carried out, you can evaluate the results and determine if changes can be made to prevent a similar incident from occurring in the future.
"Test security prevention is never fool-proof. Just when you believe you’ve covered all your bases, someone will find a new way to cheat or steal your items."
Join our mailing list
Copyright© 2019 Caveon, LLC.
All rights reserved.
Privacy Policy
Terms of Use
Interested in learning more about how to secure your testing program? Want to contribute to this magazine? Contact us.
"Breaches are going to happen. When they do, it’s important to be prepared to execute tactics that
, and
) to test security incidents."

Coming Full Circle

Now you have come full circle. You have:
  1. Put a thoughtful plan in place to prevent test cheating and theft from occurring
  2. Communicated relevant pieces of your plan to deter test maleficence
  3. Used tools to detect and confirm a test security incident has occurred
  4. Taken action and evaluated your processes to determine if anything needs to change to protect your test in the future.

Test security prevention is never fool-proof. Just when you believe you’ve covered all your bases, someone will find a new way to cheat or steal your items. However, test security, like wildfires and retail theft, needs a solid plan to lessen the impact of these malicious incidents. By thinking constructively and working diligently, you can prevent test security incidents from occurring, reduce your risk, and mitigate the incidents that do occur.