Jamie Mulkey, Ed.D.
March Edition
Risky Business:  
Determining Risk to Enhance Security

When you think of the word
, what comes to mind? Do you think of physical safety and shelter? Do you think of national defense and security? Do you think of your bank pin and passwords? As we go throughout a typical day, there are dozens of ways we protect ourselves and those close to us. Sometimes these activities are done so automatically, they are unconscious behaviors. Whether it’s walking a child across the street, locking our homes, or wearing a jacket in the winter time, we all engage in activities for the sake of protection.
For those of us who work in testing, protection involves fortifying tests with a large quantity of items, guarding against item theft, preventing cheating through a variety of ever-adapting methods, and (most importantly) upholding the shield of fairness to gain a true measure of a person’s ability. Whether a high school senior gets the score necessary to get into their dream college or a nursing student can become a practitioner, our exams represent high stakes for test takers. As such, we must protect the validity of the scores they produce. Unfortunately, test security is not one of those unconscious behaviors we engage in every day. We have to think about how to best protect our items and tests, then take deliberate actions (sometimes even extraordinary measures) to provide appropriate protection. It isn’t an easy job, but it is a vital one. Many testing professionals find the prospect of protecting their exams daunting. Where do you start? What are the priorities? How do you go about knowing what to protect? Fortunately, there is a simple way to get started. The first step for anyone facing what appears to be the overwhelming task of test security is this: conduct a risk assessment and figure out what the greatest threats are to your program.
We have to think about how to best protect our items and tests, then take deliberate actions (sometimes even extraordinary measures), in order to provide appropriate protection. It isn’t an easy job, but it is a vital one.
Test Security Risks
Testing programs today are bombarded by what seems to be an ever-growing list of threats from test fraud. A compromised test can lead to the loss of carefully-crafted test items, liability resulting from unqualified individuals performing a job, damage to a testing program’s reputation, and the loss of trust in an individual’s exam scores. In trying to mitigate these consequences, it is easy to get overwhelmed by just how many ways someone can cheat or steal questions. It is easy to feel like it is impossible to stop them all. Fortunately, for every program there are certain threats that are more dangerous than others, that hold the greatest risk because they will do the most damage. While one program might be plagued by thoughts of unauthorized access to the item banking system, another might be concerned with the unsanctioned distribution of items over the Internet. Every testing program is unique, and as such, the threats to every program are unique. The key to effective test security is figuring out which threats pose the greatest risks to your specific program, and focusing your resources there.  Let’s use an example from the non-testing world to clarify. There are numerous ways to protect a building from burglars: locks on windows and doors, video cameras, guard dogs, iron gates, security alarms, etc. However, not every building needs to employ the same security measures to effectively protect it. While it makes sense for a bank to employ armed guards and devices to detect someone tunneling into their vault, it doesn’t make much sense to have those same measures to protect your apartment – locks, alarms, and maybe a dog will suffice. The same is true for test security measures; there is no one-size-fits-all solution. To be effective, security policies need to be tailored to address the threats that are the highest risk for each specific program. Fortunately, figuring out the greatest risks to your program is straightforward. The Caveon Risk Assessment Tool is located on the Caveon website. It is quick and free, and it will help you quantify your test security threats and prioritize which actions should be taken. Every testing program, whether new to test security or long-standing, should conduct a risk assessment using this tool on a regular basis to confirm they are allocating their test security resources in the most effective ways possible.
How the Risk Assessment Tool Works
"When you’re finished completing the Caveon Risk Assessment Tool, you will have a prioritized list of test security threats specific to
your program
Make it a Group Activity
The risk assessment tool can also be used as a collaborative effort within your organization to provide you with valuable insights. You can compare your results to find answers to questions like “Does everyone on the team view the same test security threats as most important?” “Are there differences?” “And, if so, why?”
A Comprehensive Plan
The results of this analysis provide you with a personalized roadmap to protect your tests, which then allow you to choose the various test security elements that will be most effective in mitigating the biggest threats to your program. Once you’ve put a comprehensive action plan in place to address these threats, you can come back within a year’s time, complete the risk assessment again, and see if those test security priorities have changed. It is important to remember that one size doesn’t fit all. One program’s test security roadmap is not going to be the same as the next program’s. Each testing program needs to consider its budget, solution effectiveness, and resource requirements. Each program needs to set realistic expectations and tackle the most salient problems first. Test security may never become an unconscious behavior like turning off the stove. However, we can take steps to understand the most prominent threats and streamline the process of mitigating risk and providing the best form of protection for our most valuable assets — our exams.
If you open the Caveon Risk Assessment Tool (by clicking the image above) you will notice that it divides the threats every testing program faces into two categories: cheating and theft. Think of cheating as the things that an individual might do to increase or better their test score. Think of theft as the stealing of items so that other people can increase their test scores. These two categories have multiple examples of activities that describe how threats to test security occur. To complete your risk assessment, rank each example of cheating or theft; provide a score for the likelihood (How likely is this event to occur?) and the damage (If the event were to occur, how much would it damage your program?). Our comprehensive algorithm will calculate the likelihood times the damage to give each threat a combined score. When you’re finished, you will have a prioritized list of test security threats. This will help you confirm or rebuke your uncertainties regarding the security of your program and give you a path to tackle your most salient test security concerns.
The Caveon Risk Assessment Tool
Interested in learning more about how to secure your testing program? Want to contribute to this magazine? Contact us.
Join our mailing list
Copyright© 2018 Caveon, LLC.
All rights reserved. Privacy Policy | Terms of Use