Deterrence in the Wild

The Deterrence Lockbox

Various Authors

Deterrence in the Wild:

Part 1

In the words of Helen Keller, "Alone we can do so little; together we can do so much."

To help further our dialogue on deterrence and better understand how it functions in the assessment industry, we asked a few respected test security experts about their personal experiences with deterrence. How have they utilized it in their testing programs, and what value have they received in return? Let's draw on the insight and wisdom of our colleagues as we work together in pursuit of increased test security, fairness, and exam validity.

More Reads

What happens when the women of the assessment industry gather together for drinks and hors d'oeuvres? Relationships are strengthened, a cause is established, and others' lives are uplifted. Find out how you can get involved at next year's pre-ATP Women in Testing Reception.

Get to know your colleagues in the test security industry a little bit better as they answer fun and intriguing questions from Marcel Proust's 18th-century parlor game.

Contact

Interested in learning more about how to secure your testing program? Want to contribute to this magazine? Contact us.

info@caveon.com

Rory McCorkle, Ph.D. Senior Vice President of Certification and Education Services, PSI Services

Kevin Jolly Global Security and Compliance Manager, PSI Services

What are the benefits and/or drawbacks of broadcasting existing test security measures in order to discourage cheating or theft?

RM: "While the debate continues as to whether disseminating security measures reduces test cheating or IP theft, we have found that certification programs can invigorate cheating deterrence by increasing transparency. "The benefits of advising the examinee of security measures ahead of exam administration are two-fold: first, it restates to the candidate the importance of the exam, why its security needs to be taken seriously and that consequences for cheating will be severe. Second, it increases the confidence the candidate has in the certification itself, knowing that the certification program is doing all it can to uphold validity. "Potential drawbacks of communicating exam security measures include the perception of how the message will be received, as it goes to everyone, not just cheaters. Potentially it could be misconstrued as 'We don’t trust you'. And some might make a presumption that the exam has a validity problem. Or, that if you disclose what you are policing, it may reveal a weakness for a would-be bad actor to cheat. "These are largely perception-based. By providing detailed procedures and consequences, we can redefine the reality and neutralize any negative perceptions. In the end, deterrence is strengthened by the examinees themselves through their own behavior modification, informed by clearly articulated exam security policies."

What is the value of informing examinees of the specific consequences for breaking test security rules?

RM: "As we continue to work in global, multi-cultural environments, I have seen the benefits of not only informing examinees of the consequences for breaking rules, but also of specific examples of the rules. When we provide examinees clear descriptions of what constitutes these violations, we remove any doubt or confusion about what misconduct looks like. "Additionally, when adding the consequences of breaking these rules, we can then ensure examinees are forewarned about what constitutes misconduct, as well as the penalties. This is no different than what we would expect from systems that are laid out in the rule of law in any jurisdiction. "By taking these steps, we seek to dissuade the opportunistic examinee, as well as demonstrate to ‘rule-abiding’ examinees the repercussions to facilitate their participation in tip lines and other measures that empower those who play by the rules." Drawing on your experience, what is a practical action that testing programs can implement to deter individuals from cheating or stealing test content? KJ: "We recommend developing a certification exam policy and communication of the same to ensure candidates are aware of the security procedures your program has in place. This needs to be done early and often. From your program’s website to the multiple touch points with your candidate, you outline and emphasize what they should expect, and in no uncertain terms, state what the consequences are for cheating. A policy must have teeth so that its enforcement is taken seriously. "Another critical action is the use of data forensics to identify candidates who may have committed misconduct. Using sophisticated statistics, organizations can identify patterns in pass rates, similarity between candidate responses, and trends in exam time and scores that may identify irregularities, even if nothing was caught during the exam delivery process. As cloud-based data and processing power increase dramatically, this analysis can be conducted more quickly than ever before, and remediation actions taken. "A strong and holistic exam policy is one of the strongest deterrence tools a certification program possesses. Why? Because it’s one of the few variables that are completely in its control. The quality of the policy and its accessibility will, over time, help to engineer behavior modification. Greater deterrence and less cheating lead to more compliance with certification test-taking policies."

Rachel R. Watkins Schoenig

Founder & CEO, Cornerstone Strategies, LLC

"Deterrence is an important aspect of security for a high-stakes exam program. I like to think of an effective exam security program as supported by a three-legged stool of activities focused on deterrence, detection, and decision-making. Basically, programs should deter what they reasonably can, detect what they are unable to deter, and be prepared to make decisions when potential incidents do arise. Just like a three-legged stool collapses if one leg is missing, an exam security program is not as effective if the deterrence activity is missing. "Effective deterrent activities vary. For example, an effective deterrent can be as simple as ensuring item and form reviewers agree to maintain the confidentiality of exam content, thereby avoiding inadvertent disclosure due to lack of awareness. Or an effective deterrent can be as complex as capturing and comparing biometric data associated with a testing candidate to discourage attempts to engage in surrogate testing. Determining which deterrent activities are appropriate for a particular testing program depends on the unique aspects of that program."

Walt Drane, Ed.S.

Director of Education Services, Caveon Test Security

"High-stakes testing in K-12 education comes in many forms. State-wide assessments are required by the U.S. Department of Education in the areas of English Language Arts (ELA) and mathematics in grades 3-8, and once in high school, usually in the form of an End-of-Course (EOC) assessment in a subject area (i.e. Algebra I and English II). Science is required to be tested once in elementary school, once in middle school, and once in high school. Some states across the nation, such as Florida and Mississippi, use the 3rd grade ELA assessment as a promotion/retention test for students to matriculate to the 4th grade. In addition, some states have graduation requirements that include the passing of one or more EOC assessment(s). These requirements make the assessment high-stakes for students. "High-stakes also come in the form of accountability for schools and districts. All states have an accountability model that uses student performance data from the assessments previously mentioned to produce an output or label that is ultimately placed upon schools and districts. For example, Mississippi uses an accountability model that produces an A-F rating system for schools and districts. A school or district that scores an “A” is high performing, while an “F” rated school or district needs serious improvement. This rating scale resonates with the general public. In fact, business development, real estate decisions, and the community’s reputation are all dependent upon school and district accountability ratings. Therefore, all statewide assessments are high-stakes for various stakeholders and demand deterrent test security strategies to be implemented prior to and during testing. "There are many benefits to training District Test Coordinators (DTCs), School Test Coordinators (STCs), Test Administrators (TAs), and proctors on test security deterrent strategies such as data forensics, web patrol, monitoring efforts, test invalidations, and educator misconduct licensure action. In fact, states should do their due diligence and inform everyone involved in testing of the policies and procedures of testing, to include what is and is not allowed during the testing process, as well as the consequences when test security rules are not followed. Setting the expectations up front ensures everyone has been informed of the rules, and makes enforcing consequences easier when problems arise. "Just like in society, a majority of students and administrators will conform to the testing policies and procedures and do what is expected of them. For those who think about cheating, most will also think about the consequences and not engage in unethical behavior. Being informed and understanding the consequences of cheating will make most people do what is right. At the same time, elaborating on deterrent strategies like data forensics, web patrol, and test invalidations will do nothing for the small minority who have already decided to engage in cheating in some form or fashion. Therefore, it is incumbent on the state department of education to have agreements in place that require the signature of all students and administrators who participate in the testing process. "State officials should use test security incidents during the training of District Test Coordinators for the upcoming school year. Sharing these incidents without identifying those involved makes everyone aware of the potential threats that exist during the testing process, and allows for lessons to be learned in a professional manner. The goal should be for DTCs to take these real-life situations back to their home district and incorporate them into trainings of School Test Coordinators and Test Administrators. Being aware of these incidents and how they were handled becomes a natural deterrent."

Find out what other industry experts are saying about deterrent measures in testing—head over to "Deterrence in the Wild: Part 2"!