You might also like...
More Reads
Ever wonder if SmartItems actually deliver on their promises? SailPoint testifies that when used on two of their certification exams, SmartItems provided benefits such as stellar protection, reduced costs, psychometric quality, and more appropriate candidate preparation.
This dear friend has inspired nefarious creativity among us for all 105 years of its life. But due to unforeseeable advances in stratagem and technology, its effective time here on Earth has drawn to a close.
Jennifer Geraets is an Associate Director of Test Security at The College Board, where she manages test security incidents and investigations and contributes to the development of test security policies, procedures, and initiatives. She has presented at several industry conferences, including Innovations in Testing, Conference on Test Security (COTS), and National College Testing Association (NCTA). Jennifer served as chair of the security committee for ATP’s Workforce Credentialing Skills Division. She helped organize and served as chair for the Test Security Credentialing Peer Group, which is dedicated to enabling credentialing testing organizations to exchange test security related information, develop best practices, and educate others about the credentialing testing industry. Before joining The College Board, she worked at ACT, Inc. for many years and was also a bilingual kindergarten teacher in the Houston, Texas area.
caveontoc_icon_BLUEtoc_icon_BLUE_desktop
"While programs should realize that they need to commit to funding an up-front cost for prevention efforts, they also need to acknowledge that the costs of investigating a test security incident are often higher than the costs of preventing it."
3. What are some of the most valuable security measures a testing program can use to prevent item theft? Protection against item theft begins during the test development process. Testing programs need to control access to their item banks and any other systems used in the development process. Strong authentication processes should be used to ensure that only those who should have access to the systems are able to access them. Permissions should also be limited. For example, item writers should not be able to view items submitted by anyone else, and item reviewers should only be able to view the items assigned to them. Testing programs should also implement test administration policies that prevent access to cell phones, smart watches, fitness trackers, and other devices that can be used to capture test content. They must train test staff so that they are familiar with these devices and what it looks like when an examinee is using one of them. They need to ensure that no one, including test administrators, can access test content prior to test day. Booklet seals and lockboxes are common practices for paper testing. For computer testing, the test content should not be accessible until the time of the administration, and then it should only be accessed by an examinee who uses the appropriate credentials.
4. How can programs employ a secure authentication process to prevent the wrong individual from taking their exams? At a minimum, examinees should be required to present a government-issued photo identification at check-in. Staff should be trained to thoroughly inspect the ID, and in some cases, it may make sense to use equipment with document authentication capabilities. Multi-factor authentication, a combination of something that an examinee knows (like a password), something they have (a token), and/or a biometric such as palm scans or keystroke analytics, can add an additional layer of protection. Authentication processes are not one-size-fits-all capabilities. If an examinee takes only one test, then authentication methods that are designed to compare pieces of data from one test administration to another probably won’t be very valuable. Testing programs need to thoroughly analyze their needs and the structure of their program in order to determine which authentication measures make sense for them.

5. In your opinion, what is the most effective type of security measure: deterrent security measures, preventative security measures, or security measures designed to detect cheating or theft? Do these measures operate independently or together? All of these efforts are important to a comprehensive and well-designed test security plan. Testing programs should always strive to prevent and deter misconduct. In addition, since complete prevention is impossible, programs need to have measures in place that detect misconduct when it happens. It’s also important to have a plan in place for responding to misconduct when it’s detected. That plan should include the investigation procedures that will be used, documentation of all parties’ responsibilities, and the process used to determine the outcomes of an investigation.
"Testing programs should always strive to prevent and deter misconduct. Since complete prevention is impossible, programs need to have measures in place that detect misconduct when it happens."
6. What role do proctors play in test security? Do you believe their primary purpose is to prevent fraud during a test administration? Alternately, do they primarily serve to detect and control fraud?
Proctors play important roles in both prevention/deterrence roles and detection. Proctors who vigilantly monitor examinees during the test administration are critical to preventing and deterring misconduct. Equally important is their ability to detect misconduct and manage incidents when they happen. Testing programs need to ensure that test staff are thoroughly trained in vigilant proctoring practices and appropriate responses to testing misconduct.
7. How do you think the computerization of tests helps to prevent test fraud? Computer-based testing helps to prevent advanced access to test content. Paper tests must be shipped in advance of the test administration, which provides individuals with an opportunity to access the test for the purposes of sharing content with others or developing an answer key. If test content is appropriately secured during CBT delivery, no one can access the content until the examinee provides appropriate credentials to gain access to the exam at the time of the administration. Computer-adaptive testing with a sufficiently large item bank essentially eliminates the ability of examinees to copy from their neighbors, since examinees receive unique test items in a unique order. Finally, if a testing program is concerned about administrator misconduct, computer-based testing addresses the concern by eliminating the ability for administrators to change examinees’ answers after testing is complete, assuming that the test delivery system appropriately controls access to test content and responses.
"At the very least, programs should stay current on the technologies that exist, learn how they are used, and train test staff to be able to recognize the use of these devices and respond appropriately."
8. The randomization of items on computerized tests has effectively stopped copying behavior during a test session. What other test design features might have similar effects on other types of test fraud? Randomization of answer options can also eliminate the benefits of using an answer key or copying during an exam. Despite the potential test security benefits, it should be noted that a testing program would need to conduct thorough psychometric research to analyze any potential impacts on item performance before implementing such a strategy. Performance-based items also make it much more difficult for an examinee to engage in test fraud, since completion of the designated task is what determines successful completion of the exam. 9. Does the use of mobile devices make it easier or harder to prevent test fraud? What are your suggestions for how programs can approach this new technology? The use of mobile devices and in particular, small hidden devices, is creating test security challenges for many testing programs. Some programs have implemented wanding to detect these devices, but that may not be possible for everyone. At the very least, programs should stay current on the technologies that exist, learn how they are used, and train test staff to be able to recognize the use of these devices and respond appropriately. While mobile devices can provide opportunities for examinees and test administrators to engage in misconduct, testing programs should also consider whether they can be used as an asset to quickly communicate with test staff. They may facilitate quick and efficient communication between a testing program and test staff, both for general reminders and for addressing issues with a specific test administration or examinee. Programs should carefully weigh the pros and cons of an administrator using a mobile device during testing before making a decision. 10. What are typical methods that testing programs can use to prevent access to results databases, so scores cannot be altered? Testing programs need to have strong controls in place around who is granted access to sensitive data, including test scores, and what actions each of those individuals can take with the data. The system should log all actions that were taken by each individual in the system, require that a reason be provided for any edits to the data, and include a date and time stamp for each action. Data should be proactively analyzed to detect any unusual access or edits, and any anomalies should be investigated. Testing programs also need to involve IT professionals who can analyze system security, identify vulnerabilities, and recommend specific enhancements to improve security. 11. Looking to the future, are there any new or innovative preventative security measures that you are excited to implement or think will have a positive impact on test security? Advances in the area of biometrics offer a lot of possibilities for better authentication, but also raise privacy concerns. It will be interesting to follow this capability over the next few years and see which viewpoint wins out, or if there is a “happy medium” that provides some level of benefit while raising fewer concerns about privacy. It seems likely that the solution will vary depending on the value of the test results and the location of the test administration. The ability to use the data obtained through computer-based testing systems in real time to prevent testing misconduct is another area that bears watching. As more and more data is collected and accessed in real-time, testing programs may be able to detect testing misconduct early in the test session, and prevent it from continuing for the entire test. Programs would need to determine whether they are comfortable taking action based on data alone, or if the data would be used to send a message to the proctor to monitor the examinee and confirm misconduct, at which point the behavior would be addressed.

Submit
Join our mailing list
Thanks!
Copyright© 2019 Caveon, LLC.
All rights reserved. Privacy Policy | Terms of Use
Contact
Interested in learning more about how to secure your testing program? Want to contribute to this magazine? Contact us.
The Prevention Lockbox
calendar-icon_whitecircleJennifer_Geraets
Jennifer Geraets, The College Board
Ask an Expert: Jennifer Geraets
1. What is the value of prevention as a test security measure?
Prevention is an absolutely critical component of a test security program. All testing programs want fewer test scores to be impacted by misconduct in order to maintain fairness, protect the validity of the scores, and protect the value of their brand. One way to achieve that goal is to make engaging in misconduct either impossible or much more difficult. While programs should realize that they need to commit to funding an up-front cost for prevention efforts, they also need to acknowledge that the costs of investigating a test security incident are often higher than the costs of preventing it.
Prevention measures need to change as the testing environment changes. Testing programs should make a conscious effort to understand how methods for engaging in testing misconduct change over time. For example, as new technologies are introduced, or as old ones evolve, testing programs need to adapt their security plans to address the risks (and benefits) that are created. In addition, learnings from test security incidents should be analyzed to determine if there are additional prevention techniques that can be implemented to lessen the likelihood of that particular type of incident arising again. 2. What are some of the most valuable security measures a testing program can use to prevent cheating?
The use of different types of items can help to prevent cheating. For example, memorization of a key is not useful if items require task completion or another type of performance. Testing programs can take advantage of technology such as wanding, authentication devices, and lockdown browsers to prevent a variety of cheating techniques, including item harvesting, surrogacy, and using prohibited resources during testing. In addition, reducing the length of testing windows prevents cheating by reducing the amount of time and the number of possible sources an examinee has to obtain test items or answers in advance of their test session. Testing programs also need to be present at test sites during test administrations. Particularly when there are concerns about test administrator or organized examinee misconduct, a physical presence during the administration can be invaluable. While it’s not feasible to be present at every test site, sites should be chosen strategically by using data from past test administrations and/or investigations. Another benefit of site visits is word-of-mouth. Test sites talk to one another, and it won’t take long before other sites are made aware that unannounced visits do occur. The mere possibility of a visit deters misconduct and encourages proper test administration practices.