April Edition
Name It To Tame It
You might also like...
More Reads
Caveon veterans speak about their fifteen years of experience at the forefront of test security and ruminate on what the future might hold.
Read more →
Interview: Jamie Mulkey
Can Good Items Have Bad Item Statistics? Use the latest scientific research conducted by Caveon to evaluate all of your options and make the best decisions for your testing program.
Read more →
Data-Driven Success:
A Case Study: How Okta Won with Threat-Based Security
Join our mailing list
Copyright© 2018 Caveon, LLC.
All rights reserved. Privacy Policy | Terms of Use
Interested in learning more about how to secure your testing program? Want to contribute to this magazine? Contact us.
The Need:
Following five years of rapid growth, Okta, an industry leader in identity and access management, needed to develop a certification program to ensure that customers, partners, and internal employees had the appropriate knowledge and skills to support its products. Okta’s two major goals were straightforward: 1. Create a best-in-class portfolio of exams.
2. Ensure that the integrity of those exams was maintained so that the scores would be meaningful and remain so over time. To achieve these goals, Okta contacted Caveon to find out how to build high quality, secure exams. With security and innovation being the hallmarks of both companies, it was a natural fit. Together, they first determined the most dangerous security threats the young certification program would be facing. 

Five major threats were named:
It was clear that if Okta could mitigate the risk associated with these threats, the security battle was won, and Okta would be able to avoid the difficulties that have plagued IT certification programs for the past two decades.
The Security Solution
Part of the Secure Customer Solution Caveon proposed for Okta involved standard procedures such as having strong candidate agreements, developing exams in a secure environment, ensuring that only trusted individuals work on the exams, and using Web Patrol to monitor the Internet for leaked items. But the solution also included two innovative ways to defeat the anticipated threats; adopting them meant that Okta would be an industry leader in test security practices, despite their newcomer status in the certification world. Okta has always been committed to providing secure and reliable connections between people and technology, the company exercised their innovative thought leadership and became an early Caveon adopter.

What were these two innovative solutions?
1. Online proctoring 
2. Discrete Option Multiple Choice (DOMC) Online proctoring was selected to reduce the risk of harvesting from servers (Harvest Threat 1), as the exams would be delivered online and would never reside on local servers. Online proctoring would also neutralized the threat of collusion (Cheating Threat 2) since online proctoring makes it difficult for a test taker to receive help from an expert during a testing session. The second recommendation, the DOMC item type, acts as a security force because the DOMC item is stingy in revealing its content during testing. With less content exposed to test takers, the threat of memorizing questions to be used later (Harvesting Threat 2) is severely reduced, as is the threat of using  pre-knowledge (Cheating Threat 1). To handle the remaining threat (Harvesting Threat 3), using stronger non-disclosure agreements for employees and contractors deterred those inside of Okta from sharing test content.
With the decision made to implement these solutions, Caveon helped Okta develop the exams and provided technology so that the tests—with all of their security protections in place—could be securely administered through a popular online proctoring vendor.
The Happy Conclusion
IT certification programs are up against great odds; exam content is being leaked in some cases just weeks after publication. Those tests are posted on braindump sites for cheaters to buy and use. This has not been the case with Okta. Because of the sensible security measures infused into Okta’s program, the company’s exams are protected; the exam scores, meaningful. Okta’s exams have been published since August of 2016 and none have been found on the thousands of braindump sites around the world. The exams remain as protected today as they were when they were first published. Not bad for the new kid on the block!