Dennis Maynes, Chief Scientist, Caveon
June Edition
calendar-icon_whitecircle
imageedit_9_6747439116
Celebrating 15 Years with Dennis Maynes
2018 marks the 15th anniversary since Caveon Test Security first opened its (virtual) doors. Caveon has come a long way since David Foster and the small band of colleagues first recognized, and decided to address, the industry-wide need for test security. What started as a small team of enterprising pioneers has grown into a one-stop-shop for test security solutions and thought leadership. In this article, a Caveon veteran speaks to his experience at the forefront of this burgeoning field, reminisces on the past decade and a half of progress, and ruminates on what the future might hold.
How has the reality of working for Caveon differed from your expectations upon joining 15 years ago?  
When we started Caveon 15 years ago, we were enthusiastic—we knew we had important services—but we were a bit naïve. We didn’t realize just how hard it is for organizations to switch from passively endorsing test security to actively managing and promoting test security. I knew we could make an important contribution with statistical analyses—I just didn’t realize how much and how little we would accomplish.

We have brought to market world-class services, which I had hoped to produce, but never considered the total effort that would be required. On the other hand, solutions for some test security challenges and problems are still elusive. Much work remains ahead of us. These past 15 years have been professionally rewarding and challenging. It is a joy to puzzle through the data to bring solid answers to our clients.
What was the biggest obstacle Caveon faced as a young company?
In my opinion, our biggest obstacle was doubting ourselves—not believing that we could really do this. Oh yes, we had challenges that most start-ups face with staffing, organizing, managing, and marketing. After we caught the fire and paid the price for delivering these services, we have continued to progress and make significant contributions to test security and our clients.
How has your role at Caveon changed in the past 15 years?
I have the same job title that I had 15 years ago, but my job has changed. Instead of developing software and algorithms, I am now focused on cohesive client support. Instead of wading through data, I am now focused on saying the right things properly. Instead of debugging code and proving difficult computational complexity theorems, I am now focused on communicating and coordinating the solutions to test security problems. Formerly, my primary language was “C” and “Python,” now, my primary language is expressing myself properly in English.
What is your favorite Caveon memory?
I have several favorite memories. Most of them were due to knowing that we made a difference. I remember reading a tweet from a high school student in Florida which made my day. He wrote, “I used to be a big-time cheater until my FCAT got invalidated!” Those statements are magical. It means that people are taking test security seriously.

Another favorite memory was when the judge commented upon my expert witness testimony in 2017: “Maynes was an exceptionally competent and credible expert statistician…his demeanor maintained a fine balance between demonstrating mastery of the subject matter, while remaining patient and mild with both representatives. He was, in short, a highly credible and persuasive expert witness.” I never expected such a comment to be made about my work or testimony. I was overwhelmed and grateful to be able to do that work.
What do you think sets Caveon apart?
Caveon is unique in at least three ways. First: as an independent third-party, our work is fair, balanced and unbiased; despite the claims of a few journalists. Second, Caveon remains intensely motivated to strengthen exam security. This motivation permeates all our services and drives our desire to innovate and improve. Third, Caveon’s commitment to clients is exceedingly strong. Every single employee at Caveon knows the work being done is valued by our clients.
Why should people, both inside our industry and in the wider world, care about test security?
After we started Caveon and I was asked what I did for work, I would reply, “I use data to chase cheaters.” I finally understood that the primary goal of test security was NOT to change the behavior of a few. Indeed, the goal of test security is to ensure that the test scores reflect what they were designed to measure.

The number one rule is that the test scores should be trusted and they must be valid. If the test scores don’t mean anything, why do we give tests? Just, to mark an item as “complete?” No, tests and tests scores are important. They are very important to judging competence and capability. Besides this importance, the tests need to be fair. Test security violations undermine validity, trustworthiness, and fairness. Our world relies upon tests to accomplish their purpose. If these things are important to you, then you have a reason to care.
How has your outlook on security and its role in the testing industry changed over the past 15 years?
I initially thought that security was a behavioral problem. I now know it is a validity issue. I initially thought that security could be improved by simple measures. I now know that test security is best served when “a village” comes together and works cooperatively towards the common goal of strengthening test security. Everyone who touches tests has a critical role in test security. I initially thought that test security was a process that could be summarized as “prevention-detection-remediation.” I now know that test security requires a tight laser focus on protecting against threats and risks. The correct test security solutions vary between clients. There is no “one size fits all” approach to test security.
How has the world changed in the past 15 years to make test security more relevant? 
The internet has matured, evolved, and enlarged in 15 years. This means there are more nooks and crannies, and more deep, dark, hidden places where pirates, item harvesters, and cheaters lurk. It’s also become simple and easy to steal exam content with miniature cameras that are always connected to the internet. So, the theft and disclosure of secure exam content is easier by orders of magnitude than it was 15 years ago.

At the same time, many of our test security processes have remained unchanged from those that were used in the last century. The number one security concern of every testing program manager is the theft and disclosure of content that took months and years to produce at the cost of tens or hundreds of thousands of dollars. These concerns are even greater today than they were 15 years ago.
What is the most important impact Caveon has had on the testing industry and/or the world at large? 
Caveon has been the catalyst to foster myriads of test security conversations around the world. I, personally, have discussed test security with people from the USA, Canada, Mexico, all of South America, Africa, Europe, Asia, Australia, and elsewhere. Fifteen years ago, the Standards on Educational and Psychological Testing barely mentioned test security. Today, the National Conference on Measurement in Education is producing position statements on test security. Fifteen years ago, test security standards in state assessments did not exist. Today, joint publications on test security have been produced by the Association of Test Publishers and the Chief Council of State School Officers. Coincidentally with Caveon’s embarkation on this journey, awareness and discussion of test security has come to the forefront in many organizations.
Is there a lesson you have learned in the past 15 years at Caveon that you would like to pass on to those who are just starting out in this field?
Innovation is hard. It takes time. You need to prepare to run the marathon. This is not a sprint. As you pursue improved test security solutions, there will be many setbacks and challenges. Do not give up. Persist. Learn from others.
What do you think the future holds for Caveon Test Security?
The future will bring even more technological change. Caveon Test Security will either adapt and embrace the future, or it will become obsolete. Because we know we must adapt, I am counting on Caveon continuing to change, to improve, and to support the mission of testing and test sponsors. 
Caveon 2003

Submit
Join our mailing list
Copyright© 2018 Caveon, LLC.
All rights reserved. Privacy Policy | Terms of Use
Contact
Interested in learning more about how to secure your testing program? Want to contribute to this magazine? Contact us!
"The number one security concern of every testing program manager is the theft and disclosure of content that took months and years to produce at the cost of tens or hundreds of thousands of dollars. These concerns are even greater today than they were 15 years ago."
Instead of debugging code and proving difficult computational complexity theorems, I am now focused on communicating and coordinating the solutions to test security problems. Formerly, my primary language was “C” and “Python,” now, my primary language is expressing myself properly in English.