Written by Dr. David Foster, CEO, Caveon
March 14, 2017
Everyone likes a guide, likes the comfort and security that comes from following a reliable road map to a successful end. It has been my experience that successful people don’t make things up as they go along, but have learned and watched from others, and follow a proven path. The process outlined in this blog is one such roadmap, and it will help anyone in charge of test security, or anyone responsible for caring for tests and the scores they produce.
At Caveon, it has taken us years to develop and refine an exceptional test security process, and we want to share it with you. Whether you are in charge of the security of exams, play another role at the program, or are simply affected by good and bad security decisions, this 4-step process will be a valuable roadmap for you. Take it, use it, pass it along to others you know it would help.
Step 1. Recognize your security threats, calculate each threat’s risk to you, and rank them. This step will help to make sure that you are tackling the threats that are the most likely to cause you the greatest pain. Budgeted resources should be allocated to deal with the highest priorities identified.
Example: You have found out that some instructors, who also proctor the exams, are helping their students (and your test takers) cheat. This, you decide, is the threat that carries the biggest risk to you today. You are also very worried that students have been taking pictures of the test screens and sharing them over social media. These two threats, then, are where you should focus your resources.
Step 2. Protect your tests and test scores with targeted security solutions. This step has three VERY IMPORTANT sub-steps, each of which should be equally and seriously considered. These are, Prevent, Deter, and Detect/React.
Prevent. Here you put in place security measures that actually make test fraud impossible or significantly more difficult to do. For example, you could change your test administration approach so that tests are monitored by online proctors, eliminating the possibility that instructors are present to help their students cheat. For your second threat you could start a policy where cell phones are gathered up before testing begins. Or you could design your tests to limit the exposure of item content more effectively.
Deter. Each security measure you take here will make the test taker decide that he or she doesn’t want to cheat on or steal your tests. Remember that deterrence measures are only effective if they are made known, which means they should be announced and published. For example, you could publish a policy that any instructor caught helping a student cheat on a test will be fired, or announce that the new design of your tests will make it difficult to harvest questions. Even having students sign a non-disclosure agreement before taking the test will have a deterrent effect.
Detect/React. Every great defense has multiple ways of telling if an attack is imminent or has begun, and a plan to counter the attack. In the world of testing, a program manager can employ a sensitive defense to detect a breach that is about to happen or has just happened. As night follows day, it should be automatic that when such a threat is detected, a response by the program is immediate and of maximum impact. Your security plan should have very detailed reactions provided for each detection tool in place. For our examples, you could secretly insert surrogate students into the instruction/testing process. Those students immediately text you if an instructor provides inappropriate assistance to you or another student. You react by immediately replacing the instructor, launching an investigation, and firing the instructor if the allegation is proven. The important result is that you have stopped the threat in its tracks and minimized any damage.
It should be obvious that all three solution types should be used against every threat. And, remember that there are experienced individuals and companies, such as Caveon, who can help you craft these solutions if you need such help.
Step 3. Periodically evaluate security solutions and revise them as necessary. You should have several indicators (your detection system) in place to tell you how well your solutions are working. These include media reports (or lack of), web patrol reports, data forensics reports, frequency of tips, test administration logs, and others. Other types of feedback can be solicited from time to time from program employees and stakeholders. All of these steps keep you aware of the health of your program, and based on this feedback, solutions can be kept as they are, fine tuned, changed considerably, or replaced.
Step 4. Repeat all Steps 1-3 at least once each year. New threats will continue to emerge and you will want to be ready for them.
In conclusion, part of the reason that security problems are rampant today in high-stakes testing programs is that a reasonable, budgeted, planned security process is not in place. It’s time to change that.