While recently researching security methods and principles from other fields, I came across an article written by Colonel Thomas Bovet titled The Lost Principles of Security (http://www.securitymanagement.com/article/lost-principles-security-005985). Reading it, I soon learned that he was referring to securing facilities and protecting human life against terrorist attacks. Intrigued by the title and content I continued to read it and discovered many similarities to the security circumstances in our testing world (e.g., goal of protecting assets, concerns regarding specific threats, etc.).
Here are a few security “principles” that I gleaned from his article, along with my translation into testing.
Principle #1. When setting up a security system we usually follow established standards and procedures and generally discount the creativity and determination of cheaters and thieves. Most, if not all, testing programs have secured their exam content and results with standard methods for decades now, with a human proctor being the foundation of security. There is plenty of evidence that these methods have weaknesses, partially due to new technology-based methods for cheating and for the theft of test content. Today’s standard security models are vulnerable because training and tools given to proctors are insufficient to deter, prevent, and detect attacks, which are always unexpected.
Principle #2. Reaction is slower than action. Even if the attack is detected quickly, the reaction of the security team is always slower than the action of the cheaters and thieves, and much of the damage has already happened. Additionally, most testing programs have implemented processes and procedures which require decisions by committees and discussion.
Principle #3. Defense is only one of many tactical options available to the testing program. Since standard defenses are well-known and therefore vulnerable, and since the reaction to an attack is usually too late to avoid damages, what are the other tactical options? Two were described by Colonel Bovet: Design and flexibility. One “offensive-minded” tactic is to embed more security design into our tests. With a few changes to our tests, we can more quickly and more easily detect or prevent specific types of test fraud. The result would be to eliminate the element of surprise. Using “Embedded Verification Tests” or Trojan Horse items are examples of ways to increase the likelihood of early detection of specific methods of cheating . Designing tests and items to reduce exposure of test content would make stealing the tests less effective and would deter the thieves. Computerized adaptive tests (CATs) and the Discrete Option Multiple Choice (DOMC) are examples at the test and item levels, respectively. There are many others.
Principle #4. Security personnel need sufficient authority to respond in a timely manner to security threats. Bovet believes that “front-line security personnel” need to be granted flexibility in their development and application of security methods. In testing, this means that everyone, from the proctors, test administrators, test developers/designers, security managers, data forensics specialists, program management, and others, need to be able to deviate from set or traditional models, whether security or psychometric, in order to more effectively deal with new threats and to protect the tests and their results.
There is much to be learned from security methods applied by others. Banks, casinos, airports are examples of entities within other industries that have taken security to creative levels as they deal with a continuing procession of new threats. Here, at Caveon, we will continue to monitor those efforts and will report what we find.