This past week I attended the annual conference for the Society and Industrial/Organizational Psychologists (SIOP) in Honolulu, Hawaii. Many of the sessions dealt with testing issues, and one topic in particular was discussed a few times. That topic is Unproctored Internet Testing or UIT. When the internet began to be used in earnest for testing in the 1990s, industrial/organization psychologists saw a unique opportunity for such testing in the screening of individuals for hiring. For example, an organization looking to hire 100 managers for example, can provide a completely non-secure test openly worldwide on the internet, expanding the normal pool of recruits from a local group of thousands to perhaps a global group of tens of thousands or even more. In an attempt to keep the mass of test takers honest, the organization promised to have each candidate who made the “second” round prove his or her capability with a more secure “verification test.” Proponents of UIT are very passionate and insist that the model is here to stay.
I’ve known about the UIT-with-verification-test model for several years now and continue to have a couple of concerns, both related to test security. The first concern is based on the fact that UIT is used routinely without even trying to secure it, without knowing how much cheating is actually facilitated, without presenting evidence as to the effectiveness of the verification test, and without published best practices for optimizing the entire model. Without any controls the test content is obviously stolen and shared very quickly. My second concern, I believe, is more important and focuses attention on the competent candidates for the job who did not cheat and whose place in the second round was usurped by someone who was freely allowed to commit test fraud. As testing professionals, we can all agree that this kind of unfairness should not be allowed to happen.
The value of UIT to the business community is unquestioned and should occur in a manner that would allow the benefits to continue but that addresses the serious problems it unfortunately leaves behind. Here are some suggestions that should encourage a modified-UIT approach, which I refer to as SUIT, or Secure Unlimited Internet Testing. For the secure part I would make some or all of these changes:
- Use secure item and test designs that limit unnecessary exposure of test content.
- Run just-in-time data forensic analyses that detect aberrant or strange patterns of responding and that will not provide a score for those tests.
- Use inexpensive biometric authentication technologies.
- Consider new online proctoring models that monitor persons taking an internet test less expensively than traditional testing center methods.
- Monitor the web continuously to make sure that people are not stealing and sharing the test content.
There are other security measures that can be taken as well, such as the use of non-disclosure agreements and tip lines. Many of these can be implemented in automated ways at low-cost levels, supporting tests that continue to attract a large potential audience of candidates. The cost may be low enough to pass on to the candidate or to be considered part of a marketing/hiring budget. By adding low-cost security solutions to UIT, the security and fairness problems can be solved.
I applaud the creativity and the passion behind UIT that brought such a unique phenomenon to reality. I expect that that same creativity and passion could easily be brought to bear to solve its few remaining issues.