Look at any periodical these days, and you’ll note many articles taking pot-shots at our industry—Common Core! Assessment Consortia! Too much testing! Teaching to the test! Opt out movements!
Our industry is experiencing a public relations crisis. It may have crept up on us, but the signs, internal and external, are obvious. It’s not just the volume of reports that is alarming, but the fact that many reports have basis in fact.
I recently observed a classroom of 9th graders who could not begin an important online state assessment because of technical problems. Security incidents abound, and while cheaters get the blame, the environment we’ve created provides the motivation, the justification, and the means to make it easy for them. Some of the tests or items we create clearly measure unimportant or irrelevant skills. It’s no wonder that students, parents, educators, business leaders, government officials, and many others are underwhelmed with what we have to offer. We have become an easy target and often deserve the criticism. It’s our own fault that grumbling entities like FairTest have gained credibility.
Here are some specific areas where we are challenged in providing a stellar service to society:
- Our tests are unfair in some circumstances—and to some audiences, we appear unwilling to fix the problems.
- Our tests are not as secure as they need to be, and could be.
- Our tests don’t measure the important skills as well as they could.
- Testing is loathed, difficult, time consuming, inconvenient and costly.
These are serious challenges which limit our value to those whose future depend on useful test scores.
What should we do about these shortcomings?
I propose that we leverage innovative technologies in creative ways to help solve them. While we likely can’t eliminate these problems, we CAN demonstrate significant improvements for test takers, consumers of test scores, and our other constituents.
What technologies am I talking about? Here is a partial list:
- Constantly improving computing power (storage capability and amazing speed)
- High-resolution displays
- Mobile devices (tablets, smartphones, laptops)
- Internet for data transfer, communication and information gathering
- Social collaboration software; chat programs
- Digital cameras, webcams
These technologies have already demonstrated some impact in the world of high-stakes testing. Test designs, such as CAT, LOFT, random selection of equivalent forms, randomization of items, immediate score reports, and others, are good examples. Test administration is now mostly computerized (though some pockets of paper-and-pencil tests are still used today). Of course, we have a long way to go.
We need to accelerate the migration from less-effective traditional multiple-choice questions to new item types and technology improvements such as DOMC, randomization of options, innovative scoring models, etc. Technology Enhanced Items (those that involve simulations, drag and drop, etc.) have been implemented and their usefulness evaluated. Now, they must be broadly adopted. Test administration is evolving where testing is becoming more convenient, less costly, and more secure using proctors who monitor the tests from a distance through technology. There are other improvements, such as Automated Item Generation (AIG) and Automated Essay Scoring (AES), which rely heavily on technology. Security has also been improved as technology has enabled lightning-fast data forensic analyses, more effective web monitoring solutions, and very clever biometrics for authenticating test takers, among other advances.
In general, these innovations have not yet had the wide impact they should. Their adoption is slow, too slow in my opinion. We need to move faster today in order to repair our reputation and provide immediate value to society. If we don’t, we are vulnerable to both the pace of change and the changing values in society. We must not be so arrogant to believe that we cannot be replaced. There are many examples of once-thriving industries (landline phone systems, brick and mortar video stores, etc.) that did not overcome their challenges. In facing this current crisis, we must leverage technology and evolve or risk a similar demise.
Blog #2 on this topic will list some internal barriers to change we all have, and how we can and should break down those barriers.