What is assessment anyway? It is a word that is bandied about in education, but relies upon old assumptions, dusty data, and unverified hunches. Perhaps that touched a nerve. Diving deeper might help to question some of those long-held assumptions.
Recently, with the advent of generative AI, masses are wowed by the idea that they can load a reading assignment and ask the AI to create a quiz based on the content for their assessment. Yes, this is true. It is a distinct time-saver for teachers. Well, it is a time-saver for teachers who assess via quiz in order to confirm whether students have read the content material and retain factoids. Unfortunately, this assessment falls short in the rigor category. More like rigor mortis, since quizzes have been done to death.
Quizzes only hold value as formative assessment, for the purpose of confirming recognition and minimal comprehension. In truth, quizzes should not be a means of delivering grades, but of confirming the pathways of understanding.
Ideally, a quiz should provide detailed feedback about comprehension and decoding of the structures of the learning intentions. Either after each question or at the end of a short series of questions, effective quizzes must provide a confirmation of the response and the surrounding learning goal. They must also deliver information to learners who followed errant pathways. This information should clearly explain why each wrong answer was incorrect and why the correct answer is a better choice, based on the text material. Sadly, generative AI doesn’t deliver quizzes with this level of feedback because generative AI cannot think like a human or provide thinking strategies…yet.
Effective assessment must capture the essence of that the learner knows, in its full and complete cohesiveness in the same way that a slice of tissue reveals the array of healthy or diseased cells in an organism or a droplet of blood reveals the vigor and life of an individual, including infectious qualities which may have invaded it. Does anyone see this in a multiple choice response, which has a 25% likelihood to be correctly answered entirely by guesswork? The odds are greater still as most test developers invariably include an option which if far less likely to be believed than the rest, so 34% of respondents can wiggle their way through a standard quiz without knowing the content well.
The more authentic an assessment is, the more likely it will deliver meaningful results. When it comes down to it, can a student use geometry to solve a problem in the calculation of the amount of carpet needed for a flooring project? A student who can match the definition to every vocabulary word, but cannot solve the riddle of what a pattern of data suggests about the cause of an anomaly simply fails in the overall goal of scientific deduction.
Most educators work from top to bottom when they should reverse the journey and begin at the end. What is the end goal of learning for this unit? If the answer is to complete a series of specific selections from a limited pool of answers, perhaps a test is best. If the goal is to speak conversational Spanish on a specific topic, an oral assessment makes more sense.
According to Odin Westgaard, three conditions necessitate the use of a test:
- Specific skills, knowledge, and attitudes are required.
- Teachers do not have any other means of confirming if learners have the skills, knowledge, and attitudes.
- A test is the best option for assessment.
All three criteria must be true for the use of a test. Is knowing the date of the Boston Tea Party “required” for achievement of objectives? Could verbal skills be validated by another means aside from an essay test? Would a demonstration of knowledge be more effective? In most cases, tests are merely an efficient scoring method, not an ideal assessment of skills, knowledge, and attitudes.
If these premises are true, assessments should be built prior to the unveiling of the unit. Diligent effort should be expended to develop a test which covers all of the required material. In addition, the “wrong” answers should be crafted in a way to level the demonstration of knowledge to exploit key mistaken pathways in understanding, so that they can be identified and effectively addressed. Yikes! “Exploit” and “mistakes” sounds harsh, but it is as important to understand the errors students make and the reasons for them as it is to assure competency in the knowledge.
William Horton delves into the essentials of testing in E-Learning by Design, though the principles apply to all learning situations. In fact, the application of e-Learning is precise because online tests must bear an extra level of quality assurance to avoid corrupted results when they are not done in the presence of the teacher. Masterfully, Horton describes the step-by-step procedures to consider with each objective, each type of question, the language of the question, the essentials of feedback, and the language in which feedback responses are provided for the learner to yield the maximum likelihood of positive response in error situations.
Buckle up! Creating the rigorous test is only the first step. Every answer must be built into the lessons which precede it. Even the incorrect options should be covered in the coursework. When a question is asked on a test, there must be “bread crumbs” through the teaching content to provide the means for answering it accurately and knowing that the false answers are indeed false. Yes, it takes a lot of time and intentional effort, but to neglect it is to presume that students have a means of understanding something when it was not thoroughly taught to them.
There is beauty in this method. When every piece of information for the unit has a distinct outcome in the test, students are far less likely to feel that they studied something “that was on the test” or didn’t know that something was going to be covered. In fact, test questions and feedback can be embedded with the exact lesson and step in which the material was taught. Some benefits to this approach are an increased respect for the professionalism of the educator, the validity of the assessment, and the pathway for a solution.
A side benefit of the method is the succinct approach to each topic. There is no need to mention that “this will be on the test” because everything will be on the test. Therefore, wise learners know that they are best served by being engaged and attentive to the information which is being provided. When there isn’t any “fluff” or excess content, the unit progresses well from start to finish.
The bonus with a well-crafted test is that a statistical analysis after students have completed it will provide some effective answers for the quality of delivery of the content. Don’t be surprised if the first time reveals a question which was poorly mapped to the lesson content and, therefore, yielded poor results. After a bit of retooling, the lesson is ready for the next class and provides a better chance of success. That sounds like a win-win scenario.
Teacher Takeaways
Quick and easy test-making is tempting, but faulty. There is no substitute for backward planning and minute, intentional selection of questions to match goals and content styled to answer those questions.
Horton, W. (2011). E-Learning by Design. John Wiley and Sons, Inc. DOI:10.1002/9781118256039 (Link here.)
Westgaard, O. (1999). Tests that work: Designing and delivering fair and practical measurement tools in the workplace. Pfeiffer. ISBN-0-7879-4596-X.
EmpowerED 