When designing an assessment, it is important to keep a number of quality criteria in mind. These criteria not only contribute to improving the quality of education, but also help you as lecturer to assess effectively and efficiently.
Validity
An assessment is meant to measure students’ skills, knowledge and attitudes. Validity means that you measure the aspects you want to measure. It is important that the assessment actually assesses the learning objectives (in relation to level and contents) and that every learning objective is sufficiently present in the assessment(s).
Ways to safeguard the validity of an appraisal:
- In the assessment matrix you make visible how every learning objective is assessed and at what level.
- When developing the assessment it is important to create answer models, assessment criteria or rubrics in advance so you explicitly show what the performance must comply with.
Reliability
Reliability means whether the assessment measures consistently. If a student takes the same assessment twice, they should achieve the same result. But also when two lecturers grade the same assessment, they should end up at the same score.
Using four methods, you can increase the reliability of an assessment.
- The four-eyes principle: Before the assessment is taken, you can show the assessment design to a colleague. They will appraise the assessment questions or assignment based on clarity, whether the contents and level align with the learning objectives, and whether there are any mistakes.
- A second assessor: To increase the reliability of an assessment, you can use a second assessor. If the two assessors end up at different appraisals, you can assign a third assessor.
- Use of an assessment and item analysis: After the assessment, you can determine the reliability of the assessment analysing the quality of the assessment and the questions (items). With digital exams, it is often possible to carry out an automatic item analysis. Tea Evaluation Service can help you out with a more detailed report.
- Combining assessment methods: By assessing learning objectives in several ways during a course or study programme, the appraisal will not be based on just one moment and is thus more reliable.
Effectivity
Students base their learning behaviour on the assessment of a course. Because of this, you as lecturer can influence the way students learn with how you design your assessment. The quality criterion of effectivity is all about designing your assessment in such a way that it evokes the desired learning behaviour of students. Educational design based on the principle of Constructive Alignment contributes to making sure a student learns in line with the learning objectives, learning activities and assessment.
Transparency
Transparency is all about making sure it is clear for students how they are assessed and on the grounds of what assessment criteria. Among other things, this involves communicating about taking paper exams, the use of tools, how assignments are to be submitted and deadlines. Offering a practice exam and making clear what assessment criteria is used by making assessment forms and rubrics available, is also part of being transparent. Afterwards, students have the right to inspect their exam or assessment form, so it is clear how their final grade came to be and also so they can receive feedback on their performance.
Feasibility
The assessment needs to be feasible for students as well as the lecturer. Think of, for example, practicalities such as financial requirements, making sure enough time for taking the exam is scheduled and having required tools available during the exam. In addition, it is important to keep your own schedule in mind when it comes to developing the assessment and grading. If you have a large group of students write an essay, you need to make sure you have enough time to properly grade these and that it is organised well within your team.