After taking an assessment, the results are appraised. It is important when designing an assessment to determine the basis of the appraisal and record it in the form of a rubric or list of criteria.
On what basis do you appraise?
Appraisal by means of a rubric
Rubrics describe the performance of students on different levels using a number of criteria and give students and the lecturers insight into learning development and learning results. You can use rubrics for giving feedback, for appraising an assignment, paper, thesis or presentation, and with a performance assessment. You can also use the same rubrics in multiple courses within a curriculum so students gain insight into their development over the course of their studies.
Appraising with only a list of criteria
If you do not have a rubric (yet), you can also appraise by means of a list with just the assessment criteria. Behind each criterion is a blank space in which you write a qualitative judgement, or a score, preferably with feedback so the student can see how you arrived at that judgement.
Ways of grading and appraising
When grading assessment results and assignments you can make several choices that influence the reliability of the appraisal.
- Vertical grading: You grade the assessments per student.
- Horizontal grading: You grade the assessments per question or item.
- Anonymised grading: You can combine this with vertical or horizontal grading. You first anonymise the students and then start grading.
In addition, the assessment results can be appraised in different ways:
- automated, if the assessment was taken digitally;
- one lecturer grades everything;
- one lecturer grades everything and discusses some appraisals with colleagues;
- a lecturer grades the first part of an assessment and a colleague does the rest;
- a lecturer grades the whole assessment of part of the students and a colleague the other part;
- students grade each other formatively;
- students grade themselves formatively.
Assessments on paper with closed-ended questions can be graded by team Evaluation Services.
Appraising with multiple assessors
To increase the impartiality and reliability of an appraisal, you can make use of a second assessor (principle of four eyes). With end products (theses or internship reports) there is always a second assessor, but it can also be useful to use a second assessor when grading assignments, papers, presentations and oral assessments. You as supervisor and assessor appraise both the process and the end product. The second assessor only appraises the end product, unless they are also involved in the learning process.
Advantages of a second assessor
Many lecturers find it difficult to be both supervisor and (the first) assessor simultaneously. As a supervisor you might experience some subjectivity, since you guide the student through their process and towards an end product. Experiencing subjectivity is not an issue per se, the most important thing is that you are aware of it. You can try to intercept this by appraising the process separate from the content. A second assessor does not know the student or did not create the writing process, so they can merely appraise the end product.
The assessor agreement is the degree to which two assessors award a student the same appraisal. Differences can arise because both assessors use a different way of appraising or because they interpret the task of appraising differently. It is possible the appraisal method is interpreted differently and personal biases can play a role.
When the difference is one or more points by both assessors or when one appraisal is sufficient and the other insufficient, the assessor agreement is at risk and it is recommended to appoint a third assessor. When working with multiple assessors who are all grading the same assessment, a calibration session is recommended.