After the appraisal of the results of an assessment, you can determine the reliability of the assessment results by analysing the quality of the assessment and the quality of the individual assessment questions (the items). A good analysis contributes to the quality of assessment in your course. In addition, it helps you with underpinning and making decisions based on the made assessment.
An assessment and item analysis gives information that is directly usable for appraising the assessment results. Based on the statistical and content analysis and comments by students, you as an examiner decide whether the calculated score will be upheld or if a regrading needs to take place. An assessment and item analysis can also be used for adjustments in education or future improvements of the assessment.
Indicators of assessment quality
The quality of an assessment is expressed in a number of indicators. You use these indicators to identify any bottlenecks in an assessment.
A number of indicators summed up:
- The difficulty of an assessment question (p value).
- The distinctiveness of the assessment question between students who have and students who have not mastered the course material (rit value).
- The reliability of the assessment (coefficient alpha).
Questions to aid with assessment and item analysis
When analysing the assessment results, you can ask yourself a number of questions:
- How was each individual question made?
- What is the spread of the item and assessment scores?
- What is the pass rate?
- What was the difficulty of the assessment?
- What is the reliability of the assessment scores?
- Did it go as you expected?
- Is the answer key correct?
- Do you need to adjust the answer model?
Help with assessment and item analysis
An employee of Evaluation Service can help you perform an assessment and item analysis. The report can be delivered in the form of an organised assessment report in Microsoft Excel format. In the assessment report, besides students’ assessment scores and a frequency table with the distribution of assessment scores, the quality indicators of the assessment and items are also included.
When using assessment software, an assessment and item analysis is automatically conducted. The digital assessment coordinator of your faculty can take a look at the analysis report together with you.