After the appraisal of the results of an assessment, you can determine the reliability of the assessment results by analysing the quality of the assessment and the quality of the individual assessment questions (the items). A good analysis contributes to the quality of assessment in your course. In addition, it helps you with underpinning and making decisions based on the made assessment.
An assessment and item analysis gives information that is directly usable for appraising the assessment results. Based on the statistical and content analysis and comments by students, you as an examiner decide whether the calculated score will be upheld or if a regrading needs to take place. An assessment and item analysis can also be used for adjustments in education or future improvements of the assessment.
Indicators of assessment quality
The quality of an assessment is expressed in a number of indicators. You use these indicators to identify any bottlenecks in an assessment.
A number of indicators summed up:
- The difficulty of an assessment question (p value).
- The distinctiveness of the assessment question between students who have and students who have not mastered the course material (rit value).
- The reliability of the assessment (coefficient alpha).
Questions to aid with assessment and item analysis
When analysing the assessment results, you can ask yourself a number of questions:
- How was each individual question made?
- What is the spread of the item and assessment scores?
- What is the pass rate?
- What was the difficulty of the assessment?
- What is the reliability of the assessment scores?
- Did it go as you expected?
- Is the answer key correct?
- Do you need to adjust the answer model?
Support during assessment and item analysis
A member of the Evaluation Service team can help you carry out an assessment and item analysis. You will receive the report in the form of a clear assessment report in Microsoft Excel format. The report includes the students' test scores and a frequency table showing the distribution of assessment scores, as well as quality indicators for the assessment and the items.
Do you use Cirrus' assessment software? If so, an assessment and item analysis is carried out automatically. Your faculty's digital assessment coordinator(s) can then view the analysis report. If you would like to have a more detailed report, please contact team Evaluation Services.