3.2 Perspectives on testing. reliability, and validity

This video is the 9th video of a series of ten videos in the SPOC "How to write test items for measuring student progress".

The video explains two different approaches to item difficulty: EMIC and ETIC. The EMIC approach defines item difficulty based on the content expert's opinion, whereas the ETIC approach defines item difficulty based on data analysis of student responses.

The video continues to discuss two different models in testing student’s performance (achievements): an Educator's perspective, which focuses on diversity and covering all areas of the curriculum, and a psychological perspective, which focuses on psychometrics and statistics to estimate a single score reflecting the student's true ability. This perspective, aims and differences of the two approaches are presented.

The video also covers reliability and validity in measurement, which are two different concepts that are often confused. Reliability is about consistency, and validity is about accuracy. The speaker illustrates the difference between reliability and validity.
Three different types of reliability: test-retest, reliability across items, and inter-rater reliability are explained, including the use of the reliability indices, e.g. Cronebach Alpha. Similarly, three different basic types of validity are discussed, i.e. content, criterion, and construct validity. The speaker emphasizes the importance of construct validity, which is the most comprehensive type of validity.

The video concludes with a description of the practical way to assess if a measure is valid and stating that in the context of writing items for measuring student’s progress, the focus is on content validity in item writing and construct validity in the psychometric analysis.

The next and last video of this SPOC tells about test theory and item analysis with focus on the Rasch Model used in the VetRepos Project

 

COBL_litterature_40x40px_2017_18.png  Further reading