Skip to Main Content
Stony Brook University

RSOM Faculty Development Guide: Start a Teaching Renaissance

This guide contains resources for the Renaissance School of Medicine faculty.

 

Definitions of Assessment Validity and Reliability

Validity relates to the interpretation of results. Can an instructor make accurate assessments about a student’s knowledge and skills based on the student’s outcomes on any particular assessment? If the assessment tool is measuring what it is supposed to be measuring, it’s much easier for the teacher to recognize the knowledge and skills of each student.


Reliability focuses on consistency in a student’s results. If the assessment tool is reliable, the student should score the same regardless of the day the assessment is given, the time of day, or who is grading it. 

Designing Questions with Validity and Reliability

Below we look at three common question types used in medical education and see how they can be designed with high validity and reliability.

Multiple Choice Questions Short Answer Questions Essay Questions
  • Written in the form of a question
  • Question does not contain irrelevant material 
  • Question clearly indicates desired response
  • At least four plausible choices are provided
  • Choices are clear and concise 
  • “All of the above” or “none of the above” is not used
  • Higher-order questions such as application are used
  • Question clearly indicates desired response and is not misleading 
  • Item asks for no more than five elements
  • There can be multiple correct responses
  • New terms or new language is not used
     
  • Question is clear and includes multiple components
  • Clear directional verbs are used 
  • Focuses on higher-order critical thinking 
  • Question clearly indicates desired response
  • No one right answer, multiple possible responses
  • Does not ask students to just list factual pieces of information