Research & Subject Guides: RSOM Faculty Development Guide: Start a Teaching Renaissance: Measuring Success on Assessments

Measuring Success on Common Medical School Questions and Assessments

Multiple Choice Questions	Extended Matching Questions	Fill in The Blank Questions
Automatically graded. Provide explanation of correct answer. ExamSoft can do all of this.	Automatically graded. Provide explanation of correct answer. ExamSoft can do all of this.	ExamSoft can grade it but you have to double check student answers. Provide explanation of correct answer.
Essay Questions	Case-Based Learning Activity	Team-Based Learning Activity
Cannot be automatically graded by a software such as ExamSoft. Since essays have high subjectivity, it is recommended to use a rubric to grade them.	Can be assessed informally by asking student groups to report back answers to various case questions. Can be assessed formally by including cases on a graded exam or quiz.	Has two forms of assessment the iRat and gRAT. iRAT (Individual Readiness Assurance Test) is given at the beginning of a class session and tests students’ knowledge of pre work material. It should be a low stakes quiz. gRAT (Group Readiness Assurance Test) is taken in groups after students complete the iRat. It can be assessed informally by having group use scratch off cards or it can be assessed formally by giving a graded quiz for groups to complete.
Problem-Based Learning Activity	OSCE	Audience Response Questions
PBL assignments can take on many forms, some of which are a presentation or written paper. The main goal is to assess whether students have clearly communicated the problem, research methods, solutions, and resources. A rubric should be created so that students clearly know how they will be assessed.	Students can receive feedback from their standardized patient and their instructor. Standardized patients check off what students did and the SIM Capture software automatically gives a grade All patient interactions are automatically recorded and instructors are encouraged to watch the recording to give extensive feedback Rubrics or checklists should be provided to students so they know specifically what they will be graded on. There should also be a summary feedback along with the rubric.	Audience response questions are a great way to informally assess students during a lecture. They should be used to see what students have learned and what they still need to review. There will be a session on designing engaging lectures which focuses on the use of audience response questions as well as a session on how to use the CBase ARS system in the near future.

Measurement Tools: Rubrics and Checklists

Two types of measurement tools use a lot in medical education are the rubric and checklist.

Benefits of Rubrics and Checklists:

Help students understand expectations.
Provide clarity, consistency, and fairness in assessing student work.
Offer valuable feedback to guide student learning and growth.

Designing a Rubric

Rubric Example

Rubric adapted from: https://provost.fiu.edu/apa/assessment/resources/rubrics-and-curriculum-maps/_assets/rubrics/College%20of%20Medicine%20Rubric%20-%20OSCE.pdf

Rubric Example
Here is an ADA compliant version of the Rubric Example.

Considerations for Designing a Rubric

Determine what knowledge and skills the assignment is designed to assess (learning objectives).
Clearly establish criteria for knowledge and skills (performance criteria).
Divide the criteria into distinct levels of student performance (performance levels).
Develop descriptors of the criteria (performance level descriptors).
Determine the scoring method if you are giving a numeric or letter score. Note: You do not have to give a score.

Designing a Checklist

Checklist Example

Standardized Patient Checklist – Information for students and standardized patients.
Note: The N/A designation should only be selected for items that cannot be assessed on a particular case, for example, a case involving a phone call, where items related to knocking on door, closing the door, shaking hands, making eye contact, washing their hands cannot be assessed. (Note that the text in red is directed to the student and should be used by the SP to gauge what is expected of the student).

Checklist adapted from: https://www.ttuhsc.edu/medicine/academic-affairs/documents/step/Third-Year_OSCE_Manual_2019-2020.pdf

Checklist Example
Here is an ADA compliant version of the checklist example.

Considerations for Designing a Checklist

Checklists provide a basic overview of a task. Was something done or not? Was an item included or not?
Checklists provide less details than rubrics do but are less subjective.
To score checklists you should assign a point value to each item.
You can have a checklist where items get partial credit, but you should include a column for that.
You should include a description of each item or an expectation for each item (see red text in example).
You can include a space at the bottom for comments.
Checklists should be shared with students prior to the activity so that they know what is expected of them; however you may not want them to be able to refer to the checklist while they are doing the activity. That is up to you.

RSOM Faculty Development Guide: Start a Teaching Renaissance

Measuring Success on Common Medical School Questions and Assessments

Measurement Tools: Rubrics and Checklists

Designing a Rubric

Designing a Checklist

Library

Resources

Follow Us