Process
1. All DOE residencies evaluations of faculty by residents were reviewed to identify questions asked and which ones were more commonly used across programs.
2. A literature search was conducted to review examples of resident evaluations of faculty in other programs
3. A literature search was conducted to review research done on resident evaluations of faculty.
4. As a result of the first three steps, a common set of categories emerged. Including:
5. Items were developed for each of these areas. Items were reviewed by program directors, DOE leadership, residents and faculty. Feedback was obtained and a set of 14 questions became what are now called the Common Questions of Resident Evaluations. They are organized in four topics:
Goal Setting
Learning Environment
Feedback
Professionalism
6. The Common Questions were converted to a Survey Monkey and emailed to all residents to complete. At the same time, a self-assessment version of the Common Questions was emailed to all faculty to complete as a Survey Monkey. This provided residents and faculty practice with the items and an opportunity to explore resident evaluation compared to faculty self-evaluation. Results were shared in the Med Ed Day 2018 Faculty Development presentation.. Results showed considerable similarity between resident and faculty evaluations with both indicating that setting goals, formative feedback and summative feedback were our areas to prioritize for faculty development.
7. In July 2018 the Common Questions were added as the first 14 items of each resident evaluation of faculty in each ACGME residency program at MCHS. A sample analysis was conducted in October to demonstrate functionality of the system. In April 2019 data was pulled and analyzed for all resident evaluations from July 2018-April 2019. Results were presented at three levels: individual faculty member; residency program; and, all residency programs combined. Results were shared with program directors and DOE leadership. Each program director received the summarized data for each faculty member, the composite data for their program and the composite data for all of DOE. They were asked to share the individual and program level data with each faculty member and provide guidance, informed by this data. Each July since then, the data is analyzed and results shared with the program directors and DOE leadership. Program directors receive information on how their faculty performed compared to the DOE composite for each item. Areas below the composite were linked to item resources which could be used at the program level for faculty development and the faculty development lead offered to assist with implementation.
The DOE level data was also presented at the MedEd Day 2019 and 2021 Faculty Development talk
(At minutes 45:30-48 https://youtu.be/8jK6bOSlJec)