#30 Best Practices for Inclusive Interviewing

May 9, 2023 | By Molly Heublein

Audio

With Drs Chelsea Chang, Tony Dao, and Sonny Lee

Join us for an exclusive discussion at the Alliance for Academic Internal Medicine #AIM23 with Dr. Tony Dao, WashU, Dr. Chelsea Chang, UT Rio Grande Valley, and Dr. Sonny Lee, Loma Linda, on best practices for inclusive interviewing.  We review ways to prepare your program well before the interview season, how to conduct interviews on the day of to reduce bias, and how to choose the best candidates for your institution with an inclusion mindset. Recorded at Academic Internal Medicine Week 2023.

Website | Instagram | Twitter | Subscribe | Patreon | Free CME!| Youtube

 thecurbsidersteach@gmail.com

Show Segments

• Introductions: Dr Tony Dao, Dr Chelsea Chang, Dr Sonny Lee
• Case #1 from Kashlak Memorial Hospital
• Before interview day- defining diversity goals for your program
• Showcasing diversity to applicants
• Internal Medical Graduates as a Source of Diversity
• Training faculty before the interview day- implicit bias training
• Scripted interviews
• Blinded interviews
• Post-interview evaluations
• Take home points
• Outro

Best Practices for Inclusive Interviewing Pearls

1. Define diversity within your institution and generate metrics on where you are and what your goals are.
2. Encourage your faculty to examine their own implicit biases prior to interview season and provide faculty development to share best practices for inclusive interviewing. 
3. Utilize scripted or structured questions (especially situational or behavioral questions) to reduce bias. Consider including a question on DEI and how it relates to the success of a learning and working environment
4. Have at least one blinded interviewer as part of your program’s interview process with transparent messaging to the applicants on why and what information is withheld. 
5. Refrain from post-interview communication and inappropriate applicant communication (e.g. rank-order inquiries, geographic preferences, location of other interviews) as per NRMP standards and in accordance with AAIM best practices. 

Best Practices for Inclusive Interviewing Show Notes 

Tips for Pre-Interview Season

Start with thinking about what diversity means to your program. What does inclusion look like locally? Reflect on the power of diversity for your program and consider all the unique aspects of applicants–especially those who are historically under-represented in medicine–including race, gender, sexual orientation, range of abilities, international medical graduate status, etc.. that make the physician workforce even more excellent. Review your program goals for recruitment: how does diversity look at your program now, where do you want to go, and define your goals for improving diversity.  Create a vision of who you want to see on match day and what values you want your new residents to have (Gallegos 2022). 

Highlight efforts you are making to improve diversity locally, resources for under-represented groups, and/or faculty and trainees who interviewees may identify and connect with to help applicants see that they belong here! For example, Dr Dao shared his work with OutMed, helping LGBTQ+ applicants feel more welcomed and comfortable at his institution, highlighting what it’s like in the local community. Our guests encouraged listeners to use your website, social media, or application materials to share the work you’re already doing around diversity, your aims/values/metrics for selection, and your program’s commitment to inclusion (Thompson 2022). 

As recruitment and retention of a diverse workforce is an ACGME requirement, Dr. Chang noted it’s helpful to show your institutional leadership what you’re doing (describe the events you held, screenshot the website) and increase their motivation in faculty development to improve diversity and promote inclusion at your program. Faculty development for inclusive interviewing helps to ensure all interviewers are keeping program goals top of mind and considering how your program is demonstrating inclusion throughout the entire year.

International Medical Graduates (IMGs) as a Source of Diversity

According the the AMA, IMGs make up 25% of all practicing physicians in the US. 1 of 4 physicians of all specialities are IMGs.  40% of practicing internal medicine physicians are IMGs.  IMGs bring unique attributes like cultural competence, proficiency in second languages, serving rural areas to address care gaps, and yet,  IMGs tend to be under-accepted to top programs. Consider what steps can be taken at your institution to increase the interviewing, accepting, and training IMGs. 

Reducing Bias in the Interviews

Faculty training is important before interview season. AAIM published Recommendations to Promote Equity and Inclusion in the Internal Medicine Residency Interview Process to share best practices (Dao, AAIM 2022).  Implicit bias training, for example, through the Harvard IAT, is one place to start to help faculty appreciate what their implicit biases are.  Awareness through the IAT is not enough; it is important to actively work (via an action plan) to combat and address these biases (Capers 2020). 

Scripted Questions 

Scripted questions as part of structured interviews can help reduce unconscious and conscious bias within the interview process.  These questions focus on skills and qualities needed for the job (not personal attributes/connections) and reduce affect heuristic (emotional decision making).  Scripted questions as part of structured interviews are win-win: take time to develop questions, get outside consultation on the questions,  and identify biases that weren’t considered or accounted for initially. 

1. Develop questions based on the attributes you are looking for in a prospective applicant- use your mission statement, program values, and diversity goals to help inform these.
2. Ensure the question does not have a clear right or wrong answer.  The goal is to gauge how applicants would handle certain situations, particular beliefs, or how they would fit within your program.
   1. Situational questions: Intent predicts future behaviors: If you were the person on this team, how would you demonstrate to your team members that you are reliable?
   2. Behavioral questions: Past behaviors predict future behavior. Tell me about a time when you were reliable?
3. Create a point system with clear examples for interviewing faculty to use.
   1. For example, applicants can get between 1-5 points for an answer.  Faculty would have an example of anchors for the different ratings. 
4. Provide faculty development to ensure your interviewers are well prepared and understand the goals of structured interviews. 
5. Promote transparency that scripted questions are part of the interview process as part of your commitment to inclusivity. Consider adding wording to the website or in the initial email communication to the applicants to the effect of, “We will be employing standardized interview questions as an effort to reduce implicit bias.”    

Scripted questions can be felt as overly stiff by interviewees, and programs manage this differently. To address this, consider explicitly stating that the interview is scripted in an effort to reduce bias/improve inclusivity or allowing interviewers to choose from a panel of questions getting at specific attributes.

Consider asking a scripted question about DEI: What is your understanding about how DEI is related to the overall success of a program? How have you contributed to or plan to contribute to efforts in DEI? This question is key because it builds on the fact that recognition is not enough; every clinician needs to actively contribute toward DEI in their learning and working environment (it’s in everyone’s job, so how much foresight have you put into this). 

Blinded Interviews

These are an easy way to reduce bias and improve reliability in the interview process by withholding specific information (e.g. scores, medical school, hometown) from faculty to ensure they are seeking objective information (assessing attributes required for program) during the interview.  Going into an interview blind can feel uncomfortable, but it helps faculty focus on the point of an interview: to reduce potential biases that they may have toward applicants (potentially based on gender, race, and ethnicity) that are not related to the qualifications for residency. AAIM recommends at least one blinded interviewer (Dao, AAIM 2022).

Programs can decide which information to blind and at what point within the interview. The initial application reviewers could be blinded to demographics, photographs, hobbies, etc. to reduce various forms of bias that arise from having this information. Instead, the interviewers would only see the information (e.g. experiences, scholarship) that would help them make an informed decision about the resident (Bergelson, 2022).

With blinded interviews (and the questions that can come up), applicants may wonder, why are they asking me about a hardship I’ve overcome when that was in my personal statement?!? This, just like scripted questions, can also be perceived as rigid or off-putting with faculty not knowing about applicants. Our guests encourage messaging/introduction of the blinded interviewer/blinding process to the applicants during the interview day to address this possible applicant reaction and increase transparency on steps being taken by the program to reduce bias. 

Post-Interview Process

After the interview, programs typically tabulate the scores from interview days (using the anchors described earlier) and then generate a formula (e.g. 20% test scores, 40% interview, etc…) to weigh those scores and develop preliminary rank lists with potential adjustments based on DEI factors/program goals. 

Many programs will have metrics to determine the weight of different aspects of the interview. For some, the amount of scholarship done or research may be highly weighted, at other programs, test scores may be important separate from the weight of the interview and letters of recommendation. Programs also consider fit (in alignment with current culture) vs culture add (special characteristics that add diversity to program; instead of more of the same, this applicant contributes something different). 

In alignment with NRMP standards and based on AAIM best practices, Dr. Dao shares that interviewers/assessors at his institution do not engage in post-interview communication; instead, he recommends faculty direct applicants via transparent email communication to other people in the program (e.g. chief residents who can answer specific questions to help interviewees make informed decisions) who typically do not see or have bearing on final rank list (Alweis, AAIM 2019; Dao, AAIM 2022).  Clear messaging around this on the interview day to set expectations for applicants on post-interview communication is key. 

Structuring the selection committee conversations to ensure that time is spent discussing the candidates who are on the bubble of your typical rank list match line and having members ask why particular applicants are moving up or down the list can also help toward promoting inclusion and minimize bias. 

Links

Alliance for Academic Internal Medicine 2022 Promoting Inclusion and Equity in Interviewing Resources

Harvard IAT Implicit Bias Testing

Freeman CD, Guissé NF, Ceasar DR, Fakunle O, Fonseca CA, Fraz F, Gillis RP, Harris NM, Nichols AC, Oboh O, Henry TL. Reflections From Underrepresented in Medicine Applicants on the 2020 Virtual Interview Season. J Grad Med Educ. 2022 Apr;14(2):155-157. doi: 10.4300/JGME-D-21-00674.1. Epub 2022 Apr 14. PMID: 35463158; PMCID: PMC9017269.

ACGME resources around diversity in recruitment

Bergelson, I., Tracy, C. & Takacs, E. Best Practices for Reducing Bias in the Interview Process. Curr Urol Rep 23, 319–325 (2022). Doi: 10.1007/s11934-022-01116-7

Goal

Listeners will consider how to implement equitable and inclusive interview practices into their health professions education programs. 

Learning objectives

1. Outline steps to complete before interviews begin to encourage inclusivity in the application process.
2. Reframe how residency programs view international medical graduates in the selection process.
3. Employ best practices to increase equity during the interviews.
4. Identify techniques to gather interviewers’ assessments in ways to reduce bias.

Disclosures

Drs. Dao, Chang, and Lee report no relevant financial disclosures. The Curbsiders report no relevant financial disclosures. 

Citation

Dao T, Chang C, Lee S, Heublein M, Kryzhanovskaya E. “#30 Best Practices for Inclusive Interviewing”. The Curbsiders Teach Podcast. https://thecurbsiders.com/teach. May 9, 2023.