- Researchers from North Carolina State University and Microsoft published a study in July that found that developer interviews often evaluate for performance anxiety, rather than actual coding skills.
- Researchers found that students who conducted a technical interview in a private room did twice as well.
- The typical developer interview process can also introduce bias, so North Carolina State University is working with companies like Microsoft, Google, and CoderPad on ways to improve the format.
- Visit Business Insider’s homepage for more stories.
Developer interviews — which often involve having a candidate stand in front of a whiteboard and solve coding problems in front of a panel of interviewers — have long been a dreaded process.
And according to new research from North Carolina State University and Microsoft, typical developer interviews are better at testing whether a candidate has performance anxiety than whether a candidate has competent coding skills.
During a normal whiteboard interview, a candidate has solve a problem on the spot while explaining their process to interviewers. But according to the study from July, where researchers conducted technical interviews with 48 computer science students, a candidate does twice as well if there isn’t anyone watching.
“I don’t think we have any definite answers here, but I think this is fairly strong evidence there is something wrong with what we’re doing,” Chris Parnin, an assistant professor of computer science at NC State and co-author of the paper, told Business Insider.
While tech companies may have created developer interviews with good intentions to test a candidate’s programming skills, it has led to “harmful effects,” Parnin says, because developers may experience stress during interviews when someone is watching them the entire time.
“We as an industry have inadvertently reinvented a stress test instead of an actual measure of technical skills,” he said.
Now, North Carolina State University is already starting to work with companies like Microsoft, Google, and CoderPad to explore ways to make small improvements in the interview process.
Developer interviews can create unnecessary stress and anxiety
Parnin himself felt “there was something off” when he went through developer interviews in the past. This feeling stuck with him.
He eventually started doing research on programming interruptions, where he found that if a developer is interrupted in the middle of programming, it can take 10 to 15 minutes to recover.
“As I started doing more research as a professor, this kind of connected with me,” Parnin said. “This idea that if programmers consider interruptions as one of the worst things that can happen, it’s weird that the technical interview is designed around being asked questions about what you’re doing but being surveilled the whole time.”
Before beginning the study, the team gathered feedback on what developers said about technical interviews, and one of the most common things that came up was how much anxiety and stress these interviews cause.
Besides stress, the interview process could also be riddled with bias, Parnin says. For example, if two candidates solve a problem correctly, interviewers may be biased towards hiring the candidate who projects more confidence, even if that person’s solution isn’t technically better.
Some candidates might also have an advantage over others when it comes to coding interviews because of where they went to school. For example, Stanford offers a specific class that teaches students how to pass a technical interview.
“If you look at other schools, they never have done these sorts of tests so [candidates] go out and bomb them,” Parnin said. That means that candidates who weren’t privileged enough to attend a school with practice exams are automatically at a disadvantage.
“Turns out, these signals are just noise,” Parnin says. “We’re just measuring something else. If we can allow for multiple pathways for people to be interviewed, then it could potentially help reduce some of the problems associated with this.”
Also, the developer interview process often results in college students spending more time practicing interview problems to get a job than working on schoolwork about fundamental computer science concepts.
Companies like Microsoft and Google are already looking into changes in the interview process
Since Microsoft helped shape the study, the results could potentially help guide it in rethinking its own interview process, Parnin says. For example, Microsoft may look into moving into a more conversational interview where the focus is on solving a problem together, rather than being interrogated.
Microsoft has already been rolling out some changers in its interview process. Starting in early 2016, Microsoft’s developer division began rolling out the “Alternative Interview Framework” to better match an applicant’s skills to what the job really requires. It’s now used by Microsoft’s developer division to interview product managers and program managers.
Likewise, NC State is still in the early stages of working with companies like Google and CoderPad. Many companies have already adapted the whiteboard interview process to take place virtually during the coronavirus pandemic, and Twilio is even de-emphasizing it.
Parnin suggests that some potential ways to remove stress for candidates during the process include take-home exams, having candidates build a real-world project, adding private time during interviews, or having developers explain their thinking process retroactively.
“There’s not going to be one easy answer for this,” Parnin said. “Each company will have to reevaluate what’s the best way for them to tailor the interview process so that it’s more humane.”