Think about a technical, also known as pair programming, interview. A quick recap, for readers who do not come from software engineering background: during a technical interview candidate is presented with a problem definition and have to write usually a small program to solve the problem (something like this). But, just before the candidate starts, the interviewer invariably says something along the lines of: “could you also talk through your solution as you go along, so I can understand how you think about the problem.” In effect, the candidate has to think about the problem, translate their thinking into code, and at the same time - explain it.
I have seen this every time as a candidate; I also have been doing it myself, until recently (we’ll come to that). I didn't know better, and I was not even aware that it is a problem. Oh, but it is.
Thinking is rarely a linear process. I, for example, writing this post have multiple ideas flowing in random order, that I note down, and expand, edit, reorder or remove, to get to the final result. This article was not written linearly. Similarly, solving a problem, the brain considers a lot of information, before it settles on a coherent solution. Everything happens in parallel, with some thoughts being louder and some quieter, mixing and changing. If you have ever meditated, you will be more than familiar with the concept of not being able to completely blank your mind. All you can do is listen and narrow down to something specific, but you cannot shut everything down.
Out of this chaos comes a systematic, clear line of thought - the solution. But that’s just half the story - the interviewer is not content with just the solution, they want the explanation of how we got there, too. Thinking is a parallel process, but the information can only be transferred serially, through language. Here is the problem: the candidate has to do two things - wander and be focussed all at once. Solve the problem and serialise their thoughts-in-progress coherently.
Depending on your predisposition and experience, doing these two things at once can be stressful. A very common situation in such interviews is candidates seemingly unable to do neither of the two, pleading they have a brain freeze. The interviewer usually obliges a minute of silence, but the damage is already done. The candidate is anxious and stressed about their performance.
There is an interesting technical concept in software engineering, called the deadlock. It means two processes depending on each cannot proceed, because they get into a state waiting for the result of the other process. A closed loop.
Brain freeze, is the brain effectively going into a deadlock state. To move the solution forward, the mind has to wander through the chaos, but at the same time to explain the wandering, the chaos has to be serialised. When it is serialised, it is difficult to move the solution forward, because immediately questions arise of whether that’s the best way and whether there is better. It depends on wandering through the chaos.
The interviewer has the same problem. They have to simultaneously evaluate what the candidate is doing (the solution) and their thinking (the explanation), and lead the interview. Depending on the interviewer they will weigh the two differently. From my own experience, I can tell you that it is very common to hear: “don’t worry about solving the problem fully, what we are interested in is your problem solving skills and your thought process to get there.” In other words, the vast majority of interviewers weigh the explanation more than the final solution code.
The solution cannot be weighed higher, because in principle the code can be memorised. Therefore, our industry’s interviews to get a software engineering role end up prioritising people who are capable of serialising their thoughts under duress. Doing two things at once is stressful and the candidates who get hired are more likely to be dealing better with stress than be objectively better problem solvers.
If the candidate has been through this process many times - they have experience. Experience reduces stress. If you don’t have that experience - tough luck, you are more likely to fumble, get brain freeze, and not progress through the hiring process. Anyone new, trying to break into the industry, are at a massive disadvantage. If it is your first technical interview - it is hard. It is no surprise then that it is extra tough for people coming from underrepresented backgrounds and minority groups. Little to no experience, plus some other cards can unconsciously or, even worse, consciously, be stacked against them.
It is not a complete surprise. In the mid-2020s a research done by NC State University and Microsoft made small waves in technical circles. Titled “Does Stress Impact Technical Interview Performance?”, it analysed the candidate performance based on a variety of factors (including stress). They split candidates into two groups - one solved the problem in private and then retrospectively discussed the solution with the interviewer, the other - with an interviewer being present in the room talked about the solution as they were solving the problem (traditional format). The results clearly showed that mere presence of interviewer in the same room caused stress, which then impacts performance:
Our findings demonstrate that stress impacts technical interview performance; indeed, in our study, participants in a traditional technical interview format
obtained significantly lower scores,
experienced significantly higher extraneous cognitive load, and
experienced significantly higher stress levels.
To make matters much worse, high stress load was impacting the minority and underrepresented groups' performance much more. There were no women among the candidates that succeeded in the stressful interview setting.
My Takeaway For Candidates
It will sound brutal, but until the industry practices improve, this problem will persist. I think there are three ways for the candidates to try and pro-actively work to improve the industry or reduce the impact for themselves.
First, give a heads up about your performance anxiety. Interviewers usually want to support the candidates as much as possible. Reaching out to them directly or via a recruiter can make a difference. Not enough people do it, but it is how I came to my own revelation as an interviewer, that I share below.
Second, practice interviewing. interviewing.io, a paid platform to get interviewed by BigTech engineers, coined the term “technical interview practice gap.” Their data showed that having done 5+ practice interviews almost doubled the chances of passing the real interview. interviewing.io is a paid service - there are alternatives. Various volunteer driven organisations can help with that for free. One example is codebar.io. I used to volunteer with them and there were a few coaches who were willing to help with practice interviews.
Third, whatever you do - do not quit. For reference, last time I was looking for a job, I applied to 20 odd places, got rejected by 18 at various stages, and received 2 offers. It is daunting, upsetting, and I think it is only going to get worse. The best thing to do is to remember - rejection after the interview does not say anything about your engineering skills. Try to brush it off and try to get another interview.
My Takeaway As Interviewer
I have not been doing an in-person interviews in a while when this research came out. Since the pandemic started the interviews are done online, which is in some ways different from the research setup. I did not know what changes I could make at the time, until a candidate whom I had to interview reached out giving heads up that they have performance anxiety and struggle to recall information if asked a question abruptly. I remember pondering how could I accommodate the candidate, without sacrificing the interview result. I cannot leave the room, but I decided that I can let the candidate work in silence, and discuss questions at intervals.
After a few iterations, this is the rough script I use after I explain the problem:
Before you start, I want to say two things:
First, you can work in silence, and focus on solving the problem. At certain intervals I’ll ask you to pause, and we will discuss the progress.
Second, you can ask questions at any time and I will answer, but I will avoid disturbing you myself as much as possible.
How does that sound?
Behind the scenes I watch the candidate’s code and write down questions I want to ask about their thought process, when I'll pause the candidate.
As I pause them, I recap what they have done so far (and acknowledge it’s not the end of the interview yet). We then discuss the progress using the questions I have noted. In the most recent interview the problem solving took about 30 minutes. I paused the candidate 2 or 3 times during that time for discussion. The rest of the time the candidate worked mostly in silence, or asked questions.
Interestingly, I noticed, I don’t have to guess if the candidate is stuck as much - they are more likely to speak up. I can use that moment to recap their progress so far, introspect their thinking, and nudge them in the right direction, for another round of silent work.
My anecdata going through a few of these, is that it does make a difference. I sense less anxiety in candidates about talking while working. Some, on the other hand, feel a bit strange, apologise or try to explain as they go along - I politely say that they don’t have to (but won’t stop them if they insist).
We are wired to be social beings, and talking is a very natural thing we do. So when no one's talking - there is a tendency to fill in the silence with abrupt questions (“why do you think this approach is better than other?”) or answers (“I will try this approach first, instead of something else”). For the above script to work - the interviewer must be comfortable with and impress on the candidate that it is okay to not speak for a minute. So…
How comfortable are you with silence?
P.S. As I was editing this, I wanted to find the year the original research came out, and found that one of the authors published a post of their own with additional guidance on what changes could be done to reduce the stress levels.