Technical Interviews From Both Sides

An engineering team is hiring two backend engineers. It runs four rounds: an algorithm problem on a whiteboard, a system design conversation, a "culture" chat and a call with the hiring manager. Nobody wrote down what each round is meant to measure. In the debrief, the first senior person to speak usually sets the tone, and the decision follows. A candidate who freezes at the whiteboard is rejected. One who talks confidently through a design they've rehearsed is hired.

Both sides of the table lose here. The team is making a noisy decision and doesn't know it. The candidate is being assessed on something other than the job and can't see what.

This post covers what interviews can actually predict, how to run them so they predict more, and how a candidate can read a process from the outside.

What an interview can and can't predict

Decades of personnel research give a clear picture. Schmidt and Hunter's 1998 review of 85 years of findings ranked work sample tests and structured interviews among the strongest predictors of job performance. Paired with a test of general mental ability, each reached a mean validity of .63.

A 2022 re-analysis by Sackett and colleagues corrected what they showed was a systematic overcorrection in those earlier estimates. Most selection methods kept their rank, but their validity fell by .10 to .20. Structured interviews came out as the top-ranked method. The authors' conclusion: selection procedures "remain useful", but they predict performance considerably less well than previously thought.

Two things follow:

• Structure is the biggest lever you control. A structured interview asks the same job-relevant questions of every candidate and scores answers against a written standard. That's what separates the top-ranked method from a conversation.

• Even a good process is noisy. Treat a hire as a reasoned bet, and design the process to reduce the noise you can, not to find certainty.

The test: for each round in your process, can you write one sentence saying what it measures and how it's scored? A round you can't describe that way is a conversation, not an assessment.

Interviewers: write the scorecard before the first candidate

A scorecard turns "I liked them" into evidence. For each round, name two or three things that matter for the role, the question or exercise that shows them, and what weak, adequate and strong look like. Write it before you meet anyone, so the bar isn't set by whoever came first.

What you're assessing	Exercise	Weak (1)	Adequate (3)	Strong (5)
Debugging	A failing test in a small, unfamiliar service	Changes code at random until the test passes	Forms a hypothesis and checks it	Narrows it systematically, explains the cause, adds a test
Code clarity	Extend the same service with one feature	Works, but a teammate couldn't follow it	Readable, follows the existing style	Readable, and names the trade-off they made
Communication	Explain the fix to a non-specialist	Jargon, no clear cause	Clear cause and effect	Clear, and checks the listener understood
Taking feedback	Interviewer suggests another approach	Defends or ignores it	Considers it on its merits	Considers it, and improves on it

Then protect the scores. Each interviewer submits their scores and written evidence before the debrief, without seeing anyone else's. The debrief starts with the dimensions where scores differ by two or more points, not with whoever speaks first. That stops the most senior voice setting the outcome.

The test: take your last five debriefs. How often did the final decision match the first opinion voiced in the room? If it's nearly always, you're measuring that person's view, not the candidate.

Take out the stress the job doesn't have

Behroozi and colleagues ran a randomised trial with 48 computer science students. Half solved a problem on a whiteboard while thinking aloud in front of an interviewer. Half solved the same kind of problem privately. Being watched cut performance by more than half, and stress and cognitive load were significantly higher in the observed condition. The authors argue the traditional format may select for people who handle being examined, not for problem-solving ability. They note the sample was small and students, so treat it as a strong warning rather than a precise effect size.

Unless the job involves solving problems while a stranger watches, that pressure is noise. Ways to remove it without losing signal:

• Give private time first. Ten minutes to read the code or problem alone, then talk it through.

• Use the candidate's own setup. Their laptop and editor, not a whiteboard or an unfamiliar tool.

• Ask for the reasoning afterwards. Let them solve it, then walk you through what they did and why. Behroozi's team suggests this retrospective think-aloud as an alternative.

• Tell them the format in advance, including what you're scoring.

The exception: roles where real-time pressure is genuinely part of the work, such as incident response. Even then, simulate the real thing (a realistic alert and dashboards) rather than an algorithm puzzle under observation.

The test: list what's stressful about each round. For each item, ask whether the job has it. If not, remove it.

Use work samples, and size them honestly

The closer the exercise is to the work, the more it tells you. Developers' own complaints point the same way: a study of Hacker News discussions about technical interviews reported concerns including lack of real-world relevance, bias towards younger developers, and the time commitment demanded.

Format	What it shows	Cost to the candidate	Main risk
Debug or extend a small real service, in the interview	Reading unfamiliar code, reasoning, clarity	60 to 90 minutes, inside the interview	Needs a prepared, sanitised codebase
Review a pull request	Judgement, communication, standards	45 to 60 minutes	Rewards reviewers who've seen your stack
Short take-home with a time box	Code quality without an audience	Up to the time box you set	People with more free time spend more
Algorithm puzzle on a whiteboard	Recall of patterns, composure when watched	45 to 60 minutes, plus weeks of preparation	Measures preparation and stress tolerance

If you use a take-home, state the time box and score only what fits in it. A candidate who spends a weekend will produce better work than one who spent the two hours you asked for, and you'll end up rewarding free time, not skill. If the exercise takes more than a couple of hours, pay for it.

The test: for each exercise, could someone on the team do it on a normal working day without preparation? If they'd need to revise first, it measures revision.

Candidates: find out what each round measures

You can't change a company's process, but you can read it. Before the interviews, ask the recruiter or hiring manager:

1. What does each round assess, and how is it scored?

2. What format is the technical round: whiteboard, shared editor, my own machine, take-home?

3. How long should a take-home take, and is it paid?

A company running structured interviews can answer these in a sentence each. If the answer is "we just want to see how you think", the rounds are probably unstructured, and the outcome will depend more on rapport and confidence than on any rubric.

Then prepare for the format you'll actually face:

• Talking while solving. Practise narrating your reasoning out loud: what you know, what you're assuming, what you'll try first. If you're given private time, use it to read, not to rush.

• Common problem patterns, if the process does use algorithm rounds: two pointers, sliding window, breadth- and depth-first search, hash maps for lookup, dynamic programming. Recognising the pattern beats memorising solutions.

• Four or five stories for behavioural questions: a hard technical problem, a disagreement, something you led, a mistake, and a decision you'd make differently. For each, know the situation, what you did, and what happened.

• Questions that test them. "How does the team settle technical disagreements?" and "What happened to the last person in this role?" tell you more than "What's the culture like?"

The test: after the process, could you say what each round was assessing? If you couldn't, the company probably couldn't either.

When the process itself is the warning

A process tells you how a company treats people's time and judgement. Some signals are worth weighing heavily: a take-home with no time limit, rounds where nobody can explain the criteria, interviewers who are hostile or distracted, and weeks of silence between stages.

You can withdraw. "Thank you for your time. I've decided not to continue" is enough. If a company is careless with you while it's trying to impress you, expect the same after you join.

The test: would you be comfortable if your next manager treated your time the way this process does?

A worked example

Here is the composite team from the opening, over one hiring round each side of a change. The figures are illustrative.

Before: four rounds, no written criteria
  Debriefs where the decision matched the first view voiced:  9 of 10
  Candidates who withdrew at the take-home ("a weekend"):     4 of 11
  Technical round: algorithm problem on a whiteboard, observed

Changes
  Scorecard written per round before the first candidate
  Scores and evidence submitted before the debrief, unseen
  Debrief opens with dimensions 2+ points apart
  Whiteboard replaced: 10 minutes reading alone, then debug
    and extend a small service on the candidate's own laptop
  Take-home dropped; the exercise runs inside the interview

After: next hiring round
  Candidates who withdrew mid-process:                        1 of 9
  Debriefs that changed an interviewer's score after
    discussing evidence:                                      3 of 9
  Every rejection had a written reason tied to a dimension

The team didn't find a perfect predictor, and nothing here proves the new hires will do better. What changed is that the decision rests on written evidence about the job, gathered the same way for every candidate, and the format no longer filters out people who freeze when watched.

What to do this week

If you interview: write a scorecard for one round you run, with what you're assessing and what weak, adequate and strong look like. In your next debrief, ask everyone to submit scores before anyone speaks. List what's stressful about your technical round and remove one thing the job doesn't have. Hiring Engineers in an AI World covers which skills to weight now.

If you're a candidate: for your next process, ask what each round assesses and how it's scored, and what format the technical round uses. Practise one problem out loud, start to finish. Write down four stories, then ask the questions that tell you how the team works.

Final takeaway

Interviews are imperfect predictors, and the evidence says structure is what makes them better. Write down what each round measures before you meet anyone, score against it independently, and take out stress the job doesn't have, because being watched can halve performance. Use exercises close to the work, and size them honestly. As a candidate, ask what each round assesses, prepare for the format you'll face, and treat a process nobody can explain as information about the company.

Sources

• Schmidt, F. L. and Hunter, J. E., "The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings", Psychological Bulletin 124(2), 1998, pp. 262–274: the highest-validity combinations for job performance were general mental ability plus a work sample test (.63), plus an integrity test (.65), or plus a structured interview (.63). https://doi.org/10.1037/0033-2909.124.2.262

• Sackett, P. R., Zhang, C., Berry, C. M. and Lievens, F., "Revisiting Meta-Analytic Estimates of Validity in Personnel Selection: Addressing Systematic Overcorrection for Restriction of Range", Journal of Applied Psychology 107(11), 2022, pp. 2040–2068: validity estimates reduced by .10–.20; structured interviews emerged as the top-ranked selection procedure; procedures "remain useful" but relationships are "considerably lower than previously thought". https://doi.org/10.1037/apl0000994

• Behroozi, M., Shirolkar, S., Barik, T. and Parnin, C., "Does Stress Impact Technical Interview Performance?", ESEC/FSE 2020, pp. 481–492: randomised trial with 48 computer science students; performance reduced by more than half by being watched by an interviewer; higher stress and cognitive load in the traditional format. https://doi.org/10.1145/3368089.3409712 (summary: NC State University news)

• Behroozi, M., Parnin, C. and Barik, T., "Hiring is Broken: What Do Developers Say About Technical Interviews?", IEEE VL/HCC 2019: Hacker News posters reported lack of real-world relevance, bias towards younger developers, demanding time commitment, and unnecessary anxiety. https://doi.org/10.1109/VLHCC.2019.8818836

---

Related reading

• Hiring Engineers in an AI World

• The First 30 Days for a New Engineer — Mentoring Engineers, Part 1

• Code Review That Teaches

• Building Engineering Culture, Not Just Process