Interview scorecards: how to make them actually change the decisionSkip to content

Interview scorecards: how to make them actually change the decision

Most interview scorecards are filled in after the debrief. They only work when the panel measures the same thing, agrees what a score means, and scores before talking.

Dhruv Sanjay Rao, Head of ProductUpdated 7 min read

Picture a debrief. Four interviewers on a call, the hiring manager speaks first: “I really liked her.” Half the scorecards haven’t been filled in yet. By the time they are, they all agree with the hiring manager.

Most teams would say they use scorecards. Very few use them in a way that changes a decision. The common belief is that good interviewers can simply tell, and the form is paperwork you do afterwards for the file. That belief is why the form ends up as paperwork.

An interview is a measurement, taken by several people. It’s only worth something if they measure the same thing, agree what a score means, and write it down before they hear each other. A scorecard is the tool for those three things. Miss any one of them and it’s a form, not a scorecard.

It’s worth getting right. Structured interviews, where every candidate gets the same questions judged against the same standard, are among the best predictors of job performance there are. A large 2022 re-analysis of hiring research ranked them first among common selection methods, well ahead of the free-flowing conversation most interviews turn into.

Decide what you’re measuring before the role goes live

The scorecard should exist before the first interview, built by the recruiter and the hiring manager together when the role opens. Start from the job, not from a list of nice-sounding traits: what does this person actually have to do in their first six months, and which skills does that take on day one?

That usually comes down to four or five things. More than that and, in a 45-minute round, some rows get scored on impression because there was no time to gather evidence for them. Each one gets a core question that every candidate is asked, and each round owns a few of them, so the hiring manager isn’t re-asking what the recruiter already covered.

Two things tend to sneak onto scorecards that shouldn’t. The first is “culture fit”, which in practice becomes a vote on whether the interviewer liked the candidate. If you mean “comfortable without a clear brief” or “gives direct feedback”, write that, because then it can be judged. The second is anything you can’t explain in a sentence. If the hiring manager can’t say why a skill matters for this job, it isn’t a must-have.

Then keep the scorecard fixed while the role is open. Change it after the third candidate and their scores can’t be compared with anyone who comes after.

A 3 should mean the same thing to everyone

“Rate ownership from 1 to 5” gives you five interviewers using five different scales. One person’s 3 is “fine”, another’s is “worrying”. The fix is to describe, for each point, what you’d actually hear.

For ownership, a 1 might be a candidate who blames circumstances and can’t name their own part in what went wrong. A 3 owns the mistake without being prompted and explains what they did to fix it. A 5 fixed the cause, not just the symptom, and changed something so it wouldn’t happen to anyone else. Describe the 2 and the 4 as well, or interviewers will drift into them with no shared idea of what they mean.

The quickest way to find out whether your descriptions work is to have the panel score the same practice interview separately and compare. Where two sensible people land two points apart, the description is usually the problem, not the interviewer. Rewrite it until they agree.

Evidence, not adjectives

Next to every score, the interviewer writes what the candidate said or did. “Traced a slow report to a missing index, ruled out two other causes by measuring first” can be checked, argued with and remembered three weeks later. “Sharp”, “good energy” and “solid” can’t. Nobody else knows what they’re based on, and a week later neither does the interviewer.

The evidence column is also what makes a score defensible. If a rejected candidate asks why, or a hiring manager wants to overrule the panel, the conversation is about what was observed, not whose opinion carries more weight.

Score alone, then talk

This is the part that matters most, and it’s the part most often skipped.

Once people hear each other’s views, their judgements move closer together without becoming more accurate. Researchers saw exactly this when they let groups see each other’s estimates: the answers converged, the accuracy didn’t improve, and people became more confident anyway. In a debrief, the first or most senior voice sets the direction, and everyone else’s notes quietly follow.

So each interviewer submits their scorecard before the debrief, ideally straight after the interview while the details are fresh. No verdicts in the WhatsApp group, no “how did it go?” in the corridor. Then the scores are revealed together, and the discussion goes to where they differ. Where everyone agrees, there’s nothing to discuss. Where they’re far apart, each person reads out their evidence. A score changes because of new evidence, not because the room disagrees.

Don’t let the average decide

A total out of 25 is tempting because it looks objective. It also hides exactly the thing you most need to see.

Take two candidates. One scores 5, 5, 5, 1, 3: brilliant on the technical rows, but the 1 is on ownership, because they blamed their whole team for a failed launch. The other scores 4, 4, 3, 4, 3. The first has the higher total. Most hiring managers, looking at the evidence, would hire the second.

That’s why the decision rules belong on the scorecard, agreed when the role opens and not during a debrief for a candidate everyone happens to like. Which competencies are must-pass? What happens when someone scores a 1? Is an integrity concern something you add up, or something you discuss before anything else? Use the total to compare candidates who have both cleared the bar, never to decide whether they have. And keep every row’s weight equal unless you can write down why one matters more. The US Office of Personnel Management calls equal weighting the “most effective and defensible” choice without a documented reason to do otherwise.

What the scorecard looks like

Once the thinking is done, the scorecard itself is short. One per interviewer, per candidate, per round:

Competency Core question Score (1–5) Evidence
Problem solving Walk me through a problem you solved that had no obvious answer.
The role’s core skill Specific to the job
Ownership Tell me about something that went wrong on your watch.
Working with others Tell me about a time you disagreed with a teammate. How was it settled?

Below it goes an overall recommendation, with no “maybe” option, and one sentence on the evidence that decided it. The lack of a middle option is deliberate: it makes the interviewer commit, and makes the sentence below it do the work.

For freshers and campus hiring, describe the scores in terms of college projects, internships and hackathons rather than work experience, or every candidate scores a 1. And if you record interviews to check your notes against, tell candidates beforehand, explain why and how long you keep it, and give them a way to say no. That’s general good practice, not legal advice.

Why it falls apart in spreadsheets and group chats

None of this is hard to understand. It’s hard to keep doing.

The scorecard lives in a Google Doc that gets copied for each candidate, and somebody edits the copy. Two interviewers fill it in, one sends their notes on WhatsApp, and the hiring manager gives a verdict on the call before anyone has submitted anything. Three weeks later, when the candidate you rejected reapplies or the one you hired isn’t working out, nobody can find what was actually said in round two. And you can never see the pattern that matters: that one interviewer scores everyone a point higher than the rest, or that a must-pass rule gets waived every time the role is urgent.

The process only works if it happens the same way every time: the same questions in front of every interviewer, scores captured before the debrief, and the evidence kept with the candidate rather than in someone’s inbox. That’s a job for a system, not for willpower in a busy hiring month.

How Arexis helps

In Arexis, interviewers get round-specific questions drawn from each candidate’s own evidence, and rounds held on Teams are transcribed and evaluated against that round’s rubric (or you can paste a transcript or upload audio from elsewhere), so the evidence doesn’t rely on someone’s memory of the call. Every round’s scorecard sits on the candidate’s profile with the interviewer’s notes alongside, so the panel compares evidence side by side instead of reconstructing it in a debrief. See how interviews work.

Found this useful? Pass it on.

See it on your own roles.

Screening, scheduling, interviews and proctored coding tests in one workspace. 30 days free, no card.