The research evidence on interview validity is unusually clear for a domain that depends so heavily on human judgment. Across multiple decades of personnel selection research, the same finding replicated consistently: structured interviews predict job performance at roughly twice the rate of unstructured interviews. The reference point most practitioners cite is Schmidt and Hunter's 1998 meta-analysis, which synthesized data from hundreds of studies and assigned structured interviews a validity coefficient of 0.51 against job performance, compared to 0.20 for unstructured interviews. Subsequent meta-analyses have refined but not overturned that finding.
A validity coefficient of 0.51 means that structured interview scores correlate with subsequent job performance at a level that, while not deterministic, is meaningfully predictive. A coefficient of 0.20 means you are capturing roughly 4% of the variance in job performance — a level of predictive power that, in most other domains, would not justify the cost of the measurement instrument.
This article examines what drives that difference, where each approach operates well, and what the data does not tell you that practitioners often assume it does.
The Mechanism Behind the Validity Gap
The validity advantage of structured over unstructured interviews does not come from structured interviews being more intense or more rigorous in any general sense. It comes from two specific properties: consistency of measurement and separation of signal from noise.
Consistency of measurement means that every candidate answers the same behavioral questions and is evaluated against the same criteria. When questions vary across candidates, you are not comparing candidates on a common scale — you are comparing the impressions left by different conversations. The "comparison" happening in the debrief is not an apples-to-apples evaluation; it is a comparison of how comfortable each interviewer was during each respective conversation.
Separation of signal from noise means that a structured scorecard — with competency-level scores and behavioral anchors — separates the information from the evaluation. An unstructured interview collapses these: the interviewer's gut read is both the information and the evaluation, simultaneously. When you force interviewers to score specific competencies against specific behavioral anchors, you are asking them to extract the signal from their conversation rather than report on the overall feel of it. This mechanical separation is where a significant portion of the validity gain lives.
What Unstructured Interviews Do Well
We are not arguing that unstructured interviews serve no purpose. That would be inaccurate. There are specific contexts where an unstructured or lightly structured conversation produces information that a fully structured format does not capture well.
First, unstructured conversations are often better for bilateral evaluation — for the candidate assessing the role, team, and culture as much as the interviewer is assessing the candidate. A genuinely exploratory conversation can surface the candidate's real questions and concerns in a way that a structured behavioral sequence does not. Most hiring loops benefit from having at least one stage — typically a hiring manager conversation or a "culture and values" round — that gives the candidate space to lead.
Second, for roles where the primary competency is difficult to operationalize into behavioral anchors — certain types of creative or strategic roles where judgment is hard to decompose into observable behaviors — unstructured expert evaluation by a domain-fluent interviewer can carry more predictive weight than a rubric-driven process that does not capture the right signals. This is a genuine limitation of BARS methodology in domains where the job analysis itself is contested or the competency model is immature.
Third, unstructured conversations tend to produce better rapport and less candidate anxiety in certain populations — which matters because a candidate who is highly anxious will underperform against their actual capability, and the structured format does not correct for this (it may amplify it). Some organizations deliberately use an unstructured warm-up before the structured evaluation begins for exactly this reason.
Where Unstructured Interviews Fail Systematically
Despite these legitimate uses, the failure modes of unstructured interviews are severe enough to make them inappropriate as the primary selection mechanism for most hiring decisions. The validity coefficient of 0.20 is an average across many conditions; in conditions of high interviewer bias, low domain expertise, or rapid scaling (where interviewers are inconsistently trained and under time pressure), the effective validity can be substantially lower.
The patterns that drive the failure are well-documented. First-impression effects cause interviewers to form stable candidate assessments within the first few minutes of an interview — assessments that are resistant to subsequent revision even when the evidence in the remainder of the conversation contradicts them. Halo effects cause a positive impression on one dimension to inflate ratings on unrelated dimensions. Affinity bias causes interviewers to evaluate candidates who share their communication style, educational background, or professional vocabulary more favorably. None of these effects are visible to the interviewer experiencing them.
The debrief amplifies rather than corrects these biases. Unstructured debrief discussions — where interviewers share impressions without pre-submitted scorecards — consistently show anchoring effects where the first speaker's framing shapes subsequent evaluations. The result is not independent assessments synthesized through discussion; it is the first confident voice winning over subsequent, less confident ones.
A Comparison Across Two Loops for the Same Role
To make the difference concrete: consider a growing professional services firm that ran two back-to-back hiring loops for a mid-level operations analyst role — one unstructured, one structured with a BARS rubric — using different candidate cohorts but the same interviewers.
In the unstructured loop, three interviewers conducted free-form 45-minute conversations and submitted Slack notes before debrief. The debrief took 55 minutes. The eventual hire was a consensus choice — everyone liked the candidate personally, but when asked to point to specific evidence for their hire recommendation, two of the three interviewers struggled to name concrete behaviors. The hire performed adequately but not at the level the team had anticipated.
In the structured loop, the same three interviewers used a four-competency BARS rubric. Each submitted scorecards before the debrief. The debrief took 22 minutes. One strong candidate who had performed well on three of four competencies but showed a consistent gap on structured problem-solving was declined on the basis of anchored scorecard evidence — a decision that would likely not have been made in the unstructured format, where the candidate's strong interpersonal performance would have dominated the debrief impression. The hired candidate from this loop was rated higher-performing by the hiring manager at the six-month review.
This is a plausible synthetic comparison, not a controlled experiment. But it captures the directional difference that structured methodology typically produces.
What the Validity Coefficient Doesn't Tell You
The 0.51 figure is a population-level average. It does not mean that your structured interview program will produce that level of predictive validity if you implement it poorly. Validity depends on the quality of the competency model, the behavioral anchor quality, the consistency of interviewer application, and the rigor of the calibration process. A structured interview program run with a weak competency model and uncalibrated interviewers may produce better documentation than an unstructured process, but not meaningfully better hiring decisions.
Similarly, the comparison between 0.51 and 0.20 does not imply that structured interviews are good enough to be the sole selection instrument. The research consistently shows that combining multiple valid selection methods — work sample tests, structured interviews, and cognitive or skills assessments — produces higher predictive validity than any single method alone. The practical implication is that structured interviews are the floor of a valid hiring process, not the ceiling.
The data tells you where to invest. It tells you that the unstructured interview is not a reasonable baseline to accept. It tells you that consistency of measurement and behavioral anchoring are the mechanisms that produce validity gains. What it cannot tell you is how well your specific rubric, your specific competency model, and your specific interviewer team will perform against your specific role — that requires deliberate calibration and ongoing measurement of your own inter-rater reliability and hiring outcome data.
