Matching People to Jobs with Embeddings, and Knowing When Not to Trust the Model

Keyword job search has two failure modes that pull in opposite directions. It misses obviously-relevant candidates who happened to use different words, "RN" instead of "React Native," "ML" instead of "machine learning," and at the same time it rewards keyword stuffing, ranking the person who pasted the job description into their profile above the person who can actually do the work. Embeddings fix the first problem beautifully, by matching on meaning rather than exact tokens. And then they introduce a brand-new problem: they will confidently surface a beach-resort job for a backend "Java" developer, because in vector space those two things happened to drift close together and the model has no idea that pairing is absurd.

A hybrid pipeline

The design that works uses each technique only for what it is genuinely good at. Embeddings, with jobs and profiles encoded into one shared vector space and retrieved by approximate nearest-neighbour search, give you recall: a broad, semantically relevant candidate set, fast, over millions of records. Hard filters then enforce the non-negotiables a model must never be trusted to honour: location, work authorization, a seniority floor, a salary band. These are business rules, and business rules belong in code with deterministic behaviour, not in a model's mood on a given day. Finally an LLM re-ranking pass applies nuanced judgment to the filtered shortlist and produces a short explanation for each match, so a recruiter, and the candidate, can see why something surfaced rather than trusting a black box.

The ordering is deliberate. Embeddings are cheap per item and run over everything; the LLM is expensive and runs only over the shortlist the first two stages produced. Get that order wrong, by sending millions of records through the LLM, and you have built something correct and completely unaffordable.

Index time versus serve time

The subtle operational trap is that a query and the corpus must be embedded by the same model version, or their vectors live in different spaces and the similarity scores are meaningless. So changing the embedding model is not a deploy, it is a re-index of everything, done atomically with the switch. You version the model and the index together, and you never let a query embedded with version two hit an index built with version one, because nothing will error; it will just quietly return worse matches that nobody can explain.

Measuring whether it actually works

"The demo looked good" is not a metric, and matching quality is genuinely hard to eyeball, so you instrument it. You track whether surfaced matches convert into applications, interviews, and hires, and you feed that signal back into ranking and threshold tuning. Without a feedback loop you are flying blind, optimizing a number you cannot see, and you will not notice when a model change or a data shift quietly degrades the results for a whole category of candidate.

Where it breaks in production

The demo always works, because the demo uses clean inputs and obvious matches. Production is where you meet the failure modes the vector space hides. Spam profiles stuff every trending keyword and end up embedding suspiciously close to everything, so you need quality and authenticity signals that sit outside the embedding, not inside it. Very short inputs, a one-line job post or a three-bullet profile, embed into mush and match almost anything, which is the cold-start problem wearing a different hat. Highly similar roles that differ on one critical axis, a senior versus a junior version of the same title, sit almost on top of each other in vector space even though they are not interchangeable, which is exactly why the seniority filter lives in code and not in the similarity score. And regional phrasing drifts the vectors in ways that can quietly disadvantage whole groups of candidates, which is both a quality bug and a fairness one. None of these show up until real data hits the system, so you watch for them deliberately rather than assuming the model has them handled.

The cost model in practice

It is worth being concrete about why the pipeline is staged the way it is, because the economics drive the architecture. Embedding is cheap and amortized: you do it once at index time and reuse it across every query, so running it over the whole corpus is fine. Nearest-neighbour retrieval is cheap per query and scales well, so it can run on every search. The LLM re-rank is the expensive step by an order of magnitude or more, in both latency and money, so it runs only over the few dozen candidates the cheap stages already narrowed to, and never over the raw corpus. You cache its judgments where the same job and candidate pairing recurs, and you cap how many candidates reach it per request. Get this funnel backwards and the system is technically correct and financially impossible, a failure mode that does not appear in a prototype and appears immediately on the invoice.

Start simple, earn the complexity

None of this means you build the full pipeline on day one. A sensible path is to start with embeddings plus the hard filters, ship that, and measure it, because that alone already beats keyword search comfortably and it is cheap to run. You add the LLM re-rank when you have evidence that the ordering of the shortlist is costing you, and you add the feedback loop and the bias auditing as the stakes and the volume grow. The architecture here is the destination, not the starting line, and treating it as something you grow into rather than something you front-load is how you avoid building an expensive, elaborate matching system before you have learned what good matching even means for your particular users.

The constraint that matters most

A matching model trained or tuned on historical hiring data can quietly learn and then amplify the biases in that data, which in a hiring context is not just a quality problem but a legal and ethical one. So you keep protected attributes out of the features, you audit outcomes for disparate impact across groups rather than assuming the model is neutral because nobody told it to discriminate, and you lean on the per-match explanations to keep the whole thing accountable and inspectable. Put it together, embeddings for recall, code for the hard constraints, an LLM for judgment, and humans watching the metrics, and you get a system that ranks the right people without confidently recommending nonsense, and without encoding yesterday's bias into tomorrow's shortlist.