Building Haira: multi-agent resume screening with FastAPI and pgvector

Why a single prompt call was not enough

Haira automates candidate screening: parsing resumes, scoring them against a job spec, and running a live conversational technical interview. The first version was a single large prompt per resume — it worked on clean examples and fell apart on messy real-world resumes (inconsistent formatting, missing sections, non-English content). The fix was not a better prompt, it was breaking the flow into discrete, independently-verifiable steps.

Structured extraction with function calling, not free text

The parsing stage uses LLM tool/function calling with a strict JSON schema for the extracted profile (skills, years of experience per skill, education, employment gaps). Forcing schema-conformant output rather than free-text parsing eliminated most of the silent failures we saw in the early version — malformed output now fails schema validation immediately and gets retried, instead of quietly producing a wrong-shaped result downstream.

Semantic matching with pgvector

Once a resume is structured, matching it to a job spec is a hybrid of keyword filtering (must-have certifications, location, work authorization) and semantic similarity on the free-text experience descriptions. We store OpenAI embeddings for both resumes and job specs in Postgres via pgvector and run cosine similarity search alongside the keyword filters. Pure keyword matching missed strong candidates who described the same skills in different language than the job post; pure semantic matching over-ranked resumes that were superficially similar but missed hard requirements. Combining both was the actual fix.

Making the interview stage feel real-time

The conversational interview stage needed to feel like a live conversation, not a slow request/response loop. We run the interview orchestrator as an async FastAPI service backed by Celery for longer evaluation tasks, and stream model output to the Next.js client over Server-Sent Events so tokens appear as they are generated rather than after the full response completes. Session state (conversation history, current evaluation criteria) is kept in Redis so a dropped connection can resume without losing context.

The trade-off that mattered most

The biggest lesson was resisting the urge to let the model freelance. Every stage that touched a decision (pass/fail a resume, flag a red flag, score an answer) uses constrained, schema-validated output. Every stage that is genuinely conversational (the interview dialogue itself) is left open-ended. Mixing those up in either direction — constraining the conversation or freeing the scoring — is where most of the reliability problems came from during development.