When Embeddings Lie About the Facts

Look at these two summaries of the same BBC article about a drilling site in Lancashire:

output (generated):
shale gas drilling in lancashire has been suspended after a magnitude-7.5 earthquake struck.

reference (human-written):
a fracking operation in lancashire has been suspended after a gas leak was found at the site.

The topic is the same. The location is the same. The suspension event is the same. A semantic similarity metric comparing the two embeddings scores them at 0.72 — above the typical 0.7 faithfulness threshold. Verdict: PASSED.

But the reason for the suspension is completely different. No earthquake struck Lancashire that day. The output invented a detail. For faithfulness evaluation, that invented detail is the entire story.

Semantic similarity measures whether two texts talk about roughly the same thing. Faithfulness requires knowing whether the specific facts are right. Those two questions are not the same question, and the fracking example makes the gap vivid.

I ran three approaches against the same dataset — whole-text semantic similarity, semantic similarity after atomic chunk decomposition, and LLM-as-a-judge grounding — all using nexa-gauge. The fracking case threads through all three, because it is a clean example of where each approach succeeds or fails.

TL;DR

The more precisely a metric targets factual support, the fewer unfaithful outputs slip through. Whole-text semantic similarity is fast but blind to single-fact swaps — the same topic with the wrong detail still passes. Atomic chunk alignment catches those swaps and gives you a structural diff in a single run: what matched, what the output fabricated, what the reference said that the output dropped. Grounding gives the best precision but needs two separate runs to get that same full picture, and its score tells you which claims failed — not how they relate to the reference structurally. If you need one number and maximum precision, run grounding both ways. If you need to explain failures to a team, atomic alignment gives you the clearest picture with less effort.

The Setup

The source data is mtc/cleaned_xsum-faith-test-set-with-faithfulness-annotation on Hugging Face. Each row is a generated XSum-style summary for a BBC article, labeled by human annotators as faithful or not. Several generated summaries can share the same bbcid, which means we can compare outputs for the same article rather than mixing unrelated stories.

From that data, I built summary pairs grouped by article. A negative pair takes a faithful summary as the reference and an unfaithful summary as the output — the metric should fail it. A positive pair takes two faithful summaries for the same article — the metric should pass it. That gives a clean evaluation target.

The prepared set has 198 unfaithful outputs and 19 faithful outputs, 217 pairs total. The imbalance is expected: most articles have at least one faithful and one unfaithful summary, but fewer have two faithful summaries to form a positive pair. The same JSONL file fed all three metric runs.

Method 1: Whole-Text Semantic Similarity

nexa-gauge's RefAlign computes embedding-based reference alignment. With atomic chunks disabled, it operates at the coarsest level: embed the whole output, embed the whole reference, take the cosine similarity. The question it answers is whether the two summaries point at the same general thing.

For the fracking case, that is exactly the wrong question. Earthquake and gas leak are both reasons a site shuts down. They embed similarly enough that the coarse score stays above threshold.

@register_transform("add_refalign_without_atomic_chunks")
def add_refalign_without_atomic_chunks(record: dict) -> dict:
    record["refalign"] = {
        "atomic_chunks": False,
    }
    return record

uv run nexagauge run refalign \
  --input ../data/data_full/pairs.jsonl \
  --output-dir ../data/results/without_atomic_chunks \
  --extension-file ../transform.py \
  --transform add_refalign_without_atomic_chunks \
  --max-workers 8 \
  --no-cache

43 unfaithful summaries looked similar enough to pass. That is the failure mode in plain numbers. Whole-text similarity is fast and cheap, but on a faithfulness task where the hallucination is a single fabricated detail inside an otherwise accurate summary, it has a structural blind spot.

Method 2: Semantic Similarity Over Atomic Chunks

The second approach keeps embeddings but changes what gets embedded. With atomic_chunks: true, nexa-gauge first asks an LLM to split both the output and the reference into smaller factual units. It then embeds those units and aligns them with cosine similarity using a one-to-one match — each output chunk and each reference chunk can be used at most once. The score is computed as a token-weighted F1 over matched pairs.

For the fracking case, the atomic decomposition produces something the whole-text comparison could not:

{
  "matched_pairs": [
    {
      "output_text": "shale gas drilling has been suspended",
      "reference_text": "a fracking operation was suspended",
      "similarity": 0.638
    },
    {
      "output_text": "in lancashire",
      "reference_text": "the site is in lancashire",
      "similarity": 0.799
    }
  ],
  "missed_reference_chunks": [
    {
      "reference_text": "a gas leak was found at the site",
      "best_similarity": 0.382
    }
  ],
  "extra_output_chunks": [
    {
      "output_text": "a magnitude-7.5 earthquake struck",
      "best_similarity": 0.177
    }
  ]
}

Two facts match. One reference fact (the gas leak) is unaccounted for. One output fact (the earthquake) has no counterpart. The atomic F1 score is 0.667, below the threshold — the case fails correctly. The whole-text score was 0.72 and passed. Same two summaries, very different verdicts.

@register_transform("add_refalign_with_atomic_chunks")
def add_refalign_with_atomic_chunks(record: dict) -> dict:
    record["refalign"] = {
        "atomic_chunks": True,
    }
    return record

# Start a local llama.cpp server first
llama-server \
  -m /path/to/qwen-model.gguf \
  --host 127.0.0.1 --port 8080 --ctx-size 8192

uv run nexagauge run refalign \
  --input ../data/data_full/pairs.jsonl \
  --host-model-url http://127.0.0.1:8080/v1 \
  --llm-model openai/qwen \
  --output-dir ../data/results/with_atomic_chunks_qwen \
  --extension-file ../transform.py \
  --transform add_refalign_with_atomic_chunks \
  --max-workers 8 --llm-concurrency 4 --no-cache

False positives drop from 43 to 23. That is a meaningful gain for faithfulness detection, where a false positive is the evaluator declaring "looks good" on a summary that is not. The alignment trace is also directly readable: you can see exactly which output claims landed and which ones did not.

Method 3: Grounding as LLM Judge

Grounding reframes the problem entirely. Instead of asking how similar two texts are, it asks whether specific claims are supported by a context document. The pipeline extracts atomic claims from the output, sends them to an LLM judge alongside the context, and asks the judge to mark each claim as supported or not. The metric score is the fraction of supported claims.

I ran it in two directions, because each direction catches a different failure mode.

Direction 1 — output claims against the reference. The reference is the context; claims are extracted from the generated output. This catches fabrications: things the output says that the reference does not support.

@register_transform("add_grounding_output_claims")
def add_grounding_output_claims(record: dict) -> dict:
    record["context"] = record["reference"]
    record["grounding"] = {"scoring_mode": "binary_yes_no", "include_reasoning": False}
    return record

Direction 2 — reference claims against the output. The generated output is the context; claims are extracted from the reference. This catches omissions: things the reference says that the output silently dropped.

@register_transform("add_grounding_reference_claims")
def add_grounding_reference_claims(record: dict) -> dict:
    record["context"] = record["output"]
    record["output"] = record["reference"]
    record["grounding"] = {"scoring_mode": "binary_yes_no", "include_reasoning": False}
    return record

Back to the fracking example. Direction 1 extracts two claims from the output and judges each:

{
  "name": "grounding", "score": 0.5, "verdict": "FAILED",
  "result": [
    {"item": {"text": "shale gas drilling in lancashire has been suspended."}, "verdict": "PASSED", "raw_score": 1},
    {"item": {"text": "a magnitude-7.5 earthquake struck."}, "verdict": "FAILED", "raw_score": 0}
  ]
}

The earthquake claim is rejected. Direction 2 then turns the tables and extracts claims from the reference to check whether the output covers them:

{
  "name": "grounding", "score": 0.5, "verdict": "FAILED",
  "result": [
    {"item": {"text": "A gas leak was found at a fracking site in Lancashire."}, "verdict": "FAILED", "raw_score": 0},
    {"item": {"text": "A fracking operation in Lancashire has been suspended."}, "verdict": "PASSED", "raw_score": 1}
  ]
}

The missing gas leak surfaces. One direction finds the fabrication; the other finds the omission. Together they give a more complete picture than either signal alone.

# Direction 1: output claims against reference
nexagauge run grounding \
  --input ../data/data_full/pairs.jsonl \
  --host-model-url http://127.0.0.1:8080/v1 \
  --llm-model openai/qwen \
  --output-dir ../data/results/with_grounding \
  --extension-file ../transform.py \
  --transform add_grounding_output_claims \
  --max-workers 8 --llm-concurrency 4 --no-cache

# Direction 2: reference claims against output
nexagauge run grounding \
  --input ../data/data_full/pairs.jsonl \
  --host-model-url http://127.0.0.1:8080/v1 \
  --llm-model openai/qwen \
  --output-dir ../data/results/with_grounding_rev \
  --extension-file ../transform.py \
  --transform add_grounding_reference_claims \
  --max-workers 8 --llm-concurrency 4 --no-cache

False positives drop to 9 and 11 respectively — the best result across all three methods, and by a noticeable margin.

Putting the Numbers Together

All runs used the same 217 pairs and a postprocessing threshold of 0.7.

The progression is not accidental. Each step changes what the metric measures. Whole-text similarity asks whether two passages are topically related. Atomic RefAlign asks whether the specific factual units align. Grounding asks whether individual claims are supported. The more precisely the question targets factual support, the fewer unfaithful outputs slip through.

What grounding gives up is structural context. The per-claim verdicts show which claims failed, but not how the output and reference relate to each other — whether a claim has no counterpart at all, or contradicts something specific. You also need two separate runs to see both directions: fabrications and omissions. RefAlign's alignment trace — matched pairs, missed reference chunks, extra output chunks — gives that structural picture in one shot, and it reads like a diff.

What This Means in Practice

These three metrics are not competing for the same job. They cover different ground.

Whole-text semantic similarity is the right starting point when you need a fast, cheap signal that the output is on-topic. It will not catch every factual swap, but it will catch cases where the model drifted into an entirely different subject.

Atomic RefAlign is the right call when you need a score you can explain. The alignment trace shows which output facts matched, which were unsupported, and which reference facts went missing. It costs an LLM call for decomposition but gives you something auditable.

Grounding is the right call when unsupported claims are the primary risk. Running both directions — output claims against reference and reference claims against output — catches fabrications and omissions independently. The cost is that the averaged score sometimes needs per-claim inspection to understand the failure.

The fracking example illustrates why no single number tells the whole story. Whole-text similarity saw two topically related summaries. Atomic RefAlign saw one extra fact and one missing fact. Grounding, in two directions, found the fabricated earthquake and the dropped gas leak. That is the difference between measuring whether two texts sound alike and measuring whether the output is faithful.

Resources

• Dataset: mtc/cleaned_xsum-faith-test-set-with-faithfulness-annotation
• Background paper: On Faithfulness and Factuality in Abstractive Summarization
• Semantic similarity reference: BERTScore
• nexa-gauge RefAlign docs
• nexa-gauge Grounding docs