How Your Page Gets Retrieved

Here is the uncomfortable part. When an AI assistant answers a question and cites a source, no model read your page, not the way a person does, not top to bottom. Your page was pulled apart into passages, each converted into a list of numbers and stored in an index. The question became numbers too, and the system went looking for the closest match.

The unit that competed was never your page. It was a fragment of it, maybe three hundred words long, sitting alone in a database with no title, no navigation, and no memory of the paragraph above it. Teams still write for the page, measure at the page level, and audit at the page level, meanwhile the thing being judged is a passage they never saw in isolation.

This piece walks the whole pipeline, because the shape of the machine determines the shape of the content that wins inside it. Once you can see where passages get cut, where meaning leaks out, and where the ranking actually happens, a lot of vague advice about "writing for AI" collapses into something specific, the mechanical layer beneath how RAG actually works.

01Why does your page get cut up at all?

Feed it one tight paragraph about pricing tiers and the vector strongly encodes pricing tiers. Feed it your entire product page and the vector encodes something like "general B2B software marketing page", with the pricing detail smeared into near-invisibility. That is not a bug, it is the arithmetic of compression: specific numbers, named entities and precise claims lose signal strength relative to the dominant topic of the document. Chunking is the fix, split the document into smaller passages before encoding and each passage gets its own full budget.

02What are the four ways your page gets cut?

03What is context rupture, and why does good content fail?

Now audit your own writing: pronouns carrying the subject across paragraphs, section three referring to "this approach" from section two, a case study where the client name appears once in the intro and never again, comparison tables where product names live in the header row and the rows say "it" and "the platform". All of that is fine for a human reader. All of it produces orphaned chunks, the same failure that makes AI misdescribe a brand it can't cleanly resolve.

04How do retrieval systems patch the problem?

Fix one: generative contextual augmentation

Before embedding a chunk, send it to a language model with the full parent document and ask for a short prefix explaining what the chunk is about, then glue that prefix on and index the combined text. A chunk reading "Operating margins expanded by 240 basis points" becomes "This chunk is from the Q2 2023 financial statement for ACME Corporation... Operating margins expanded by 240 basis points". The rewritten passage resolves the pronouns, names the entity, anchors the timeframe, and the explicit terms become searchable in the keyword index too. This is Anthropic's contextual retrieval.

<document>
{{WHOLE_DOCUMENT_TEXT}}
</document>

<chunk>
{{TARGET_CHUNK_TEXT}}
</chunk>

Please give a succinct context (50-100 tokens) to situate this
chunk within the overall document, to improve search retrieval.
Answer only with the contextual prefix and nothing else.

The measured gains are not marginal. On standard evaluation sets, contextual prefixes on dense embeddings cut top-20 retrieval failure by 35% against a naive pipeline; pair them with contextual keyword search and failures drop 49%; add a reranking stage and the reduction reaches 67%. The whole funnel runs once per sub-query produced by query fan-out.

Read those numbers as a diagnosis, not a benchmark: two thirds of retrieval failures in a naive system are caused by problems that have nothing to do with content quality, context loss, vocabulary mismatch, and bad ranking. All three have content-side counterparts. And it is cheap: key-value prompt caching gives cache reads a ~90% token discount, so ingestion lands near $1 per million document tokens, which means you should assume the systems reading your content already do this and are already compensating for some of your ambiguity. Some, not all.

Fix two: late chunking

The more elegant approach reverses the order of operations. Traditional pipelines split first and encode second, so attention only ever sees one isolated chunk. Late chunking feeds the entire document through a long-context encoder in one pass, so every token vector already encodes global context, and only then applies chunk boundaries, pooling across the token slice. It preserves cross-chunk dependencies and document-level context without changing the vector's dimensions, and adds no query-time cost.

05What happens inside the vector space?

# What dense retrieval actually computes, stripped to essentials
import numpy as np

def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

query  = embed("how long does onboarding take")
chunks = [embed(c) for c in page_chunks]

scores  = [cosine(query, c) for c in chunks]
ranking = np.argsort(scores)[::-1]

# Only the top handful ever reach the model.
top_k = [page_chunks[i] for i in ranking[:5]]

Two refinements matter for content. Matryoshka embeddings nest meaning hierarchically, coarse topic in the leading dimensions, fine detail later, which enables a cheap first pass on a 64-dimensional prefix before rescoring the shortlist on full vectors. The blunt consequence: the first filter your chunk faces is deliberately coarse, judging on broad topical signal alone, so passages whose topic is unmistakable in the opening lines pass it. And multi-vector models like ColBERT keep token-level vectors and score with MaxSim, which rewards passages containing the literal terminology of the question, not just its general vibe. The exact words still matter.

06What funnel does your chunk have to survive?

Dense vectors are excellent at intent and synonyms and weak at exact lookups (part numbers, SKUs, proper nouns), so BM25 keyword search runs alongside. BM25's term-frequency component saturates, so the tenth mention of a keyword adds almost nothing and dilutes your passage on the vector side, there is no version of this system where stuffing works. The two score scales are incompatible, so Reciprocal Rank Fusion throws away raw scores and operates on rank positions, commonly weighting dense 80% and keyword 20%. The passage that wins is rarely the best on either axis, it is the one that is good on both.

# Reciprocal Rank Fusion, the whole idea in a few lines
K = 60

def rrf(rankings, weights=None):
    scores = {}
    for i, ranked_ids in enumerate(rankings):
        w = (weights or [1] * len(rankings))[i]
        for rank, doc_id in enumerate(ranked_ids, start=1):
            scores[doc_id] = scores.get(doc_id, 0) + w / (K + rank)
    return sorted(scores, key=scores.get, reverse=True)

# Dense search carries more weight than keyword search
final = rrf([dense_results, bm25_results], weights=[0.8, 0.2])

The reranker is where most outcomes are decided. Dense retrieval uses a bi-encoder that embeds query and passage separately (what makes pre-indexing possible); a cross-encoder runs both through one transformer together, letting every query word attend to every passage word. Far more accurate, far too expensive to run over the whole index, so it runs as a second pass over 100 to 150 candidates and narrows them to the 5 to 20 that build the final context.

07What are the three failure modes, and their fixes?

Clever renaming of a known category is a lexical mismatch you are inflicting on yourself. And cross-encoders reward passages that answer the query head-on: a passage that opens with three sentences of throat-clearing loses to one that opens with the answer, the same shape as any high-citation page.

08How do you write content that survives the cut?

Front-loading the answer helps at three separate stages at once, coarse first-pass filtering, cross-encoder scoring, and the moment a model decides which passage to quote, which is exactly what the answer-block optimizer checks. Naming the subject in every section directly reduces fragmentation, the highest-leverage change a content team can make, and it is the same explicitness that makes you resolvable as an entity.

09How do you tell whether any of this is working?

You will not run RAGAS on someone else's index, but a manual equivalent is genuinely useful: build 30 to 50 questions your buyers actually ask in their words; run each through the assistants that matter and record whether you appear and which passage got quoted; for the misses, read the passage on your site that should have answered it in isolation, with no title and no surrounding text. In most cases the reason is visible in ten seconds, the passage does not name its subject, does not state its qualifier, or buries the answer four sentences deep. Fix the passage, not the page, then re-run the same question set in four to six weeks. Different assistants weight these stages differently, which is part of why engines recommend different vendors.

10What's the takeaway?

That is not a hack, and it will not stop working when the architectures change. It is simply what it looks like to write for a reader who arrives in the middle, with no context, and only a few seconds of attention to spend, which, as it turns out, describes most human readers too.