A genuinely busy few weeks for structured-data research: a viral duck, a controlled study with thousands of evaluations, a twelve-chatbot rendering test, and a field report on broken audit tools. Four methods, four angles. You’d expect that much evidence to settle something. Mostly, it sharpened the picture of how much we still don’t know, while the confidence surrounding some of the interpretations often outpaced the evidence itself.
Every time a new AI capability ships, the explanations arrive faster than the evidence. Within a week, there’s a confident post, a tidy diagram, and a screenshot of a chatbot returning a fact under the caption “proof that LLMs read your schema.” None of that is malicious; most of it is people trying, in good faith, to give clients an answer in a space that changes monthly. But the gap between how sure the slides sound and how sure the data lets us be is wide, and this recent wave of research is a good chance to sit honestly in that gap.
Four pieces landed close together. Read fairly, and they each deserve to be read fairly, yet they don’t crown a winner. They map the edges of the problem well and leave the center roughly as dark as it was. Let’s walk through what each one actually shows, where the competing camps disagree, and what, if anything, we can now say out loud.
First, The Distinction Worth Keeping Straight
Before anyone argues about whether “schema works,” it helps to agree on what the word even means — because three different things routinely travel under one label.
| Term | What it actually is | Example |
|---|---|---|
| Structured data | The umbrella category: any data in a machine-readable, standardized format | A JSON-LD block, an HTML <table>, RDFa attributes, an XML sitemap, a CSV |
| Schema.org | A shared vocabulary — the agreed meaning of types and properties, maintained by Google, Microsoft, Yahoo & Yandex since 2011 | @type: Organization, streetAddress, telephoneNumber |
| JSON-LD (also Microdata, RDFa) | A syntax — one of several ways to serialize a vocabulary onto a page | <script type=”application/ld+json”>{ … }</script> |
These are three different axes, not three names for one thing. You can write Schema.org vocabulary in JSON-LD, Microdata, or RDFa; you can write valid JSON-LD using a vocabulary that has nothing to do with Schema.org; you can have structured data that uses none of them. “Add schema” usually means “add JSON-LD using the Schema.org vocabulary” — a reasonable shorthand, as long as everyone remembers it’s shorthand. A surprising number of disagreements in this space dissolve once you notice the two people are arguing about different layers. For readability, I’ll use “schema” loosely below and name the precise layer wherever it carries the argument.
Four Recent Studies
Exhibit A — The Duck. In this Search Engine Journal article, Mark Williams-Cook described how he built a fake T-shirt company, DUCK YEA, and put a deliberately broken JSON-LD block in the page head: invalid @context, a made-up @type of MallardEnterprise, and not one real Schema.org property among flockName, waddleStyle, and quackVolume. He then asked ChatGPT and Perplexity for the company’s address, and both returned it with Perplexity attributing the answer to the page’s “embedded structured data.”
Exhibit B — The Paper. Volpini et al. at WordLift ran a controlled experiment — 2,439 valid evaluations across 349 queries, four domains, seven conditions — testing whether structured data improves Retrieval-Augmented Generation. It’s a structured-data company publishing numbers on the limits of structured data, which earns it a careful read.
Exhibit C — The Decoy. Andre Alpar handed twelve AI assistants a page rigged like a polygraph: the “reference number” visible in the raw HTML was a decoy, while the real number existed only after a line of JavaScript fetched it from a second endpoint — nowhere present in the HTML or the script’s source. A unique canary string flagged any bluffing, and the server logs recorded who actually ran the code.
Exhibit D — My Client Response. My own earlier article on why some “agentic SEO auditors” report healthy schema as “missing” — the testing tool’s own browsing layer strips or never reaches the code, and the absence gets blamed on the website rather than the tool.
Four methods, four authors, four motives. Here’s where they seem to agree, where they pull apart, and where they all go quiet.
Where The Evidence Seems To Lean
Treat this as the weight of current evidence, not a verdict. Each finding below has a caveat in the next section, and the caveats matter.
Visible text does most of the work — across very different setups
The clearest common thread is that these systems rely heavily on text that a reader can actually see and treat hidden or wrapped data as far less special than their proponents hope. The duck’s address appeared because it was readable text that happened to be inside a <script> tag. The WordLift study’s biggest gains came not from adding schema but from what the authors call link materialization — resolving data that lived behind opaque knowledge-graph URIs and rendering it as plain prose on the page. Their enhanced entity pages produced +29.6% accuracy in standard RAG and +29.8% in the full agentic pipeline (Cohen’s d = 0.60 and 0.61). And the domain breakdown is the honest, telling part:
| Domain | Was the baseline already fact-rich in visible HTML? | Gain from enhanced pages (C6+ vs C1) |
|---|---|---|
| BlackBriar (e-commerce, n=137) | Yes — price, name, availability already rendered | +0.07 |
| Express Legal (legal, n=111) | Partly | +1.50 |
| SalzburgerLand (travel, n=79) | No — geo, hours, “contained in” lived in the graph | +2.47 |
| WordLift Blog (editorial, n=22) | No — “about” / “related” relationships lived in the graph | +2.73 |
Where the facts were already visible, the structured treatment added almost nothing; where they were hidden in the graph, surfacing them as text helped a great deal (factual queries climbed from 2.74/5 to 4.57). Alpar’s decoy points the same way from the opposite end: the real answer was never written as readable text, and the major Western assistants — reading raw HTML — simply reported the decoy. Three very different experiments, one recurring pattern: surface knowledge as visible text, and you tend to win; bury it and you tend to lose.
Structured data’s most defensible job is disambiguation — strongest for thin-footprint entities
There’s a reasonable reading, advanced by Williams-Cook and supported in direction by the WordLift domains, that schema’s real value isn’t “citations” but disambiguation — helping a system resolve which entity you mean. That would predict exactly the WordLift pattern (gains largest where the entity was least resolved) and exactly the Ahrefs result: 1,885 pages newly adding JSON-LD against ~4,000 controls showed no citation lift — but, as Gianluca Fiorelli noted, those pages already carried 100+ AI Overview citations before treatment. Testing disambiguation on already-resolved entities can’t show much. The untested, genuinely interesting case is new and challenger brands — and nobody has run it cleanly.
There are validation paths that sidestep the HTML pipeline entirely
One practical area of agreement: you don’t have to win the HTML-parsing argument to prove your data is live. Dereferenceable URIs with content negotiation, a /.well-known/mcp.json discovery file, a working /mcp endpoint, an NLWeb /ask query that returns a coherent answer — these confirm a structured-data setup end to end without depending on any chatbot reading a <script> tag. That’s a tool, not a theory.
Where Each Study Stops Short
Before discussing what these studies did not answer, it is worth recognizing what they contributed. Williams-Cook challenged a common assumption about how AI systems interpret structured data. Volpini and the WordLift team published one of the largest controlled studies we have seen in this space. Alpar developed an elegant methodology for separating retrieval behavior from model explanations. Even my audit tool rant examples exposed weaknesses in how organizations currently evaluate AI readiness. None of these studies failed. They simply answered different questions than the industry hoped they would.
None of these results is as clean as its headline, and saying so isn’t cynicism — it’s reading the papers’ own limitations sections, which are admirably candid.
The duck rules out a bad inference, not the schema itself
The duck kills one specific, very popular claim: “an LLM returned a fact that’s in my schema, therefore it parses my schema.“ It can’t survive a nonsense schema producing the right answer. But it does not prove that these systems ignore the schema. A system that consulted a valid schema and fell back to text extraction would produce the identical result on the duck page, because the address was also readable text. So the duck lowers our confidence in the loudest pro-schema “proof” without raising our confidence that the schema is inert. It narrows the argument; it doesn’t end it.
The WordLift numbers carry three confounds that the authors flag themselves
The +29.6% is real in their setup, and they’re scrupulous about why it might not mean what a carousel will say it means:
- The null on JSON-LD may be an artifact of their pipeline. Their headline “JSON-LD alone barely helps” result (d = 0.18) sits next to an admission that their embedding step truncates documents at ~20k characters, that 82% of HTML and 88% of JSON-LD documents exceeded that limit, and that the JSON-LD block began at a median character position of 18,510 — right at the cliff. In most documents, the schema was cut before it was embedded. So the honest reading isn’t “schema doesn’t help RAG”; it’s “schema this pipeline deleted didn’t help this pipeline.”
- Circularity. Ground-truth answers were derived from the same knowledge-graph data used to build the enhanced pages, and the same model family generated queries, answers, and judgments. Conditions presenting that data more directly may score higher, partly because the grader rewards proximity to it.
- Domain weighting. The aggregate +1.10 blends near-zero gain in the domain that’s 39% of queries with very large gains in one that’s 6% (n=22). The authors explicitly ask readers to use the per-domain table, not the headline.
Alpar’s test is clean and deliberately narrow
His server-log method is the most bluff-proof of the four, and the finding is striking: on the live, paste-a-URL path, the entire US top tier read only raw HTML.
| Assistant | Origin | Ran the JavaScript? | Reported |
|---|---|---|---|
| ChatGPT, Claude, Gemini, Perplexity, Meta AI | US | No | Decoy (raw HTML) |
| Microsoft Copilot | US | Downloaded the script (via Diffbot) but didn’t execute it | Decoy |
| Grok | US | Executed on one node, then ignored the result | Decoy |
| DeepSeek, ERNIE, Qwen, Kimi | China | Yes | Real number |
| Mistral | Europe | Yes | Real number |
But he draws the boundary himself, and it’s a real one: this is a single-pass, user-triggered live fetch, not the background indexing crawlers (GPTBot, ClaudeBot, and friends) that build training and search corpora, and those “may well behave differently.” It’s also a snapshot in time on one site. So it tells you a great deal about what happens when a user pastes your link today, and very little about what the indexing pipelines did with your page last month. Two questions, one answered, one open.
There’s a genuinely uncomfortable secondary finding worth carrying forward, though: self-reports don’t match logs. Perplexity told the user it “couldn’t access” a page its own crawler had fetched with an HTTP 200, and Grok ran the JavaScript and then answered from the raw HTML anyway.
This is not a new phenomenon. As Simon Willison has noted in his analysis of Claude and Brave Search, the visible behavior of AI systems and the retrieval mechanisms operating underneath them are often difficult to infer from the model’s own explanations. The explanation is generated text; it is not telemetry.
A model’s narration of its own behavior is generated text, not telemetry, which is also why I wouldn’t treat any chatbot’s account of its own retrieval backend as evidence, mine included.
The audit-tool problem is real, but it’s a tooling fault, not proof of anything about schema
The field-report article is about bad measurement, not bad websites: an LLM-driven auditor that hits proxy defenses, strips scripts, or chokes on non-HTML can report your healthy setup as broken. Useful to know when a vendor PDF lands on your desk. But it’s a caution about instruments, and it shouldn’t be stretched into a claim about what schema does or doesn’t do for AI — that would be the same overreach, pointed the other way. (For scale on why the crawlers are touchy in the first place: TollBit and Cloudflare put AI bot traffic up ~300% year on year, around 1 in 31 site visits.)
The Hole In The Middle: The Question None Of Them Answered
Here’s what should keep everyone honest. Stack all four, and notice what they have in common: each one measures, or describes, the flat-text and live-fetch world, and every one of them merely asserts the dedicated-parser world.
Both the WordLift paper (its “Two Worlds of AI Search” section) and my own field report lean on the same dichotomy: a flat-text RAG pipeline that treats JSON-LD as more text, versus a crawl-time parser (Google, Bing) that extracts <script type=”application/ld+json”> as a separate signal and feeds it into entity understanding and a knowledge graph. Multiple authors independently landing on that split is a decent sign that it’s the right shape. But look at what’s tested versus claimed: the paper tests its Vertex pipeline and discusses how Google may process structured data; my article tests chatbots and discusses the indexing layer; Alpar measures the live path and explicitly excludes the background crawlers; Williams-Cook is skeptical of the whole chain. Nobody behaviourally demonstrated the schema → knowledge-graph → citation pipeline that the entire optimization pitch rests on. We have strong evidence about the edges and essentially none about the center.
And the competing camps remain live, not resolved. The structured-data optimists can fairly say: materialized knowledge demonstrably helps, dereferenceable URIs enable traversal, invest accordingly. The skeptics can fairly say: the citation lever is unproven, and most “proof” is confounded. The flat-text realists can fairly say: on the paths users actually hit, it’s visible HTML or nothing. All three are partly right, and none of them has closed the case. That’s not a cop-out but just where the evidence honestly sits.
A recent comment from Microsoft’s RV Guha, one of the creators of Schema.org, highlights an important distinction that often gets lost in these discussions. Guha noted that AI systems have been trained on millions of examples of Schema.org markup and therefore understand and benefit from its semantics. That observation is difficult to dispute. Google, Microsoft, and others have spent more than a decade encouraging the adoption of structured data, and Schema.org has become one of the most widely implemented machine-readable standards on the web.
Large-scale training datasets reinforce that likelihood. FineWeb, one of the largest openly documented web-scale corpora used for AI research, was built from 96 Common Crawl snapshots and contains roughly 15 trillion tokens. At that scale, it is difficult to imagine modern language models failing to develop an understanding of Schema.org concepts after being exposed to millions of examples across the web.
The open question is not whether AI systems can understand Schema.org concepts. It is whether, when interacting with a page today, they actively parse and use that markup as an independent signal in retrieval, citation, recommendation, or generation workflows. Those are related questions, but they are not the same question.
I completely agree that AI systems understand Schema.org semantics because they have been trained on millions of examples. My question is narrower: what evidence do we have that modern AI retrieval systems actively parse and use Schema.org properties from a page as independent signals at retrieval or generation time?
Industry case studies continue to suggest that structured data, entity modeling, and knowledge-graph work may influence AI visibility and representation. The challenge is that these projects rarely change a single variable, making it difficult to determine precisely which intervention produced the observed outcome.
That distinction sits at the center of the current debate. A model can understand what an Organization, Product, or LocalBusiness is because it has seen millions of examples during training. That does not necessarily mean it is reading and incorporating the JSON-LD block on your page when generating an answer today. Understanding the vocabulary and actively consuming the markup are related capabilities, but they are not equivalent.
So, What Did We Actually Learn?
Less than the timeline will tell you, and that’s the point worth making.
- The most certain thing we learned is how relentless the certainty is. Four careful, caveated studies produced weeks of uncaveated takes. The single most reliable pattern in this space remains the speed at which provisional findings become confident slides.
- Visible text appears to do most of the work — provisionally. Three independent setups point that way, which is more than we had. But each has a confound, and “appears to” is doing real work in that sentence.
- The headline RAG number is real and heavily qualified. +29.6% in one pipeline, with truncation, circularity, and domain-weighting caveats that the authors raise themselves. Cite it with its asterisks or don’t cite it.
- On the live fetch path, the major Western assistants read raw HTML. Strong, clean, and scoped — it says nothing certain about the background crawlers, which is exactly the layer the citation pitch depends on.
- The core question is still unmeasured. Schema → knowledge graph → citation was asserted by everyone and tested by no one. Until someone runs it, treat both the bullish and bearish versions as conjecture.
- The defensible practices haven’t changed, and they’re cheap. Put your key facts in the raw HTML, high on the page; server-side render anything you need to read; keep your Schema.org valid because the parser-based engines may well use it; expose dereferenceable URIs and endpoints you can verify directly. None of this requires believing any particular theory — it’s robust to being wrong about all of them.
- Trust logs over narration. When you want to know what a system did with your page, read your server logs, not the chatbot’s description of itself.
The honest summary is unsatisfying and probably correct: we know more about the edges of this problem than we did two weeks ago, almost nothing new about its center, and we’re roughly as far from a confident answer as when we started — only now with the uncertainty better mapped. The research is progressing slowly and carefully. The hype is not waiting for it.
These studies answered important questions. They improved our understanding of retrieval behavior, rendering limitations, disambiguation, and knowledge representation. What they did not answer is the question that sparked much of the debate in the first place: whether AI systems read and use Schema.org properties as independent signals, as many practitioners assume.
The research is doing its job slowly and carefully. The hype is not waiting for it.
