TIME Serves AI Crawlers a Separate Markdown Site With Ads
TIME's split delivery shows publishers are beginning to treat AI crawlers as a distinct audience with its own ad inventory, a shift that could reshape how web content is monetized.
Reporting from 1 source: GIGAZINE.
Software developer Vincent Schmalbach found that TIME delivers a simplified Markdown version of its site to some AI crawlers, including ClaudeBot and PerplexityBot, while human browsers get the full HTML page. The AI version includes ads from Mobian and sponsor content that do not appear on the human-facing site.
Software developer Vincent Schmalbach tested TIME's site by changing the User-Agent header. A normal browser received about 303KB of HTML, while ClaudeBot and PerplexityBot got about 13KB of Markdown with a content-type of text/markdown.
The AI version carries ads from Mobian, an AI advertising technology firm, and includes a per-visit ad display ID. Sponsored pages insert sponsor information in a question-and-answer format, labeled as sponsored content, so that generative AI can read it easily. These ads do not show on the human-facing pages.
TIME said that on many days, AI bot traffic exceeds human traffic. Schmalbach called the case an early example of what happens when the web's main readers are AI models.
Synthesized by Yomimono from the 1 cited source below, including Japanese-language reporting where cited, then editorially reviewed before publishing.