AUTO-UPDATED

Media Briefing: The case for and against markdown for AI bots

Publishers are increasingly creating simplified markdown versions of their websites to improve AI bot accessibility, though experts remain divided on whether this strategy effectively boosts visibility or revenue.

Key Points

  • Publishers like Time are stripping away JavaScript and visuals to create markdown pages, which require less compute power for AI systems to process.
  • Data marketplace TollBit reports that converting content to markdown can reduce token usage by approximately 90%, potentially lowering AI crawl costs.
  • Time has begun experimenting with serving ads directly to AI agents within these markdown files to monetize bot traffic.
  • Critics argue that markdown makes it easier for AI to scrape content without permission, potentially leading to a loss of publisher control.
  • Google’s official guidelines state that markdown is not necessary for search optimization, as its systems do not rely on these files for indexing.
  • Research from Promptwatch suggests there is currently no clear evidence that using markdown increases the frequency of AI citations.

Why it Matters

This debate highlights a critical strategic shift as publishers attempt to balance the need for AI visibility with the desire to protect and monetize their intellectual property. While markdown offers a technical path to efficiency, the lack of proven ROI leaves many media companies questioning whether it is a sustainable long-term solution or merely a speculative marketing tactic.
Digiday Published by Sara Guaglione
Read original