Reddit Restricts the Internet Archive's Wayback Machine
August 2025
In August 2025 Reddit moved to block the Internet Archive's Wayback Machine from archiving anything beyond its homepage, citing AI scrapers exploiting the archive — a step critics warned damages the public record to protect Reddit's data-licensing revenue.
What happened
On August 11, 2025, Reddit announced that it would sharply restrict the Internet Archive's Wayback Machine, with the change taking effect the following day. Under the new policy, the Wayback Machine could continue to archive only Reddit's homepage; it was blocked from preserving individual post detail pages, comment threads, subreddit pages, and user profiles — in other words, nearly all of the substantive content that makes Reddit worth archiving.
Reddit's stated rationale was anti-AI-scraping enforcement. Spokesperson Tim Rathschmidt told The Verge that 'Internet Archive provides a service to the open web, but we've been made aware of instances where AI companies violate platform policies, including ours, and scrape data from the Wayback Machine,' framing the restriction as a response to AI firms using the archive as a back door to harvest Reddit content while avoiding Reddit's policies and licensing fees. The move marked a reversal of Reddit's earlier 2024 position, when the company had said it would not block 'good faith actors' and had explicitly named the Internet Archive among them.
The restriction followed Reddit's lucrative data-licensing deals — including the roughly $60-million-per-year Google agreement and the OpenAI deal — and aligned with a broader strategy of locking down its content through robots.txt directives and access controls. Critics argued that the explanation, while not implausible, primarily served Reddit's commercial interests: the same content that AI companies might scrape from the archive was content Reddit was now selling, so restricting the archive protected the paid data pipeline while collaterally degrading a vital public-interest resource used by researchers, journalists, and ordinary users.
The most pointed objection concerned accountability. The Wayback Machine is one of the primary tools people use to preserve and later retrieve content that Reddit users or moderators delete or edit. By restricting it, Reddit weakened an independent record of its own platform's history, making it harder to document what was posted and later removed — whether by individual users, by moderators, or by the company. For a site whose discussions are frequently cited, quoted, and contested, the loss of a durable third-party archive removed a check on both the platform and its moderators, and concentrated control over the historical record in Reddit's own hands at precisely the moment it was monetizing that record.
Observers placed the decision within Reddit's wider 2024–2025 campaign to gate its data: blocking most search-engine crawlers, restricting third-party archiving, and signing exclusive AI-licensing arrangements. The throughline was a transformation of Reddit from an openly accessible public square into a tightly controlled, monetized data asset — a shift that the Wayback Machine restriction made unusually stark, because it traded away part of the public record itself.
Impact
The restriction curtailed independent archiving of Reddit's posts and comments, weakening a public-interest preservation and accountability resource in service of Reddit's AI-data monetization strategy. By limiting the Wayback Machine to Reddit's homepage, it undermined a key tool for documenting content that users and moderators later delete or edit, concentrating control over Reddit's historical record in the company's own hands and reversing Reddit's earlier pledge not to block 'good faith actors.'
Sources
- 01
- 02
- 03
- 04
- 05