AUTO-UPDATED

Strengthening our defenses against distillation attacks

Anthropic has implemented advanced security measures to combat industrial-scale distillation attacks, successfully reducing the unauthorized harvesting of Claude’s internal reasoning data by approximately 90% to 99%.

Key Points

  • Anthropic introduced a two-stage constitutional classifier system to detect and block requests attempting to extract hidden model reasoning.
  • New "preserved thinking" protocols prevent API users from editing prior context, limiting the ability of proxy networks to manipulate model outputs.
  • The company has banned tens of thousands of fraudulent accounts used by distillers to replicate frontier AI capabilities.
  • Distillation attacks involve using large-scale networks to mimic high-level AI performance, often for use in foreign military or surveillance systems.
  • These defensive measures have resulted in a false positive rate of less than 0.1% for legitimate user requests.

Why it Matters

Preventing distillation is critical to national security and public safety, as illicitly copied models often lack the robust safeguards necessary to prevent misuse in cyberattacks or biological research. By sharing these defensive strategies, Anthropic aims to set an industry standard that encourages other AI developers to prioritize security over the short-term revenue generated by high-volume, malicious traffic.
Anthropic.com Published by Anthropic
Read original