Anthropic has implemented advanced security measures to combat industrial-scale distillation attacks, successfully reducing the unauthorized harvesting of Claude’s internal reasoning data by approximately 90% to 99%.
Key Points
- Anthropic introduced a two-stage constitutional classifier system to detect and block requests attempting to extract hidden model reasoning.
- New "preserved thinking" protocols prevent API users from editing prior context, limiting the ability of proxy networks to manipulate model outputs.
- The company has banned tens of thousands of fraudulent accounts used by distillers to replicate frontier AI capabilities.
- Distillation attacks involve using large-scale networks to mimic high-level AI performance, often for use in foreign military or surveillance systems.
- These defensive measures have resulted in a false positive rate of less than 0.1% for legitimate user requests.