AUTO-UPDATED

Ask a model if code is malicious and it reaches for its morals

Modern language models evaluate code for malice by routing information through internal "moral" pathways, suggesting these systems weigh security questions using machinery typically reserved for ethical judgment.

Key Points

  • Researchers analyzed mixture-of-experts (MoE) models, including OLMoE and DeepSeek-V2-Lite, to track how they process questions about malicious code.
  • When asked to identify malicious code, models consistently activate the same internal expert networks used for questions regarding morality and ethics.
  • Routing experiments showed that forcing a model to use a different question's expert path significantly alters its final answer.
  • Pruning studies revealed that while these moral pathways are consulted during decision-making, the actual judgment is not stored within those specific experts.
  • The study utilized 240 code samples, including both malicious and benign PyPI packages, to map how models frame security-related queries.

Why it Matters

This research demonstrates that AI models do not treat security analysis as a purely technical task, but rather integrate it with broader, abstract reasoning frameworks. Understanding these internal routing behaviors is critical for developers aiming to improve the reliability and safety of AI-driven code analysis tools.
Manifold.security Published by Cody Nash
Read original