Manifest has officially deprecated its LLM router, concluding that using a single, battle-tested model is more effective for performance and consistency than dynamically routing requests between different AI models.

Key Points

• Manifest launched its LLM router in March and officially shut down the service on September 1st.
• The company analyzed data from 7,000 cloud users, finding that prompt complexity is often impossible to determine accurately before execution.
• Cache-based optimization proved more cost-effective than routing, offering 75% to 90% savings on input tokens.
• Dynamic routing was found to break behavioral consistency and hinder engineers from mastering specific model capabilities.
• The company determined that the unpredictability introduced by routers complicates maintenance for automated agentic workflows.

Why it Matters

• This shift highlights a growing industry skepticism toward automated model selection as a primary method for cost reduction. By prioritizing model consistency, developers can better manage system predictability and long-term maintenance costs in complex AI applications.