AUTO-UPDATED

Realtime transaction fraud detection – with an LLM?

This guide demonstrates how to build a real-time transaction fraud detection system using open-weight AI models and Canonical’s open-source MLOps stack on a local Kubernetes cluster.

Key Points

  • The project utilizes the "system-one-qwen3.5-4b-scorer" model to perform classification tasks instead of generating natural language.
  • Developers use KServe on a MicroK8s cluster to deploy the model, ensuring production-ready resilience and automation.
  • Activation-aware Weight Quantization (AWQ) is applied to compress the model, reducing VRAM requirements by up to 75% for CPU-based execution.
  • The system processes transaction data to return probability scores for fraud risk, bypassing the latency associated with traditional generative chatbots.
  • The implementation requires Ubuntu 24.04 LTS, 32GB of RAM, and tools including Juju, Terraform, and vLLM.

Why it Matters

Transitioning from generative text models to specialized classification models allows businesses to integrate AI directly into high-speed middleware for tasks like fraud detection. By leveraging open-source MLOps tools, organizations can deploy these efficient, self-governed AI systems behind their own firewalls to maintain data privacy and reduce reliance on external cloud services.
Ubuntu.com Published by Rob Gibbon (Rob Gibbon)
Read original