This guide demonstrates how to build a real-time transaction fraud detection system using open-weight AI models and Canonical’s open-source MLOps stack on a local Kubernetes cluster.
Key Points
- The project utilizes the "system-one-qwen3.5-4b-scorer" model to perform classification tasks instead of generating natural language.
- Developers use KServe on a MicroK8s cluster to deploy the model, ensuring production-ready resilience and automation.
- Activation-aware Weight Quantization (AWQ) is applied to compress the model, reducing VRAM requirements by up to 75% for CPU-based execution.
- The system processes transaction data to return probability scores for fraud risk, bypassing the latency associated with traditional generative chatbots.
- The implementation requires Ubuntu 24.04 LTS, 32GB of RAM, and tools including Juju, Terraform, and vLLM.