Design an AI Fraud Detection Agent
A hellointerview-style system design deep dive into real-time AI fraud detection systems like Stripe Radar, PayPal, and Featurespace. Covers requirements, core entities, the two-speed scoring architecture, and three production deep dives: real-time scoring pipeline, feature engineering, and adversarial adaptation with cold-start handling. Each deep dive walks through naive, better, and production-grade approaches with trade-offs.
Quick Reference
- →Two-speed architecture: fast ML scores every transaction in under 100ms, slow LLM agent investigates flagged patterns asynchronously
- →Feature engineering across transaction, velocity, behavioral, device, historical, and network signals is the core differentiator
- →Sub-100ms latency constraint means no LLM in the hot path — use pre-computed features and optimized model serving
- →Ensemble scoring combines rule-based, gradient boosting, and neural network models with a feature store for pre-computed signals
- →Adversarial adaptation through online learning with concept drift detection — fraudsters change tactics in response to your model
- →Cold-start handling uses device intelligence, behavioral biometrics, and progressive trust that expands as account history builds
- →The label delay problem — chargebacks arrive 30 to 90 days later — requires proxy labels and models that generalize to new patterns
- →Feature stores are the product: invest 80 percent of engineering effort in features, 20 percent in model architecture
Understanding the Problem
A fraud detection system evaluates every financial transaction in real-time and decides whether to approve, decline, or flag it for investigation. The system must score millions of transactions per day with sub-100ms latency while maintaining a false positive rate below 0.1 percent — because every legitimate transaction you block is lost revenue and a frustrated customer. Products like Stripe Radar, PayPal Risk Management, and Featurespace ARIC process billions of dollars in transactions daily, catching fraud patterns that range from simple stolen card usage to sophisticated multi-account synthetic identity schemes. From a system design perspective, this is a fascinating problem because it combines real-time ML inference at extreme scale, adversarial dynamics where attackers actively adapt to your defenses, temporal feature engineering across multiple time horizons, and the cold-start problem where new accounts have no behavioral history. The constraints are sharp: scoring must be fast enough that the customer never notices a delay, accurate enough that fraud losses stay below acceptable thresholds, and adaptive enough that detection rates hold even as fraudsters evolve their tactics weekly.
Stripe Radar processes hundreds of millions of transactions and uses a gradient boosting ensemble trained on features computed across all Stripe merchants — giving it a network effect where fraud patterns detected at one merchant improve detection for all merchants. PayPal combines traditional ML scoring with graph-based analysis that traces relationships between accounts, devices, and IP addresses to detect fraud rings. Featurespace pioneered adaptive behavioral analytics that build a behavioral profile for each customer and flag deviations from established patterns. All three systems share the same fundamental architecture: a fast scoring pipeline for every transaction backed by a rich feature store, with asynchronous investigation for flagged cases.
This is fundamentally about building a two-speed system: a fast ML pipeline that scores every transaction in under 100ms using pre-computed features, and a slower investigation layer that analyzes flagged patterns across accounts and time. The three hardest sub-problems are: (1) achieving sub-100ms scoring with rich feature computation, (2) engineering temporal and behavioral features that capture fraud signals, and (3) adapting to adversarial drift where fraudsters change tactics in response to your model.