The Problem

Credit card fraud detection has a hard real-time constraint: every transaction must be scored before the customer removes their card. In practice, this means sub-100ms end-to-end latency. Batch inference won't cut it — you need a streaming architecture where features are computed online, predictions are served in real-time, and every prediction is observable.

This post walks through the architecture of a real-time fraud detection pipeline built with Redpanda (Kafka-compatible streaming), XGBoost for online inference, and Grafana for live monitoring.

Architecture Overview

Transaction Source
      |
      v
  Redpanda Topic (raw_transactions)
      |
      v
  Feature Engineering Worker
      |  - rolling window aggregates (30/60/90 min)
      |  - velocity checks (count, amount)
      |  - merchant category encoding
      |  - time-since-last-transaction
      v
  Redpanda Topic (enriched_transactions)
      |
      v
  XGBoost Inference Worker
      |  - model: 30-feature booster (500 rounds)
      |  - output: fraud probability + SHAP explanation
      v
  Redpanda Topic (scored_transactions)
      |
      +---> PostgreSQL (persistent storage)
      +---> Grafana (live dashboards)

Component Breakdown

1. Streaming Layer — Redpanda

Redpanda is a Kafka-compatible event streaming platform written in C++. Compared to Kafka, it offers:

  • Lower latency — no JVM overhead, single-binary deployment
  • Faster topic creation — no ZooKeeper dependency
  • Better resource utilization — uses < 50% of Kafka's memory for the same throughput

Three topics form the pipeline:

  • raw_transactions — ingested from the transaction gateway
  • enriched_transactions — after feature computation
  • scored_transactions — after XGBoost inference

2. Feature Engineering

Each transaction arrives as a JSON blob with amount, merchant_id, timestamp, card_id, location.

The feature worker maintains rolling window state using Python's deque (backed by Redis for production durability):

features = {
    'amount': tx.amount,
    'avg_amount_1h': rolling_avg(tx.card_id, 3600),
    'tx_count_1h': rolling_count(tx.card_id, 3600),
    'velocity_change': tx.amount / (avg_amount_1h + 1),
    'merchant_freq': merchant_freq[tx.merchant_id],
    'hour_of_day': tx.timestamp.hour,
    'is_weekend': tx.timestamp.weekday() >= 5,
    # ... 30 features total
}

3. XGBoost Inference

The model is a standard XGBoost classifier trained on historical fraud data with 30 engineered features. Training happens offline; the serialized model.json is loaded at worker startup.

Inference path:

model = xgb.Booster(model_file='model.json')

def score(features: np.ndarray) -> dict:
    dmat = xgb.DMatrix(features.reshape(1, -1))
    prob = model.predict(dmat)[0]
    # Application-level thresholding
    if prob > 0.9:
        action = 'block'
    elif prob > 0.5:
        action = 'flag'
    else:
        action = 'allow'
    return {'fraud_probability': float(prob), 'action': action}

Latency: <50ms per transaction (including deserialization + feature fetch).

4. Observability — Grafana

Every scored transaction is written to PostgreSQL. Grafana dashboards show:

  • Fraud volume over time (flagged vs blocked vs allowed)
  • Prediction distribution — histogram of fraud probabilities
  • Potential loss averted — cumulative sum of blocked transaction amounts
  • Pipeline latency — p50/p95/p99 latency from ingestion to score

Results

Metric Value
End-to-end latency (p50) 28ms
End-to-end latency (p99) 94ms
Throughput 2,400 tx/sec per worker
Fraud recall (test set) 0.91
Precision (test set) 0.87

Key Lessons

  1. Feature engineering is the bottleneck, not inference. The model inference takes ~2ms. The remaining 26–92ms is spent computing rolling window features.
  2. Redpanda simplifies operations. A single Go binary replaces Kafka + ZooKeeper + Schema Registry.
  3. Monitor everything. The Grafana dashboard caught a 3× latency spike caused by Python GC pausing — switching to pypy resolved it.

The complete pipeline is on GitHub with a docker compose up one-command setup.