jorgeasmz/fraud-stream-detector

Fraud stream detector

Scores a card transaction as it arrives and ranks it for a review team that works a fixed number of alerts a day. Fitted on 872,795 transactions over 91 days of the simulator published with the Fraud Detection Handbook (Le Borgne and Bontempi, Université Libre de Bruxelles).

Metrics

Measured on the following 85 days, 813,843 transactions, at a budget of 100 alerts a day.

Metric	Value
Precision at the operating point	0.532
Alerts a day at the operating point	120
Precision ranking each day, top 100	0.620
Card precision ranking each day	0.561
Recall, an amount above a threshold no legitimate transaction reaches	0.916
Recall, a terminal compromised for about four weeks, at ordinary amounts	0.664
Recall, a card compromised for about two weeks, at multiplied amounts	0.819

Each fraud pattern is reachable through a different family of features, so recall per pattern says which signal the detector is using rather than reporting one aggregate.

The first row is what a deployment reaches. The last two rank each completed day and take its best 100, which is an offline view: a service decides on a transaction when it arrives, without the rest of the day to compare it against. Reporting only the ranked figure overstates the served result by 0.088.

The operating point also emits 120 alerts a day rather than 100. It is a quantile of the training scores, and the score distribution moves over the following months, which is what an operating point set once and never revisited does.

Card precision counts a card once however many of its transactions were flagged, since a team investigates cards rather than transactions.

Accuracy is not reported. At a fraud rate below one percent it is a measure of the base rate, and a detector that flags nothing scores above 0.99 on it.

Operating point

0.016430, the score whose expected alert rate equals 100 alerts a day at the training period's volume. A review team is a rate rather than a probability, so the operating point is a quantile of the training scores and an output of the fit rather than a constant in the serving code.

Inputs

30 features per transaction, in the order recorded in decision.json: the amount, the hour, a weekend flag, rolling counts and mean amounts per card and per terminal over 1, 7, 30 days, the ratio of the amount to the card's window mean, and fraud rates per card and per terminal over the same windows.

Every window is prior-only: a scorer has not seen the transaction it is judging, so including it in its own window would report a quality the live path cannot reach.

The fraud-rate features end 7 days before the transaction they describe, which is the delay a dispute takes to resolve. Without them the detector cannot reach the compromised-terminal pattern at all: no configuration fitted without labels exceeds chance on it, and with them its recall is 0.664.

Limitations

The corpus is simulated. Its three fraud patterns are documented and separable, which is what makes recall per pattern meaningful, and it is not evidence of behaviour on real card traffic.

The artifact is a pickle and executes on load. Load it only from a repository you control.

Trained by Fraud-Stream-Detection.