Churn Prediction

Churn prediction is the use of statistical or machine-learning models to estimate, for each customer, subscriber or player, the probability that they will stop using a product within a defined future period. It is one of the most common applications of predictive analytics, because retaining an existing customer is usually cheaper than acquiring a new one and because the inputs, behavioural logs and account records, already exist. The modeling is the easy part. The persistent difficulties are defining churn sensibly, avoiding label leakage, evaluating on honest time-ordered splits, and recognising that an accurate risk score does not by itself tell anyone what to do about it.

Defining Churn and Building Labels

In a contractual setting, such as a subscription, churn is an observed event: a cancellation or a failed renewal. In a non-contractual setting, such as retail, a free app or a free-to-play game, nobody announces their departure; churn has to be defined by inactivity, for example "no session in the next 14 days." The threshold is a judgment call: too short and people on holiday count as churners; too long and the label arrives too late to act on.

A training example is built from three windows: an observation window from which features are computed, an optional gap representing the lead time needed to intervene, and an outcome window in which churn is or is not observed. Every example is anchored to a snapshot date.

Two refinements are worth knowing. Voluntary churn (a decision to leave) and involuntary churn (an expired card) have different causes and remedies and should usually be modelled separately. And customers who are still active at the end of the data are censored, not survivors; survival analysis handles this directly by predicting time-to-churn. Periáñez et al. (2016), for instance, applied survival ensembles to mobile games and reported improved accuracy and robustness over Cox regression.

Leakage

Leakage is the most frequent cause of churn models that look excellent offline and fail in production. Kaufman, Rosset and Perlich (KDD 2011) define it as the introduction of information about the target that "should not be legitimately available to mine from," and recommend strict separation between what is known at prediction time and what is learned afterwards. Typical churn leaks include features computed over a window that overlaps the outcome window; fields that are written as part of the cancellation process (a support-ticket category, a "last login" that is by construction old for churners); snapshot tables overwritten with current values rather than stored as they were on the snapshot date; and random train/test splits that put the same customer, or a later period, on both sides. The remedy is point-in-time correctness: every feature must be reconstructable as of the snapshot date, which is the main purpose of a feature store.

Features and Model Families

Useful features fall into familiar groups: recency, frequency and intensity of use; trends in those quantities (declining session length is more informative than its level); tenure and lifecycle stage; payment history and plan; breadth of features used; social ties to other users; and support or complaint contacts.

Family	Strengths	Weaknesses
Logistic regression and survival models	Interpretable, well calibrated, handle censoring (survival)	Depend on hand-built features; miss interactions
Gradient-boosted trees (XGBoost, LightGBM, CatBoost)	The long-standing default for tabular features; fast to score; mature tooling	Need feature engineering and tuning
Sequence models (recurrent networks, transformers, behavioral foundation models)	Learn directly from raw event streams; one representation can serve many tasks	Costly to train and serve; public evidence of gains over tuned trees is limited
Tabular foundation models (for example TabPFN)	No per-dataset training or tuning; strong on small and medium tables	GPU inference; licence limits on the newest weights; little published churn-specific evidence

The last row is recent. Vendor reports from 2026 claim leading results on benchmarks that include grouped and temporal data, the structure typical of customer snapshots, but those are general benchmarks, not churn studies. The only reliable test is a comparison on one's own data under the evaluation protocol below.

Evaluation with Time-Ordered Splits

Churn data is not independent and identically distributed: behaviour drifts, products change, and the same customers recur. Evaluation should therefore mimic deployment. Train on snapshots up to a cut-off date, validate on later snapshots, and test on a period later still, ideally repeating the exercise across several cut-offs (rolling-origin evaluation).

Churners are usually a minority, so accuracy is uninformative. Precision-recall curves and lift in the top deciles describe ranking quality where it matters. Calibration matters when scores feed expected-value calculations, since a predicted 30% should mean that about three in ten such customers leave.

From Prediction to Retention

A churn score identifies who is at risk, not who can be saved. Some high-risk customers will leave whatever is offered; some would have stayed regardless; a few are prompted to cancel by the outreach itself. Eva Ascarza's "Retention Futility" (Journal of Marketing Research, 2018) made the point with field experiments in two industries: targeting customers by their responsiveness to the intervention reduced churn more effectively than targeting those at highest risk.

The practical consequence is that churn prediction is one input to a decision, not the decision. Estimating who responds requires randomised holdouts and uplift modeling; choosing among several possible interventions and adapting over time is a job for contextual bandits; and deciding how much a save is worth requires customer lifetime value. The churn model's own success metric should, in the end, be the measured retention difference between treated customers and a random control group, not its AUC.