How we compute probability and uplift.

A propensity score alone answers “how likely is this customer to take action A?”. That is not the same as “will A happen because we contacted them?”. Next best action requires the second question, otherwise budget is spent on sure things and lost causes instead of persuadables.

For each candidate action a (MOT reminder, Motoring Club upgrade, Bike Care booking, EV charger quote, …) and each customer i, we estimate two quantities:

p is the calibrated probability shown in the UI. τ is the causal uplift used for ranking. They agree only when the base rate is near zero; otherwise they diverge, which is why we always model both.

Models are trained on a customer-level table joined from Motoring Club membership, transaction history, Autocentre service records, MOT dates (DVSA-derived where consented), online sessions, and fitting-bay bookings. Feature engineering falls into six families:

• Recency / frequency / monetary — visits, spend, time since last service, basket composition.
• Vehicle context — vehicle age, months to next MOT, fuel type, historical fail codes, aftermarket parts fitted.
• Cycling context — bike ownership, e-bike flag, months since last Bike Care visit, seasonality.
• Membership — Motoring Club tier, tenure, benefit utilisation rate, renewal window.
• Engagement — email open/click decay, app opens, store vs. online mix.
• Exogenous — local weather, fuel price index, school-holiday calendar, store catchment demographics.

Features are computed as of a fixed reference date so no post-treatment information leaks into training. Categorical values use target encoding with out-of-fold smoothing; numerics are winsorised at the 1st/99th percentile before tree models see them.

The propensity head p_i,a is a gradient-boosted decision tree (LightGBM, occasionally XGBoost) trained with log-loss. GBDTs win on tabular retail data with mixed types, missingness, and non-linear interactions. Deep tabular models (TabNet, FT-Transformer) are benchmarked but only shipped when they beat GBDT by ≥2% calibrated log-loss on a holdout, which is rare.

Raw GBDT scores are not probabilities. We calibrate with isotonic regression on a held-out fold, then verify with a reliability diagram and the Expected Calibration Error:

We ship at ECE ≤ 0.03 across deciles. Anything above that and marketing spend allocation becomes systematically biased — usually over-contacting the top decile.

Uplift τ_i,a is the average treatment effect conditional on covariates. We use two families depending on data availability:

• Meta-learners on RCT data — where we have a clean holdout (e.g. Motoring Club renewal campaign), we fit T-learner and X-learner variants with GBDT base models. X-learner dominates when treatment is rare.
• Causal forests on observational data — for always-on channels (app push, in-store attach) we use a doubly- robust causal forest with propensity-of-treatment as a nuisance model. Assumes overlap and unconfoundedness given the feature set.

Formally, for treatment T ∈ {0,1} and outcome Y:

Ranking uses τ̂, not p̂. This is what stops the model recommending an MOT reminder to someone who has already booked online — high propensity, zero uplift.

For each customer we compute expected value per action and rank:

Eligibility encodes hard business rules (consent, stock at the local Autocentre, MOT window). Fatigue caps how often a customer can be contacted per channel per fortnight. Capacity constraints solve as a bipartite assignment when a channel is oversubscribed — e.g. Saturday fitting-bay slots — using a greedy top-k with fairness reshuffling.

Models graduate from lab to production only when they beat the incumbent on all four of: calibrated log-loss, Qini, top-decile lift, and holdout revenue delta — measured on an out-of-time window, not a random split.

• Overlap. Every customer has non-zero probability of receiving each action. Violated if a channel is only ever used for one segment; we detect via propensity histograms and trim.
• Unconfoundedness (observational only). Treatment assignment is independent of the outcome given the features. We stress-test with sensitivity analysis (Rosenbaum bounds) and prefer RCT holdouts wherever the commercial cost allows.
• SUTVA. One customer’s treatment doesn’t affect another’s outcome. Broken by household-level effects (two cars, one email); we cluster households and treat at household level.
• Stationarity within horizon. The joint distribution of features and outcomes is stable over the scoring horizon. Enforced by monthly retraining and PSI monitoring on the top 30 features.
• Label fidelity. Conversions are attributed within a defined window (e.g. 14 days for MOT booking, 30 days for Club upgrade). Attribution rules are frozen before training.
• Ethics & consent. Models never use protected attributes as features. Suppression lists (bereavement, complaint, regulatory) are applied post-ranking and audited quarterly.

• Population Stability Index per feature; alert at PSI > 0.2, retrain at PSI > 0.25.
• Calibration drift weekly on the last 4 weeks of scores vs. realised outcomes.
• Uplift decay from the permanent holdout; if the Qini coefficient drops > 20% vs. launch, the action is paused and re-estimated.
• Guardrail metrics — unsubscribe rate, complaint rate, average contacts per customer per week. Any breach reverts to the previous policy automatically.