Ride Demand Forecasting
A spatiotemporal demand model, an evaluation bug it caught red-handed, and what changed shipping it to production
The problem
This started as a take-home exercise for a ride-hailing company's data science interview, built against ride data they provided for a specific city. Reusing a company's exact prompt and dataset publicly isn't appropriate, so this is a full rebuild: the same methodology, applied honestly end to end to the public Kaggle NYC Taxi Trip Duration dataset (1.46 million trips). The original dataset included a fare field; this public one doesn't, so ride duration stands in as the per-ride signal throughout.
Architecture
Exploration
Production service
The eval bug
The modelling unit is (pickup zone, hour), and the original methodology used a straightforward 80/20 train/test split by row count. That looked fine until the numbers didn't add up: raw ride counts per zone-hour averaged 2,429 on the training set but only 607 on the test set, a 4x gap, for what should be the same underlying demand pattern.
The cause: ride volume is roughly uniform per day, so an 80/20 split by row count isn't an 80/20 split by time. Training ended up covering 145 days, testing only 38, so a zone-hour's raw ride count was being compared across two completely different-length windows. Trained and evaluated directly on that raw sum, the model's error would have looked far worse than it actually was, for a reason that had nothing to do with the model.
The fix: aggregate into an average daily ride count for that split, rather than a raw sum, which is also the number that matters for driver guidance (how many rides typically happen in this zone during this hour). After the fix, train and test means both land around 16 rides a day, and the baseline evaluates to 3.24 MAE and 5.09 RMSE, about a fifth of the mean and roughly 22% of the median target value.
What the exploration found
Ride volume peaks at 6pm and troughs at 5am, a 6x gap, with Friday busiest and Monday quietest. One sharp exception stands out: January 23, 2016 crashed to about 1,600 rides from a normal 7,000 to 9,500, which lines up exactly with Winter Storm Jonas, a real external demand shock the feature set has no way to see coming. IQR-based outlier detection on coordinates flagged 4 to 6% of rows, too aggressive for geospatial data. An OpenStreetMap administrative-boundary check found only 0.09% genuine geo-outliers instead: GPS glitches landing near Sacramento and out over the Atlantic. For zone clustering, silhouette analysis peaks at k=20, half the k=40 the original methodology guessed by eye.
Model validation, and what didn't help
The XGBoost baseline was tuned with a randomised search over time-series cross-validation folds, then compared against an equally tuned LightGBM on a richer day-level dataset. LightGBM won by a margin that doesn't mean much: 4.626 MAE against 4.627 for XGBoost.
A notebook-only research extension then tested recency features (the last observed count and a trailing rolling mean) against two time-series baselines, a naive rolling mean and Holt-Winters. Recency features were the single biggest lever tested, roughly a 20% MAE improvement. The service can't use them yet, because it is stateless and would need to track recent counts per zone and hour.
A second extension added daily weather and a US federal holiday flag, motivated by the Winter Storm Jonas crash. Weather made the model worse under the chronological split, MAE 4.626 to 5.189, and the notebook diagnoses why: the test window (late May to June) has no snow days, so the split can't reward learning a snow effect. Weather is left undecided rather than rejected. The holiday flag alone gave a small gain, 4.626 to 4.580, but it rests on a single holiday in the test window.
From notebook to production
Porting the notebook's logic into a real service surfaced issues that only matter once you're actually serving predictions. First, the notebook fit its pickup-zone KMeans on train and test coordinates combined. The production training pipeline fits it on training data only, then assigns zones to the test set with the already-fitted model.
Second, the notebook trained on average ride duration and distance as if they were known ahead of time, but those are outcomes of rides that haven't happened yet: a real caller can't supply them for a future prediction window. The production pipeline instead builds a (zone, hour) to historical-average lookup from training data, so the API only ever needs a zone (or raw coordinates) and an hour.
Two more changes concern training data and dependencies. The service trains on one row per (zone, hour, date), tens of thousands of real daily observations instead of a few hundred pre-averaged numbers, which also removes the window-length mismatch behind the eval bug. Distance uses a vectorised NumPy haversine instead of a row-wise geodesic call, a city-scale difference under 0.5%, which drops geopy, osmnx, geopandas, contextily and xgboost from the service's runtime dependencies.
Results
| Pipeline | MAE | RMSE |
|---|---|---|
| Notebook baseline, XGBoost (zone-hour average rate) | 3.24 | 5.09 |
| Notebook, tuned LightGBM (day level) | 4.63 | 6.64 |
| Production service (day level, leakage-free) | 4.56 | 6.78 |
All values are rides per day. The last two rows score the same day-level target and are directly comparable. The first row scores a pre-averaged rate, which is a smoother, easier target by construction, so it isn't comparable with the others.
Example request
Real responses from the committed model artefact, rounded to two decimals for display. No Kaggle download needed to run this.
curl -X POST localhost:8000/predict \
-H "Content-Type: application/json" \
-d '{"pickup_zone": 5, "hour": 18}'{"pickup_zone": 5, "hour": 18, "predicted_avg_daily_ride_count": 19.63}curl 'localhost:8000/rankings?hour=18&top_n=3'{"hour": 18, "rankings": [
{"pickup_zone": 0, "predicted_avg_daily_ride_count": 61.67},
{"pickup_zone": 15, "predicted_avg_daily_ride_count": 44.86},
{"pickup_zone": 19, "predicted_avg_daily_ride_count": 42.47}
]}Testing
55 tests cover the data, feature and clustering pipeline plus the FastAPI service, run with pytest. CI runs ruff (lint and format), mypy and the full test suite on every push and pull request, plus a separate job that builds the Docker image, starts the container and smoke-tests the API.
Limitations
- ▸Public data stands in as a structural analog, not the original dataset: same shape (timestamped pickup/dropoff coordinates), a different city, and no fare field, so ride duration substitutes for ride value throughout.
- ▸Stateless serving leaves accuracy on the table. Recency features cut MAE by about 20% in the notebook, but using them needs a service that tracks recent counts per zone and hour.
- ▸Weather and holidays aren't used at serving time. The weather result is negative but confounded, and using it would need a live forecast dependency. The holiday gain would need a date input the API doesn't take, and one holiday is thin evidence for changing the API.
- ▸About six months of data from one city. The Kaggle file covers January to June 2016, so seasonal patterns outside that window are untested.
- ▸Retraining is a manual step, and the committed model is trained on the 2016 data. Moving training to current monthly NYC TLC data is in progress: ingestion and rolling-window training exist, but the committed model hasn't been replaced yet.