Autonomous ML experiment loop — advertising response curve fitting
1,073 product familiesWLS with 1/sales weighting, gradient descent tuning, Huber loss, LR decay optimization. Discovered that reducing GD iterations improved WMAPE by reducing overfitting.
Dropped Hill/Exp models entirely — linear-only WLS won. Added power law, recency weighting, quadratic, saturation models, cached blending. Extensive hyperparameter micro-optimization converging on diminishing returns.
Weekly aggregation of daily data produced a massive 30.9% WMAPE reduction in a single experiment. Smoothing noisy daily ad data removed measurement noise that was limiting model quality.
Biweekly then monthly (4-week) aggregation. Holdout split optimization (15% optimal). Recency and blend parameter refinement on the new data granularity.
Selection margin removal, ensemble methods, three-way blending, cubic recency. Additional post-TSV experiments brought WMAPE from 0.1322 to 0.1228 (current best).
| # ▲ | Commit ▲ | WMAPE ▲ | Best ▲ | Time(s) ▲ | Status ▲ | Description ▲ |
|---|
4-week temporal aggregation collapsed noisy daily data into cleaner monthly signals. 0.189 → 0.169 (-10.6%)
Weekly aggregation harness was the most impactful single change in the entire experiment series. 0.314 → 0.217 (-30.9%)
Dropping Hill and Exp models and using linear-only WLS improved WMAPE. Fewer GD iterations also improved results by reducing overfitting. Simpler is better.
250+ discarded experiments in the micro-optimization plateau proved the current approach is near-optimal for this data. The search space has been thoroughly explored.