Event Date
SPEAKER: Lee Richardson, Senior Data Scientist, Google
TITLE: "Proxy Metrics at YouTube"
ABSTRACT: North star metrics and online experimentation play a central role in how technology companies improve their products. In practice, however, evaluating experiments based on the north star metric directly can be difficult. The two biggest issues are 1) low sensitivity of the north star metric and 2) differences between the short-term and long-term impact on the north star metric.
A common solution is to rely on Proxy metrics rather than the north star in experiment evaluation and launch decisions. Existing literature on Proxy metrics concentrates mostly on the estimation of the long-term impact from short-term experimental data. We instead focus on the trade-off between the estimation of the long-term impact and the sensitivity in the short-term. We propose a suite of methods to find optimal proxies: ranging from heuristic based approaches to a more principled statistical framework.
This talk starts by giving an overview of experiments at Google, then zooms in on the specific topic of proxies.
SEMINAR DATE/TIME: Thursday December 7, 2023, 4:10pm, at MSB 1147
Refreshments : 30:30pm, MSB Courtyard