Live Session
Session 10: Graph Learning
Industry
Country-diverted experiments for mitigation of network effects
Lina Lin (Google), Changping Meng (Google), Jennifer Brennan (Google Research), Jean Pouget-Abadie (Google Research), Ningren Han (Google), Shuchao Bi (Google) and Yajun Peng (Google)
Abstract
We describe the process of conducting a country-diverted experiment on a major content platform to mitigate the interference often observed in user-diverted A/B experiments. In particular, we propose a heuristic measure of leakage based on cosine similarity between treatment and control groups, which is used to select suitable country diversions, paired with a synthetic control approach to estimate the total treatment effect. We demonstrate the success of our approach through a live experiment on a key user engagement metric, as compared to a previous user-diverted experiment.