Senior Data Scientist- Experimentation & Causal Inference
RemoteSan Francisco, California, United States
Job Summary
Design and maintain automated anomaly detection systems for live experiments, including Sample Ratio Mismatch (SRM) detection and traffic allocation validation. Implement variance reduction methodologies such as CUPED and covariate adjustment to reduce metric noise and shorten average experiment duration from 14 days to 9 days. Operate continuous A/A testing programs to validate the integrity of the end-to-end measurement pipeline and baseline metric stability. Collaborate with Product, Engineering, and Leadership teams on experimental design, sample size calculations, and causal inference using Difference-in-Differences or Synthetic Control methods. Translate statistical methodologies into technical specifications and validation scripts to enable scalable self-service experimentation while establishing organization-wide standards.
Required Qualifications
- Master's or PhD in Statistics, Biostatistics, Economics (Econometrics), Computer Science, Mathematics, or a related quantitative discipline
- 3+ years of industry experience designing and analyzing online controlled experiments at scale within a technology company
- Demonstrated experience working with large-scale digital experimentation programs involving meaningful user traffic
- Experience beyond survey-based or clinical trial experimentation
- Strong foundation in hypothesis testing
- Strong foundation in power analysis
- Strong foundation in multiple testing correction
- Strong foundation in sequential testing
- Strong foundation in experimental design
- Hands-on experience with CUPED or other variance reduction techniques
- Hands-on experience with Sample Ratio Mismatch (SRM) detection
- Hands-on experience with continuous experimentation quality monitoring
- Hands-on experience with experiment health diagnostics
- Advanced Python proficiency
- Experience with SciPy
- Experience with Statsmodels
- Experience with NumPy
- Experience with equivalent statistical libraries
- Strong SQL skills
- Experience with window functions
- Experience with Common Table Expressions (CTEs)
- Experience with data validation and quality analysis
- Experience building simulations and statistical validation frameworks
- Ability to explain complex statistical concepts to technical and non-technical stakeholders
- Strong capability to translate business questions into rigorous experimental methodologies
- Comfortable writing clear specifications that engineers can implement in production systems
- Extensive hands-on experience using AI-assisted development tools such as Claude Code, OpenClaw, Cursor, or similar AI coding assistants
- Proven ability to leverage AI tools to accelerate analysis, automate repetitive workflows, improve productivity, and build self-service tooling
- Fluency in both English and Chinese
- Ability to collaborate effectively with engineering, product, and data teams across APAC and North America
- Comfortable serving as a communication bridge between the US R&D Center and regional teams
- Comfortable joining important meetings during early mornings or evenings when necessary
Desired Qualifications
- Experience with Bayesian experimentation frameworks
- Familiarity with multi-armed bandit methodologies
- Understanding of streaming data ecosystems and their impact on experimentation validity, including Kafka, Flink, and real-time analytics architectures
- Publications or conference presentations related to experimentation, causal inference, statistical methodology, or machine learning evaluation
- Experience in Fintech, cryptocurrency, trading platforms, or other high-scale consumer technology environments
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.