Cartoons, Captions, and Confidence Intervals

Abstract:

Adaptive experimental design (AED) methods, often colloquially referred to as Multi-Armed Bandit (MAB) algorithms, have emerged as a promising paradigm for data collection under-sampling budget constraints. AED promises to deliver statistical significant in a fraction of the time compared to traditional A/B/N testing. In this talk, we will discuss some lessons learned from real world applications of AED. These include crowdsourced voting on the New Yorker Caption Contest and large scale experimentation for online platforms. In addition, recommended best practices for practitioners seeking to implement AED will be presented.


Lalit K. Jain is an assistant professor in the Foster School of Business. His research is focused on the theory and implementation of machine learning algorithms for large-scale data collection with an emphasis on ”human in the loop” and crowdsourcing applications. His work has been applied to a variety of applications including optimizing crowdfunding and microlending platforms, measuring conceptual perception in cognitive psychology, and detecting humor.