Do participation rates vary with participation payments in laboratory experiments?

This paper reports a series of experiments designed to evaluate how the advertised participation payment impacts participation rates in laboratory experiments. Our initial goal was to generate variation in the participation rate as a means to control for selection bias when evaluating treatment effects in common laboratory experiments. Initially, we varied the advertised participation payment to 1734 people from $5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\$5$$\end{document} to $15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\$15$$\end{document} using standard email recruitment procedures, but found no statistical evidence this impacted the participation rate. A second study increased the advertised payment up to $100\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\$100$$\end{document}. Here, we find marginally significant statistical evidence that the advertised participation payment affects the participation rate when payments are large. To combat skepticism of our results, we also conducted a third study in which verbal offers were made. Here, we found no statistically significant increase in participation rates when the participation payment increased from $5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\$5$$\end{document} to $10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\$10$$\end{document}. Finally, we conducted an experiment similar to the first one at a separate university. We found no statistically significant increase in participation rates when the participation payment increased from $7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\$7$$\end{document} to $15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\$15$$\end{document}. The combined results from our four experiments suggest moderate variation in the advertised participation payment from standard levels has little impact on participation rates in typical laboratory experiments. Rather, generating useful variation in participation rates likely requires much larger participation payments and/or larger potential subject pools than are common in laboratory experiments.