In this short article, I want to summarize recent studies that have dealt with trends in experimental economics by highlighting problems and future hopes. Certainly, experimental economics, after decades of growth is changing. Partly due to the advent of online experiments, and partly due to new qualitative and quantitative standards of research that seek to ensure reproducibility. From this moment, from all the analysis done, we need to treasure in order to propose an experimental economics that takes into consideration all that has been done but is able to look to the future of the economy and adapt.

Recent trends

Number of publications

Nikiforakis and Slonim (2015) in the first issue of the Journal of the Economic Science Association capture the growth in the number of experimental papers produced during a 40-year period by tracing the trajectory of experimental research published in the “top-5” economics journals between 1975 and 2014. They did, however, also record an unexpected fall following 2010. In the 2019 analysis, as shown in the graph, they noticed that since 2000, the number of field experiments has been continuously increasing while the number of “lab experiments” published has continued to decline. However, nothing to worry about because further analysis showed that overall between 2011 and 2018, there has been a 10.5% increase in the number of papers published in the top 5 (Nikiforakis, N., & Slonim, R., 2019).

So what is happening? There are at least two explanations for these data as pointed out by the authors:

1. The decrease in experimental papers published in the AER, which began in 2011, may be to blame for the drop. This happened at the same time as the editorial board of the journal changed.
2. The other journals all allocate a comparable percentage of their pages to experimental studies, with the exception of Econometrica, which gives a higher share of its pages to the lab than to field experiments.

Moreover,

1. In the Top 5 publications, the percentage of lab research involving student subjects has sharply decreased (Reuben et al., 2022).

Good news. Between 2000 and 2021, the percentage of non-lab experimental articles (primarily field experiments) published increased, and the percentage of lab research involving student subjects in Experimental Economics, JEEA, and EJ remained stable (Reuben et al., 2022).

Partial good news. Even though fewer experimental articles were published, their influence consistently outweighs that of recent papers in other domains (Fréchette et al., 2022) however, as pointed out by Reuben et al. (2022) lab experiments receive fewer citations than other types of experiments.

Reuben et al. (2022), to better understand the data tracked down the publications also in JEEA and EJ confirming the overall results: the percentage of LAB experiments in Top journals (apart from AER) has stayed constant. The percentage of LAB experiments has increased in JEEA+EJ increased to 6.8% in the second decade from 4.6% in the first.

Topics

Quantitative methods papers have increased when comparing the first and second halves of the decade. Work focusing on financial issues is declining, while work focusing on health and education-related issues is increasing. Additionally, we observe a decline in publications exploring mechanism design and an increase in papers exploring individual consumer choice (Fréchette et al., 2022).

Web-based revolution

Only around 5% of experiments reported in the top 5 journals between 2010 and 2014 included an online treatment. However, that percentage rose to 23% between 2015 and 2019 (Fréchette et Al., 2022).

The benefit of virtual laboratories is that they enable researchers to conduct experiments even without access to physical laboratories. However, recent findings indicate that online platform observations may be noisier than traditional lab observations (Snowberg E., Yariv L., 2021; Brodeur et al., 2022). Why? Inattention is one possible avenue through which noise might be produced, according to Gupta, Rigotti, and Wilson (2021).

Issues for future improvements

To improve the validity of experimental studies and to try to solve the replication crisis (that, however, hit economics differently than psychology) have been suggested and in some cases already implemented (Fréchette et Al., 2022):

• Pre-registration and pre-analysis plans. Pre-registration is only helpful inasmuch as it is being monitored. There is a chance that unchecked pilots will proliferate with the introduction of online experimental platforms: the use of design-hacking instead of p-hacking.
• Lower p-value thresholds for significance. A decrease in type-I errors might be offset by a rise in type-II errors, and some experimental designs and findings might not be published because of their lower relevance.
• An effort to replicate existing studies.

Conclusion

We can summarize the evidence in a few points:

1. There is not a general drop in experimental publications but rather a shift. There is a shift from laboratory experiments to field experiments.
2. A mighty increase in online experiments that nonetheless suffer (or may suffer) from the noise that vitiates their meaningfulness and generalizability.
3. For laboratory experiments to remain valid tools of analysis for all fields of economics, it is necessary to incorporate certain standards that have been shown to be important.
4. In the years to come, more information comparing various experimental platforms will be gathered, especially as experimental designs advance to speak to a wider range of participants.
5. The apparent rise in designs that contain a battery of elicitations at the conclusion of experimental sessions without any a-priori reasons is one trend that requires further examination.
6. We need to further investigate the fact that lab experiments receive fewer citations than other types of experiments. As suggested by someone the topics studied in lab experiments may be less well-known or less related to other areas of economics than those explored in other types of experiments, which would imply that non-experimental economists regard lab studies to be less persuasive (Reuben et al., 2022).

Sources

• Brodeur, Abel, Nikolai Cook, and Anthony Heyes. We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments. No. 1157. GLO Discussion Paper, 2022.
• Fréchette, G. R., Sarnoff, K., & Yariv, L. (2022). Experimental economics: Past and future. Annual Review of Economics, 14, 777-794.
• Gupta, Neeraja, Luca Rigotti, and Alistair Wilson (2021). The Experimenters’ Dilemma: Inferential Preferences over Populations.
• Nikiforakis, N., & Slonim, R. (2015). Editors Preface: Introducing JESA. Journal of the Economic Science Association, 1(1), 1–7.
• Nikiforakis, N., & Slonim, R. (2019). Editors’ Preface: Trends in experimental economics (1975–2018). Journal of the Economic Science Association, 5(2), 143-148.
• Reuben, E., Li, S. X., Suetens, S., Svorenčík, A., Turocy, T., & Kotsidis, V. (2022). Trends in the publication of experimental economics articles. Journal of the Economic Science Association, 1-15.
• Snowberg E., Yariv L. 2021. Testing the waters: Behavior across participant pools. American Economic Review 111(2):687–719