Methodological and Reporting Quality in Laboratory Studies

The replication crisis is currently a hot topic within the field of Psychology and the credibility of many findings are being challenged [1]. Reliable and valid research methods are crucial to good scientific practice, and ill practice of these fundamental characteristics may be where Psychology research falls short.

A team of researchers from The University of Liverpool and Radboud University (Netherlands) recently reviewed studies that had investigated human eating behaviour in the lab [2]. Laboratory studies allow for highly controlled experiments to identify factors that predict and causally influence objectively measured energy intake [3]. Therefore, studying eating behaviours in the lab is important for nutrition science. The aim of review was to examine the basic methodology and reporting practices in recent laboratory studies of human eating behaviour.

The team examined 140 laboratory studies of human eating behaviour that were published in 2016. They extracted the basic methodology (e.g., sample size, use of participant blinding) and reporting practices (e.g., information on participant characteristics) of each study.

They found that the majority of studies (>90%) demonstrated good practice by reporting information relating to participant characteristics (e.g., age, gender) and study methodology (e.g., length of washout periods in within-subjects studies). However, 25.7% of studies did not provide information regarding participant weight status (which is of primary importance when investigating eating behaviours). Also, approximately a third of studies failed to report participant eligibility criteria. With regards to design, Robinson and colleagues reported that often the studies did not appear to standardize pre-test meal appetite or attempt to blind participants to study aims.

Furthermore, how the authors of papers determined their sample size was frequently not reported. The average sample size of studies was small (between-subjects design studies in particular) and the primary statistical analyses in a sizeable minority of studies (24%) were reliant on very small sample sizes that would only be sufficient to detect large effect sizes.

The researchers concluded that there are sub-optimal basic methodology and reporting practices in the laboratory study of human eating behaviour which is likely to affect the validity and replicability of research. For access to the article and the full results click here.

Their findings contribute to the discussion around the  replication crisis in Psychology and the need for researchers to be as transparent as possible in their reporting and methods. Also, as aspiring academics it is a reminder to read the full article and not just the abstracts!

References