Relevance

Statistical testing of null hypotheses is a daily routine for many researchers. Researchers often do not sufficiently realize that with null hypothesis significance testing (NHST) they do not draw conclusions about their data directly, but indirectly. As a result, conclusions from scientific research are often incorrect. Methodologists have been pointing this out for years. Nevertheless, the practice of NHST only seems to be increasing, while there are a number of better alternatives available. We wonder which perceptions or beliefs are at the base if this.

The Study

In this study, we have, by means of interviews and focus groups, investigated researchers’ views on the use of NHST or alternative methods and what they see as the advantages and disadvantages of these methods. We also identified researchers who are, or want to be, pioneers in promoting the use of better alternatives to NHST. Together, we are developing strategies to implement these alternatives into scientific research.

Objectives of the project

Our study consists of two parts. In the first part we aim to gain insight into the views of stakeholders on the use of Null Hypothesis Significance Testing. The aim is to gain knowledge about the perceptions of stakeholders of null hypothesis significance testing and to map their motives for using this method. With the second part of the study, we aim to develop strategies that will overcome the default use of Null Hypothesis Significance Testing and that will stimulate use of better alternatives.

 

In the first part of the study, we held interviews with representatives of several stakeholder groups from within the science system: researchers, editors, lecturers of statistics classes and funding agencies. The results of these interviews served as input for focus groups with the same groups of stakeholders. In the focus groups, alternatives to null hypothesis significance testing were further explored and concretized.

 

In the second part of the research, a search conferences was held in which researchers discussed strategies to change current ubiquitous use of NHST in science and higher education.

Timeline

Spring 2017: Application approved

August-November 2017: Rewrite protocol and registration

November 2017- June 2018: Interviews

September-December 2018: Focus groups

March 2019: Search conference

April-December 2019: data analysis and reporting