Researchers are often disappointed when their work yields "negative" results, meaning that the null hypothesis cannot be rejected. However, negative results are essential for research to progress. Negative results tell researchers that they are on the wrong path, or that their current techniques are ineffective. This is a natural and necessary part of discovering something that was previously unknown. Solving problems that lead to negative results is an integral part of being an effective researcher. Publishing negative results that are the result of rigorous research contributes to scientific progress.
There are three main reasons for negative results:
- The original hypothesis was incorrect
- The findings of a published report cannot be replicated
- Technical problems
Here, we will discuss how to write about negative results, first focusing on the most common reason: technical problems.
Writing about technical problems
Technical problems might include faulty reagents, inappropriate study design, and insufficient statistical power. Most researchers would prefer to resolve technical problems before presenting their work, and focus instead on their convincing results. In reality, researchers often need to present their work at a conference or to a thesis committee before some problems can be resolved.
When presenting at a conference, the objective should be to clearly describe your overall research goal and why it is important, your preliminary results, the current problem, and how previously published work is informing the steps you are taking to resolve the problem. Here, you want to take advantage of the collective expertise at the conference. By being straightforward about your difficulties, you increase the chance that someone can help you find a solution.
When presenting to a thesis committee, much of what you discuss will be the same (overall research goal and why it is important, results, problem(s) and possible solutions). Your primarily goal is to show that you are well prepared to move forward in your research career, despite the recent difficulties. The thesis defense is a defined stopping point, so most thesis students should write about solutions they would pursue if they were to continue the work. For example, "To resolve this problem, it would be advisable to increase the survey area by a factor of 4, and then…" In contrast, researchers who will be continuing their work should write about possible solutions using present and future tense. For example, "To resolve this problem, we are currently testing a wider variety of standards, and will then conduct preliminary experiments to determine…"
Putting the "re" in "research"
Whether you are presenting at a conference, defending a thesis, applying for funding, or simply trying to make progress in your research, you will often need to search through the academic literature to determine the best path forward. This is especially true when you get unexpected results—either positive or negative. When trying to resolve a technical problem, you should often find yourself carefully reading the materials and methods sections of papers that address similar research questions, or that used similar techniques to explore very different problems. For example, a single computer algorithm might be adapted to address research questions in many different fields.
In searching through published papers and less formal methods of communication—such as conference abstracts—you may come to appreciate the important details that good researchers will include when discussing technical problems or other negative results. For example, "We found that participants were more likely to complete the process when light refreshments were provided between the two sessions." By including this information, the authors may help other researchers save time and resources.
Thus, you are advised to be as thorough as possible in reviewing the relevant literature, to find the most promising solutions for technical problems. When presenting your work, show that you have carefully considered the possibilities, and have developed a realistic plan for moving forward. This will help a thesis committee view your efforts favorably, and can also convince possible collaborators or advisors to invest time in helping you.
Publishing negative results
Negative results due to technical problems may be acceptable for a conference presentation or a thesis at the undergraduate or master's degree level. Negative results due to technical problems are not sufficient for publication, a Ph.D. dissertation, or tenure. In those situations, you will need to resolve the technical problem and generate high quality results (either positive or negative) that stand up to rigorous analysis. Depending on the research field, high quality negative results might include multiple readouts and narrow confidence intervals.
Researchers are often reluctant to publish negative results, especially if their data don't support an interesting alternative hypothesis. Traditionally, journals have been reluctant to publish negative results that are not paired with positive results, even if the study is well designed and the results have sufficient statistical power. This is starting to change—especially for medical research—but publishing negative results can still be an uphill battle.
Not publishing high quality negative results is a disservice to the scientific community and the people who support it (including tax payers), since other scientists may need to repeat the work. For studies involving animal research or human tissue samples, not publishing would squander significant sacrifices. For research involving medical treatments—especially studies that contradict a published report—not publishing negative results leads to an inaccurate understanding of treatment efficacy.
So how can researchers write about negative results in a way that reflects its importance? Let's consider a common reason for negative results: the original hypothesis was incorrect.
Writing about negative results when the original hypothesis was incorrect
Researchers should be comfortable with being wrong some of the time, such as when results don't support an initial hypothesis. After all, research wouldn't be necessary if we already knew the answer to every possible question. The next step is usually to revise the hypothesis after reconsidering the available data, reading through the relevant literature, and consulting with colleagues.
Ideally, a revised hypothesis will lead to results that allow you to reject a (revised) null hypothesis. The negative results can then be reported alongside the positive results, possibly bolstering the significance of both. For example, "The DNA mutations in region A had a significant effect on gene expression, while the mutations outside of domain A had no effect. Don't forget to include important details about how you overcame technical problems, so that other researchers don't need to reinvent the wheel.
Unfortunately, it isn't always possible to pair negative results with related positive results. For example, imagine a year-long study on the effect of COVID-19 shelter-in-place orders on the mental health of avid video game players compared to people who don't play video games. Despite using well-established tools for measuring mental health, having a large sample size, and comparing multiple subpopulations (e.g. gamers who live alone vs. gamers who live with others), no significant differences were identified. There is no way to modify and repeat this study because the same shelter-in-place conditions no longer exist. So how can this research be presented effectively?
Writing when you only have negative results
When you write a scientific paper to report negative results, the sections will be the same as for any other paper: Introduction, Materials and Methods, Results and Discussion. In the introduction, you should prepare your reader for the possibility of negative results. You can highlight gaps or inconsistencies in past research, and point to data that could indicate an incomplete understanding of the situation.
In the example about video game players, you might highlight data showing that gamers are statistically very similar to large chunks of the population in terms of age, education, marital status, etc. You might discuss how the stigma associated with playing video games might be unfair and harmful to people in certain situations. You could discuss research showing the benefits of playing video games, and contrast gaming with engaging in social media, which is another modern hobby. Putting a positive spin on negative results can make the difference between a published manuscript and rejection.
In a paper that focuses on negative results—especially one that contradicts published findings—the research design and data analysis must be impeccable. You may need to collaborate with other researchers to ensure that your methods are sound, and apply multiple methods of data analysis.
As long as the research is rigorous, negative results should be used to inform and guide future experiments. This is how science improves our understanding of the world.
