Open Science: A Shift Away From Standard Practices

An article published in the eighth issue of Honours Review. Honours Review is a publication of students at the University of Groningen, the Netherlands.

As a scientist, you are expected to be open and transparent about the way you practice research.(1) These core values of the scientific enterprise have not changed over the last decades. However, some things have changed. Present-day scientists have excellent tools at their disposal to adhere to these values in a systematic manner as a result of technological improvements. For example, a present-day scientist can, quite literally, with a click of a button, share research data, analyses scripts, and any other relevant study materials. In contrast, scientists of previous eras had to hassle with physical papers to share their work, which usually meant that journals could keep record of only the actual paper rather than of the whole research process.(2)

The importance of openness

Formally put, science is an overarching system that studies nature in a collaborative fashion through scientific discovery. Painting an ideal science is difficult since it is a continuously developing enterprise which faces many challenges.(3) In an ideal scientific world, we are not only open with our research practices, but also with every other aspect of science. “No one scientist or scientific body is the arbiter of truth” (4) – we are more likely to discover the truth by remaining open and transparent.

Openness and transparency are considered to be core values of science,(1) values which support science’s objectives of building a shared body of knowledge and supporting the systematic search for knowledge whose validity does not depend on the particular individual but is open for anyone to check or rediscover.(5) Science is so tightly woven into Western societies that it is almost impossible to imagine a life without it – it has become part of our modern secular world. Indeed, common sense is usually accompanied by scientific knowledge and it seems as if there is a general consensus about science being open, objective, and trusted unconditionally. Not only is there a lack of public questioning of scientific work and does science receive high scores for trust, but most people also have the view that science has led us to a better world in terms of welfare, health, and freedom.(6,7) However, this view should also be nuanced. As the Rathenau Instituut emphasises in their recent reports on trust in science, “science comes under pressure once scientific insights threaten to affect daily life and are given concrete form. As soon as scientists become involved in government policymaking or take commercial assignments then trust in them falls”.(8;3)

Technological developments gave scientists exceptional means by which they could study the human psyche and, thanks to those dedicated scientists, we have made mountainous leaps in our understanding of humans. Indeed, improved technologies, such as more powerful computers and better-equipped laboratories, largely helped scientists work more efficiently, handle more data, and share their work. The driving force of science’s progress, however, is not only the result of improved technologies, but also of scientists working together and deliberating over each other’s work. No scientist would deny that science progresses through open debates and the exchange of thoughts and results. These are the most efficient ways of discovering the truth about nature and can be the source of an ‘eureka’ moment.

If the contemporary scientist chose to adhere to the values of openness and transparency, it would be beneficial for science: It would facilitate replication efforts and large scale meta-analyses in the natural sciences, and it would stimulate sharing large repositories of texts in the humanities. Replication studies can be more easily conducted when scientists have a clear understanding of the original methods and materials, and science would gain much more clarity if scientists were open about their methods, analyses, and data. Moreover, in transparent and open science, all results, rather than only those that show statistical significance, can be accessed and discussed among scientists. The values of openness and transparency are closely related to each other: If a scientist makes data and procedures openly and freely available, this leads to more transparency.

It remains a challenge for scientists to align their day-to-day practice with the core values of science. Although scientific norms (rules which describe appropriate behaviour within science), such as openly sharing data, being transparent about research procedures, and replicating studies for independent verification, are readily accepted by most scientists,(9) increasing evidence suggests that adhering to these norms is not part of scientists’ daily routines.(10) Moreover, in a recent study on data-sharing, Vanpaemel et al. found that the availability of psychological research data is still far away from an open science ideal.(11) After requesting data of 394 psychology papers, only approximately one third of the approached authors shared their data. Although scientists shared their data more often than an earlier estimation suggested,(12) this is still an alarmingly low rate. Why, then, do we not yet have an open and transparent culture within science despite its clear benefits?

It hinges on the academic rewards system

One popular answer to the question of why science is not yet open and transparent is: ‘Because of the academic reward system.’ Scientists are often rewarded for the funding they attract, the amount of studies they publish, the amount of citations they receive, and the sexiness (i.e., novelty and counter intuitiveness) of their research. Currently, journals have the tendency to favour publishing studies that have statistically significant results as opposed to studies that do not.(13) This is especially true for the domains of psychology and psychiatry.(14) For science, both significant and non-significant results are of equal value – but this is often not the case for the individual scientist. For example, it is equally valuable to know that some quirks of the human mind (e.g., memory) differ between some groups of people (e.g., young and older people) but not others (e.g., males and females). Alas, a lack of significant results can lead to a career of short durance since non-significant studies are less likely to be published, and publishing is vital to an academic career. Thus the current system leads to a loss of interesting results which are often ‘filed into a drawer,’ never to be accessed again.

In other words, scientists are rewarded for personal achievements. As a consequence, scientists tend to neglect the core values of science, including openness, transparency, and reproducibility, in order to gain a personal advantage. For example, scientists might prefer to keep their data secret rather than openly sharing them to protect themselves.(2) Scientists are rewarded personally for discovering something, so why would they share their data if someone else could beat them to it using this shared data? (15)

Science, however, does not care about who discovers what, but about finding the truth in an efficient way – it cares about the what. If our goal in science is to discover the truth, then there are clear benefits to endorsing open scientific practices. Open practices not only have the potential to smoothen collaboration, but also to grease the wheels of up-and-coming research practices, such as large scale meta-analyses and reproducibility projects.(16) Taken together, the culture within science seems to rely on a contradiction between its ideal framework that is built around collaboration and its tendency to reward egoism.(4) Scientists are expected to navigate in a web of multiple, contradictory incentives. This begs the question: Is there a way to a culture of openness in science?

The path to openness

There are many ways of creating conditions that help openness flourish because science consists of several layers of hierarchical orders. Firstly, there are the higher-order systems (e.g., top-down), such as the journals, universities, and politics (http://www.openaccess.nl/en). Secondly, there are lower-order systems (e.g., bottom-up), which constitute the contribution of individual scientists.

Two scholars in particular – Daniel Lakens and Felix Schönbrodt – have written about what individual scientists can do to support open science (see http://daniellakens.blogspot.nl/ and http://www.nicebread.de/). Their suggestions boil down to a few things. Firstly, scientists need to be aware of what openness as a core value of science signifies, why more openness is necessary, and where in the research process openness is missing. Scientists also need to be aware of why openness is beneficial for the individual scientist, science, and societies, and who deserves to have open access to scientists’ work (e.g., do we provide open access to people who pay taxes, which are being used for funding of research projects?). Secondly, an important step for individual scientists is to organise their research related materials and to make use of existing initiatives that facilitate open scientific practices, such as the Center for Open Science (COS; see, https://cos.io/). Thirdly, scientists today can already openly share their preprints. Fourthly, scientists need to find the degree of openness they are comfortable with at the beginning because we cannot expect scientists to shift their gears towards openness from one moment to the next. Fifthly, Lakens and Schönbrodt encourage individuals to get involved with pre-registering. The whole idea of pre-registration is to work more efficiently by applying for a so called ‘in-principle acceptance’ of the introduction, method, and analysis plan before actually doing the study. Sixthly, when scientists are in a position in which they evaluate the work of others, such as peer-reviewers, they can foster an open culture by encouraging others to publish with open access as a standard practice. Lastly, one of the biggest reasons that scientists currently do not use open scientific practices is because they have never learned them properly. Teaching openness to young academics is, therefore, crucial.

Conclusion

Open access has the potential to polish the window through which the workings of human nature are observed. Science is a collaborative enterprise, and it is important to remind ourselves of scientific values, especially openness and transparency. The Dutch government even has this as a policy instrument in order to have “maximum impact in society and industry”.(8) However, the current academic system predominantly rewards individual accomplishments as opposed to open debates. This has the inevitable consequence of fuelling egotistical tendencies, such as striving for a great quantity of publications and citations. But engaging in open scientific practices increases our knowledge as a common good, and various ways of encouraging them exists for larger institutions as well as individual scientists. Indeed, openly exchanging ideas and data is the most useful way of practicing science in the long run – a rising tide lifts all boats.

Notes

The Rathenau Instituut, an organisation formed by the Dutch government in 1978, “promotes the formation of political and public opinion on science and technology”. To achieve this, the Institute studies among others the organisation and development of science systems. Moreover, the institute publishes about social impact of new technologies, and organises debates on issues and dilemmas in science and technology.

The contents of this article proved controversial: ‘science’, ‘openness’ and ‘transparency’ are terms that need to be defined clearly. The current article does not follow the normative understanding of science in the line of Karl Popper: As science as a means to explain how nature works, which is not necessarily the discovery of truth. A subtle but im­portant difference the reader might want to reflect on.

References

1. Merton, Robert K. 1942. “A note on science and democracy.” Journal of Legal and Political Sociology. 1: 115-126.

2. Morey, Richard D.; Chambers, Christopher D.; Etchells, Peter J.; Harris, Christine R.; Hoekstra, Rink et al. 2016. “The Peer Reviewers’ Openness Initiative: Incentivising Open Research Practices through Peer Review.” Royal Society Open Science. 3: 150547.

3. Belluz, Julia; Plumer, Brad Resnick, Brian. 2016. “The 7 Biggest Problems Facing Science, According To 270 Sci­entists.” Vox. Retrieved from http://www.vox.com/2016/7/14/12016710/science-challeges-research-fund­ing-peer-review-process. Accessed Dec. 3, 2016.

4. Nosek, Brian A. & Bar-Anan, Yoav. 2012. “Scientific Communication Is Changing and Scientists Should Lead the Way.” Psychological Inquiry. 23: 308-314.

5. “Science and Pseudo-Science.” Stanford Encyclopedia of Philoso­phy. 2014. http://plato.stanford.edu/entries/pseudo-science/. Accessed Dec. 3, 2016.

6. Toulmin, Stephen. 1990. Cosmopolis: The Hidden Agenda of Modernity. New York: The Free Press.

7. Bem, Sacha & De Jong, Huib L. 2013. Theoretical issues in psychology: An introduc­tion. 3rd edition. 1-4. Thousand Oaks, CA, US: Sage Publications, Inc..

8 de Jonge, Jos. 2016. “Trust in Science in the Netherlands 2015”. The Hague: Rathenau Instituut. https://www.rathenau.nl/en/publication/trust-sci­ence-netherlands-2015. Accessed Dec. 3, 2016.

9. Miguel, E.; Camerer, C.; Casey, K.; Cohen, J.; Esterling, K. M. et al. 2014. “Promoting Transparency in Social Science Research.” Science. 343: 30-31.

10. Anderson, Melissa S.; Martinson, Brian C.; & De Vries, Raymond. 2007. “Normative Dissonance in Science: Results from a National Survey of U.S. Scientists.” Journal of Empirical Research on Human Research Ethics. 2: 3-14.

11. Vanpaemel, Wolf; Vermorgen, Maarten; Deriemaecker, Leen & Storms, Gert. 2015. “Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm.” Collabra. 1: 1-5.

12. Wicherts, Jelte M.; Borsboom, Denny; Kats, Judith & Molenaar, Dylan. 2006. “The poor availability of psychological research data for reanalysis.” American Psychologist. 61: 726-728.

13. Song, Fujian; Parekh-Bhurke, Sheetal; Hooper, Lee; Loke, Yoon K.; Ryder, Jon J. et al. 2009. “Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies.” BMC Medical Research Methodology. 9: 79.

14. Fanelli, Daniele. 2010. “‘Positive’ Results Increase Down the Hierarchy of the Sciences.” PLoS ONE. 5: e10068.

15. TU/e symposium Open Access 2015 - morning sessions: Workshop 1 Daniel Lakens. Lakens, Daniel. 2015-4-16. https://www.youtube.com/watch?v=O_x7EPsGnjk. Accessed Dec. 3, 2016.

16. Aleksic, Jelena; Alexa, Adrian; Attwood, Teresa K.; Hong, Neil C.; Dahlö, Martin et al. as part of the AllBio: Open Science & Reproducibility Best Practice Workshop. 2014. “An Open Science Peer Review Oath”. F1000Research 3, 271.