cognitive science
and more
Taking The High Road to publication: my experience with pre-prints and data sharing

Scientists should do lots of things. Just search for #openscience on Twitter. It’s buzzing with reform!

Scientists should make their papers freely available, and no longer hide them behind the paywalls that are put up by commercial publishers. They should make their datasets available, so that analyses can be independently verified. They should post their ongoing work to pre-print servers (as many from the exact sciences already do), where it can be discussed, shared, and debated without unnecessary publication delays. They should spend more time replicating each others findings. They shouldn’t care about journal impact factors, but judge quality on a per-manuscript basis, using altmetrics. And the list goes on!

Sharing can be a little scary at first. But a young generation of scientists is doing it more and more. (Source)

But despite all the buzz, actual scientific practice has hardly changed. And I’m at fault here as much as anyone. I’ve written a few blogs on the subject, but I haven’t really conducted much #openscience at all. So, with our new-years resolutions fresh in mind, my colleagues and myself decided to put our money where our mouth is, and take the ‘high road’ to what will hopefully become our next publication.

As a first step, I posted all experimental materials to a GitHub repository. You can think of GitHub as a public DropBox with infinite history. All versions of all files within a specific project remain available, and you can easily inspect the changes from one version to the next. Versioning systems like GitHub are mostly used by software developers for whom they are indispensable. But versioning is also useful when writing papers and conducting analyses. After all, in many ways doing science is similar to software development.

I have been making my datasets publicly available for a while now, but this is the first time that I’ve also uploaded the analysis pathway. I’ve found that doing so is really helpful, because the knowledge that my scripts were going to be public forced me to keep them well organized–and presumably less buggy. Another benefit is that papers for which data is available are cited more often than papers for which it is not. One can debate causality here, but I found this correlation quite interesting.

Next, I uploaded the latest draft of our manuscript to a pre-print server. Although the paper is in a good state, it will surely undergo revision and it may take some time before it is actually published in a journal. Posting a pre-print prior to publication is a way to bypass these delays, at least partly, and to communicate your latest research quickly. And who knows, you may receive valuable feedback on a pre-print. For me, posting a public pre-print serves the same purpose as discussing my research at a conference, or sending a manuscript draft to colleagues by e-mail.

There are a number of pre-print servers. The best-known ones are ArXiv, FigShare, PeerJ PrePrints, and F1000 Research. We chose PeerJ PrePrints, for no particular reason. I had assumed that I could simply upload the manuscript and that it would be immediately available on the website. But PeerJ PrePrints actually takes things a bit more seriously than that. There is a submission process that requires you to provide information about funding, ethics, author contributions, etc. The manuscript is also checked by a PeerJ employee to make sure that it is a serious pre-print. This check is superficial, of course, but I think it may help to avoid pre-print servers from becoming a dump for academic waste material. Also, I commend PeerJ on the user-friendliness of their web interface. It’s hands down the best submission system I’ve seen.

I should point out that not all publishers accept manuscripts that have been previously made available as a pre-print. Some do, some don’t. Usually you can find a pre-publication policy on the journal’s website. (Tip by PeerJ: see also this Wikipedia page.) If there isn’t any, I think the safest assumption is that they don’t take kindly to pre-prints. So it is important to check beforehand whether your favorite journal is compatible with a pre-print. If not, you should reconsider whether you really want to post a pre-print, or reconsider your favorite journal. I suggest the latter.

Currently, we are also in the traditional process of publishing our manuscript. This will take time and a lot can happen before then. But at least we got our work out there quickly. In preliminary form, but in its entirety! Waiting for your harsh comments …

You can find the pre-print here:

Mathôt S, van der Linden L, Grainger J, Vitu F. (2014) The pupillary light response reflects eye-movement preparation. PeerJ PrePrints, 2, e238v1 doi:10.7287/peerj.preprints.238v1">doi:10.7287/peerj.preprints.238v1