Research data management at Elsevier: Supporting networks of data and workflows

2.1.Data should be stored

First of all it is imperative that data not be thrown away. Data management plans aim to ensure that research groups define the ways to store their datasets in advance of their experiments. Use or simple programs like Word or Excel will allow researchers to store information about their experiment. Additionally, new technologies such as electronic lab notebooks present a viable option for storing the observations and results of experiments.

As an example of efforts to encourage electronic data storage, we worked with a neuroscience lab at Carnegie Mellon University to develop a tailored Electronic Lab Notebook to allow metadata to be stored during data creation [30]. Similarly, the Hivebench Electronic Lab notebook (http://hivebench.com) was developed with the goal of sharing protocols in life science settings (note: Hivebench was acquired by Elsevier on June 1, 2016).

2.2.Data should be preserved

Information can only be valuable when it is in a format that we can use in the long term. Even if there are no measures in place to ensure that data were properly preserved, data can in some cases still be rescued.

Elsevier collaborated on a number of different projects to ‘rescue’ data sets and save them from oblivion. In a collaboration with Carnegie Mellon University, we helped pilot the Olive Executable Archive project [27] to bring a number of arcane operating systems back to life as Virtual Machines. The user can load these locally and view the software and data created within the old systems, ensuring that these historical collections which were created in an obsolete operating system can still be accessed. As a further example of an effort in data preservation, we worked together with the Lamont–Doherty Research Center at Columbia University in New York to run the International Data Rescue Award in the Geosciences, an initiative that encourages and rewards efforts to preserve data that were not initially saved for the long [14]. Organizations such as the Data Archiving and Networked Services, DANS (http://www.dans.knaw.nl/en) in the Netherlands, provide an infrastructure for data preservation. Mendeley Data (http://data.mendeley.com) has teamed up with DANS to ensure long-term preservation of their Datasets (https://blog.mendeley.com/2015/11/09/put-your-research-data-online-with-mendeley-data/).

2.3.Data should be accessible

If data is stored and preserved, this does not necessarily mean that it is automatically accessible. Both researchers and machines may want to access the data, for example for meta-analyses or other kinds of re-use. Researchers are increasingly being required by their institution or funder to make their data accessible, which has caused researchers to start thinking about solutions.

In Mendeley Data (http://data.mendeley.com) researchers create private data sharing spaces which can be opened to larger communities, or opened up to the wider public. Also, we recently launched an open data pilot (http://www.elsevier.com/about/open-science/research-data/open-data) to make raw research data (as submitted with an article) openly-accessible alongside the article for any web user.

2.4.Data should be discoverable

Elsevier and other publishers support various mechanisms to support the discoverability of datasets, for instance through inclusion of data DOI’s that link to associated data in public databases (an overview is given in http://www.elsevier.com/databaselinking). Recent funding proposals encourage the development of data discoverability. The first of the data Principles drawn up by the FAIR Data group are to make data Findable (as well as, Accessible, Interoperable and Reproducible – see Ref. [33]). Initiatives such as the National Data Service and the Data Discovery Index (http://grants.nih.gov/grants/guide/rfa-files/RFA-HL-14-031.html) aim to provide a data discovery layer over disparate data collections.

In a project co-funded by a National Science Foundation EAGER Grant, Elsevier collaborated with the Carnegie Mellon School of Computer Science to develop superior ways by which to access and query tabular content extracted from articles and imported from research databases (see http://boston.lti.cs.cmu.edu/eager/deusre/ for an overview).

Working within the Research Data Alliance (RDA) Publishing Data Services Working Group (https://rd-alliance.org/groups/rdawds-publishing-data-services-wg.html) we worked to bring together different stakeholders to agree on common standards, combine links from disparate sources, and create the Data-Literature Interlinking Service, a universal, open service for collecting and sharing such links. A prototype of this service is now available (http://dliservice.research-infrastructures.eu/) that demonstrates the population of and access to a graph of dataset–literature links collected from a variety of major data centers, publishers, and research organizations. Next to this prototype, the Working Group has developed ‘Scholix’ (http://www.scholix.org/) see also [5,6], – a high-level interoperability framework for exchanging information about the links between scholarly literature and data. It aims to build an open information ecosystem to understand systematically what data underpins literature and what literature references data, and developed a set of guidelines for these practices, which are aimed to be developed and implemented in a subsequent RDA working group.

2.5.Data should be citable

If datasets are to be promoted to be first-class knowledge assets (and, for example, be counted for researcher’s assessments), better methods of citation of datasets are needed.

FORCE11 has developed a set of principles [15] to describe how data should be cited. Elsevier has been one of the first publishers to implement these FORCE11 principles. In several data repositories datasets now get their own DOIs which can be used to cite the dataset.

2.6.Data should be comprehensible

To enable data reuse it is essential that user understands what units of measurements were used, how the data was collected, and what abbreviations and parameters were used. One way to make sure that the future user understands how a dataset was created is by publishing an extensive description of how the data was collected, analyzed, and otherwise manipulated One way of delivering this description is by writing a so-called ‘data article,’ describing exactly that.

Several publishers now publish specific data journals. Elsevier has developed a new article type, called ‘Research Elements,’ which contain data and software articles. Research Elements can be a part of an existing journal, but we have also started several Open Access journals specifically devoted to data papers. One of these is ‘Data in Brief’ (http://www.journals.elsevier.com/data-in-brief) which allows the author(s) to provide a thorough description of their datasets. For data already published within the article, we have developed a suite of tools to improve data comprehension, such as in-article data visualizations, like interactive plots (http://www.elsevier.com/books-and-journals/content-innovation/iplots). These interactive content elements allow readers to manipulate datasets by for example, hovering over a plot to see the value of a data point or by switching from a graphical to a tabular view to inspect the data in greater detail (for other examples see https://www.elsevier.com/books-and-journals/content-innovation).

2.7.Data should be reviewed

To be trusted (a necessary step towards being reusable) it is important that research data is viewed, at least, and ideally, reviewed. This review can take different forms: a data set might be manually checked for complying with domain standards or having the proper metadata appended to it; and it might be checked automatically to make sure it renders to a standard format or methodology. In other cases, the data might be validated for having a proper description attached as metadata – with which the data can be fully understood and re-used.

In the Open Data Pilot (http://www.elsevier.com/about/open-science/research-data/open-data), reviewers are asked to check that the submitted files are raw data that can be parsed and are commonly used within the relevant domain; for the data journals described in the previous section, data are more thoroughly checked by reviewers of the paper.

2.8.Data should be reproducible

It has been argued that there is a ‘reproducibility crisis’ in research, as many domains find that experimental results are difficult or impossible to reproduce [11,20]. Irreproducibility often originates from missing elements to research data, which are needed in order to achieve the same research results. As an example of this, there is often inadequate reporting of key elements of a dataset or research method. For instance, Vasilevsky et al. found that for example antibodies, model organisms, and other data resources reported in the biomedical literature often lack sufficient detail to enable reproducibility or reuse [32].

We have contributed to, and are actively implementing the guidelines established by the Force11 Resource Identification Initiative, which aims to enable resource identification within the biomedical literature through a pilot study promoting the use of unique Research Resource Identifiers (RRIDs) [3]. By active participation in the Reproducibility Initiative (http://validation.scienceexchange.com/#/), we support efforts to validate key experimental results via independent replication. The group has obtained funding to actually reproduce a series of key experiments and report on the statistical significance of a result when it combines its data with that of the original experiments [31]. Another way in which we explored reproducibility is through the Executable Paper Challenge: an open challenge for projects that showed the execution of a piece of software inside a paper [13].

2.9.Data should be reusable

Only if research data is comprehensible, trusted, and reproducible will other researchers consider reusing it. But one further factor that inhibits reuse is uncertainty of the legal or copyright status of the dataset. Ideally, the legal status of a dataset should be identified at the moment of storage or sharing.

In line with recommendations by, for example, the UK’s Digital Curation Centre [2], we therefore ask all researchers using Mendeley Data (https://data.mendeley.com/) to select a Creative Commons license (https://creativecommons.org/licenses/by/4.0/) under which their data is made available. Similarly, our data and software journals allow for selection of a data license during submission (see for example, https://www.elsevier.com/journals/data-in-brief/2352-3409/open-access-journal).