Published Versions 2 Vol 2 (1) : 285–292 2019
Download
Considerations for the Conduction and Interpretation of FAIRness Evaluations
95 3 0
Abstract & Keywords
Abstract: The FAIR principles were received with broad acceptance in several scientific communities. However, there is still some degree of uncertainty on how they should be implemented. Several self-report questionnaires have been proposed to assess the implementation of the FAIR principles. Moreover, the FAIRmetrics group released 14, general-purpose maturity for representing FAIRness. Initially, these metrics were conducted as open-answer questionnaires. Recently, these metrics have been implemented into software that can automatically harvest metadata from metadata providers and generate a principle-specific FAIRness evaluation. With so many different approaches for FAIRness evaluations, we believe that further clarification on their limitations and advantages, as well as on their interpretation and interplay should be considered.
Keywords: FAIR metrics; FAIR principles; FAIR data; FAIRness; Implementation; Maturity indicators; Scientific community
Acknowledgements
M. Dumontier was supported by grants from NWO (400.17.605;628.011.011), NIH (3OT3TR002027-01S1; 1OT3OD025467-01; 1OT3OD025464-01), H2020-EU EOSClife (824087), and ELIXIR, the research infrastructure for life-science data. R. de Miranda Azevedo was supported by grants from H2020-EU EOSClife (824087), and ELIXIR, the research infrastructure for life-science data.
[1]
M.D. Wilkinson, M. Dumontier, I. Jan Aalbersberg, G. Appleton, M. Axton, A. Baak, N. Blomberg … & B. Mons. The FAIR Guiding Principles for scientific data management and stewardship. Nature 3(2016), 160018. doi:10.1038/sdata.2016.18.
[2]
D. Spichtinger & J. Siren. The Development of Research Data Management Policies in Horizon 2020. Research Data Management – A European Perspective. 2017. doi: 10.1515/9783110365634-002.
[3]
FAIR principles & FAIRmetrics – ZonMw. Available: https://www.zonmw.nl/en/research-and-results/fair-data-data-management/data-management-at-zonmw/fair-principles-fairmetrics/.
[4]
T. Hill. Turning FAIR into reality: Review. Learned Publishing 3(32) (2019), 283-286. doi: 10.1002/leap.1234.
[5]
B. Mons, C. Neylon, J. Velterop, M. Dumontier, L.O. Bonino da Silva Santos & M.D. Wilkinson. Cloudy, increasingly FAIR; revisiting the FAIR Data guiding principles for the European Open Science Cloud. Information Services & Use 37(1) (2017), 49–56. doi: 10.3233/ISU-170824.
[6]
M.D. Wilkinson, S.A. Sansone, E. Schultes, P. Doorn, L.O. Bonino da Silva Santos & M. Dumontier. A design framework and exemplar metrics for FAIRness. Scientific Data 5(2018), 180118. doi: 10.1038/sdata.2018.118.
[7]
5 Star Data Rating tool – OzNome. Available at: https://research.csiro.au/oznome/tools/oznome-5-star-data/.
[8]
FAIR data self-assessment tool. Available at: https://www.ands.org.au/working-with-data/fairdata/fair-data-self-assessment-tool.
[9]
FAIR data assessment tool-Data Impact Blog. Available at: http://blog.ukdataservice.ac.uk/fair-data-
assessment-tool/.
[10]
J. Daniel, B. Clarke, L.L. Wang, A. Jones, P.L. Wojciechowicz, D. Torre, K.M. Jagodnik … & A. Ma’ayan. FAIRshake: Toolkit to evaluate the findability, accessibility, interoperability, and reusability of research digital resources. bioRxiv 2019. Availabe at: https://www.biorxiv.org/content/10.1101/657676v1.
[11]
R. David. How to operationalize and to evaluate the FAIRness in the crediting and rewarding processes in data sharing: A first step towards a simplified assessment grid. Available at: https://hal.archives-ouvertes.fr/hal-01943521/.
[12]
GARDIAN FAIR Metrics. Available at: https://gardian.bigdata.cgiar.org/metrics.php#!/.
[13]
M.D Wilkinson, M. Dumontier, S.A. Sansone, L. O. Bonino da Silva Santos, M. Prieto, P. McQuilton … & E. Schultes. Evaluating FAIR-compliance through an objective, automated, community-governed framework. bioRxiv. 2018. Available at: https://www.biorxiv.org/content/10.1101/418376v2.abstract.
[14]
Results of an analysis of existing FAIR assessment tools. Available at: https://www.rd-alliance.org/group/fair-data-maturity-model-wg/outcomes/results-analysis-existing-fair-assessment-tools.
[15]
FAIRassist.org. Available at: http://fairassist.org.
[16]
Checklist to evaluate FAIRness of data(sets). Available at: https://docs.google.com/forms/d/e/1FAIpQLSf7t1Z9IOBoj5GgWqik8KnhtH3B819Ch6lD5KuAz7yn0I0Opw/viewform.
[17]
Manual FAIRness assessment. Available at: https://docs.google.com/spreadsheets/d/1mJgYmiyLQdXrR_CWF6Al9z_1pgp9R8JXfMwhXBOB6E0/edit?usp=sharing.
[18]
The FAIR maturity evaluation service. Available at: https://w3id.org/AmIFAIR.
[19]
D. Butler. Scientists: Your number is up. Nature 485(7400) (2012), 564. doi:10.1038/485564a.
[20]
S.A. Sansone, P. McQuilton & P. Rocca-Serra. FAIRsharing: Working with and for the community to describe and link data standards, repositories and policies. bioRxiv, 2018. Available at: https://www.biorxiv.org/content/10.1101/245183v1.abstract.
[21]
N. Juty, C. Laibe & N. Le Novere. Identifiers.org/MIRIAM Registry annotation and cross-referencing frame-work. Nature Proceedings. 2011. doi: 10.1038/npre.2011.6396.1.
[22]
P.L. Whetzel, N.F. Noy, N.H. Shah, P.R. Alexander, C. Nyulas, T. Tudorache & M.A. Musen. BioPortal: Enhanced functionality via new Web services from the National Center for Biomedical Ontology to access and use ontologies in software applications. Nucleic Acids Research 39(2011), W541–5. doi: 10.1093/nar/gkr469.
Article and author information
Cite As
R. de Miranda Azevedo & M. Dumontier. Considerations for the conduction and interpretation of FAIRness evaluations. Data Intelligence 2(2020), 285–292. doi: 10.1162/dint_a_00051
Ricardo de Miranda Azevedo
R.de Miranda Azevedo (r.demirandaazevedo@maastrichuniversity.nl) took the lead in writing themanuscript.
r.demirandaazevedo@maastrichuniversity.nl
Ricardo de Miranda Azevedo is an independent and pragmatic researcher with a backgroundin psychology and clinical epidemiology who is happy to work in multidisciplinary settings. He isa great multilingual communicator who is enthusiastic about scientific research but also forcomputers and teaching different topics. He is often labeled as a creative mind with a problemsolver attitude.
0000-0002-7641-6446
Michel Dumontier
M. Dumontier (michel.dumontier@maastrichuniversity.nl) supervised and reviewed themanuscript.
Michel Dumontier is the Distinguished Professor of Data Science at Maastricht University andco-founder of the FAIR (Findable, Accessible, Interoperable and Reusable) data principles. Hisresearch focuses on the development of computational methods for scalable and responsiblediscovery science. Previously at Stanford University, Dr. Dumontier now leads the interfacultyInstitute of Data Science at Maastricht University to develop socio-technological systems foraccelerating scientific discovery, improving human health and well-being, and empoweringcommunities with ethical data-driven decision making.
0000-0003-4727-9435
Publication records
Published: None (Versions2
References
Data Intelligence