Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research
Author
Superchi, Cecilia
Hren, Darko
Blanco de Tena Davila, David
Rius, Roser
Recchioni, Alessandro
Boutron, Isabelle
González, José Antonio
Publication date
2020ISSN
2044-6055
Abstract
Objective: To develop a tool to assess the quality of peer-review reports in biomedical research. Methods We conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer-review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5-point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis. Results: A total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle-aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer-review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1–5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer’s comments). Conclusion: Assessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers’ work, and also as an outcome when evaluating interventions to improve the peer-review process.
Document Type
Article
Document version
Published version
Language
English
Subject (CDU)
61 - Medical sciences
Keywords
Investigació biomèdica
Editors biomèdics
Eines per avaluar la qualitat dels informes
Investigación biomédica
Editores biomédicos
Herramientas para evaluar la calidad de los informes
Biomedical research
Biomedical editors
Tools for evaluating the quality of reports
Pages
10
Publisher
BMJ Publishing Group
Collection
10;
Is part of
BMJ Open
Citation
Superchi, Cecilia; Hren, Darko; Blanco de Tena Davila, David [et al.]. Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research. BMJ Open, 2020, 10, e035604. Disponible en: <https://bmjopen.bmj.com/content/10/6/e035604>. Fecha de acceso: 15 sep. 2021. DOI: 10.1136/bmjopen-2019-035604
This item appears in the following Collection(s)
- Ciències de la Salut [738]
Rights
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by/4.0/