Abstract
Scientific editors’ policies, including peer review, are based mainly on tradition and belief. Do they actually achieve their desired effects, the selection of the best manuscripts and improvement of those published? Editorial decisions have important consequences—to investigators, the scientific community, and all who might benefit from correct information or be harmed by misleading research results. These decisions should be judged not just by intentions of reviewers and editors but also by the actual consequences of their actions. A small but growing number of studies has put editorial policies to a strong scientific test. In a randomized, controlled trial. blinding reviewers to author and institution was usually successful and improved the quality of reviews. Two studies have shown that, contrary to conventional wisdom, reviewers early in their careers give better reviews than senior reviewers. Many studies have shown low agreement between reviewers but there is disagreement about whether this represents a failing of peer review or the expected and valuable effect of choosing reviewers with complementary expertise. In a study of whether manuscripts are improved by peer review and editing, articles published in Annals of Internal Medicine were improved in 33 of 34 dimensions of reporting quality, but published articles still had room for improvement. Because of the central place of peer review in the scientific community and the resources it requires, more studies are needed to define what it does and does not accomplish.
Similar content being viewed by others
References
Lock S. & Wells F. (1993) eds. Fraud and misconduct in medical research, BMJ Publishing Group, London.
Schneiderman, L. J., Kronick, R., Kaplan, R. M., Anderson, J. P. & Langer, R. D. (1992) Effects of offering advance directives on medical treatments and costs. Annals of Internal Medicine 117: 599–606.
Patterson, K., Bailar, J. C. III (1985) A review of journal peer review. In: Warren, K.S., ed. Selectivity in Information Systems: Survival of the Fittest. Praeger, New York.
Lock, S. (1985) A difficult balance. Editorial peer review in medicine. ISI Press, Philadelphia.
Fletcher, R. H. & Fletcher, S. W. (1992) Medical journals and society: threats and responsibilities. Journal of Internal Medicine 232: 215–221.
Guarding the guardians: Research on editorial peer review. Selected proceedings from the First International Congress on Peer Review in Biomedical Publication (1990) Journal of the American Medical Association 263: 1309–1456.
The Second International Congress on Peer Review in Biomedical Publication (1994) Journal of the American Medical Association 272: 91–170.
Chubin, D. E. & Hackett, E. J. (1990) Peerless science. Peer review and U.S. science policy. State University of New York Press, Albany.
Lock, S. (1991) ed. The future of medical journals. British Medical Journal, London.
Daniel, H.-D. (1993) Guardians of Science. Fairness and Reliability of Peer Review. VHC, Weinheim.
Speek, B. W. (1993) ed. Publication peer review. An annotated bibliography. Greenwood Press, Westport, CT.
Rennie, D. & Flanigin, A. (1994) Authorship! Authorship! Guests, ghosts, grafters and the two-sided coin. Editorial, Journal of the American Medical Association 271: 469–471.
Fletcher, R. H. & Fletcher, S. W. (1993) Who’s responsible? Editorial, Annals of Internal Medicine 118: 645–646.
Chalmers, A. F. (1982) What is this thing called science? University of Queensland Press, New York.
Fletcher, S. W. (1991) Research agenda for medical journals. In: Lock, S. ed. The future of medical journals. British Medical Journal, London.
McNutt, R. A., Evans, A. T., Fletcher, R. H. & Fletcher, S. W. (1990) The effects of blinding on the quality of peer review. Journal of the American Medical Association 263: 1371–1376.
Fisher, M., Friedman, S. B. & Strauss, B. (1994) The effects of blinding on acceptance of research papers by peer review. Journal of the American Medical Association 272: 143–146.
Yankauer, A. (1991) How blind is blind review? American Journal of Public Health 81: 843–845.
Blank, R. M. (1991) The effects of double-blind versus single-blind reviewing: Experimental evidence from The American Economic Review 81: 1041–1067.
Stossel, T. P. (1985) Reviewer status and review quality. Experience of the Journal of Clinical Investigation. New England Journal of Medicine 312: 658–659.
Evans, A. T., McNutt, R. A., Fletcher, S. W. & Fletcher, R. H. (1993) The characteristics of peer reviewers who produce good-quality reviews. Journal of General Internal Medicine 8: 422–428.
Goodman, S. N., Berlin, J., Fletcher, S. W. & Fletcher, R. H. (1994) Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Annals of Internal Medicine 121: 11–21.
Roberts, J.C., Fletcher, R. H. & Fletcher, S. W. (1994) Effects of peer review and editing on the readability of articles published in Annals of Internal Medicine. Journal of the American Medical Association 272: 119–121.
Bailar, J. C. III & Patterson, K. (1985) Journal peer review: The need for a research agenda. New England Journal of Medicine 312: 654–657.
Uniform requirements for manuscripts submitted to biomedical journals. International Committee of Medical Journal Editors. JAMA 1993; 269: 282–286.
Author information
Authors and Affiliations
Additional information
This paper is based on a presentation at a workshop, “Advances in Peer Review Research”, American Association for the Advancement of Science Meeting, Baltimore, MD, February 9, 1996.
Rights and permissions
About this article
Cite this article
Fletcher, R.H., Fletcher, S.W. Evidence for the effectiveness of peer review. SCI ENG ETHICS 3, 35–50 (1997). https://doi.org/10.1007/s11948-997-0015-5
Received:
Revised:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11948-997-0015-5