Description: First edition. | Hoboken, NJ : John Wiley & Sons, Inc., 2021. | Includes bibliographical references and index.
Identifiers: LCCN 2020051750 (print) | LCCN 2020051751 (ebook) | ISBN 9781119794141 (paperback) | ISBN 9781119794189 (adobe pdf) | ISBN 9781119794196 (epub)
Subjects: MESH: Research Design–standards | Randomized Controlled Trials as Topic | Systematic Reviews as Topic | Evidence‐Based Practice–standards | Treatment Outcome
Classification: LCC R852 (print) | LCC R852 (ebook) | NLM W 20.5 | DDC 610.72–dc23
LC record available at https://lccn.loc.gov/2020051750 LC ebook record available at https://lccn.loc.gov/2020051751
Cover Design: Wiley
Cover Image: © Govindanmarudhai/DigitalVision Vectors/Getty Images
Preface
Evidence is central to the practice of medicine. The amount of published medical evidence is immense, but there are widespread concerns about the quality of many studies. Deficiencies in the conduct of research studies often result in misleading estimates of the benefits of treatments, so that ineffective treatments may be used in clinical practice. In addition, many studies are wasted because they have been so poorly designed and conducted.
This book explores the nature of the deficiencies and flaws in the evidence about the effectiveness of treatments. It is the result of a career‐long interest in medical evidence. A recent career change provided an opportunity to read the extensive literature on research quality that has emerged in recent years. As befits a book on evidence, references are cited to support the statements made. Reviews and commentaries are used where available, although many landmark studies are also referenced. The approach taken is to cite sufficient papers to support a point, rather than give a comprehensive a review of it. The number of references cited reflect the wealth of evidence on the deficiencies in medical research.
I am grateful to the University of Dundee for providing the facilities to research and write this book, and to my colleagues for their support and encouragement. Special thanks are due to my long‐time colleague and friend Linda Irvine: she made several gently delivered, trenchant criticisms of early drafts and picked out factual, logical and grammatical failings in later drafts. The irony of writing a book about evidence is that it may contain errors of fact or interpretation. Equally there may be errors of omission. I take full responsibility for all flaws.
Aims of this Book
The advent of the randomised controlled trial (RCT) provided a method to generate reliable evidence on the effectiveness of treatments. It enables a fair comparison of groups [1], and should identify which treatments are beneficial and which are of little or no value. Since the early randomised controlled trials (RCTs) of the mid‐twentieth century, the design of trials has been progressively refined, making them the bedrock on which modern medicine is built. The findings of RCTs are used by regulatory authorities across the world to licence effective treatments. Other methods of testing treatments have been proposed, but none matches the ability of high quality randomised trials to provide good estimates of treatment effectiveness [2]. The RCT deserves its status as the gold standard for assessing the effectiveness of treatment.
The RCT now occupies a leading place in medical research, with tens of thousands of trials being published annually [3, 4]. These studies should provide high quality evidence across all fields of medicine, but that promise has not been fulfilled. At issue is the quality of the evidence. Concerns are growing ‘about the reliability and validity of the underlying research that supports regulatory and clinical decision‐making’ [5], with authors describing ‘the pervasiveness of poor quality clinical evidence’ [6] and concluding that ‘much of the published medical research is apparently flawed, cannot be replicated and/or has limited or no utility’ [7]. In a particularly trenchant comment, Ian Roberts and colleagues concluded that ‘the knowledge system underpinning healthcare is not fit for purpose and must change’ [8]. This book evaluates that proposition.
To investigate the knowledge system for healthcare, this book poses five questions: 1) what are the problems; 2) how common are they; 3) to what extent do they bias evidence; 4) why do they occur, and 5) how can they be prevented? Medical research is a vast enterprise, so the book focuses on the two most important research methods: the randomised controlled trial and the systematic review. Answers to the first three questions will provide a broad assessment of the quality of evidence (Chapters 2–5). The answer to the fourth question, on the causes of poor quality, highlights issues around misconduct and how the structures and incentives in the research environment influence the quality of evidence (Chapters 6 and 7). The final chapter presents an approach for developing a comprehensive strategy to the quality problem. This is supported by an Appendix, which lists the initiatives that have been proposed to improve research evidence. First, to introduce the nature of evidence in medicine, Chapter 1 provides a brief review of the rationale for treatments from ancient times to the present day.
This book focuses on the flaws in medical evidence. It does not review the important findings from the many high quality studies that have provided convincing evidence on the effectiveness of a wide range of treatments. Instead, the book describes the variety of deficiencies that afflict much research. This assessment could lead to an overly negative assessment of the state of medical evidence. The reality is that clinical research is spread across a spectrum from high to very low quality studies. The aim of the book is to use a detailed review of the flaws and their causes, to develop a strategy to prevent poor quality and misleading research. If implemented, the strategy could prevent some of the more egregious studies from being published, and rectify some with less reprehensible weaknesses. In this way, it could help move the spectrum upwards in quality. The book should be read from a constructive perspective of what is going wrong and how can we change it, in which the focus on flaws also provides the motivation for action.
REFERENCES
1 1. Chalmers, I. (2011). Why the 1948 MRC trial of streptomycin used treatment allocation based on random numbers. J. R. Soc. Med. 104: 383–386.
2 2. Byar, D.P., Simon, R.M., Friedewald, W.T. et al. (1976). Randomized clinical trials. Perspectives on some recent ideas. N. Engl. J. Med. 295: 74–80.
3 3. Bastian, H., Glasziou, P., and Chalmers, I. (2010). Seventy‐five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. https://doi.org/10.1371/journal.pmed.1000326.
4 4. Viergever, R.F. and Li, K. (2015). Trends in global clinical trial registration: an analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013. BMJ Open https://doi.org/10.1136/bmjopen‐2015‐008932.
5 5. Wallach, J.D., Gonsalves, G.S., and Ross, J.S. (2018). Research, regulatory, and clinical decision‐making: the importance of scientific integrity. J. Clin. Epidemiol. 93: 88–93.
6 6. Ioannidis, J.P.A., Stuart, M.E., Brownlee, S. et al. (2017). How to survive the medical misinformation mess. Eur. J. Clin. Investig. 47: 795–802.
7 7. Eshre, C.W.G. (2018). Protect us from poor‐quality medical research. Hum. Reprod. 33: 770–776.
8 8.