Journal of Evaluation in Clinical Practice

Leape, Lucian L.

doi: 10.1046/j.1365-2753.1999.00162.xpmid: 10468378

Three papers in this issue of the Journal illustrate the challenges of measuring and reporting therapeutic mishaps. Using a broad definition of ‘adverse incidents’ that embraces injuries (including death) from any cause, drug errors, unexpected events (e.g. convulsions) or findings (e.g. Hb8gm/dl), and worrisome indicators (e.g. LOS5 days), Stanhope et al. (1998) found 196 incidents identified from 500 consecutive deliveries in two obstetric units. While many (most?) of these ‘incidents’ were not actual injuries, the fact that 39% of patients had something other than a perfectly smooth course is very disturbing. It is an unpleasant reminder that, even in 1996, the most normal of biological events, childbirth, is still fraught with hazard. Also disturbing, but less surprising, is their finding that of incidents that were discoverable on later record review only 23% were reported (as required) when they occurred. In their search for explanations for non‐reporting by survey of the staffs of these same two units, Vincent et al. (1998) found that one‐third of physicians did not know how to find the list of reportable incidents and 15% did not know how to file a report. No single reason for non‐reporting among 12 reasons suggested by the investigators was accepted by a majority of respondents, suggesting that the problem is multi‐factorial or, more probably, denial of reality by the respondents. The authors also note the role of perceived lack of necessity and concern about disciplinary action as factors in non‐reporting.

Stanhope, Nicola; Crowley‐Murphy, Margaret; Vincent, Charles; O'Connor, Anne M; Taylor‐Adams, Sally E.

doi: 10.1046/j.1365-2753.1999.00146.xpmid: 10468379

Abstract: To examine the reliability of adverse incident‐reporting systems we carried out a retrospective review of the mother and baby case notes from a series of 250 deliveries in each of two London obstetric units. Notes were screened for the presence of adverse incidents defined by lists of incidents to be reported in accordance with unit protocols. We assessed the percentage of adverse incidents reported by staff to the maternity risk manager at each unit; the percentage of incidents detected by each risk manager, but not reported; and the percentage of incidents identified only by retrospective case note review. A total of 196 adverse incidents was identified from the 500 deliveries. Staff reported 23% of these and the risk managers identified a further 22%. The remaining 55% of incidents were identified only by retrospective case‐note review and not known to the risk manager. Staff reported about half the serious incidents (48%), but comparatively few of the moderately serious (24%) or minor ones (15%). The risk managers identified an additional 16% of serious incidents that staff did not report. Drug errors were analysed separately; only two were known to the risk managers and a further 44 were found by case‐note review. Incident‐reporting systems may produce much potentially valuable information, but seriously underestimate the true level of reportable incidents. Where one risk manager covers an entire trust, rather than a single unit, reporting rates are likely to be very much lower than in the present study. Greater clarity is needed regarding the definition of reportable incidents (including drug errors). Staff should receive continuing education about the purposes and aims of clinical risk management and incident reporting and consideration should be given to designating specific members of staff with responsibility for reporting.

Vincent, Charles; Stanhope, Nicola; Crowley‐Murphy, Margaret

doi: 10.1046/j.1365-2753.1999.00147.xpmid: 10468380

Abstract: A previous study (Stanhope et al. 1998) established that staff in two obstetric units reported less than a quarter of designated incidents to the units’ risk managers. A questionnaire was administered to 42 obstetricians and 156 midwives at the same two obstetric units, exploring the reasons for low rates of reporting. Questions concerned their knowledge of their unit's incident reporting system; whether they would report a series of 10 designated adverse obstetric incidents to the risk manager; and their views on 12 potential reasons for not reporting incidents. Most staff knew about the incident‐reporting system in their unit, but almost 30% did not know how to find a list of reportable incidents. Views on the necessity of reporting the 10 designated obstetric incidents varied considerably. For example, 96% of staff stated they would always report a maternal death, whereas less than 40% would report a baby's unexpected admission to the Special Care Baby Unit. Midwives said they were more likely to report incidents than doctors, and junior staff were more likely to report than senior staff. The main reasons for not reporting were fears that junior staff would be blamed, high workload and the belief (even though the incident was designated as reportable) that the circumstances or outcome of a particular case did not warrant a report. Junior doctors felt less supported by their colleagues than senior doctors. Current systems of incident reporting, while providing some valuable information, do not provide a reliable index of the rate of adverse incidents. Recommended measures to increase reliability include clearer definitions of incidents, simplified methods of reporting, designated staff to record incidents and education, feedback and reassurance to staff about the nature and purpose of such systems.

Karson, Andrew S.; Bates, David W.

doi: 10.1046/j.1365-2753.1999.00158.xpmid: 10468381

Abstract: Adverse events (AEs) in medical patients are common, costly, and often preventable. Development of quality improvement programs to decrease the number and impact of AEs demands effective methods for screening for AEs on a routine basis. Here we describe the impact, types, and potential causes of AEs and review various techniques for identifying AEs. We evaluate the use of generic screening criteria in detail and describe a recent study of the sensitivity and specificity of individual generic screening criteria and combinations of these criteria. In general, the most sensitive screens were the least specific and no small sub‐set of screens identified a large percentage of adverse events. Combinations of screens that were limited to administrative data were the least expensive, but none were particularly sensitive, although in practice they might be effective since routine screening is currently rarely done. As computer systems increase in sophistication sensitivity will improve. We also discuss recent studies that suggest that programs that screen for and identify AEs can be useful in reducing AE rates. While tools for identifying AEs have strengths and weaknesses, they can play an important role in organizations’ quality improvement portfolios.

Pronovost, Peter J.; Kazandjian, Vahé A.

doi: 10.1046/j.1365-2753.1999.00160.xpmid: 10468382

Abstract: The emphasis provided by quality improvement strategies on performance measurement and evaluation often results in our understanding of „processes of care and, perhaps, better outcomes. There are different references for process evaluation: external peers, regional profiles of „performance or a trending of one's own performance patterns. This paper proposes a methodology that enables learning from the daily practice of medicine by comparing alternative care processes and outcomes. Since it is estimated that 15–20% of medical practices are based on rigorous scientific data establishing their effectiveness, we have much to learn. We propose to learn from our daily practice by combining clinical research methods with quality improvement tools. The products comprise modified clinical trial and case–control studies. In a modified clinical trial, we would use a practice guideline as a control group and modify the guideline to create an experimental group. This method would maintain the internal validity of efficacy research while maintaining the external validity of effectiveness research. In the case–control method, it is possible to quantitate risk for a given outcome and focus improvement effort on factors associated with that outcome. We believe physicians will accept this learning approach because it is a more valid learning method than traditional quality improvement and, unlike randomized clinical trials, learning will occur in the daily practice of medicine.

Needham, Charles W.

doi: 10.1046/j.1365-2753.1999.00166.xpmid: 10468383

Science has nothing to do with ethics. Science is blind to good and evil. Ethics is a division of philosophy, not science. Socrates was the father of ethics, not science. The science of his day taught him little about human nature. Searching his soul enlightened. It is therefore odd to find certain modern scientists employing the Socratic method. They ask probing questions. Their questions begin a conversation, as though they know nothing. They do know there are serious limits to knowledge. Like Socrates they may admit their ignorance. They define terms. Then they differentiate between real knowledge and mere opinion. They argue, in order to demolish false opinion. Truth is what is left when the rest has been destroyed by compelling logic.

Bardsley, Martin; Cleary, Robert

doi: 10.1046/j.1365-2753.1999.00152.xpmid: 10468384

Abstract: The past few years have seen a growth of interest in outcome measurement in a variety of settings including audit, health care management and commissioning – besides the traditional applications in research work. This paper reports on a study of the outcomes of total knee replacement in an acute hospital where the outcomes were studied as part of an audit process. The outcome measures used included clinical and symptomatic measures as well as generic health status scales. The initial study in one hospital was expanded to include a number of others in the same region and a comparative database of outcomes developed. Examples of the results are shown. The technical measures using knee scores and general health status measure show significant improvement from pre‐operatively to 3 months later. This improvement was maintained up to the 1‐year follow‐up on both measures. Although the information systems for collecting and measuring outcomes has been successful, the ability of such measures to lead to behavioural change has been limited. The problems in using outcome measures are discussed in particular in the context of an audit within hospitals, and for purchasing agencies.

Walker, Alexander R.P.; Charlton, Karen E.

doi: 10.1046/j.1365-2753.1999.00143.xpmid: 10468385

Introduction: In view of current international interest and research on the elderly, numerous questions present themselves. Examples are: how rapidly are populations ageing? Is physical and functional decline in old age inevitable? Can morbidity be compressed into the final years of life? Are social, medical and other resources available for coping with the increasing numbers of elderly? Not least, how do old people themselves feel about ageing?

Al‐Shammari, Sulaiman A.; Ghani, Hamza Abdul

doi: 10.1046/j.1365-2753.1999.00149.xpmid: 10468386

Introduction: Upper respiratory tract infection (URTI) is a common health problem among patients attending primary health care (PHC) clinics (Kuyvenhoven et al. 1993; Khattab et al. 1997) and sore throat is one of the most common presenting symptoms in PHC clinics. It imposes a heavy burden on resources and time and results in the prescription of an oral antibiotic in 20% to 90% of cases (Kuyvenhoven et al. 1993; Carr et al. 1994). Two‐thirds of all antimicrobial drugs prescribed by Dutch PHC physicians, for example, are prescribed for URTI (Kuyvenhoven et al. 1993). PHC physicians prescribe antibiotics for sore throat for different reasons, including the „prevention of complications (e.g. rheumatic fever, glomerulonephritis, sinusitis, otitis media, etc.), the relief of symptoms and for psychosocial „reasons (Little & Williamson 1994). The majority of PHC physicians working in Saudi Arabia are expatriates contracted from outside the Kingdom. These colleagues come with varied backgrounds and attitudes to prescribing. A minority of patients presenting with sore throat have acute tonsillitis and only one‐third appeared to harbour Group A b‐haemolytic streptococci (GABHS), including carriers (Dangnelie et al. 1996). Thus a large number of patients may be unnecessarily exposed to risk of antibiotics and development of resistance with a concomitant drain on the health budgets. Furthermore, it is common for broad‐spectrum antibiotics, particularly amoxycillin, to be chosen, despite consistent advice that penicillin or erythromycin are still the most appropriate antibiotics for sore throat of bacterial origin (Fry 1993).Local studies that have evaluated the presentation and management pattern of such problems in PHC clinics in Saudi Arabia are rare. This study was initiated to address the following research questions:1 What is the pattern of presentation and management of sore throat in PHC clinics?2 Could clinical presentation guide us in the initiation of antimicrobial therapy?3 What is the outcome of this minor illness employing such management?

Mant, Jonathan; Hicks, Nicholas R.; Dopson, Sue; Hurley, Pauline

doi: 10.1046/j.1365-2753.1999.00169.xpmid: 10468387

To the Editor: There are a large number of studies addressing the impact of continuing medical education (Davis et al. 1995) and clinical guidelines (Grimshaw & Russell 1993) on changing clinical practice, but comparatively little on the impact of purchasers of health care. There is some evidence that financial incentives and penalties can influence practice (Greco & Eisenberg 1993). It has been suggested that purchasers can play an important role in implementing research findings (Haines & Jones 1994). Indeed, the separation of purchasers and providers has been viewed as an opportunity to use knowledge about effectiveness to improve health services (Dunning et al. 1994). We report a study the aim of which was to explore the impact that a specific brief external intervention might have on a detailed aspect of clinical practice: the use of corticosteroids in preterm labour. The intervention was initiated by public health physicians on behalf of a Health Authority. It formed a part of the Getting Research Into Practice (GRIP) initiative in the old Oxford Health Region, UK. The administration of corticosteroids to mothers expected to deliver prematurely reduces neonatal mortality and morbidity (Crowley et al. 1990). The first trial which suggested that corticosteroids were effective in this role was published in 1972, and evidence from 12 trials was assembled in a systematic review published in January 1990 (Crowley et al. 1990). Despite the accumulating evidence, in 1991 many women delivering prematurely in the UK and elsewhere were not receiving corticosteroids (Anon 1992;Donaldson 1992). The apparent failure of obstetricians to make full use of this treatment has been cited as an example of the delayed implementation of research findings that can occur in clinical practice (Haines & Jones 1994; Enkin 1996).