Ability of laypersons to estimate short time intervals in cardiac arrest.

STUDY OBJECTIVE: Estimates of time intervals by bystanders are considered critical in cardiac arrest, and are often used in other disorders such as stroke and myocardial infarction. Because they have never been previously studied, we sought to determine their accuracy. METHODS: This study was performed by prospective collection of bystander estimates (made at the time of the arrest) of the time from calling 911 to the arrival of urban fire department first responders, and comparison with actual measured response interval from computerized records, in all out-of-hospital cardiac arrests from January 1996 through June 1998. RESULTS: The fire department responded to 1,015 patients in cardiac arrest during the study period. First responders arrived before advanced life support providers to 831 patients, who thus met study entry criteria. Bystander estimates were obtained in 497 of these 831 patients, who did not differ in key characteristics from those lacking estimates. The bystander's average estimated fire department response interval was 5.6 minutes (95% confidence interval [CI] 5.2 to 5.9 minutes) and the actual measured interval to the patient's side from computer records was 6.1 minutes (95% CI 5.9 to 6.4 minutes). However, the median error of the bystander estimate (1.3 minutes) was 32% of the median of the actual measured on-scene interval, and there was no correlation between the bystander estimates and the measured interval in individual cases (R

Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance.

STUDY OBJECTIVE: To determine whether a fictitious manuscript into which purposeful errors were placed could be used as an instrument to evaluate peer reviewer performance. METHODS: An instrument for reviewer evaluation was created in the form of a fictitious manuscript into which deliberate errors were placed in order to develop an approach for the analysis of peer reviewer performance. The manuscript described a double-blind, placebo control study purportedly demonstrating that intravenous propranolol reduced the pain of acute migraine headache. There were 10 major and 13 minor errors placed in the manuscript. The work was distributed to all reviewers of Annals of Emergency Medicine for review. RESULTS: The manuscript was sent to 262 reviewers; 203 (78%) reviews were returned. One-hundred ninety-nine reviewers recommended a disposition for the manuscript: 15 recommended acceptance, 117 rejection, and 67 revision. The 15 who recommended acceptance identified 17.3% (95% confidence interval [CI] 11.3% to 23.4%) of the major and 11.8% (CI 7.3% to 16.3%) of the minor errors. The 117 who recommended rejection identified 39.1 % (CI 36.3% to 41.9%) of the major and 25.2% (CI 23.0% to 27.4%) of the minor errors. The 67 who recommended revision identified 29.6% (CI 26.1% to 33.1%) of the major and 22.0% (CI 19.3% to 24.8%) of the minor errors. The number of errors identified differed significantly across recommended disposition. Sixty-eight percent of the reviewers did not realize that the conclusions of the work were not supported by the results. CONCLUSION: These data suggest that the use of a preconceived manuscript into which purposeful errors are placed may be a viable approach to evaluate reviewer performance. Peer reviewers in this study failed to identify two thirds of the major errors in such a manuscript.

Effect of attendance at a training session on peer reviewer quality and performance.

STUDY OBJECTIVE: To determine whether attendance at a voluntary training workshop improves quality ratings of medical journal peer reviewers. METHODS: Peer reviewers for Annals of Emergency Medicine who completed two or more reviews during the 20 months before or the 20 months after October 1995 were eligible. Reviews were routinely rated by editors on a subjective 5-point quality scale. Comparisons were made between reviewers who chose to attend a 4-hour workshop on peer review sponsored by the journal in 1995 (attendees) and 2 groups of reviewers who did not attend: controls matched for review quality and number of reviews completed before the workshop, and unmatched controls. Guest reviewers were excluded. RESULTS: A total of 298 reviewers completed 1906 reviews before the workshop and 2,194 after the workshop; 2,117 of these reviews were rated by editors. Forty-five attendees participated in the workshop, 39 of whom had sufficient ratings for analysis. Matched controls were almost identical in performance to attendees, but unmatched controls had performed fewer reviews and had lower average ratings before the workshop. There was no significant change in any performance measurement after the workshop, including average quality rating, percent change in quality rating, odds ratio for recommending acceptance, and odds ratio for congruence with editor's decision. CONCLUSION: In a self-selected group of experienced reviewers who attended a 4-hour workshop on peer review, no effect could be identified in subsequent performance as measured by editors' quality ratings or reviewer performance statistics.

Reliability of editors' subjective quality ratings of peer reviews of manuscripts.

CONTEXT: Quality of reviewers is crucial to journal quality, but there are usually too many for editors to know them all personally. A reliable method of rating them (for education and monitoring) is needed. OBJECTIVE: Whether editors' quality ratings of peer reviewers are reliable and how they compare with other performance measures. DESIGN: A 3.5-year prospective observational study. SETTING: Peer-reviewed journal. PARTICIPANTS: All editors and peer reviewers who reviewed at least 3 manuscripts. MAIN OUTCOME MEASURES: Reviewer quality ratings, individual reviewer rate of recommendation for acceptance, congruence between reviewer recommendation and editorial decision (decision congruence), and accuracy in reporting flaws in a masked test manuscript. INTERVENTIONS: Editors rated the quality of each review on a subjective 1 to 5 scale. RESULTS: A total of 4161 reviews of 973 manuscripts by 395 reviewers were studied. The within-reviewer intraclass correlation was 0.44 (P<.001), indicating that 20% of the variance seen in the review ratings was attributable to the reviewer. Intraclass correlations for editor and manuscript were only 0.24 and 0.12, respectively. Reviewer average quality ratings correlated poorly with the rate of recommendation for acceptance (R=-0.34) and congruence with editorial decision (R=0.26). Among 124 reviewers of the fictitious manuscript, the mean quality rating for each reviewer was modestly correlated with the number of flaws they reported (R=0.53). Highly rated reviewers reported twice as many flaws as poorly rated reviewers. CONCLUSIONS: Subjective editor ratings of individual reviewers were moderately reliable and correlated with reviewer ability to report manuscript flaws. Individual reviewer rate of recommendation for acceptance and decision congruence might be thought to be markers of a discriminating (ie, high-quality) reviewer, but these variables were poorly correlated with editors' ratings of review quality or the reviewer's ability to detect flaws in a fictitious manuscript. Therefore, they cannot be substituted for actual quality ratings by editors.

Masking author identity in peer review: what factors influence masking success? PEER Investigators.

CONTEXT: In a previous study, we found that masking success was higher at a journal that masked reviewers to author identity. We hypothesized that masking policy or other factors could be associated with masking success. OBJECTIVES: To evaluate differences in success of masking reviewers to author identity at 7 biomedical journals and to identify factors associated with these differences. DESIGN: Written questionnaire. PARTICIPANTS: Reviewers at 3 journals with a long-standing policy of masking author identity (Annals of Emergency Medicine, Epidemiology, and Journal of the American Geriatrics Society) and 4 journals without a policy of masking author identity (Annals of Internal Medicine, JAMA, Obstetrics & Gynecology, and Ophthalmology). MAIN OUTCOME MEASURES: Masking success (percentage of reviewers successfully masked) and reviewer characteristics associated with masking. RESULTS: There was no significant difference in masking success between journals with a policy of masking (60%) and those without (58%) (P= .92). We found no association between masking success and a policy of masking when adjusted for the reviewer characteristics of age, sex, years of reviewing experience, number of articles published, number of articles reviewed, percentage of time spent in research, editorial experience, or academic rank (odds ratio [OR], 1.3; 95% confidence interval [CI], 0.64-2.8; P=.43). In multivariable analysis of reviewer characteristics, reviewers spending a greater percentage of time in research, the only significant predictor of masking success, were less likely to be successfully masked (OR, 1.01; 95% CI, 1.00-1.02) (P=.04). CONCLUSIONS: Masking success appears unrelated to a journal policy of masking, but is associated with reviewers' research experience and could be affected by other characteristics. Using reviewers with less research and reviewing experience might increase masking success, but the effect on review quality is unknown.

Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting.

CONTEXT: Studies with positive results are more likely to be published in biomedical journals than are studies with negative results. However, many studies submitted for consideration at scientific meetings are never published in full; bias in this setting is poorly studied. OBJECTIVE: To identify features associated with the fate of research abstracts submitted to a scientific meeting. DESIGN AND SETTING: Prospective observational cohort, with 5-year follow-up of all research submitted for consideration to the major annual 1991 US research meeting in the specialty of emergency medicine. PARTICIPANTS: All research abstracts submitted for consideration at the meeting for possible presentation. MAIN OUTCOME MEASURES: Characteristics associated with acceptance for presentation at the meeting and subsequent publication as a full manuscript. RESULTS: A total of 492 research abstracts were submitted from programs in emergency medicine and other specialties affiliated with 103 US medical schools. A total of 179 (36%) were accepted for presentation and 214 (43%) were published in 44 journals. Of the 179 abstracts accepted for presentation, 111 studies were published. Scientific quality of abstracts or prestige of the journal in which the study was eventually published did not predict either of these outcomes. The best predictors (by logistic regression) of meeting acceptance were a subjective "originality" factor (odds ratio [OR], 2.07; 95% confidence interval [CI], 1.13-3.89) and positive results (OR, 1.99; 95% CI, 1.07-3.84), and, for publication, meeting acceptance (OR, 2.49; 95% CI, 1.49-4.35) and large sample size (OR, 2.26; 95% CI, 1.23-4.31). Forty-nine percent (241) of abstracts did not report on blinding, and 24% (118) did not report on randomization. Acceptance and publication were both more likely for positive outcomes (P=.03). Funnel plots showed the classic distribution of positive-outcome ("publication") bias at each of the submission, acceptance, and publication phases. Meeting acceptance predicted publication with a sensitivity of only 51%, specificity of 71%, positive predictive value of 57%, and negative predictive value of 66%. CONCLUSIONS: Positive-outcome bias was evident when studies were submitted for consideration and was amplified in the selection of abstracts for both presentation and publication, neither of which was strongly related to study design or quality.

Unpublished research from a medical specialty meeting: why investigators fail to publish.

CONTEXT: It is not known whether peer review of research abstracts submitted to scientific meetings influences subsequent attempts at publication. OBJECTIVE: To determine why research submitted to a scientific meeting is not subsequently published. We hypothesized that authors of abstracts rejected by a meeting are less likely to pursue publication than those whose abstracts are accepted, regardless of research quality. DESIGN AND PARTICIPANTS: Blinded review of abstracts submitted to a medical specialty meeting in 1991 and not published as full manuscripts within 5 years. In 1996, authors of 266 unpublished studies were asked to complete questionnaires. MAIN OUTCOME MEASURES: Submission of a full manuscript to a journal between 1991 and 1996; failure to submit a manuscript to a journal because the investigator believed it would not be accepted for publication. RESULTS: A total of 223 (84%) of the unpublished investigators returned the questionnaire. Only 44 (20%) had submitted manuscripts to a journal. Manuscript submission was not associated with abstract quality (odds ratio [OR], 1.16; 95% confidence interval [CI], 0.80-1.64), positive results (OR, 0.75; 95% CI, 0.31-1.57), or other study characteristics. Having an abstract accepted for presentation at the meeting weakly predicted submission of a manuscript to a journal (OR, 1.88; 95% CI, 0.84-4.10). Authors of accepted abstracts were significantly less likely to believe a journal would not publish their manuscript than were authors of rejected abstracts (OR, 0.23; 95% CI, 0.0001-0.61). CONCLUSIONS: Study characteristics do not predict attempts to publish research submitted to a scientific meeting. Investigators whose research is rejected by a meeting are pessimistic about chances for publication and may make less effort to publish.

Collaboration in emergency medicine research: a consensus statement.

The role of collaboration in medical research and how it applies to emergency medicine (EM) research are discussed. The guidance of the Division of Health Sciences Policy of the Institute of Medicine is reviewed. Application of the principles of collaborative efforts and recognition of each individual's contribution are overviewed. Emergency physicians can and should be invaluable contributors to collaborative research. Collaborative research relationships, whether established at individual institutions or through national clinical trials, must be developed deliberately. The specialty of EM must make the necessary commitment of time and resources to ensure that these occur.

Collaboration in emergency medicine research: a consensus statement.

The role of collaboration in medical research and how it applies to emergency medicine research are discussed. The guidance of the Division of Health Sciences Policy of the Institute of Medicine is reviewed. Application of the principles of collaborative efforts and recognition of each individual's contribution are overviewed. Emergency physicians can and should be invaluable contributors to collaborative research. Collaborative research relationships, whether established at individual institutions or through national clinical trials, must be developed deliberately. The specialty of emergency medicine must take the necessary commitment of time and resources to ensure that these occur.

Quality-of-life and formal functional testing of survivors of out-of-hospital cardiac arrest correlates poorly with traditional neurologic outcome scales.

STUDY HYPOTHESIS: The traditional (and unvalidated) five-point Cerebral Performance Category (CPC) score at hospital discharge does not correlate with the results yielded by a validated functional status instrument and subjective quality-of-life assessment. METHODS: We compared CPC scores with the results of prospective standardized testing after discharge in survivors of out-of-hospital cardiac arrest. Consenting survivors were tested with the validated Functional Status Questionnaire (FSQ), a subjective quality-of-life assessment, and traditional CPC scoring. RESULTS: Of the 3,130 arrests during the 52 months of the study, 93 patients survived. Thirty-five patients were tested (71% of those eligible at the time of follow-up). Of these patients, 34% said their quality of life was worse, 38% said it was the same, and 28% said it was better than before the cardiac arrest. Fifty-four percent of patients scored normally on all FSQ subscales, but the remainder had an average 2.1 areas (of 6) with significant impairment. CPC score correlated very poorly with quality-of-life rating and with all scores and subscores on the FSQ. A CPC of 1 on discharge (supposedly normal function) had a sensitivity of 78%, a specificity of 43%, a positive predictive value of 64%, and a negative predictive value of 60% for quality of life the same as or better than that before arrest. With regard to ability to predict the presence of any major areas of impairment on the FSQ, the respective figures were 32%, 43%, 43%, and 32%. CONCLUSION: The CPC score, relied on as a measure of functional outcome in cardiac arrest, correlates poorly with subsequent subjective quality of life and with validated objective functional testing instruments, and conclusions based on it are suspect. Future researchers should employ standardized testing instruments.

Research directions in emergency medicine.

The goal of emergency medicine is to improve health while preventing and treating disease and illness in patients seeking emergency medical care. Improvements in emergency medical care and the delivery of this care can be achieved through credible and meaningful research efforts. Improved delivery of emergency medical care through research requires careful planning and the wise use of limited resources. To achieve this goal, emergency medicine must provide appropriate training of young investigators and attract support for their work. Promotion of multidisciplinary research teams will help the specialty fulfill its goals. The result will be the improvement of emergency medical care which will benefit not only the patients emergency physicians serve but also, ultimately, the nation's health.

A randomized clinical trial of active compression-decompression CPR vs standard CPR in out-of-hospital cardiac arrest in two cities.

OBJECTIVE: To compare the effectiveness of active compression-decompression (ACD) cardiopulmonary resuscitation (CPR) with a handheld suction device vs standard manual CPR in victims of out-of-hospital cardiopulmonary arrest. DESIGN: Prospective randomized clinical trial with crossover group design. SETTING: Emergency medical services (EMS) of a large (San Francisco) and medium-sized (Fresno) city in California. PATIENTS: All normothermic adult victims of out-of-hospital, nontraumatic cardiac arrest on whom CPR was performed by first responders. INTERVENTION: Patients were randomized to receive either standard manual CPR according to American Heart Association guidelines or ACD CPR, on first-responder contact. MAIN OUTCOME MEASURES: Return of spontaneous circulation, admission to the intensive care unit, survival to hospital discharge, and neurological function at hospital discharge. RESULTS: The ACD group (n = 117 in Fresno; n = 297 in San Francisco) and standard group (n = 136 in Fresno; n = 310 in San Francisco) were similar with regard to demographic and prognostic variables, such as age, witnessed arrest and bystander CPR frequency, and initial cardiac rhythm. Average interval from 911 call activation to EMS responder arrival was 6.4 minutes in Fresno and 4.0 minutes in San Francisco. In Fresno, there was no difference between the ACD group and standard CPR group in return of spontaneous circulation (17% vs 20%; P = .68), hospital admission (16% vs 20%; P = .56), hospital discharge (5% vs 7%; P = .64), or cerebral performance category score at discharge (1.5 vs 1.6; P = .90). Similarly, in San Francisco there was no difference between the ACD group and standard CPR group in return of spontaneous circulation (19% vs 21%; P = .65), hospital admission (13.5% vs 14.5%; P = .79), hospital discharge (4.7% vs 5.5%; P = .80), or cerebral performance category score at discharge (2.2 vs 2.6; P = .31). There was no increase in significant complications associated with the use of ACD CPR. CONCLUSION: There was no improvement in outcome with ACD CPR in out-of-hospital cardiac arrest in these two cities. Differences in study design, demographics, EMS systems, response intervals, training, and technique performance may contribute to the lack of improvement in initial resuscitation with ACD CPR compared with previous studies. Future research needs to control these variables to determine the reason for these differences in outcome.

Evaluation of active compression-decompression CPR in victims of out-of-hospital cardiac arrest.

OBJECTIVE--Active compression-decompression (ACD) cardiopulmonary resuscitation (CPR) appears to improve ventilation and coronary perfusion when compared with standard CPR. The objective was to evaluate potential benefits of this new CPR technique in patients with out-of-hospital cardiac arrest in St Paul, Minn. DESIGN--Ten-month, prospective, randomized parallel-group design. SETTING--St Paul, Minn, population 270,000. PATIENTS--All normothermic victims of nontraumatic cardiac arrest older than 8 years who received CPR. MAIN OUTCOME MEASURES--Return of spontaneous circulation, admission to the intensive care unit (ICU), return of baseline neurological function (alert and oriented to person, place, and time), survival to hospital discharge, survival to hospital discharge with return of baseline neurological function, and complications. RESULTS--Seventy-seven patients received standard CPR and 53 patients received ACD CPR. The mean emergency medical services call response interval was less than 3.5 minutes. When all patients were considered, a higher percentage of ACD CPR patients had a return of spontaneous circulation and were admitted to the ICU vs standard CPR (45% vs 31%, and 40% vs 26%, respectively), but these trends were not statistically significant (P < .10 and P < .10). No statistically significant differences were found between hospital discharge rates (12 [23%] of 53 for ACD CPR vs 13 [17%] of 77 for standard CPR), return to baseline neurological function (10 [19%] of 53 for ACD CPR vs 13 [17%] of 77 for standard CPR), or return to baseline neurological function at hospital discharge (nine [17%] of 53 for ACD CPR vs 12 [16%] of 77 for standard CPR). Return of spontaneous circulation, ICU admission, and neurological recovery in both CPR groups were highly correlated with downtime (time from collapse to emergency medical system personnel arrival to the scene in witnessed arrests). With less than 10 minutes' downtime, survival to the ICU was 59% (19/32) with ACD CPR and 33% (16/49) with standard CPR (P < .02), return to baseline neurological function was 31% (10/32) with ACD CPR and 20% (10/49) with standard CPR (P = .27), and hospital discharge rate was 38% (12/32) with ACD CPR and 20% (10/49) with standard CPR (P = .17). Complication rates in patients admitted to the hospital were similar in both groups. CONCLUSIONS--This study demonstrates that ACD CPR appears to be more effective than standard CPR in a well-defined subset of victims of out-of-hospital cardiac arrest during the critical early phases of resuscitation. Based on this study, a larger study should be performed to evaluate the potential long-term benefits of ACD CPR.

Physiology of blood flow during cardiopulmonary resuscitation. A transesophageal echocardiographic study.

BACKGROUND: There are two competing theories of the mechanism of blood flow during cardiopulmonary resuscitation. The "cardiac pump" theory postulates that blood flows because the heart is squeezed between the sternum and the spine. The "thoracic pump" theory postulates that blood flows from the thorax because intrathoracic pressure exceeds extrathoracic vascular pressure and that flow is restricted to the venous-to-arterial direction because of venous valves that prevent retrograde flow at the thoracic inlet. To determine which mechanism is operative during actual cardiopulmonary resuscitation, 20 patients were imaged with transesophageal echocardiography during resuscitation. METHODS AND RESULTS: Transesophageal two-dimensional and pulse Doppler echocardiography was begun within 7 minutes of initiation of cardiopulmonary resuscitation. In the 18 patients who could be analyzed, the mitral valve opened during the release phase (diastole) and closed during the compression phase (systole) of cardiopulmonary resuscitation. Mitral velocity-time integral measured 8 +/- 3 cm during diastole. There was compression of right and left ventricular cavities with significant reduction in measured left ventricular volume during cardiopulmonary resuscitation. In five patients, mitral regurgitation was present. CONCLUSIONS: Transesophageal echocardiography performed during actual cardiopulmonary resuscitation showing mitral valve opening during cardiac release, reduction of ventricular cavity size with compression, and atrioventricular regurgitation support the cardiac pump theory of cardiopulmonary resuscitation. This study demonstrates the feasibility and usefulness of transesophageal echocardiography during cardiopulmonary resuscitation.