Total marks: 23
Time allowed: 90mins
Paper: High-sensitivity versus conventional troponin in the emergency department for the diagnosis of acute myocardial infarction
1. Provide a summary / abstract for the paper. (Up to 5 marks)
Answer
Answer
This should include all or some of the following points:
Background: Recently, newer assays for cardiac troponin (cTn) have been developed which are able to detect changes in concentration of the biomarker at or below the 99th percentile for a normal population.
Objective: The objective of this study was to compare the diagnostic performance of a new high-sensitivity troponin T (HsTnT) assay to that of conventional cTnI for the diagnosis of acute myocardial infarction (AMI) according to pretest probability (PTP).
Design: A prospective observational study of consecutive patients who presented to emergency departments in France with chest pain suggestive of AMI.
Setting: Three French Emergency Departments.
Participants: Adult patients presenting with chest pain suggestive of acute myocardial infarction with onset within the previous 6 hours.
Outcome measures: Levels of HsTnT were measured at presentation, blinded to the emergency physicians, who were asked to estimate the empirical pretest probability (PTP) of AMI.
The discharge diagnosis (of AMI or not) was adjudicated by two independent experts (both ED physicians) on the basis of all available data up to 30 days post presentation.
Results: A total of 317 patients were included, comprising 149 (47%) who were considered to have low PTP, 109 (34%) who were considered to have moderate PTP and 59 (19%) who were considered to have high PTP.
AMI was confirmed in 45 patients (14%), 22 (9%) of whom were considered to have low to moderate PTP and 23 (39%) of whom were considered to have high PTP (P < 0.001).
In the low to moderate PTP group, HsTnT levels ≥ 0.014 μg/L identified AMI with a higher sensitivity than cTnI (91%, (95% CI 79 to 100), vs. 77% (95%
CI 60 to 95); P = 0.001), but the negative predictive value was not different (99% (95% CI 98 to 100) vs. 98% (95% CI 96 to 100)).
There was no difference in area under the receiver operating characteristic (ROC) curve between HsTnT and cTnI (0.93 (95% CI 0.90 to 0.98) vs. 0.94 (95% CI 0.88 to 0.97), respectively).
Conclusion: In patients with low to moderate PTP of AMI, HsTnT is slightly more useful than cTnI. Our results confirm that the use of HsTnT has a higher sensitivity than conventional cTnI.
2. Give three strengths and three weaknesses of the study design (up to 3 marks)
Answer
Answer
Strengths:
- Prospective study; consecutive patients
- Done in a European ED setting
- Pragmatic approach to other aspects of patient care (very real world)
- Outcome measures clearly defined (AMI, unstable angina, other)*. At least two independent people decided upon the outcome.
- Treating and enrolling physicians blind to the results of HsTnT assay.
- Biologists assessing the level of HsTnT blind to patient data.
Weaknesses:
- Small numbers of patients
- Different analysers measured the HsTnT levels
- *ED physicians (not cardiologists) diagnosed AMI and other outcomes.
- Not all patients were admitted – its unclear if positive attempts were made to contact discharged patients to find out what happened to them at 30 days or if the authors simply used what information they had to decide upon outcome. I suspect the later.
- Different timings for the HsTnT level based only on when the patient arrived.
- Gestalt used to risk stratify patients (rather than a standardised scoring system)
3. Explain why this study may have been subject to “incorporation” or “work up” bias. Suggest two ways in which this could have been avoided in this study (up to 4 marks)
Answer
Answer
It appears to me that the result of the conventional troponin test (cTnI) was used in some cases to determine the need for admission (or not). Since all patients had a conventional troponin test, those with a low pre-test probability of AMI and a negative cTnI were probably discharged. This seems to amount to 39% of the patients (61% were admitted).
Thus the cTnI result helps to determine whether or not further investigations are carried out. It is not too great a leap to assume that the results of cTnI and those of HsTnT are highly correlated.
Ideally the diagnosis of the reference standard (which in this case is based on a review of notes and subsequent investigations in hospital) should be made entirely independently of the interpretation of the diagnostic test under evaluation.
The test under evaluation is the troponin. However, some of the investigations in hospital were only arranged if the troponin was positive. Other patients were discharged. Hence some of the patients may have been “deprived” of the opportunity to be diagnosed with AMI
Incorporation bias thus results in an over estimation of the diagnostic accuracy of a test.
Further explanation of incorporation bias / work-up bias can be found here: http://www.cjem-online.ca/v10/n2/p174
Ways to reduce the influence of incorporation bias in this study include:
Changing the primary outcome measure to, for example, death by 30 days and obtaining the answer for all patients (but probably fewer deaths and hence a bigger study would be needed)
Admitting all patients and performing a standardised set of investigations regardless of the troponin result.
4. The following is an excerpt from the methods section:
“We followed most of the recommendations concerning the reporting of diagnostic studies set forth by the Standards for Reporting of Diagnostic Accuracy initiative”
Give 4 elements of the STARD guidelines which should be reported in a diagnostic study such as this. (Up to 4 marks)
Answer
Answer
STARD checklist for reporting of studies of diagnostic accuracy (version January 2003)
| Section and Topic | Item# | On page # | |
| TITLE/ABSTRACT/KEYWORDS | 1 | Identify the article as a study of diagnostic accuracy (recommend MeSH heading "sensitivity and specificity"). | |
| INTRODUCTION | 2 | State the research questions or study aims, such as estimating diagnostic accuracy or comparing accuracy between tests or across participant groups. | |
| METHODS | |||
| Participants | 3 | The study population: The inclusion and exclusion criteria, setting and locations where data were collected. | |
| 4 | Participant recruitment: Was recruitment based on presenting symptoms, results from previous tests, or the fact that the participants had received the index tests or the reference standard? | ||
| 5 | Participant sampling: Was the study population a consecutive series of participants defined by the selection criteria in item 3 and 4? If not, specify how participants were further selected. | ||
| 6 | Data collection: Was data collection planned before the index test and reference standard were performed (prospective study) or after (retrospective study)? | ||
| Test methods | 7 | The reference standard and its rationale. | |
| 8 | Technical specifications of material and methods involved including how and when measurements were taken, and/or cite references for index tests and reference standard. | ||
| 9 | Definition of and rationale for the units, cut-offs and/or categories of the results of the index tests and the reference standard. | ||
| 10 | The number, training and expertise of the persons executing and reading the index tests and the reference standard. | ||
| 11 | Whether or not the readers of the index tests and reference standard were blind (masked) to the results of the other test and describe any other clinical information available to the readers. | ||
| Statistical methods | 12 | Methods for calculating or comparing measures of diagnostic accuracy, and the statistical methods used to quantify uncertainty (e.g. 95% confidence intervals). | |
| 13 | Methods for calculating test reproducibility, if done. | ||
| RESULTS | |||
| Participants | 14 | When study was performed, including beginning and end dates of recruitment. | |
| 15 | Clinical and demographic characteristics of the study population (at least information on age, gender, spectrum of presenting symptoms). | ||
| 16 | The number of participants satisfying the criteria for inclusion who did or did not undergo the index tests and/or the reference standard; describe why participants failed to undergo either test (a flow diagram is strongly recommended). | ||
| Test results | 17 | Time-interval between the index tests and the reference standard, and any treatment administered in between. | |
| 18 | Distribution of severity of disease (define criteria) in those with the target condition; other diagnoses in participants without the target condition. | ||
| 19 | A cross tabulation of the results of the index tests (including indeterminate and missing results) by the results of the reference standard; for continuous results, the distribution of the test results by the results of the reference standard. | ||
| 20 | Any adverse events from performing the index tests or the reference standard. | ||
| Estimates | 21 | Estimates of diagnostic accuracy and measures of statistical uncertainty (e.g. 95% confidence intervals). | |
| 22 | How indeterminate results, missing data and outliers of the index tests were handled. | ||
| 23 | Estimates of variability of diagnostic accuracy between subgroups of participants, readers or centers, if done. | ||
| 24 | Estimates of test reproducibility, if done. | ||
| DISCUSSION | 25 | Discuss the clinical applicability of the study findings. |
5. The following figure is taken form the results section of the paper:
Briefly describe how the ROC curve is generated. Broadly what does the “area under the curve” (AUC) tell you about the test? What value for AUC would be given for a perfect test and for a completely useless test? (Up to 3 marks)
Answer
Answer
ROC curves can be generated by using different cut off values to represent “positive” and “negative” for tests with continuous data points (e.g. quantitative D-Dimer, troponin I). Several points are chosen and then the sensitivity and specificity of the test for each cut off point is calculated. The ROC curve is simply a plot of the results with 1 – specificity on the x-axis and sensitivity of the y-axis (as above).
The ROC curve gives an assessment of the overall performance of the test at different cut off points for positive and negative. The larger the area under the curve then the better the test. Good tests will have curves which tend towards the top left of the graph.
A perfect test will have an AUC of 1.0 and a useless test will have an AUC of 0.5 (a straight diagonal line at 45 degrees to the origin, representing a 50:50 chance or a test no better than tossing a coin).
You will note that the 95% confidence interval for the AUC of HsTnT ranges from 0.881 to 0.971. i.e. the test appears to have good utility in the diagnosis of AMI.
6. The following is a portion of one of the results tables:
a Write one sentence explaining the results in each of the 4 columns pertaining to all patients with a positive cTnI. (4 marks)
Answer
Answer
71% of patients who are diagnosed with AMI will have a positive cTnI at presentation in the ED. Thus a negative test is not very good at ruling out likely development of AMI (SnOUT). It is not a very sensitive test when taken this early.
97% of patients who are not diagnosed with AMI will have a negative cTnI at presentation in the ED. cTnI is quite a specific test i.e. a positive test tends to rule AMI in (SpIN).
Only 78% of patients with a positive cTnI at presentation will be diagnosed with AMI (the PPV).
95% of patients with a negative test will not be diagnosed with AMI (the NPV).
The 95% confidence intervals around these levels indicate the range of plausible results (i.e. the range within which we are 95% certain that the real result lies).
b Briefly explain how the prevalence of the target condition in the population influences the sensitivity of a test and its negative predictive value (2 marks)
Answer
Answer
The sensitivity of a test is unaffected by the prevalence of the disease as it only relates to patients who actually have the condition. Similarly the specificity of a test is unrelated to the prevalence of the disease as it only relates to those without the condition.
The negative predictive and positive predictive value, however, are both influenced by the prevalence of the disease in the population. If the prevalence of a condition in a population is small then the reported NPV will be higher than when using the same test in populations with a greater prevalence of the disease. This is one reason why it is important to ensure that the study patients are similar to your own before implementing the results of a study into your practice.
Put another way, if there are hardly any patients with the condition in any case then useless tests such as e.g. tossing a coin, might have quite a good NPV (because there aren’t that many cases to miss anyway). If however, half the patients had the disease tossing a tail wouldn’t always be associated with those without the disease and the NPV will fall.
Conversely, PPV for the same test is higher in populations with a high prevalence of disease and lower if the disease is uncommon.
7. The following is another section from the same results table:
Write down the formulae for calculating the positive and negative likelihood ratios.
What do the positive and negative likelihood ratio results given in the top row mean? (Up to 3 marks)
Answer
Answer
Positive likelihood ratio: LR+= sensitivity/1-specificity
Negative likelihood ratio: LR-=1-sensitivity/specificity
LR + = 0.71 / 0.03 Approximately 21.5
LR- = 0.29 / 0.97 Approximately 0.32
LR+ above 10 means that a positive test will significantly increase the pre test probability enough to make the test worth doing. Figures like 21.5 mean that a positive test (cTnI) significantly increases the chance of predicting AMI (and are thus helpful in decision making).
LR – less than 0.1 means that a negative test will significantly decrease the pre test probability enough to make the test worth doing. Thus, the figure of 0.32 means that a negative test (cTnI) is not at all helpful in excluding subsequent AMI. (Mainly because it is done at arrival in ED and not the usual 12 hours).
Fagans nomogram is used to convert pre test to post test probabilities based on LRs.
Fagans nomogram
8. What are your conclusions overall? Is this paper going to influence your practice? (Up to 2 Marks)
Answer
Answer
The HsTnT does appear to have a better sensitivity (93%) than the traditional troponin when taken on arrival in ED. This may be helpful in improving time to discharge following attendance with chest pain and may reduce admissions.
However, the 95% CIs for the sensitivity are 89 – 100, and the possible sensitivities are thus too low to allow it to be confidently used to rule out MI based on this study.
Further studies are required.
Note: The answers are not done by me, they were given to me when I prepared my exam. If anyone have any questions, just drop a line below and we all can discuss.
Critical Appraisal Practice Paper 2
No comments:
Post a Comment