Critical Appraisal Practice Paper 4 (Diagnostic)

Total marks: 23
Time allowed: 90 mins

You might wish to download the paper. Do it in 90 minutes and then compare with the answers provided here.

Paper: Mallampati test as a predictor of laryngoscopic view

Biomed Pap Med Fac Univ Palacky Olomouc Czech Repub. 2010 Dec;154(4):339-43.

Download (PDF, 65KB)

1. Provide a summary / abstract for the paper. (Up to 5 marks)

Answer

Answer

This should include all or some of the following points:

Aim. To determine the accuracy of the modified Mallampati test for predicting the difficulty of subsequent tracheal intubation.

Design. A cross-sectional, clinical, observational, non-blinded study. A quality analysis of anaesthetic care.

Setting. Operating theatres and department of anaesthesia in a university hospital in the Czech Republic.

Material and methods. Following local ethics committee approval and patients’ informed consent to anaesthesia, all adult patients (> 18 yrs) presenting for any type of non-emergency surgical procedures under general anaesthesia requiring endotracheal intubation were enrolled.

Prior to anaesthesia, Samsoon and Young"s modification of the Mallampati test (modified Mallampati test) was performed.

Following induction, the anaesthetist described the laryngoscopic view using the Cormack-Lehane scale. Classes 3 or 4 of the modified Mallampati test were considered a predictor of difficult intubation. Grades 3 or 4 of the Cormack-Lehane classification of the laryngoscopic view were defined as impaired glottic exposure.

The sensitivity, specificity, positive and negative predictive value, relative risk, likelihood ratio and accuracy of the modified Mallampati test were calculated on 2x2 contingency tables.

Results. Of the total 1,518 patients enrolled, 48 had difficult intubation (3.2%).

We failed to detect as many as 35.4% patients in whom glottis exposure during direct laryngoscopy was inadequate (sensitivity 64.4%).

Compared to the original article by Mallampati, we found lower specificity (82.4% vs. 99.5%), lower positive predictive value (0.107 vs. 0.933), higher negative predictive value (0.986 vs. 0.928), lower likelihood ratio (3.68 vs. 91.0) and accuracy (0.819 vs. 0.929).

Conclusion. When used as a single examination, the modified Mallampati test is of limited value in predicting difficult intubation in elective surgery patients.

2. Give three weaknesses of the study design and suggest improvements for these (up to 3 marks)

Answer

Answer

Strengths - there aren’t that many!

• They did get ethical approval


• The sample size was large


• Sample recruited from a good cross section of patients undergoing elective surgery

Weaknesses – there are lots of these!

• Patients studied are elective not emergency patients, so use in emergency situations cannot be inferred.


• It’s unclear who assessed the Mallampati Score but it’s likely to be the same anaesthetist who also assessed the outcome (laryngoscopic grade).


• It’s unclear what grade of anaesthetist assessed Mallampati or laryngoscopic grade.


• ? only one person assessed Mallampati Score.


• The person assigning the Cormack / Lehane grade was not unaware of the previously assigned Mallampati Score. In diagnostic studies it is important that the outcome is assigned without knowledge of the intervention test result.

The study could have been improved (for an EM readership) by:

• Studying a group of patients needing emergency airway control.


• The authors could have described in more detail who exactly was assigning the Mallampati Score e.g. grade, level of training etcc.. and how this was done.


• More than one assessor could have done this. The authors could have got two people to do this to ensure consistency and assessed agreement using a Kappa statistic or similar. This would help the reader to assess if the Mallampati Score was reproducible enough to make it worth doing amongst a wider range of clinicians.


• Someone else (who didn’t do the Mallampati Score) should have assigned the Cormack & Lehane Grade. The C&L grade given could have been influenced by knowledge of the Mallampati Score.

3. Name one checklist which is useful in evaluating the quality of diagnostic papers such as this. Give two further questions / points within this checklist not covered by the weaknesses you have mentioned in question 2. above. (up to 3 marks)

Answer

Answer

Common checklists include QUADAS and STARD (see below)

The STARD statement can be found here (it is similar to QUADAS):

http://www.stard-statement.org/

In this study the “index test” is the Mallampati Score and the “reference standard” is the Cormack & Lehane Grade. You should go through this checklist with the study and see how many weaknesses you can now identify!

4. The table 3 below is taken from the results section.

 Summarise the results in the table in one sentence. What is the Mann-Whiney U test? (Up to 2 marks)

Answer

Answer

Men are taller and heavier than women!

The Mann Whitney U Test is a test used to compare continuous (or ordinal) data in two independent groups, when the data is non parametric (i.e does not follow a normal distribution). It is analogous to a t-test which does the same thing but for normally distributed data.

 5. Construct a 2 x 2 table illustrating the main data from the current study (not Mallampati’s original). (Up to 2 marks)

Answer

Answer

		Actual Difficulty of Intubation(by Cormack Lehane Grade)
		Difficult (3/4)	Easy (1/2)
Predicted Difficulty of Intubation (Mallampati Class)	Difficult (3/4)	31	258	289
	Easy (1/2)	17	1212	1229
		48	1470	1518

6. Use your table in 5. above to demonstrate how the positive likelihood ratio and the accuracy were calculated. Explain how you would interpret the positive likelihood ratio in this study. (4 marks)

Answer

Answer

Positive LR = Sensitivity / 1 – Specificity

Sensitivity = a / a+c = 31 / 48 = 0.646

Specificity = d / b+d = 1212 / 1470 = 0.824

Positive LR = 0.646 / 0.176 = 3.67

Accuracy = total number (%) of “correct” predictions = a + d / a + b + c + d

Accuracy = 1243 / 1518 = 81.9%

LR+ above 10 means that a positive test (i.e. a higher Mallampati Score) will significantly increase the post test probability (of a difficult intubation) enough to make the test worth doing. Figures below 10 (like 3.67) mean that a positive test doesn’t really alter your chances of predicting the outcome enough to make it worth doing.

 

7. The authors used Fishers Exact Test to statistically compare their results with the results of Mallampati. Describe the indications for using this test as opposed to a Chi Squared Test? (Up to 2 marks)

Answer

Answer

Fishers Exact Test is used for 2 x 2 data when the expected count in any of the 4 boxes is “low”. The rule of thumb is if the expected number is less than 10 in any box then Fishers Exact Test should be used.. The expected number in any box can be calculated by multiplying the total of the column by the total row value and dividing by the overall number of patients / data points.

The authors appear to have used 2 x 2 tables for each of the possible outcomes. An example for “true negative” is illustrated below.

		Study
		Adamus et al	Mallampati et al
Observed Result	True Negative	1212	181	1393
	Not True Negative	306	29	335
		1518	210	1728

Thus for box d (value 29), the expected value is 210 (the total column value) x 335 (the total row value) / 1728 (the overall total). This is 40.1. In this case all the expected values are greater than 10 and so Chi Squared could have been used. However, for some of the others (e.g. false positives), the expected values will be low so I suppose the authors went for Fishers Test for consistency!

8. What are your conclusions overall? Is this paper going to influence your practice? Briefly suggest any ideas for future research in this area? (Up to 2 Marks)

Answer

Answer

Overall this paper is not great and is unlikely to influence your practice. There are multiple weaknesses and potential areas of bias. In addition the results are very different from the original Mallampati Study.

The authors hint at the other factors which allow a good assessment of the airway (e.g. weight or patients etc..). A better study might look at the whole LEMON acronym which you are probably familiar with from ATLS. It could be done in an ED setting with independent assessment of LEMON and the final C&L grade / ease of intubation.

Finally given that the study results are so different from the original Mallampati Study you could propose some secondary research (a systematic review, Best BET or even CTR!) to answer the question posed.

Mallampati test as a predictor of laryngoscopic view. Biomed Pap Med Fac Univ Palacky Olomouc Czech Repub. 2010 Dec;154(4):339-43.

A clinical sign to predict difficult tracheal intubation: a prospective study. Can Anaesth Soc J. 1985 Jul;32(4):429-34.

 

Critical Appraisal Practice Paper 4 (Diagnostic)