Inter‐Rater Reliability in Pre‐Anaesthetic Airway Assessment: A Protocol for a Systematic Review

Abstract

BACKGROUND: Pre-anaesthetic airway assessment relies on subjective clinical judgement, whether performed through direct physical examination, such as Mallampati classification or thyromental distance, or through composite scoring systems that combine several such components, including the Simplified Airway Risk Index, the El-Ganzouri Risk Index and the Wilson score. The clinical value of these assessments depends not only on their predictive accuracy but also on whether they can be applied consistently across assessors. However, evidence on the inter-rater reliability, agreement or variability of these composite tools and their individual components remains scattered across the literature and has not previously been systematically synthesised. This systematic review therefore aims to identify, assess and synthesise the available literature on inter-rater reliability, agreement or variability in composite airway assessment tools and individual airway assessment components in the context of pre-anaesthetic or preoperative assessment. Based on previous inter-observer studies of individual airway assessment components, we hypothesise that inter-rater reliability of these tools and components will predominantly range from moderate to excellent. METHODS: This systematic review protocol has been developed in accordance with Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) and informed by the PRISMA 2020 statement and the Cochrane Handbook for Systematic Reviews. A systematic search will be conducted in MEDLINE, Embase, CINAHL and CENTRAL, supplemented by reference list screening and forward citation tracking of foundational articles. Studies involving adult patients or healthy volunteers, in which two or more assessors independently evaluated a composite airway assessment tool or individual airway assessment component through physical assessment of a real participant, will be eligible for inclusion. Two reviewers will independently screen studies and extract data using a standardised, piloted form. Methodological quality will be assessed using a checklist adapted from the Guidelines for Reporting Reliability and Agreement Studies. Findings will be synthesised primarily narratively; quantitative synthesis may be considered where studies are judged sufficiently homogeneous. CONCLUSION: This systematic review will synthesise the available literature on inter-rater reliability, agreement or variability in composite airway assessment tools and individual airway assessment components and may help clarify the consistency of these routinely used pre-anaesthetic assessments across both clinical and research populations.

The authors' abstract, as published at the source. Acta Anaesthesiologica Scandinavica, 2026 · DOI ↗