Comparison of Readability Scores for Written Health Information Across Formulas Using Automated vs Manual Measures

Abstract

Assessing the readability of written health information is a common way to evaluate whether patients are likely to understand it.1 Readability is an objective measure that estimates a text’s equivalent school-grade reading level and is increasingly recommended globally in health policies.2,3 Several formulas for calculating readability exist, and scores can vary substantially depending on the formula applied.4 There has also been a proliferation of automated online calculators that provide readability estimates within seconds. However, the accuracy and consistency of automated calculators have not been evaluated.

The aims of this study were to assess (1) the variability of readability scores across automated calculators, (2) the association of text preparation with score variability, and (3) the level of agreement of automated readability scores with the reference standard (manually calculated scores) using the Simple Measure of Gobbledygook (SMOG) Index, the Flesch Kincaid Grade Level (FKGL), and the Automated Readability Index (ARI).