Bladder Cancer Health Literacy: Assessing Readability of Online Patient Education Materials

INTRODUCTION

A bladder cancer diagnosis has heavy implications in the physical, psychological, and emotional aspects of the lives of patients. In this digital age, it is common for patients to explore treatment options online before making their decisions [1]. Nine in ten US adults use the Internet, and an estimated fifty-nine percent of US adults use the internet to search health related information [1, 2]. As the number of internet users continues to rise, it is clear that urologists must become aware of health information online pertaining to the various treatment options of bladder cancer and their associated morbidities and financial costs.

Health literacy is defined as the “ability to obtain, comprehend, and act on medical information” [3]. Health literacy requires an individual to have functional, communicative, and critical skills. Functional skills include reading and writing. Communicative skills refer to extracting and applying new information in varying and evolving situations. Critical skills allow for the analysis of information and reflecting on advice [3]. The National Assessment of Health Literacy estimates that one-third of patients in the United States have limited health literacy, with the average U.S. adult reading at a seventh to eighth grade reading level [3]. The National Institutes of Health (NIH) and American Medical Association (AMA) recommend patient education materials be written at a sixth-seventh grade reading level to reach a greater patient base and account for the national average reading level [4]. The Joint Commission recommends that patient education materials be written at a fifth-grade level [5]. When applied to the urological surgery patient population, health literacy is crucial for comprehending the nature, risks, and benefits of surgical procedures, adhering to strict perioperative rules, and making complex care decisions about interventions or lack thereof. Poor health literacy may lead to patient nonadherence, higher healthcare costs, increased mortality, and poor surgical outcomes [6, 7]. Low health literacy is associated with $106–236 billion annually in increased hospitalizations [6].

With an abundance of patient education materials available online, a nonprofit organization termed Health on the Net Foundation (HON) was created in 1995 to combat false health information online [8]. The Health on the Net Foundation Code of Conduct (HONcode) was created by the HON Foundation to grant certification to human health online content containing reliable and credible health information [9]. The certification is designed to set basic ethical standards in the presentation of health information and clearly identify the source and purpose of the data on patient-centered websites [9]. HONcode certification is reviewed by a committee of medical professionals [8]. The HONcode does not rate the quality of information provided by websites and requires an application in order to be considered for certification [10].

Prior health literacy studies have looked at broad urological topics, collecting patient education materials from websites of major urologic organizations [11] and analyzing readability of the articles [4, 12]. Additionally, a 2018 study by Azer et al. reviewed accuracy and readability of kidney and bladder cancer websites [13]. This study determined readability of kidney and bladder cancer to fall between 10–11^th grade reading levels [13]. A second study by Pruthi et al. analyzed resources on bladder, prostate, kidney, and testicle cancer from major urologic organizations, determining readability to average between an 11–12^th grade reading level [12]. However, these studies have not isolated bladder cancer resources, comparing materials from the American Urological Association’s Urology Care Foundation compared with internet search engines. This study aims to analyze patient resources specific to bladder cancer by characterizing online materials, assessing for HONcode certification, and evaluating for readability.

MATERIALS AND METHODS

Free patient education resources on bladder cancer from the Urology Care Foundation, the official foundation of the American Urological Association (AUA), and from the top 50 websites using search engine results queried on July 25, 2020. A total of 8 articles related to bladder cancer patient education were obtained from the AUA website. The search term “bladder cancer” was queried using Google. The search browser was cleared for prior search engine history, cookies, location tools, and user account information to limit potential result bias. The first 50 websites were used, with materials within one click of opening the website collected for data analysis. Only the main text of the article was included, with advertisements, external links, references, contact forms and addresses excluded from the analysis. Sponsored or duplicate websites were not included in the analysis. Only articles written in English were included in the analysis.

The text collected from each website was analyzed for readability using four different validated scales to maintain reliability in the readability assessment process [14]. These scales included the Coleman-Liau Index, which assesses average number of letters and average number of sentences [4]; the SMOG (Simplified Measure of Gobbledygook) Readability Formula, which analyzes average number of words with three or more syllables and average number of sentences [15]; the Gunning Fog Index, which looks at number of sentences, number of words, and number of words with three or more syllables [16]; and the Flesch-Kincaid Grade Level, which looks at average number of syllables per word and average number of words per sentence [16].

The top 50 websites were further classified by category including institutional or reference (academic and hospital centers, government sponsored, and evidenced-based), commercial (private practice or funded), charitable (non-profit and recognized medical society), or personal sites. Each website was also assessed for HONcode certification.

Statistics were calculated utilizing JMP 15.1 Software (SAS Institute Inc, SAS Campus Drive, Cary, NC) and Microsoft Excel Version 16.35 (Microsoft Corporation One Microsoft Way Redmond, WA). Mean readability and standard deviation were calculated for each of the four assessment tools. Statistical analysis was performed to assess for significance between the AUA and top 50 websites using a two-tailed t test. To further analyze the top 50 websites by category, a one-way analysis of variance (ANOVA) was used to assess for significance. P value was set at 0.05. Unless specified, data are reported as means±Standard Deviation (SD).

RESULTS

The mean readability scores of resources on the AUA website include a Coleman-Liau Index of 9.4±0.7 (9^th grade), SMOG Readability of 7.5±0.5 (7^th–8^th grade), Gunning Fog Index of 8.9±1.2 (8^th–9^th grade), and Flesch-Kincaid of 8.1±0.8 (8^th grade). The average readability of AUA materials by the four assessment tools was 8.5±0.8 (8^th–9^th grade).

The top 50 websites by search engine results are listed in Table 2. Scores included a Coleman-Liau of 11.7±2.0 (11–12^th grade), SMOG Readability of 10.1±2.7 (10^th grade), Gunning Fox Index of 13.4±3.6 (13–14^th grade), and Flesch-Kincaid of 11.8±3.5 (11–12^th grade). The average readability of these websites by the four assessment tools was 11.7±2.9 (11–12^th grade). Results are displayed in Table 1. A two-tailed t test comparing the average readability between the AUA (8.5±0.8) and top search engine results (11.7±2.9) demonstrated statistical significance (P = 0.0001). Readability by assessment tools of the AUA-UCF and top search engine results are displayed in Fig. 1.

Fig. 1

Readability of AUA-UCF and Top 50 Website Results by Readability Tools. Bar graph showing the mean reading grade level for online bladder cancer patient information from the American Urological Association-Urologic Care Foundation (AUA-UCF) websites (orange bars) and the top 50 search engine websites (blue bars). Reading grade level was assessed with four validated metrics. SMOG, Simplified Measure of Gobbledygook. ^*p < 0.05.

The majority of the top 50 websites were categorized as institution/reference (33/50, 66%), followed by commercial (10/50, 20%), and charitable (7/50, 14%). None of the websites were categorized as personal, and this category was excluded from further analysis. Only 9/50 (18%) websites were HONcode certified, mainly in the commercial (5/9, 56%) and institutional/reference (4/9, 44%) categories (Table 2). HONcode certification was not associated with a difference in readability (P = 0.41). A subgroup analysis (Table 2) was performed between the institutional/reference, commercial, and charitable websites. ANOVA demonstrated significant differences in mean readability scores across website categories (P = 0.0003).

An example of an AUA-UCF material written at a 9^th grade reading level includes the description of cystoscopy: “this very common test lets your doctor see inside your bladder using a thin tube that has a light and camera” [17]. In contrast, a 12^th grade reading level example from one of the top 50 websites from Cancer.Net reads: “a surgeon inserts a cystoscope through the urethra into the bladder” [18]. While both accurately describe the procedure, the AUA-UCF material includes simple, commonly used words.

DISCUSSION

The American Medical Association, U.S. Department of Health and Human Services, and Healthy People 2020 have made specific aims to improve identification and accommodation of patients with low health literacy [4]. Patient understanding can be improved by providing patient education materials written at a grade level that will be most widely understood by a wide spectrum of education and reading levels. In this study, bladder cancer patient online education materials were examined as a unique search term allowing for specificity of search results. With more than half of US adults utilizing the internet for patient education materials, this investigation aimed to analyze and characterize the readability of health information online related to bladder cancer and its treatment to best serve this unique patient population [1, 2].

The readability tests utilized in this analysis were chosen as they are among the most commonly used readability formulas and have been validated by several prior studies [19]. These tests analyze word complexity (based on number of syllables), character count, and sentence length. The Coleman-Liau Index analyzes number of characters and sentences per 100 words. This index is often chosen for its simplicity and ease in assessing readability of health-related materials [19]. The Flesch-Kincaid Grade Level calculates results based on total number of words, sentences, and syllables with results corresponding to an overall grade level [20, 21]. This tool is often selected for analyzing readability given its accuracy and simplicity [19]. The Gunning Fog Index analyzes average sentence length and proportion of complex words with 3 or more syllables in comparison to regular words to determine readability [20, 21]. The Gunning Fog Index is a preferred tool for assessing health and medicine information materials [19]. The SMOG Readability Formula is calculated by number of sentences and words with 3 or more syllables [20, 21]. Higher scores correspond with lower readability. SMOG aims at predicting 100%comprehension, whereas other formulas aim for 50–75%comprehension [22]. SMOG Readability Formula has been referred to as the preferred readability tool for assessing consumer-oriented health material given reliability and consistency in testing [19, 23]. As supported by previous readability studies, multiple readability tools were utilized to maintain reliability of the results [19, 21, 22, 24].

After readability analysis with these four validated instruments, this study demonstrated that the level of bladder cancer materials on the AUA website is between one to two grade levels above the recommended sixth-seventh grade reading levels, falling between an 8–9^th grade reading level. The AUA is the largest urologic association worldwide and strives to provide educational materials for both providers and patients [25]. With free access to websites such as the AUA’s Urology Care Foundation, providers are able to confidently provide patient education materials on bladder cancer that reach a large range of reading levels. Of note, the online materials provided by the AUA’s Urology Care Foundation mirrored the print materials and can be printed by medical professionals to distribute to patients during office visits. In contrast to the reading level of the Urology Care Foundation, search engine from top 50 websites results demonstrated greater deviation from the average American reading level, on the order of at least three grade levels (average 11–12^th grade reading level). The difference between these two resources were statistically significant (P < 0.01). The findings of this study align with prior studies investigating readability levels of urological materials [4, 12, 13]; however, this is the first study that has specifically investigated bladder cancer materials and compared top 50 websites results with one of the most widely utilized urology foundations [25].

Delving further into general organization of websites on bladder cancer, broad topic categories could be described as: defining bladder cancer, signs and symptoms, causes and risk factors, diagnostic testing, and management. Based on knowledge of readability tools and how reading grade levels are scaled (number of words per sentence, polysyllables, and character count) these categories were further explored to determine areas that were more or less complex in respect to health literacy. While many websites tended to define and explain in detail a bladder cancer overview with definitions and symptoms, complexity of the text increased when naming diagnostic testing and management. For example, hematuria was often defined as “bleeding in the urine,” assisting patients in understanding the terminology [26]. In contrast, diagnostic testing was often listed without further descriptors, such as “urinalysis with microscopy,” “voided urinary cytology,” and “cystoscopy” [27]. While names of these particular tests may be appropriate, adding a sentence to describe these tests may aid patients with lower health literacy.

Additionally, this study demonstrated variation between website categories, with the highest reading level identified in commercial websites and lower reading levels within charitable websites. This study also found that the majority of websites (82%) were not HONcode certified, a similar result seen in prior studies [28]. Websites holding this certification were commercial and institution websites. However, it is noteworthy that certification requires an application process and fee, possibly limiting non-profit organizations and websites with limited budgets [29]. HONcode certification was not associated with a difference in readability, also similar to prior studies [8]. This study demonstrates the need for continued efforts in improving patient education materials to meet a broader patient base; however, guiding patients to resources accessible online written at or near the recommended grade level, such as the AUA and similar health professional-led patient-focused non-profit organizations, may contribute to greater understanding of their diagnosis and potential treatment options.

Readability refers to understanding of a text [30]. The average US resident reads at a 7–8^th grade reading level [3], and the average Medicare beneficiary reads at a 5^th grade reading level [31]. A lower readability has been advocated for as “far more people have been put off reading a text because they could not understand it” than because the reading level was viewed as beneath their reading level [32]. Health literacy barriers are found among all ethnicities, races, and social classes; however, there has been a link between health literacy and education and income levels [31]. Prior studies have demonstrated that a large majority of patients understand patient education materials written at a lower reading grade level. In a diabetes patient education study by Overland et al., patients were randomized to read food care information at 6^th, 9^th, or 11^th grade reading levels. 60%of patients understand the 6^th grade materials, 21%understood the 9^th grade reading level, and 19%understood the 11^th grade materials independently [33]. Other studies have demonstrated similar results [31, 34]. Thus, while online patient education materials can be a valuable resource, it is important to note that these materials are often written at a level much higher than the readability of many, if not most, patients. The National Institutes of Health (NIH) note that many meritorious patient education campaigns are designed at a 9–10th grade reading level and are written for the general public. However, defining the target audience of the patient education materials may assist in communication and target messages towards patients with low health literacy. The NIH notes that readers appreciate materials “conveyed simply and clearly.” It is advised that patients who desire greater detail and in-depth information can be directed to these sources [35]. Online patient materials, both from the Urology Care Foundation and top Web sites, should be modified to the recommended 6–7^th grade reading level. Additionally, many of the websites, including the AUA-UCF and top search results, supplement their materials with photographs or videos on bladder cancer along with treatment options, which may also aid to patient understanding and should continue to be utilized in the assemblance of patient education materials. References or links to resources containing greater detail should be available for patients who desire in-depth explanations at higher reading grade levels. These improvements can positively enhance patient awareness and understanding of their diagnosis and options, increase patient engagement, and improve upon physician-patient communication [36].

Beyond patient education materials in their written forms, several other validated methods may be employed to enhance patient education and understanding in the clinic. Examples include using nonmedical language during patient counseling, explaining difficult topics slowly or repeating information multiple times, visual aids, audiovisual resources, written supplements, and the teach-back method. Among these resources, the method most proven to enhance and ensure patient understanding is the teach-back method. In this style of patient education, the provider explains information to the patient using plain language and asks the patient to restate the information provided in their own words. If the explanation is incorrect, the provider may then choose to rephrase or repeat the information or may employ different strategies of communication with the patient. The process of repeating is continued as appropriate until the patient is able to offer the correct explanation. Once correct, the provider should acknowledge and reinforce appropriate understanding [37].

This study had certain limitations as only online, written materials were assessed. Oftentimes in clinical practice many methods of communication are utilized, including direct discussion with the patient, audio, images and figures, or video which may add to the greater picture of patient understanding. Additionally, as only English written materials were included, results are biased to English-speaking patients. More studies are needed to assess how implementing materials written at the recommended reading level may lead to improved surgical outcomes in the bladder cancer patient population.

CONCLUSION

Patient education materials online related to bladder cancer on the American Urological Association (AUA) website is written slightly above the reading ability of most U.S. adults by one to two grade levels. However, websites on bladder cancer obtained from top 50 website results are written above the recommended readability by at least three grade levels, often far higher. Urologists should assist patients by improving and providing patient education materials to fit a wider range of reading abilities, at a recommended sixth-seventh grade reading level. Additionally, evidenced-based teaching methods including the teach-back method may be employed to greater enhance physician-patient communication. With a greater focus on health literacy and readability of patient education materials, urologists may enhance provider-patient relationships and contribute to improved surgical outcomes and lowered health care expenditures.