P. Orlandoni, N. Jukic Peladic
OU Clinical Nutrition, Referral Center for Home Artificial Nutrition, IRCCS-INRCA Ancona, Ancona, Italy
Corresponding Author: Paolo Orlandoni MD, OU Clinical Nutrition, Referral Center for Home Artificial Nutrition, IRCCS-INRCA Ancona, Via della Montagnola 81 , 60100 Ancona, Italy, Tel/fax: 0039/071/8003653; Mob: 0039/335/450987; e-mail: p.orlandoni@inrca.it
Abstract
Background: Oropharyngeal dysphagia negatively affects the quality of life of patients. It may lead to malnutrition, dehydration, aspiration pneumonia and death, especially in older people. Dysphagia and its level of severity have to be assessed accurately and in a timely fashion, because only early intervention can prevent the onset of complications. There are numerous self-administered questionnaires to monitor both the severity of dysphagia and the effectiveness of therapeutic approaches. The objective of this article is to conduct a literature review and to illustrate the characteristics of various self-assessment questionnaires for oropharyngeal dysphagia. Methods: A search of observational studies of adult populations with dysphagia, published from 1990 to June 2014, was performed in the electronic database Pubmed. Results: A total of 23 self-assessment questionnaires, on Health-related Quality of Life and Functional Health status, were identified. Fourteen questionnaires were excluded from the analysis for the following reasons: the questionnaire was written in a language other than English or Italian (n=3); the questionnaire was specific for caregivers (n=1); the questionnaires were not specific for oropharyngeal dysphagia (n=10). Nine questionnaires, validated in adult populations, were examined. Only two self-assessment questionnaires on quality of life – DHI (Dysphagia Handicap Index) and SWAL-QOL (Swallowing Quality Of Life) – were correctly validated; other questionnaires had methodological errors. Conclusions: A specific self-assessment questionnaire for older adults was not found. Almost all of the currently available questionnaires need to be improved methodologically. Furthermore, new questionnaires specifically for older people should be developed.
Key words: Oropharingeal dysphagia, self-reported questionnaires.
Introduction
Swallowing is the ability to transport solids and liquids, safely and effectively, from the oral cavity to the stomach (1, 2). Functionally, swallowing can be divided into three phases: the preparatory phase, which includes conscious effort to ingest food and reflexes in the oral cavity that help prepare the bolus to be swallowed; the transfer phase, which involves reflex activities in the oral and pharyngeal cavities; the transport phase, which involves moving the swallowed food bolus through the esophagus into the stomach (3). Anatomically, swallowing has been divided into three phases: oral, pharyngeal, and esophageal. The oral phase includes preparatory and early transfer phases.
Dysphagia is defined as a difficulty to swallow liquids and/or solids (4). It is classified as oropharyngeal or esophageal, depending on where it occurs. In oropharyngeal dysphagia, symptoms arise from the dysfunctional transfer of a food bolus in the pharynx past the upper esophageal sphincter into the esophagus. Esophageal dysphagia is caused by disordered peristaltic motility or conditions that obstruct the flow of a food bolus through the esophagus into the stomach. Dysphagia may be caused by physiological changes in the mechanism of oropharyngeal or oesophageal swallowing of healthy older adults (Presbyphagia) or by pathological factors of various origins (neurological, muscular, neoplastic, etc.). Oropharyngeal dysphagia is most common in older and institutionalized subjects, especially if there are additional stressors such as acute illnesses or certain medications (5). Literature data show that swallowing difficulties are present in approximately 13% of the general older population and in more than 50% of institutionalized seniors (6). Oropharyngeal dysphagia occurs most often in people with neurologic conditions or skeletal muscular disorders. Thirty percent of older patients develop dysphagia following a stroke and of those, 10 to 30% have permanent symptoms (7). Approximately 60-80% of people with neurodegenerative diseases also develop dysphagia.
Oropharyngeal dysphagia negatively impacts patients’ quality of life and can lead to increased risk of malnutrition, dehydration, aspiration pneumonia and death (8,9,10). It is therefore essential to diagnose dysphagia, in a timely fashion, in order to implement appropriate treatment and to avoid the onset of complications (11).
Dysphagia assessment in older adults relies on the same modalities as in other age groups: taking a clinical history, assessment and instrumental examinations with videofluoroscopy (VFS) and fiberoptic endoscopic evaluation of swallowing (FEES), which are considered the gold standards to determine specific swallowing impairments, patient compensation and the response to rehabilitation (12-15). However, given that symptoms of oropharyngeal dysphagia are quite specific – choking or coughing at mealtime, pain on swallowing and nasal regurgitation – numerous patient self-administered questionnaires have been developed over the last few decades to measure their health-related quality of life (HR-QoL) and functional health status (FHS) . These questionnaires are used to assess oropharyngeal dysphagia and its severity, as well as to measure the impact of dysphagya and of therapeutic and rehabilitative interventions on the social, psychological and functional aspects of a patient. Since questionnaires are non-invasive diagnostic tools, are easy to understand and quick to perform, their use for older patients should be encouraged.
The purpose of this review is to provide an overview of available HR-QoL and FHS questionnaires, with indications on their target populations, their feasibility and the domains they assess, as well as their psychometric properties. A list of self-administered questionnaires containing items to evaluate dysphagia, which are not specific to oropharyngeal dysphagia, is also provided.
Materials and methods
This review is based on a qualitative analysis of articles cited in Pubmed. A defined search strategy was applied, with specific keywords for dysphagia tests (self-assessment, functional impairment, quality of life, dysphagia symptoms, older adults). The review was restricted to observational studies with adult populations with dysphagia published from 1990 to June 2014. Reference lists of all included articles were searched for additional literature. Information about self-assessment questionnaires was extracted from eligible studies. The examination of self assessment tools was limited to articles published in English and Italian. The target populations, the feasibility and the domains that were assessed by each questionnaire are provided. Information on the psychometric properties of questionnaires provided by authors and some reviews are presented.
Results
A total of 23 self-assessment questionnaires on Health-related Quality of Life and Functional Health status for different populations were identified. Fourteen questionnaires were excluded from the analysis for the following reasons: the questionnaire was written in a language other than English or Italian (n=3); the questionnaire was specific for caregivers of dysphagic patients (n=1); the questionnaires were not specific for oropharyngeal dysphagia (n=10). Nine questionnaires found in articles published in English and/or Italian and validated in adult populations with different pathologies were examined (Table 1).
Table 1 Quality of Life and Functional Health Status Questionnaires for Oropharyngeal Dysphagia included in the review
Questionnaires on functional health status and quality of life of patients with oropharyngeal dysphagia
Characteristics of the target populations and the feasibility of self assessment questionnaires for oropharyngeal dysphagia are summarized in Table 2.
Table 2 Characteristics of Quality of Life and Functional Health Status Questionnaires of Oropharyngeal Dysphagia
MD Anderson dysphagia inventory (MDADI)
MD Anderson dysphagia inventory (MDADI) is a self assessment questionnaire composed of 20 items designed to evaluate the impact of dysphagia on the quality of life of patients with head and neck cancer, who are undergoing surgery and chemo-radiotherapy (A. Y. Chen, R. Frankowshi, J. Bishop-Leone et al, 2002). The items assess four aspects of quality of life: emotional, physical, functional and overall conditions. Only four items explicitly investigate the presence of dysphagia throughout the analyses of physiological aspects: “It takes me longer to eat because of my swallowing problem”, “I cough when I try to drink liquids”, “I limit my food intake because of my swallowing difficulty”, “I cannot maintain my weight because of my swallowing problem”. This study involved 100 patients with head and neck cancer, while the reproducibility was evaluated in a sample of 29 patients. Construct validity was determined by comparing the results of the MDADI questionnaire with the SF-36, while the criterion validity was tested with the Performance Status Scale. Completing the questionnaire does not require much time, but scoring is complicated. The authors reported good internal consistency (Cronbach α varies between 0.85 and 0.93) and discrete validity and reproducibility (Cronbach α=0.69 for the global scale; Cronbach α=0.88 for the emotional and functional scales; Cronbach α=0.86 for the physical scale) of MDADI.
Self-Report symptom questionnarie
The Prototype Self-report Symptom Inventory (19 items) proposed by Wallace at (K. L. Wallace, S. Middleton, and I. J. Cook, 2000), gave rise to the Modified Self-report Inventory (17 items). The Modified Self-report Inventory was derived from a cross sectional study of a sample of 45 patients with oropharyngeal dysphagia of neuromyogenic etiology, 11 patients with oropharyngeal dysphagia caused by Zenker’s diverticulum and 19 nondysphagic controls. According to the authors, the 17 question inventory shows strong test-retest reliability over 2 weeks as well as face, content and construct validity, but its responsiveness to treatment in neuromyogenic dysphagia was not defined.
SWAL-QOL/SWAL-CARE
SWAL-QOL/SWAL-CARE is a self-assessment questionnaire which was administered to 386 patients with oropharyngeal dysphagia of different etiologies (vascular, neurodegenerative, neoplastic, traumatic) (McHorney CA, Robbins J, Lomax K, Rosenbek JC, Chignell K, Kramer AE, et al., 2002). It consists of two parts. The first part, SWAL-QOL, is composed of 44 items which assess the impact of dysphagia on quality of life. There are also 14 questions that investigate the presence of specific signs of dysphagia such as tracheobronchial aspiration and laryngeal penetration. The second part, SWAL-CARE, comprises 15 items which examine the effectiveness of a therapeutic intervention. This questionnaire was validated against video fluoroscopy. It is very long and its compilation is time consuming (it takes at least 20 minutes). These characteristics could negatively affect patients’ compliance. The authors reported very good internal consistency (Cronbach α between 0.80 and 0.95) and reproducibility (Cronbach α=0.60-0.91).
Deglutition handicap index
This questionnaire, based on the Voice Handicap Index, is composed of 3 groups of 10 items each (A. K. Silbergleit, L. Schultz, B. H. Jacobson, T. Beardsley, and A. F. Johnson, 2012). The first group of questions assesses physical signs of swallowing difficulties, the second group evaluates the functional status of patients (nutritional status and eventual pulmonary complications, such as aspiration pneumonia), the third part appraises emotional aspects and QoL. The construct validity of the first part of the questionnaire was tested with videofluoroscopy. The validity of items concerning functional aspects was tested against data on the nutritional status of patients. For emotional aspects, patients’ quality of life was assessed. The authors found good construct and content validity and very good internal consistency and reproducibility.
Eating assessment Tool (EAT-10)
Eating assessment Tool (EAT-10) is a self-assessment questionnaire designed by Belafsky (P. C. Belafsky, D. A. Mouadeb, C. J. Rees et al., 2008). The validation of the original questionnaire in English involved a total of 235 subjects; 21% of patients had oropharyngeal dysphagia, 11% displayed esopgaheal dysphagia, 22% suffered from phonatory disorders, 18% had head and neck cancer and 28% experienced gastroesophageal reflux. The questionnaire is validated in different languages (39). It consists of 10 items. Two items (items n. 2 and n. 7) assess the impact of disphagia on patients’ quality of life; all other specifically investigate the presence and severity of dysphagia. The brevity of EAT 10 makes it simple and quick to complete (approximately 2 minutes). It has very good internal consistency (Cronbach α=0.96) and good reproducibility, which, however, was evaluated only in the group of patients with esophageal dysphagia.
Swallowing outcome after laryngectomy (SOAL)
To develop this questionnaire, a cross sectional study was performed on a relatively small group of volunteer laryngectomees from the UK (n=19) (mean 5 years post-surgery), 19 dysphagic and 20 non-dysphagic patients (R. Govender, M. T. Lee, T. C. Davies et al., 2012). The questionnaire is composed of 17 items. Only two items: “Has your enjoyment of food reduced?” and “Do you feel self-conscious eating with other people?” assess patients’ QoL. All other items measure oropharyngeal dysphagia-related functional health status. Responses from the dysphagic group were confronted with the results of the modified barium swallow examination and a strong positive correlation was found with the subjective SOAL score (r=0.5; p=0.03). According to the authors, the questionnaire has excellent internal consistency (Cronbach α=0.91 for laryngectomees; Cronbach α=0.96 for all groups together).
Sidney swallow questionnaire (SSQ)
Sidney swallow questionnaire (SSQ) published by Dwivedi et al in 2010, is identical to the Modified Self-report inventory of Wallace et al (R. C. Dwivedi, S. S. Rose, J. W. G. Roe et al. , 2010). It is a 17 item questionnaire that was administered to 62 patients with head and neck cancer during a follow up period. Terminal phase patients, and those with other malignances or diagnoses of neuromuscular diseases, were excluded from the study. All items investigate, specifically, the presence of oropharyngeal dysphagia, with foods of different texture, difficulties to start the swallowing process, reduced oral and pharyngeal clearance and signs of tracheobroncheoral aspiration. Only the last item – “How much does your swallowing problem interfere with your enjoyment or quality of life?” – investigates the impact of swallowing deficiencies on patients’ quality of life. This questionnaire was validated with MDADI, which the authors considered a gold standard among self administered questionnaires. The authors reported excellent internal consistency (Cronbach α=0.95) and good reproducibility, calculated with Spermann’s correlation coefficient (0.71-0.83). Eighty-seven percent of the patients completed the questionnaire.
Dysphagia in multiple sclerosis (DYMUS)
Dysphagia in multiple sclerosis (DYMUS) is the first specific questionnaire to precociously identify dysphagia in patients with multiple sclerosis (MS) (R. Bergamaschi, P. Crivelli, C. Rezzani et al., 2008). It comprises 15 items exclusively evaluating signs of dysphagia and patients’ functional status. It was administered to 226 consecutive MS patients. According to the authors, DYMUS showed very good internal consistency (Cronbach α =0.88 for all questions; Cronbach α=0.85 for solids; Cronbach α=0.87 for liquids).
Dysphagia screening questionnaire
Dysphagia screening questionnaire consists of 14 items, and it was used in a cross sectional study on 1313 older adults living at home, to determine the prevalence of dysphagia and to clarify the relation between dysphagia and the level of daily living competence (Kei Kawashima, Yutaka Motohashi, Ichiro Fujishima, 2004). The questionnaire showed good internal consistency (Cronbach α=0.83). It evaluates six factors: past history of pneumonia, nutritional status, and pharyngeal, oral, esophageal and airway protective functions.
Discussion
This review provides information on currently available questionnaires for the self assessment of both HR-QoL and FHS in oropharyngeal dysphagia and a brief overview of questionnaires containing items to evaluate dysphagia, but focusing on other primary diseases associated with dysphagia.
According to the authors, all the self assessment questionnaires for oropharyngeal dysphagia showed satisfactory psychometric properties. In contrast to this, two recent reviews, one on HR-QoL (Timmerman AA et al,2014) and another on FHS questionnaires (Speyer R et al, 2014), presented quite different results. Those two reviews analyzed the psychometric properties of the majority of the questionnaires for oropharyngeal dysphagia shown in Table 2 (MDADI, DHI, SWAL-QOL, EAT-10, SOAL, Self reported inventory and SSQ) using the COSMIN checklist and the 4-point rating scale according to Terwee et al. (40,41). The methodological quality of each questionnaire, was evaluated for internal consistency, reliability, measurement error, content, construct and criterion validity and responsiveness. According to the Health-related Quality of Life review, only the DHI and SWAL-QOL questionnaires showed strong properties, especially for all validity aspects and the interpretability of the questionnaires (42). The SWAL-QOL homogeneously measures what it claims to, in an appropriate target population of patients with dysphagia. It differentiates relevant clinical characteristics such as normal swallowing, the manner of food intake, food texture, liquid consistency and symptom severity. Shortcomings were found in its reproducibility and floor and ceiling effects for the score ranges (over and under estimating). The DHI received positive ratings on almost all validity aspects and on the ability to distinguish between patients at one point, but its face validity was questionable. Regarding the review of psychometric characteristics of Functional Health Status, all the questionnaires analyzed (EAT-10, SOAL, Self reported inventory and SSQ) had poor overall methodological quality scores (43). The latter was due to very small samples of patients used to validate the questionnaires, inaccurate evaluation of all psychometric properties and inappropriate tests used for the analyses.
Conclusion
The availability of valid questionnaires is very important in daily clinical practice to facilitate early diagnosis of dysphagia and to assess the impact of therapies on patients’ quality of life. The use of non-invasive, self administered questionnaires is even more crucial for geriatric patients who have developed dysphagia and are too frail to undergo some invasive instrumental examinations.
Given that numerous self administered HR-QoL and FHS questionnaires for oropharyngeal dysphagia are already available, but their measurement properties and their methodological characteristics are not always robust, the first step should be to improve the characteristics of these existing tools. Secondly, given that the self assessment of oropharyngeal dysphagia is particularly important in geriatric settings, developing specific questionnaires for older adults, and caregivers who assist them, could play a critical role in the precocious recognition of dysphagia.
Conflict of interest statement: none declared
Fundings: none
Ethics standards: This study was performed in compliance with the current Italian laws.
References
1. Guerrier Y, Uziel A. Fisiologia delle vie aereodigestive superiori. Milano: Masson.1986;
2. Schindler O, Ruoppolo G, Schindler A. Degluttologia. Torino: Omega edizioni. 2011;
3. Robbins J, Bridges AD, Taylor A. Review: Oral, pharyngeal and esophageal motor function in aging, Part 1 Oral cavity, pharynx and esophagus. GI Motility online 2006; doi:10.1038/gimo39;
4. Ekberg O, Hamdy S, Woisard V et al. Social and psychological Burden of Dysphagia: its impact on diagnosis and treatment. Dysphagia 2002; 17: 139-146;
5. Ney D, Weiss J, Kind A et al. Senescent Swallowing: Impact, Strategies and Interventions. Nutr Clin Pract 2009; 24(3): 395–413; doi: 10.1177/0884533609332005;
6. Park YH, Han HR, Oh BM et al. Prevalence and associated factors of dysphagia in nursing home residents. Geriatr Nurs 2013; 34(3): 212-7; doi: 10.1016/j.gerinurse.2013.02.014;
7. Tippet DC. Clinical challenges in the evaluation and treatment of individuals with poststroke dysphagia. Top Stroke Rehabilh; 2011; 18:120-133;
8. Altman KW, Richards A, Goldberg L et al. Dysphagia in stroke, neurodegenerative disease, and advanced dementia. Otolaryngol Clin North Am 2013; (6):1137-49. doi: 10.1016/j.otc.2013.08.005. Epub 2013 Oct 10;
9. Daniels SK, Brailey K, Priestly DH et al. Aspiration in patients with acute stroke. Archives of Physical Medicine and Rehabilitation 1998; 79 (1): 14-9;
10. Marik P. Aspiration pneumonitis and aspiration pneumonia. N. Engl. J Med 2000; 344(9): 665-71;
11. Gustafsson B, Tibbling L, Theorell T. Do physician care about patients with dysphagia? A study on confirming communication. Fam Pract 1992; 9: 203–209;
12. Rosenthal DI, Lewin JS, Eisbruch A. Prevention and treatment of dysphagia and aspiration after chemoradiation for head and neck cancer. J Clin Oncol 2006; 24(17):2636-43;
13. Goldsmith T. Videofluoroscopic evaluation of Oropharyngeal swallowing; 4th ed St. Louis: Mosby 2003: 1727–1753;
14. Hinchey JA, Shephard T, Furie K et al. Formal dysphagia screening protocols prevent pneumonia. Stroke 2005; 36(9):1972-6;
15. Donovan SJ, Daniels SK, Edmiaston J et al on behalf of the American Heart Association Council on Cardiovascular Nursing and Stroke Council Dysphagia Screening: State of the Art: Invitational Conference Proceeding From the State-of-the-Art Nursing Symposium, International Stroke Conference. Stroke 2013;44:e24-e31, originally published online February 14, 2013 doi: 10.1161/STR.0b013e3182877f57;
16. Chen AY, Frankowshi R, Bishop-Leone J et al. The development and validation of a dysphagia-specific quality-of-life questionnaire for patients with head and neck cancer: the M. D. Anderson Dysphagia inventory. Archives of Otolaryngology—Head and Neck Surgery2001, 127 (7): 870–876;
17. Wallace KL, Middleton S, Cook IJ. Development and validation of a self-report symptom inventory to assess the severity of oral-pharyngeal Dysphagia. Gastroenterology 2000, 118 (4): 678–687;
18. McHorney CA, Robbins J, Lomax K et al. The SWAL-QOL and SWAL-CARE outcomes tool for Oropharyngeal dysphagia in adults: III. Documentation of reliability and validity. Dysphagia 2002;17: 97–114;
19. Silbergleit AK, Schultz L, Jacobson BH. The dysphagia handicap index: development and validation. Dysphagia 2012; 27 (1): 46–52;
20. Belafsky PC, Mouadeb DA, Rees CJ et al. Validity and reliability of the eating assessment tool (EAT-10). Annals of Otology, Rhinology and Laryngology 2008; 117 (12): 919–924;
21. Govender R, Lee MT, Davies TC et al. “Development and preliminary validation of a patient-reported outcome measure for swallowing after total laryngectomy (SOAL questionnaire),” Clinical Otolaryngology 2012; vol. 37, pp. 452–459;
22. Bergamaschi R, Crivelli P, Rezzani C et al. “The DYMUS questionnaire for the assessment of dysphagia in multiple sclerosis,” Journal of the Neurological Sciences 2008; vol. 269, no. 1-2, pp. 49–53;
23. Dwivedi RC, Rose SS, Roe JWG et al. “Validation of the Sydney Swallow Questionnaire (SSQ) in a cohort of head and neck cancer patients,” Oral Oncology 2010; vol. 46, no. 4, pp. e10–e14;
24. Kawashima K, Motohashi Y, Fujishima I. Prevalence of Dysphagia Among Community-Dwelling Elderly Individuals as Estimated Using a Questionnaire for Dysphagia Screening. Dysphagia, 2004; 19:266-271;
25. Woisard V, Andrieux M. P, Puech M. Validation of a self assessment questionnaire for swallowing disorders (Deglutition Handicap Index). Revue de Laryngologie Otologie Rhinologie 2007; vol. 127, (5)315–325;
26. Colodny N. Validation of the Caregiver Mealtime and Dysphagia Questionnaire (CMDQ). Dysphagia 2008 vol. 23 (1), 47–58, ;
27. Cohen JT, Manor Y. Swallowing disturbance questionnaire for detecting Dysphagia. Laryngoscope 2011; vol. 121, no. 7, 1383–1387;
28. Martin S, Catarina I, Therese E et al. The Dysphagia short questionnaire: an instrument for evaluation of Dysphagia – a validation study with 12 months follow-up after anterior cervical spine surgery. Spine 2012; vol. 37 (11), 996– 1002;
29. Grudell ABM, Alexander JA, Enders FB et al. Validation of the Mayo Dysphagia Questionnaire, Diseases of the Esophagus 2007; 20, 202–205;
30. Belafsky PC, Postma GN, Koufman JA. Validity and reliability of Reflux Syndrom Index. Journal of Voice 2002; vol. 16 (2): 274-277;
31. Dauer E, Thompson D, Zinsmeister AR et al. Supraesophageal reflux: validation of a symptom questionnaire. Otolaryngol Head Neck Cancer 2006; 134: 73–80;
32. Eypasch E, Williams J I, Wood-Dauphinee S et al. Gastrointestinal quality of life index: development, validation and application of a new instrument. Br J Surg 1995; 82: 216–22;
33. Locke G R, Talley N J, Weaver A L, Zinsmeister A R. A new questionnaire for gastroesophageal reflux disease. MayoClin Proc 1994; 69: 539–47;
34. Weymuller EA, Alsarraf R, Yueh B et al. Analyses of the Performance Characteristics of the University of Washington Quality of Life Instrument and its Modification (UW-QOL-R), Arch Otolaryngol Head and Neck Surg 2001; n. 127: 489-493;
35. Slade GD, Spencer AJ. Development and evaluation of the Oral Health Impact Profile. Community Dent Health 1994;11(1):3-11;
36. Pace-Balzan A, Cawood JI, Howell R et al. The Liverpool Oral Rehabilitation Questionnaire: a pilot study. J Oral Rehabil 2004;31:609-17;
37. Blazeby JM, Conroy T, Bottomley A et al on behalf of the European Organisation for Research and Treatment of Cancer Gastrointestinal and Quality of Life Groups. Clinical and psychometric validation of a questionnaire module, the EORTC QLQ-STO 22, to assess quality of life in patients with gastric cancer. European Journal of Cancer 2004; 40. 2260–2268;
38. Bjordal K, Hammerlid E, Ahlner-Elmqvist M et al for the European Organization for Research and Treatment of Cancer Quality of Life Study Group. Quality of Life in Head and Neck Cancer Patients: Validation of the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-H&N35, JCO 1999; Mar 1:1008;
39. Schindler A, Mozzanica F, Monzani A et al . Reliability and validity of the Italian eating assessment tool (I-EAT-10), Ann Otol Rhinol Laryngol 2013;122(11):717-24;
40. Mokkink LB, Terwee CB, Patrick DL et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Quality of Life Research 2010;19:539-549;
41. Terwee CB, Mokkink LB, Knol DL et al. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Quality of Life Research; vol. 21; 651-657;
42. Timmerman AA, Speyer R, Heijnen BJ, Klijn-Zwijnenberg IR. Psychometric Characteristics of Health-Related Quality-of-Life Questionnaires in Oropharyngeal Dysphagia 2014; vol. 29 (2); 183-198;
43. Speyer R, Cordier R, Kertscher B, Heijnen B. Psychometric Properties of Questionnaires on Functional Health Status in Oropharyngeal Dysphagia: A Systematic Literature Review. BioMed Research International 2014; art ID 458678; http://dx.doi.org/10.1155/2014/458678.