JHQ 150 - Validating a Patient Satisfaction Survey Translated into Spanish

Penny J. Miceli, PhD
Keywords: Cultural competence, Information management, Linguistic diversity, Management and leadership, Satisfaction surveys
July/August 2004

One of the many challenges facing U.S. healthcare providers is ensuring that the care provided to patients is delivered in a manner linguistically appropriate for the growing numbers of patients with limited English proficiency (Office of Minority Health [OMH], 2003). Consider, for example, the more than 13 million Spanish-speaking residents in the United States who speak English only with difficulty or not at all (U.S. Census Bureau, 2003). For these individuals, language may be a significant barrier to effective healthcare (Carrasquillo, Orav, Brennan, & Burstin 1999; Doty, 2003; Timmins, 2002).

Title VI of the Civil Rights Act of 1964, which has served as the foundation of the movement to provide linguistically appropriate care, is widely interpreted to mean federally funded healthcare providers must offer language assistance when necessary for the patient to have meaningful access to care (Executive Order, 2000; Office of Minority Health, 2003; U.S. Department of Health and Human Services, 1998). In addition, the Joint Commission on Accreditation of Healthcare Organizations cites “effective communication” as a fundamental patient right and cites the provision of written materials in the patient’s language, as well as the facilitation of translation services, as fundamental elements for ensuring this right (JCAHO, 2004). Healthcare providers are therefore faced with the challenge of locating the best resources possible to bring their services in line with their patient base.

A growing part of delivering healthcare today is following up with assessments of patient satisfaction. The Centers for Medicare and Medicaid Services (CMS) requires participating hospitals to demonstrate that they have a program in place for assessing and improving quality of care and patient satisfaction (U.S. Department of Health and Human Services, 2003). Accommodating linguistic diversity adds an additional level of complexity to these efforts. Although many surveys are available to measure patient satisfaction, each of which could likely be translated into Spanish, evidence of the equivalence of Spanish translations of English surveys is for the most part lacking and represents an often overlooked part of the survey research process (Morales, Reise, & Hays, 2000; Napoles-Springer & Perez-Stable, 2001).

The practical implication of this state of affairs is that it is difficult to know whether differences in satisfaction levels of English-speaking and primarily Spanish-speaking minority patients reflect true differences in satisfaction or are a result of an instrumentation problem. Studies show contradictory findings with regard to Hispanic patients’ perceptions of care. For example, although 56% of the Hispanic population questioned in one study were “very satisfied” with their care, Hispanics more often than whites felt their race/ethnicity, language skills, and ability to pay for care negatively influenced the care they received (Doty, 2003). To date, however, studies of patient satisfaction broken down by language have been seriously limited by the lack of demonstrated equivalence between English-language and Spanish-language patient satisfaction measures.

A valid and reliable Spanish-language tool for measuring patient satisfaction is essential as healthcare providers strive to deliver linguistically appropriate services to their Spanish-language population. This is a critical first step in the quality improvement process. In addition, if healthcare providers are to understand how the needs of this population differ from those of their English-language counterparts, then equivalent tools must be used to measure the perceptions of each and to make fair comparisons of the satisfaction levels of the two. This study evaluated the validity and reliability of a Spanish translation of the Press Ganey Inpatient Survey, a measure originally developed in English to assess patient satisfaction with hospital experiences. After measure equivalence between the two versions of the survey was established, the study then compared satisfaction levels among English-language and Spanish-language survey respondents.

Design and Participants

This study was a retrospective database study, drawing on patient survey responses maintained in the Press Ganey National Database. The following criteria were prerequisite for a patient’s survey to be included in the data set:

• The patient was treated at a hospital collecting patient satisfaction responses in both English and Spanish.
• The patient returned a survey to Press Ganey between January 1, 2002 and December 31, 2002.
• The language of the survey (i.e., English or Spanish) was recorded clearly in the national database.

Surveys meeting these criteria included 10,134 Spanish-language surveys and 220,026 English-language surveys returned by patients treated at 134 U.S. hospitals. In accordance with Health Insurance Portability and Accountability Act (HIPAA) regulations, this investigation was conducted on a fully de-identified data set.

Instrument

The Press Ganey Inpatient Survey is a self-administered measure of patient satisfaction with the inpatient care experience. Soon after discharge, the patient receives the questionnaire with an appropriate cover letter and return envelope. A one-wave mail methodology is used rather than phone or face-to-face interviews to respect patient privacy, to allow patients time to carefully consider the questionnaire items before responding, and to minimize the risk of acquiescence bias (Hall, 1995).

The 49 survey items, when combined, assess specific aspects of the care experience and provide a comprehensive measure of patient satisfaction. Items are worded such that patients are asked to provide a numeric rating of a concept (e.g., “speed of admission process,” “amount of attention paid to your special or personal needs”), rather than to agree or disagree with a statement or to respond directly regarding level of satisfaction. Each item is rated by the patient on a 1–5 Likert-type scale: 1 = very poor, 2 = poor, 3 = fair, 4 = good, 5 = very good. For data analysis, responses are linearly converted to a 0–100 scale where 0 = very poor, 25 = poor, 50 = fair, 75 = good, and 100 = very good.

The items of the survey are arranged into ten subscales, each representing a specific dimension of the care experience (e.g., admissions, room, meals, nursing, tests and treatments, physicians, visitors and family, discharge, personal issues, and overall ratings). Subscale scores are calculated by averaging across items within each survey subscale at the patient level. A summary Overall Satisfaction Score is calculated for each respondent by averaging across the subscale scores. Information on the psychometric properties of the English version of the instrument (i.e., validity and reliability) is available elsewhere and therefore not repeated here (Kaldenberg, Mylod, & Drain, 2003).

The survey was translated into Spanish by a professional translating service. The intent was to produce a Spanish version of the survey to provide the closest word-for-word translation possible, while retaining the intended meaning of the English version. In addition, the translation is “generic” and appropriate for use regardless of dialect. The translation process included the following steps:

• The English version was translated into Spanish by a native Spanish speaker.
• The Spanish translation was then reviewed independently by another native Spanish speaker for accuracy.
• The Spanish translation was then reviewed by a native English speaker to confirm that no contextual errors occurred.

Spanish Translation Validation Results
Construct Validity.

A principal components analysis with oblique rotation was performed on the Spanish-language survey data. Overall Assessment questions were omitted from the analysis because of predicted high intercorrelations with other items and the fact that they might conceptually be considered outcomes of the constructs indexed by the survey. The results identified the same underlying factor structure for the Spanish translation (see Table 1) as has been reported previously for the original English-language version (Kaldenberg, Mylod, & Drain, 2003), with each item showing only a meaningful loading with its theoretically expected factor.

In survey construction, it is expected that items will be more highly correlated with their own parent subscale than with other subscales on the measure. As shown in Table 2, the average item-subscale correlation was higher than the average item-non-subscale correlation. Thus, the construct validity of the translated survey is verified across multiple statistical tests, and the underlying theoretical structure mirrors that of the English version of the survey (Kaldenberg, Mylod, & Drain, 2003).

Reliability.

The internal consistency among the items of each subscale was examined using Cronbach’s alpha. Alpha levels for the subscales of the Spanish translation (see Table 2) surpassed the traditional criteria of 0.70 for reliable measures (range 0.86–0.95). Cronbach’s alpha for the entire survey was 0.98. Thus, the Spanish translation, like the English version, demonstrates high reliability (Kaldenberg, Mylod, & Drain, 2003).

Concurrent Validity.

The item “likelihood of your recommending this hospital to others” is included on the survey as a proxy for positive word of mouth. If the survey is a valid measure of patient satisfaction, it should relate to theoretically important criteria known to be associated with satisfaction, such as patients’ intention to recommend the hospital to others. Each individual item on the survey was significantly correlated to patients’ likelihood to recommend the hospital to others—correlations ranged from 0.40 to 0.84, statistically significant at the p < .01 level. Multiple regression analysis revealed that the items on the survey accounted for 81% of the variance in patients’ likelihood to recommend the hospital to others (F(48, 2312) = 205.044, p < .001; adj. R² = 0.81).

Comparison of Ratings by English-Language vs. Spanish-Language Respondents

Distribution of Responses.

Differences in the distribution of responses on each question of the survey obtained from Spanish-language and English-language respondents were examined by calculating the X² statistic. Because the X² statistic is highly sensitive to sample size, 500 respondents were chosen at random (i.e., 250 English-language respondents and 250 Spanish-language respondents) for this analysis. Because of the large number of X² statistics being calculated, alpha was set at p < .001 in order to maintain family-wise alpha at 0.05. As shown in Table 3, the proportion of responses falling within a given response category for English-language versus Spanish-language respondents generally did not differ for the two groups. There were, however, three items for which the distribution of responses differed significantly: how well things worked (e.g., television, call button, lights, bed, and so on), food temperature (e.g., cold foods cold, hot foods hot), and food quality.

Mean Scores.

Table 4 presents the difference in the item and subscale means and standard deviations for the English-language and Spanish-language surveys. A series of t tests for independent samples was performed to identify statistically significant mean differences, which are represented by the gray-scale bars in the graph. Alpha was set to p < .001 in order to maintain family-wise alpha at 0.05.

The general pattern shows Spanish-language respondents reporting significantly higher levels of patient satisfaction across most aspects
of care than English-language respondents. There were, however, some exceptions to this pattern. Spanish-language respondents rated the skill of the physician significantly lower than English- language respondents. Moreover, Spanish-language respondents rated the courtesy of the admissions personnel, the person who took blood, and the person who started
the IV significantly lower than their English-
language counterparts.

Discussion

The first goal of this study was to validate a Spanish translation of the Press Ganey Inpatient Survey. Psychometric testing of the translated survey suggested a high degree of similarity to the English survey. The factor analysis grouped items along the same dimensions previously identified with the English survey (e.g., admissions, room, meals, nursing, tests and treatments, physicians, discharge, and personal issues). Item analyses revealed a high degree of internal consistency (i.e., reliability) for each of the subscales, as well as appropriate item-subscale and item-non-subscale correlations. Furthermore, ratings on survey items—both individually and collectively—were related to Spanish-language respondents’ intention to recommend the hospital to others. These results support the notion that the Spanish translation of the survey measures patient perceptions of care along the same dimensions previously identified with the English survey, and that these dimensions are meaningful to Spanish-language respondents inasmuch as they are related to theoretically important criteria such as patients’ intention to recommend the facility.

The second goal of the study was to examine differences in perceptions of care among English-language and Spanish-language respondents. On the vast majority of issues, patients returning Spanish-language surveys reported higher mean levels of satisfaction than English-language respondents. Spanish-language respondents may be less willing to voice negative opinions when it comes to evaluating their healthcare than English-language respondents. Healthcare providers typically are afforded a high level of respect within the Hispanic cultures, and directly criticizing care via a patient survey may not be considered acceptable (The National Alliance for Hispanic Health [NAHH], 2001). However, this is not an entirely adequate explanation for two reasons. First, the distribution of responses revealed that respondents in both languages used the negative rating options when deemed appropriate, and with few exceptions, the distribution of responses for individual survey items did not differ for the two groups of respondents. Second, the fact that Spanish-language respondents on average rated certain items—most notably, “skill of the physician”—significantly lower than English-language respondents
suggests that they are willing to criticize their caregivers to some degree—but perhaps do so very selectively.

Higher mean ratings offered by Spanish-language respondents on the majority of the survey items may reflect lower expectations for healthcare, formed by their experiences with healthcare in their country of origin. Some of the greatest disparities in satisfaction level occurred on items regarding amenities (room and meals), with Spanish-language respondents reporting more positive perceptions than English-language respondents. If the standard of care in their country of origin was to provide the minimal accommodations necessary, then the amenities offered within the U.S. healthcare system may be perceived as surpassing their expectations. Future research following Spanish-language patients across time to assess how expectations may change as these patients become increasingly acculturated is warranted.

The issue of perceived courtesy of the caregivers was a notable exception to the general pattern of higher mean ratings by Spanish-language respondents. Specifically, staff who admitted patients, took blood, and started IVs were all rated as significantly less courteous by the Spanish-language respondents than the English-language respondents. Spanish-language respondents also showed a lower mean rating on friendliness and courteousness of the nurses, although this difference failed to reach statistical significance. Thus, this study identifies an important opportunity for improving the healthcare experience of Hispanic patients with limited proficiency in English—staff courtesy.

What is not possible to discern from this study, however, is exactly why Spanish-language respondents viewed members of the healthcare team as less courteous. Cross, Bazron, Dennis, and Isaacs (1989) have suggested that institutions, including those related to healthcare, progress along a continuum with regard to ability to provide culturally proficient care. The beginning phases of the continuum are marked by hostile attitudes toward and/or an inability to help those from minority cultures. The most advanced phases are marked by respect for cultural differences and a focus on meeting the unique needs of those of various cultures. In between these two opposite poles of the continuum, however, lies a “culturally blind” middle ground (i.e., treating patients the same regardless of their culture), a phase eventually replaced by attempts to reach out to different groups in culturally appropriate ways.

Patients in this study treated at facilities with current organizational cultures within the beginning phases of this continuum may have experienced interactions with staff that were indeed less courteous by U.S. standards, indicating a lack of respect toward patients from minority cultures. Alternatively, staff may have interacted with patients in a “culturally blind” manner. Two concepts within Hispanic cultures—respeto and personalismo—may help explain why “culturally blind” care is not likely to be viewed as particularly courteous care by Hispanic patients (NAHH, 2001).

Respeto refers to unwritten social conventions of Hispanic cultures that dictate behavior in social situations based on social status, age, gender, and authority (NAHH, 2001). For example, the degree of formality expected by a Hispanic patient during interactions with caregivers in part depends on the difference in ages between the two individuals (NAHH, 2001), regardless of the fact that the U.S. healthcare system has traditionally been set up to uniformly afford the caregiver more power in the interaction. Violation of such culturally defined social conventions can undermine patient-caregiver relationships—and could have contributed to the perceived lack of courtesy reported by Spanish-language respondents in the present study.

Personalismo refers to the emphasis Hispanic cultures place on personal relationships (NAHH, 2001). Patients from Hispanic cultures may expect healthcare providers to adopt a more personal approach in their interaction than is customary in the United States, including actions such as standing closer, touching more, and expressing interest in the events of the patient’s personal life (NAHH). In today’s healthcare environment, however, there is often little time for cultivating such personal relationships. A perceived lack of personalismo may have contributed to the lower ratings of courtesy provided by Spanish-
language respondents in the present study.

As important as respeto and personalismo are as individual concepts, the real value of cultivating these qualities with Hispanic patients is that over time they establish confianza (i.e., trust; NAHH, 2001). When the care provider respects a patient and takes a personal interest in the patient’s life and well-being, then that patient is more likely to trust that the provider is making appropriate recommendations regarding care. The fact that Spanish-language respondents in this study rated the skill of the physician significantly lower than the English-language respondents may indicate that many Spanish-language respondents had not yet established confianza in their physicians.

Limitations

Important limitations to this study must be acknowledged. First, the hospitals represented in this data set were not selected randomly. They are hospitals that (a) measure patient satisfaction and (b) do so using both English-language and Spanish-language surveys. Therefore, the results cannot be generalized to all hospitals.

Second, this study did not assess language proficiency. Thus, it cannot be stated with certainty whether patients received the questionnaire that was most appropriate to them (English vs. Spanish). Presumably, the hospitals made accurate determinations based on patient need or stated preference—but this cannot be verified in this study. Therefore, the magnitude of the differences in mean scores for the two groups may have been masked to some extent.

Third, this study is constrained by the original survey development process, which did not specifically seek to identify the issues contributing to patient satisfaction within various cultural subgroups in the United States. Although the translated survey proved to be valid and reliable across multiple tests, it is still possible that the survey did not assess certain other issues that might be important to Hispanic patients. As important as it is to demonstrate the equivalence of measures translated into different languages, it is also important to recognize that additional questions may need to be incorporated into patient satisfaction measures to understand fully how well an organization meets the needs of various patient subgroups.

Conclusions

Efforts to improve the experience of care from the patient’s perspective depend on the availability of valid and reliable patient satisfaction measures. Likewise, understanding how different segments of a patient population—such as English-language and Spanish-language patients—differ in their needs and perceptions of care requires a tool that is valid and reliable for use with both populations. This study verified the validity and reliability of the Spanish translation of the Press Ganey Inpatient Survey. Like the original English survey, the Spanish version demonstrated a high degree of validity and reliability, and thus offers a means for healthcare professionals to assess how well they meet the needs of each of these populations.

In comparing the evaluations of care offered by Spanish-language and English-language respondents to the survey, this study also identified a key area for improving the healthcare experiences of primarily Spanish-speaking patients—staff courtesy. The first step in addressing this service opportunity is for hospital leadership to believe in and convey to others the powerful role of cultural influences in healthcare settings. A second and equally important step is to equip frontline staff with the tools necessary to effectively interact with those from other cultures. Providing educational opportunities designed to increase staff understanding of cultural definitions of courtesy and understanding the importance of building trust with those of different cultures are essential. This understanding of how patient culture influences expectations regarding the interpersonal and technical aspects of care allows care to be delivered in a more effective manner.

Take a test on the article you just read for continuing education credit!

Author's Biography
Penny J. Miceli is a research associate at Press Ganey Associates. She received her doctorate in developmental psychology from the University of Notre Dame. Dr. Miceli’s work is published in Developmental Psychology, Infant Behavior and Development, Journal of Pediatric Psychology, Men’s Total Health Digest, American Journal of Hospice and Palliative Care, and Caring. She has presented her research at numerous national conferences and is a member of the American Psychological Association.

Acknowledgements
Thanks are extended to Anabel Navarro, Dennis Kaldenberg, Maxwell Drain, Robert Wolosin, Sabina Gesell, Deirdre Mylod, and three anonymous reviewers for helpful comments on earlier drafts of this manuscript, to Paul Clark for facilitating the collection of background literature, and to Aimee Williams for preliminary analyses.

References
Carrasquillo, O., Orav, J., Brennan, T., & Burstin, H. (1999). Impact of language barriers on patient satisfaction in an emergency department. Journal of General Internal Medicine, 14, 82–87.

Cross, T.L., Bazron, B.J., Dennis, K.W., & Isaacs, M.R. (1989). Toward a culturally competent system of care: A monograph on effective services for minority children who are severely emotionally disturbed. (Monograph). Washington, DC: Child and Adolescent Service System Program (CASSP) Technical Assistance Center, Center for Child Health and Mental Health Policy, Georgetown University.

Doty, M. (2003). Hispanic patients’ double burden: Lack of health insurance and limited English. Report of the Commonwealth Fund 2001 Health Care Quality Survey. Retrieved February 28, 2003, from www. cmwf.org

Executive Order 13166. (2000). Improving access to services for persons with limited English proficiency. Federal Register, 65(159).

Hall, M. (1995). Patient satisfaction or acquiescence? Comparing mail and telephone survey results. Journal of Healthcare Marketing, 15, 54–61.

Joint Commission on Accreditation of Healthcare Organizations (JCAHO). 2004 Hospital accreditation standards: Pre-publication edition. Retrieved April 12, 2004, from www.jcaho.org/accredited+organizations/hospitals/standards/new+standards/ri_hap.pdf

Kaldenberg , D., Mylod, D., & Drain, M. (2003). Patient-derived information: Satisfaction with care in acute and post-acute care environments. In N. Goldfield, M.
Pine, & J. Pine (Eds.), Measuring and managing health care quality (pp. 469–489). New York: Aspen Publishers.

Morales, L., Reise, S., & Hays, R. (2000). Evaluating the equivalence of health care ratings by whites and Hispanics. Medical Care, 38, 517–527.

Napoles-Springer, A. & Perez-Stable, E. (2001). The role of culture and language in determining best practices. Journal of General Internal Medicine, 16, 493–495.

National Alliance for Hispanic Health. (2001). A primer for cultural proficiency: Towards quality health services for Hispanics. Washington, DC: Estrella Press.

Office of Minority Health. Assuring cultural competence in health care: Recommendations for national standards and an outcomes-focused research agenda. Retrieved February 17, 2003, from www.omhrc.gov/CLAS/cultural1a. htm

Timmins, C. (2002). The impact of language barriers on the health care of Latinos in the United States: A review of the literature and guidelines for practice. Journal of Midwifery & Women’s Health, 47, 80–96.

U.S. Census Bureau. Table DP-2 profile of selected social characteristics: 2000. Retrieved February 18, 2003, from http://factfinder.census.gov/servlet/

U.S. Department of Health and Human Services Centers for Medicare & Medicaid Services (2003). Medicare and Medicaid programs; Hospital conditions of participation: Quality assessment and performance improvement. Federal Register, 68(16). [Online]. Retrieved April 13, 2004, from www.access.gpo.gov/su_docs/fedreg/ a030124c.html

U.S. Department of Health and Human Services Office for Civil Rights. (1998). Guidance memorandum: Title IV prohibition against national origin discrimination—persons with limited-English proficiency. Retrieved February 17, 2003, from www.hhs.gov/ocr/lepfinal. htm

Objectives
Core CPHQ Examination Content Area
I. Management and Leadership
II. Information Management