New Assessment Strategy
Learn who to ask and what to ask them in this section on improving quality-of-life and care preference assessments in nursing homes.
Every year Nursing Facility A sends questionnaires to its residents’ family representatives, asking them to respond, on behalf of their loved ones, to a series of satisfaction questions: How satisfied is your resident with the food here? With the social activities offered? With the staff? With the care they receive? The responses received back are stunning: Almost everyone—at least eight of ten respondents—reports high levels of satisfaction with each and every item on the questionnaire. Sound too good to be true? It probably is, especially considering that Facility A, like an estimated 90% of all nursing homes in the nation, has too few workers to provide quality care to residents.
But if Facility A’s actual quality of service does not deserve such high ratings, then what accounts for them? The questionnaire’s design—one commonly used by nursing homes—and in particular its choice of respondents and its reliance on direct satisfaction questions. Let’s take a look at how these design features influence responses.
Back to Top
Rule #1: Residents Before Their Reps
With half of all residents showing some degree of cognitive impairment, many nursing homes believe that asking respective family members and significant others to assess quality of care is both more time-efficient and reliable than asking the residents themselves. Presumably healthy (or healthier) and cognitively intact, family members can capably respond to a mailed questionnaire (no need to interview them in person) with meaningful answers.
There are two objections to this reasoning. First, it assumes that residents and their family members share the same preferences for service and perceptions of care quality. Often they don’t, however. Writes Social Work professor Scott Miyake Geron of Boston University (1), “…the findings of researchers who have explored consumer perceptions of long-term care (are) that consumers’ definitions of quality of long-term-care services are simply different from those of professionals, family members, and other stakeholders (pg. 69).” Similar perceptions? Maybe. Identical? Hardly. Adds Kane (2): “Proxy inaccuracy may be compounded for nursing home residents if families visit infrequently or staff are not well acquainted with residents (pg. 32).”
Also objectionable is the presumption that all cognitively impaired residents are suspect evaluators of care quality and, in particular, of their own preferences for care. Recent research, by us and others, shows that the majority of residents with mild to moderate cognitive impairment can indeed provide useful, reliable information about the care they receive, the services they prefer, and their quality of life. In one study, Kane et al. (3) were able to interview 1,988 residents from 40 nursing homes in five states and, based on the results, develop Quality-of-Life scales for about 60% of them. “This was achieved,” the authors write, “even though at least half of the sample included the more impaired levels on a cognitive performance scale; only 19% of the sample had a perfect cognitive score, and 17% had the worse possible cognitive score (pg. M245).”
Our research shows similar results. In one study, we interviewed 111 incontinent residents to assess their satisfaction with incontinence and mobility care processes (4). To qualify for the study, incontinent residents had only to pass a simple responsiveness screen: They were asked to state their name or identify two common items, and were given chances on two separate days to pass the screen. An analysis of their responses showed that 75% of the sample provided logically consistent and stable answers to our interview questions about their received and preferred care levels.
In two other studies, we set out to identify a simple cognitive screen that would accurately identify residents capable of providing meaningful responses to satisfaction and preference questions (5, 6). Both studies showed that residents who score two or more on the Minimum Data Set Recall subscale can accurately describe the care they receive. Our more recent research shows even more encouraging results: About half of residents who score 1 on the MDS Recall subscale can reliably self-report pain and symptoms of depression, express meaningful preferences for daily care (they can tell you, for example, what activities they like or where they would like to have their breakfast or if they prefer to take an afternoon nap), and accurately describe care they receive on a daily basis (they can recall, for example, if staff helped them to the bathroom or provided walking assistance).
Back to Top
The "Gold Standard"
Based on these findings, there is growing consensus that residents’ self-reports represent the “gold standard” for measuring their care preferences as well as their quality of life—an inherently subjective construct that includes such domains as relationships, autonomy, privacy, and enjoyment—and are integral to quality of care assessments. After all, residents are the direct recipients of long term care—not their respective family members or health care providers.
In addition, there’s a side benefit to interviewing residents. Writes Kane et al. (3), “The very act of asking resident directly about their (quality of life) could engage staff directly and systematically with residents’ opinions about their daily existence in a way that seldom occurs in a typical (nursing facility). Such a process mitigates the tendency to depersonalize residents, and to view them merely as care recipients rather than people who live out their lives in difficult circumstances (pg. 247).”
The implications for nursing home care providers are inescapable: If you want to evaluate consumer satisfaction, quality of life, quality of care, call it what you will, then you must capture the voices of residents, including cognitively impaired residents, in your assessment. Surveying family members is an acceptable practice; they are important stakeholders in long-term care. But don’t canvass them at the cost of excluding their loved ones. Our Interview Protocol on the next page presents guidelines for selecting residents for interview.
Back to Top
Rule #2: Question the Questions
We turn now to the question of the questions themselves. Direct satisfaction questions, like those used by Nursing Facility A (“How satisfied are you with…[fill in the blank]?”), are a staple of consumer satisfaction surveys, but they are not an ideal choice for querying nursing home residents for two reasons.
The first is that they are prone to an acquiescent response bias; that is, residents will tend to respond favorably to these questions, despite known problems with the quality of care they are receiving. In effect, residents are giving answers they think you want, not necessarily expressing their own views.
Any questionnaire can inadvertently elicit an acquiescence bias among respondents, but consumer satisfaction surveys conducted with nursing home residents are especially likely to do this for several reasons. Older adults, and women in particular, tend to report higher rates of satisfaction with health care services; thus, there is a good chance that extremely old and frail nursing home residents, who are predominantly female, will report high rates of satisfaction with substandard or inadequate care. In addition, many residents lower their expectations for care as they reside in the facility over time. Their actual experience teaches them to expect and accept substandard quality of care. The fact that residents also are dependent on staff for daily care and many are isolated from family and friends can only decrease their willingness to express dissatisfaction with care due to fear of reprisal.
In order to collect data useful for quality improvement, your assessment questions must reveal both your facility’s strengths and weaknesses. You will not be able to identify areas that need improvement if resident responses to all or most of your questions cluster at the “highly satisfied” end of the scale, as responses to direct, forced-choice questions about satisfaction tend to do (4,7,8). You need, therefore, to ask questions that are more sensitive to differences in satisfaction levels.
Your questions also should elicit information that will help guide improvement efforts. A second problem with direct satisfaction questions is that they fail to do this. Though they may be able to tell you whether residents are generally satisfied or dissatisfied with a certain aspect of care, they shed no light on how to correct identified problems or how to tailor care to the individual; that is, how to provide more resident-centered care. Does the person want more privacy or less? Does she want to eat in the dining room or her own room? Does he receive enough help with toileting or does he want more? With quality improvement, as with many things in life, the devil is in the details. But the details are absent in direct satisfaction questions.
Back to Top
Four Types of Questions...
What are good alternatives to direct satisfaction questions? In two studies, we evaluated various interview strategies to identify questions that both tempered acquiescence response tendencies among residents and provided information useful for improving care (4, 7). We asked residents these four types of questions:
Back to Top
...Yield Mixed Responses...
Answers to our questions about walking assistance (4) are typical of the responses we received in other ADL care areas:
From 80% satisfied, it now appears that 81% are dissatisfied, at least with the amount of walking assistance they receive.
Back to Top
...But Analyses Show That Some Question Top Others
What do we make of these results? Satisfaction with care is subjective; so when a resident tells us that overall, she is satisfied with the amount of walking assistance she receives but would like more of it, we are obliged to accept both statements. (And no, our analyses show that this response pattern is not related to a resident’s cognitive status.) That said, given our goal of collecting information useful for improvement efforts and given what we know of acquiescence, reduced expectations, and fear of reprisal among nursing home residents, it is appropriate to question the questions. How effective is each type? Findings from our studies show the following (4,7):
The proportion of residents reporting unmet needs for ADL care are significantly higher with the discrepancy and open-ended questions compared to the direct satisfaction questions. This suggests that the former questions are more sensitive to differences in satisfaction levels and that the latter questions are more limited by acquiescent response biases.
In sum, the direct satisfaction questions—most commonly used in nursing home surveys—proved the least useful and reliable. The discrepancy and open-ended questions are better choices for both care planning and quality improvement purposes.
In the next section, we present a quality-of-life assessment protocol that takes into account these findings as well as the mandate, born of research, not regulations, to interview residents as the best reporters of their care preferences and related quality of life. You can use this protocol to develop and implement an effective assessment strategy for your facility.
Back to Top
Try This Quick Assignment
Back to Top or proceed Interview Protocol
- A Common Consumer Survey
- Rule #1: Residents Before Their Reps
- The “Gold Standard”
- Rule #2: Question the Questions
- Four Types of Questions…
- …Yield Mixed Responses…
- …But Analyses Show That Some Questions Top Others
- Try This Quick Assignment
Every year Nursing Facility A sends questionnaires to its residents’ family representatives, asking them to respond, on behalf of their loved ones, to a series of satisfaction questions: How satisfied is your resident with the food here? With the social activities offered? With the staff? With the care they receive? The responses received back are stunning: Almost everyone—at least eight of ten respondents—reports high levels of satisfaction with each and every item on the questionnaire. Sound too good to be true? It probably is, especially considering that Facility A, like an estimated 90% of all nursing homes in the nation, has too few workers to provide quality care to residents.
But if Facility A’s actual quality of service does not deserve such high ratings, then what accounts for them? The questionnaire’s design—one commonly used by nursing homes—and in particular its choice of respondents and its reliance on direct satisfaction questions. Let’s take a look at how these design features influence responses.
Back to Top
Rule #1: Residents Before Their Reps
With half of all residents showing some degree of cognitive impairment, many nursing homes believe that asking respective family members and significant others to assess quality of care is both more time-efficient and reliable than asking the residents themselves. Presumably healthy (or healthier) and cognitively intact, family members can capably respond to a mailed questionnaire (no need to interview them in person) with meaningful answers.
There are two objections to this reasoning. First, it assumes that residents and their family members share the same preferences for service and perceptions of care quality. Often they don’t, however. Writes Social Work professor Scott Miyake Geron of Boston University (1), “…the findings of researchers who have explored consumer perceptions of long-term care (are) that consumers’ definitions of quality of long-term-care services are simply different from those of professionals, family members, and other stakeholders (pg. 69).” Similar perceptions? Maybe. Identical? Hardly. Adds Kane (2): “Proxy inaccuracy may be compounded for nursing home residents if families visit infrequently or staff are not well acquainted with residents (pg. 32).”
Also objectionable is the presumption that all cognitively impaired residents are suspect evaluators of care quality and, in particular, of their own preferences for care. Recent research, by us and others, shows that the majority of residents with mild to moderate cognitive impairment can indeed provide useful, reliable information about the care they receive, the services they prefer, and their quality of life. In one study, Kane et al. (3) were able to interview 1,988 residents from 40 nursing homes in five states and, based on the results, develop Quality-of-Life scales for about 60% of them. “This was achieved,” the authors write, “even though at least half of the sample included the more impaired levels on a cognitive performance scale; only 19% of the sample had a perfect cognitive score, and 17% had the worse possible cognitive score (pg. M245).”
Our research shows similar results. In one study, we interviewed 111 incontinent residents to assess their satisfaction with incontinence and mobility care processes (4). To qualify for the study, incontinent residents had only to pass a simple responsiveness screen: They were asked to state their name or identify two common items, and were given chances on two separate days to pass the screen. An analysis of their responses showed that 75% of the sample provided logically consistent and stable answers to our interview questions about their received and preferred care levels.
In two other studies, we set out to identify a simple cognitive screen that would accurately identify residents capable of providing meaningful responses to satisfaction and preference questions (5, 6). Both studies showed that residents who score two or more on the Minimum Data Set Recall subscale can accurately describe the care they receive. Our more recent research shows even more encouraging results: About half of residents who score 1 on the MDS Recall subscale can reliably self-report pain and symptoms of depression, express meaningful preferences for daily care (they can tell you, for example, what activities they like or where they would like to have their breakfast or if they prefer to take an afternoon nap), and accurately describe care they receive on a daily basis (they can recall, for example, if staff helped them to the bathroom or provided walking assistance).
Back to Top
The "Gold Standard"
Based on these findings, there is growing consensus that residents’ self-reports represent the “gold standard” for measuring their care preferences as well as their quality of life—an inherently subjective construct that includes such domains as relationships, autonomy, privacy, and enjoyment—and are integral to quality of care assessments. After all, residents are the direct recipients of long term care—not their respective family members or health care providers.
In addition, there’s a side benefit to interviewing residents. Writes Kane et al. (3), “The very act of asking resident directly about their (quality of life) could engage staff directly and systematically with residents’ opinions about their daily existence in a way that seldom occurs in a typical (nursing facility). Such a process mitigates the tendency to depersonalize residents, and to view them merely as care recipients rather than people who live out their lives in difficult circumstances (pg. 247).”
The implications for nursing home care providers are inescapable: If you want to evaluate consumer satisfaction, quality of life, quality of care, call it what you will, then you must capture the voices of residents, including cognitively impaired residents, in your assessment. Surveying family members is an acceptable practice; they are important stakeholders in long-term care. But don’t canvass them at the cost of excluding their loved ones. Our Interview Protocol on the next page presents guidelines for selecting residents for interview.
Back to Top
Rule #2: Question the Questions
We turn now to the question of the questions themselves. Direct satisfaction questions, like those used by Nursing Facility A (“How satisfied are you with…[fill in the blank]?”), are a staple of consumer satisfaction surveys, but they are not an ideal choice for querying nursing home residents for two reasons.
The first is that they are prone to an acquiescent response bias; that is, residents will tend to respond favorably to these questions, despite known problems with the quality of care they are receiving. In effect, residents are giving answers they think you want, not necessarily expressing their own views.
Any questionnaire can inadvertently elicit an acquiescence bias among respondents, but consumer satisfaction surveys conducted with nursing home residents are especially likely to do this for several reasons. Older adults, and women in particular, tend to report higher rates of satisfaction with health care services; thus, there is a good chance that extremely old and frail nursing home residents, who are predominantly female, will report high rates of satisfaction with substandard or inadequate care. In addition, many residents lower their expectations for care as they reside in the facility over time. Their actual experience teaches them to expect and accept substandard quality of care. The fact that residents also are dependent on staff for daily care and many are isolated from family and friends can only decrease their willingness to express dissatisfaction with care due to fear of reprisal.
In order to collect data useful for quality improvement, your assessment questions must reveal both your facility’s strengths and weaknesses. You will not be able to identify areas that need improvement if resident responses to all or most of your questions cluster at the “highly satisfied” end of the scale, as responses to direct, forced-choice questions about satisfaction tend to do (4,7,8). You need, therefore, to ask questions that are more sensitive to differences in satisfaction levels.
Your questions also should elicit information that will help guide improvement efforts. A second problem with direct satisfaction questions is that they fail to do this. Though they may be able to tell you whether residents are generally satisfied or dissatisfied with a certain aspect of care, they shed no light on how to correct identified problems or how to tailor care to the individual; that is, how to provide more resident-centered care. Does the person want more privacy or less? Does she want to eat in the dining room or her own room? Does he receive enough help with toileting or does he want more? With quality improvement, as with many things in life, the devil is in the details. But the details are absent in direct satisfaction questions.
Back to Top
Four Types of Questions...
What are good alternatives to direct satisfaction questions? In two studies, we evaluated various interview strategies to identify questions that both tempered acquiescence response tendencies among residents and provided information useful for improving care (4, 7). We asked residents these four types of questions:
- Direct satisfaction questions about Activities of Daily Living (ADL) care (e.g., “Overall, are you satisfied with how often someone helps you to walk?”)
- Discrepancy questions that compared residents' preferences for ADL care frequency to their perceptions of the ADL care actually delivered (e.g., “How many times during the day would you like staff to help you walk?” vs. “How many times during the day do staff help you to walk?”)
- A second type of discrepancy question that compared residents’ preferences for ADL care frequency to how often they actually received care based on research staff observations
- Open-ended questions that asked what residents wanted changed about ADL care.
Back to Top
...Yield Mixed Responses...
Answers to our questions about walking assistance (4) are typical of the responses we received in other ADL care areas:
- When asked the direct satisfaction question, “Overall, are you satisfied with how often someone helps you to walk?” 80% of 111 residents interviewed said “yes,” a finding that suggests the facility is meeting the vast majority of residents’ needs in this area.
- Responses to the discrepancy questions suggest otherwise, however. Overall, 81% of the respondents reported a preference for more walking assistance than was provided by staff. Specifically, their reported preferences showed that they wanted an average of two more walk assists per day than staff were actually providing to them.
- Open-ended comments spontaneously provided by residents revealed a desire for change in aspects of care other than the frequency of assistance. One resident told us, for example, that she “would like to have somewhere important to go [when walking], such as an activity she was interested in attending, as opposed to just walking down the hall.”
From 80% satisfied, it now appears that 81% are dissatisfied, at least with the amount of walking assistance they receive.
Back to Top
...But Analyses Show That Some Question Top Others
What do we make of these results? Satisfaction with care is subjective; so when a resident tells us that overall, she is satisfied with the amount of walking assistance she receives but would like more of it, we are obliged to accept both statements. (And no, our analyses show that this response pattern is not related to a resident’s cognitive status.) That said, given our goal of collecting information useful for improvement efforts and given what we know of acquiescence, reduced expectations, and fear of reprisal among nursing home residents, it is appropriate to question the questions. How effective is each type? Findings from our studies show the following (4,7):
The proportion of residents reporting unmet needs for ADL care are significantly higher with the discrepancy and open-ended questions compared to the direct satisfaction questions. This suggests that the former questions are more sensitive to differences in satisfaction levels and that the latter questions are more limited by acquiescent response biases.
- Open-ended questions produce the most useful information for individualizing aspects of technical care and assessing the interpersonal quality of care.
- Discrepancy questions elicit specific information useful for changing the frequency or occurrence of ADL care; and, these questions are most sensitive to care quality improvements.
- Direct satisfaction questions are the least useful for designing improvement interventions and the most unreliable (when residents were re-interviewed within a day or two of their first interview, they were most likely to change their answers to questions of this type).
In sum, the direct satisfaction questions—most commonly used in nursing home surveys—proved the least useful and reliable. The discrepancy and open-ended questions are better choices for both care planning and quality improvement purposes.
In the next section, we present a quality-of-life assessment protocol that takes into account these findings as well as the mandate, born of research, not regulations, to interview residents as the best reporters of their care preferences and related quality of life. You can use this protocol to develop and implement an effective assessment strategy for your facility.
Back to Top
Try This Quick Assignment
- Choose an ADL care area in need of improvement in your facility.
- Develop a discrepancy question set that assesses residents’ preferences for care in this area. One question, for example, might ask how often the resident receives care in this area. The companion question would then ask how often the resident would like to receive care in this area. For examples, see our quality-of-life assessment forms.
- You can score such questions by subtracting the second answer from the first. For example, if a resident says he receives a shower 3 times per week but prefers a shower 5 times per week, then the discrepancy score is -2 (i.e., 3-5 = -2). The negative difference signals unmet needs.
- To assess resident satisfaction with other aspects of the care process (e.g., the way staff actually provides showers), pose a structured open-ended question: “If you could change something about your shower schedule or the way staff help you with your shower, what would it be?” You may find, for example, that in addition to preferring 5 showers per week, the resident also prefers that his shower be given in the morning before breakfast and that staff are not always careful about keeping him covered when transporting him to and from the shower room.
Back to Top or proceed Interview Protocol
