Clinician and Group CAHPS® Survey
In our previous posting, we discussed some background on the CAHPS® Clinician and Group Survey (CG CAHPS and identified the survey items where we found the largest differences between the physicians rated highest and lowest on the overall ratings question.
We might also note (failed to mention in our previous post) that PPACA requires Medicare to include “an assessment of patient experience and patient caregiver and family engagement” on the Medicare Physician Compare Website by January 2013. Further, the Star bonus system, being applied by CMS to all MA plans, calls for a number of measures that address the quality of the member / patient experience.
In this post, we report the results of looking at the same information in a different way. Please keep in mind that our analysis continues to be exploratory in nature and the specific findings are based on the data we used and cannot be generalized to all CG CAHPS results.
However, the approaches we are using can be applied to any set of results to arrive at a tailored direction for improving the physician-patient experience (at least as measured by CG CAHPS®). You can search the web and find plenty of advice for physicians and others interested in the physician-patient experience on how to improve that experience.
However, almost all of this advice is based on common sense and anecdotes. Our goal is to provide approaches that can be used by specific physicians and physician groups to get clear direction for improving the physician-patient interaction in an objective, intelligent and reasoned manner.
As described in our previous post, for this analysis, we used CG CAHPS® patient survey results for approximately 300 physicians. We looked at two kinds of items, those with response options categorical yes/no and those with response options never, almost never, sometimes, usually, almost always, and always.
All data were collected by mail survey. A sample of the questionnaire can be found at http://www.cahps.ahrq.gov/content/products/CG/PROD_CG_CG40Products.asp#V- S_Instrument .
For this analysis we used what is often referred to as a key driver approach. In this approach, we used the overall rating of the physician by the patient as the dependent variable and all of the ratings and yes / no questions as predictors. A complete listing of all the questions, in shortened form, can be found in the first of the following tables. Complete text can be found at the link provided above. In the table, we indicate whether each question is a yes / no or a ratings question. For the ratings questions, a 6-point scale is used with the options never, almost never, sometimes, usually, almost always and always. Please note that the CAHPS® Clinician and Group Survey has not been totally standardized and is currently available in 4- and 6-point scale versions. If you are making a decision regarding which option to choose, we recommend the 6-point option.
Our analysis included both factor and regression analyses. Without getting side tracked into a long discussion on technical issues, suffice it to say that factor analysis was used to address several issues; primary among these issues is the need to meet the non-collinearity (predictor variables are not correlated) regression assumption.
Factor analysis was used to group items with similar response patterns (or items that are correlated) together. The following table (Table 1) provides the factor analysis results that we got with this data set. The factors are actually composite variables created from the original input variables.
Factor Analysis Results for First 17 Factors
The table shows results for the first 17 factors. These 17 factors explain about 70% of the variance in the 48 total original variables used in the analysis. The percent of variance explained by each factor is provided in the last column.
The factors are listed in order of the percent of variance explained with the factor providing the most explanatory power listed first, second most second and so on. As you can see, the factors have a certain face validity. For example, the items most heavily loaded or correlated with the first factor relate to doctor listening carefully, doctor showing respect for what you said, doctor spending enough time with you, doctor explaining things so they are easy to understand, doctor giving easy to understand instructions about problems and doctor knowing important information about your medical history. Items most heavily correlated with the second factor also have a certain logical consistency and include doctor gave all information regarding care plan, doctor gave all information regarding treatment, doctor gave all information regarding care choices, doctor gave all information regarding follow-up care and doctor gave all information regarding medications. So, the first factor might be interpreted to as being related to Communication and Respect conveyed by the physician in the physician- patient interaction. The second appears to be related to the Information provided by the physician and so on for the other factors. We have provided names for each of the 17.
In the second phase of the analysis, we performed regression analysis using overall satisfaction with the physician as the dependent variable and the factors identified in phase one as predictors. Again, avoiding a lot of technical details, though we did analysis on the factors, it is possible to track back to the contribution of the individual variables that made up each factor. The results with the regressive coefficients we scaled on a zero to 100 scale to facilitate interpretation are provided in the table (Table 2) below.
First of all, let’s revisit the results of the analysis provided in our previous blog. For that analysis, we looked at differences on ratings questions and yes / no questions for those physicians in the top 20% based on overall patient satisfaction and those in the bottom 20%.
The five ratings ques
tions where we found the biggest differences were:
Complete text for the questions is as follows:
- Q22 In the last 12 months, when this doctor ordered a blood test, x-ray or other test for you, how often did someone from this doctor’s office follow up to give you those results?
- Q18 In the last 12 months, how often did this doctor seem to know the important information about your medical history?
- Q12 In the last 12 months, when you phoned this doctor’s office after regular office hours, how often did you get an answer to your medical question as soon as you needed?
- Q20 In the last 12 months, how often did this doctor spend enough time with you?
- Q19 In the last 12 months, how often did this doctor show respect for what you had to say?
The yes / no questions where we found the greatest differences between the top and bottom rated physicians were:
Complete text for the questions is as follows:
- Q17f In the last 12 months, did you and this doctor talk about things in your life that worry you or cause you stress?
- Q20f Choices for your treatment or health care can include choices about medicine, surgery, or other treatment. In the last 12 months, did this doctor tell you there was more than one choice for your treatment or health care?
- Q17e In the last 12 months, did you and this doctor talk about the exercise or physical activity you get?
- Q14a In the last 12 months, were any of the explanations this doctor gave you hard to understand because of an accent or the way he or she spoke English?
- Q17d In the last 12 months, did you and this doctor talk about a healthy diet and healthy eating habits?
Comparing these results to the regression results we find:
Three of the five ratings items where we found the biggest differences between top and bottom rated physicians are among the top six in importance in driving overall patient satisfaction with physician, including:
- Q19. How often doctor showed respect for what you said.
- Q20. How often doctor spent enough time with you.
- Q21. How often doctor knew important information about you medical history.
One of the items from the top five in the previous research fell to the middle of the pack:
- Q22. How often you received follow-up on test results.
And one fell near the bottom in importance:
- Q12. How often you received an answer to medical questions after hours.
Turning to the yes/no questions where we found the biggest differences between top and bottom rated physicians in our previous analysis we found:
Four of the five items fell fairly close together in the lower middle of the 48 items:
- Q17e. Doctor talked about exercise or physical activity.
- Q17d. Doctor talked about healthy diet and eating habits.
- Q17f. Doctor talked about worries and stressors.
- Q20f. Doctor indicated there was more than one treatment choice.
One fell near the bottom of the list in terms of overall impact on overall satisfaction with the physician:
- Q14a. Doctor was hard to understand because of accent.
In general, the yes / no questions performed more poorly in the regression analysis simply because they are binary in nature and have a limited range of variation.
Some conclusions that arise from the two analyses are:
- Showing respect for what patients have to say, spending enough time with them and knowing important information about their medical histories are shown to be highly important in both analyses.
- Issues related to wellness, though less important, still are shown to be important contributors to the patient experience in both analyses.
- Talking to patients about exercise or physical activity, discussing a healthy diet and eating habits and talking to them about worries and stressors contribute to their satisfaction with the patient experience.
Finally, two items related to tests and treatment are shown to positively impact the patient experience in both analyses:
- Making sure you follow up with test results.
- Offering patients treatment options and letting them participate in the decision making process.
® CAHPS is a registered trademark of the Agency for Healthcare Research and Quality (AHRQ).