Patients with higher postoperative pain after ambulatory shoulder surgery reported lower satisfaction: a prospective observational study

Introduction

In recent years, there has been an increase in the utilization of patient reported outcome measures (PROMs) to strengthen evidence-based treatments for the general population (1). This change has been concurrent with the US government implementing major changes in the way that healthcare is reimbursed, with emphasis now being placed on the quality and value of healthcare through value-based payments (VBP) (2,3). The Consumer Assessment of Healthcare Providers and Systems (CAHPS) was created to provide a standardized nation-wide data collection tool to measure patient’s perception of their hospital experience (4). The 2010 Affordable Care Act made one such survey, the Hospital CAHPS (HCAHPS), a mandatory portion of the calculation of hospitals total performance score (TPS). The TPS can determine as high as 2% of Medicare severity diagnosis-related group payments (3,5). Currently, HCAHPS and an adaptation of the CG (Clinician and Groups) CAHPS (6) are required by the Center for Medicaid and Medicare Services (CMS). However, the CMS plans to extend their assessment of patient satisfaction of care to outpatient surgical centers with a recently released CAHPS, the Outpatient Ambulatory Surgery CAHPS (OAS-CAHPS) (7).

There has also been a reported shift of performing orthopaedic surgery in an inpatient setting to ambulatory surgery centers or as outpatients (8). Shoulder surgery, in particular, has increased dramatically in the outpatient setting with early reports showing a 272% increase in a 10-year period (9). With policy changes incentivizing the use of ambulatory surgery centers and increased reports of comparable outcomes with inpatient surgery, the rates of outpatient shoulder surgery are only expected to increase (10,11).

Previous studies on CAHPS measurements in orthopaedics have centered around patient specific and hospital policy factors in H-CAHPS scores of total joint arthroplasty and spine patients (12-16). These studies have found higher satisfaction scores with shorter hospital stays, male gender, and private hospital settings. These factors are notably independent of surgical outcomes or techniques. Studies that have focused on CAHPS satisfaction in shoulder care have been limited to the role of patient-specific factors in satisfaction at clinic visits, satisfaction after inpatient total shoulder arthroplasty, and web-based education (7,17,18). However, CAHPS scores in outpatient shoulder surgery, specifically in sports medicine procedures, have not been reported. These scores will have eventual relevance for reimbursement and evaluation of quality of care. Thus, it is important for the orthopaedic surgeon to understand and determine the contributing factors of these scores so that they can adjust their practice as needed to maximize performance. The purpose of this study is to report CG-CAHPS and OAS-CAHPS scores after outpatient shoulder surgery. A secondary purpose is to identify how these satisfaction scores relate to other PROMs used to predict shoulder surgery outcomes as well as any related patient and surgery specific factors. We hypothesize that patient and surgery specific factors will not be associated with CAHPS satisfaction ratings based on the previous literature findings (12-16).

Methods

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the institutional review board (IRB) of the University of Cincinnati FWA #00003152 and informed consent was taken from all the patients (UC IRB Protocol #2019-0673). All patients who underwent outpatient shoulder surgery by the senior author (Grawe) from August 2019-March 2020 were eligible for this study. Patients undergoing total joint arthroplasty were excluded from this study. A total of 75 enrolled patients were approached pre-operatively to complete the following forms: a demographics form, Pain Catastrophizing Scale (PCS), short form-12 (SF-12), resiliency survey (RS-11), PROMIS Upper Extremity 7a (PROMIS UE), and the Shoulder Surgery Expectation Survey (SSES) adopted from Mancuso et al. (19). PCS quantifies an individual’s pain experience, SF-12 assesses the general health and quality of life, RS-11 measures resilience or the capability to adapt positively to adverse conditions, and PROMIS UE and SSES are joint specific surveys to assess function. These preoperative surveys were selected as they help capture important clinical factors that have the potential to influence patient satisfaction after surgery (17,18). Patients in this study had surgery at one of two ambulatory surgery centers, a surgery center on the campus of a Level I trauma center (Main Campus) or a satellite hospital. All patients in this study received an anesthesiologist-administered interscalene block, a standardized multimodal pain protocol, and were queried for a Visual Analog Scale (VAS) pain score at time of discharge.

After surgery, patients were contacted directly to evaluate their satisfaction levels using the CG-CAHPS and OAS-CAHPS surveys. We ensured there were no missing survey data. Retrospective chart review was performed to obtain preoperative diagnosis. There were five CG-CAHPS and five OAS-CAHPS domains included in the analysis. The five CG-CAHPS domains are as follows: Getting Timely Appointments, How Well Providers Communicate with Patients, Providers’ Use of Information in Care, Helpful, Courteous, Respectful, Office Staff, and Overall Rating of the Provider. The five OAS-CAHPS domains were: Facilities and Staff, Communication about the Procedure, Preparation for Discharge and Recovery, Overall Rating of the Facility, Recommendation of the Facility. Average CG-CAHPS and OAS-CAHPS were calculated as previously described (18). To evaluate the relationship between patient or surgery specific factors and satisfaction scores the top box scoring method was employed. This method developed by the Agency for Healthcare and Research Quality guidelines converts all categorical and numerical responses into binary responses. The top-box score is the most positive score(s) for a given items response scale (20). This score was assigned a value of one and all other values were assigned values of zero. These values were then used to calculate an overall satisfaction score for each survey.

Statistical analyses

The relationship between patient and surgery specific factors and satisfaction was evaluated using OAS-CAHPS and CG-CAHPS top-box scores. A cut off of four was used for VAS due to previous report of the minimal clinically important difference for pain control after adequate analgesia (21). Overall satisfaction scores for each of the respective CG-CAHPS and OAS-CAHPS domains were compared using Welch’s F test due to violation of the homoscedasticity assumption needed for ANOVA testing. Post-hoc analysis between CG-CAHPS and OAS-CAHPS respective domains was performed using the Games-Howell method.

To evaluate the impact of pre-operative PROMs on overall CG-CAHPS and OAS-CAHPS scores, the survey results were converted to binary levels using a 33^rd percentile threshold as previously described (18-22). A linear regression analysis was then performed to evaluate the influence of age, body mass index (BMI), gender, smoking status, preoperative function as evaluated by the PROMIS UE score, and VAS pain score at discharge on CG-CAHPS and OAS-CAHPS satisfaction status. Age, BMI, gender, and smoking status were chosen due to their demonstrated importance in previous literatures (12-18). The PROMIS UE and VAS pain scores were chosen due to their significance on univariate analysis. Statistical analyses were performed using R (R Foundation for Statistical Computing, Vienna, Austria). P value of less than 0.05 has been decided as the threshold to demonstrate statistical significance.

Results

A total of 68 (90.7%) patients were able to be reached to obtain satisfaction scores. The average age of included patients was 51.7±16.2 (range, 15–82). There were 44 (64.7%) men and 24 (35.2%) women. The average CG-CAHPS score was 91.3±11.0 (range, 42.1–100) and the average OAS-CAHPS score satisfaction was 93.4±6.2 (range, 69.1–100). Top-box CG-CAHPS and OAS-CAHPS satisfaction was 71.6±19.2 (range, 14.1–95.8) and 80.3±12.0 (range, 29.2–95.8), respectively. Welch’s F test of scores between CG-CAHPS (P<0.001) and OAS-CAHPS (P<0.001) domains demonstrated that there were significant differences between each respective survey’s categories (Tables 1,2). Post-hoc analysis of the CG-CAHPS domain scores revealed that “Getting Timely Appointments” had significantly lower satisfaction than all other domains. OAS-CAHPS “Preparation for Discharge and Recovery” had significantly lower scores than the “Facilities and Staff”, “Communication about the Procedure”, and “Recommendation of the Facility”.

Average VAS pain at the time of discharge was 2.57±2.87 (range, 0–9). There were no significant results between the groups in terms of age, BMI, gender, smoking status, ASA class, or preoperative diagnosis on overall top-box CG-CAHPS or OAS-CAHPS satisfaction scores (Tables 3,4). VAS pain greater or equal to four at the time of discharge had a statistically significant lower satisfaction score for both the CG-CAHPS (P=0.05) and OAS-CAHPS survey (P=0.03). There was no significant difference in OAS-CAHPS scores for each between surgery performed at the main campus or satellite hospital.

The mean pre-operative total SF-12, PCS, RS-11, SSES, and PROMISE UE scores were 87.3±13.4 (range, 53.9–117.3), 13.8±12.4 (range, 0–46), 66.6±14.4 (range, 13–77), 42.1±13.5 (range, 22–85), and 36.6±8.4 (range, 16.3–50.9). Only PROMIS UE scores demonstrated a significant difference between higher and lower OAS-CAHPS score groups (Tables 5,6).

The relationship of age, BMI, smoking status, shoulder function as assessed by the PROMIS UE, and VAS pain score at discharged were assessed using linear regression modeling. While demographic variables and shoulder function were not significantly associated with CG-CAHPS or OAS-CAHPS survey results, a VAS pain score of four or greater at discharge had a statistically significant relationship with both lower CG-CAHPS (P=0.05) and OAS-CAHPS (P=0.008) top-box scores (Table 7).

Discussion

The results of the present study demonstrated that patient satisfaction with the clinician and ambulatory surgery center is not associated with demographic factors, surgical characteristics, or preoperative shoulder function, instead being independently associated with pain levels at discharge. CG-CAHPS domain analysis revealed that patients included in this study were less likely to be satisfied with the providers ability to provide timely appointments. OAS-CAHPS scores were likely to be lower due to lack of information regarding discharge procedures and recovery. Patient satisfaction does not appear to depend on location of their procedure. Overall, these results coupled with the decreased satisfaction seen with higher pain levels indicates that patient CAHPS satisfaction after outpatient shoulder surgery is primarily based on postoperative pain levels and access to timely appointments.

Previous studies on outcomes of shoulder surgery and patient expectations, PROMIS UE, SF-12 scores, and resiliency have shown that there are patient-specific factors can predict outcomes following shoulder surgery (23-26). In addition, a study on non-operative shoulder pain found higher CAHPS satisfaction with improved function and PROMIS scores (18). While this study did show that PROMIS scores were higher in patients who were more satisfied according to the OAS-CAHPS survey, this effect was not independent of patient demographics.

The findings in this study coupled with the absence of relationship with pre-operative diagnosis and CAHPS satisfaction lead the authors to believe that the CAHPS satisfaction scores seen in an outpatient surgery setting are a product of experiences with the clinician and surgery center rather than the patient’s personal outlook on their injury or condition. The relationship between pain and satisfaction after shoulder surgery is not novel (23), however, coupled with the findings of a lack of correlation with PCS scores and significantly lower OAS-CAHPS “Preparation for Discharge and Recovery” domain scores suggest that this again stems from an overall unfavorable patient institutional experience. In general, the CAHPS satisfaction in this study seemed to be more dependent on the processes of care rather than other variables.

A previous study on shoulder surgery and CG-CAHPS satisfaction found that patients who were smokers were likely to have lower scores. However, the smoking restrictions in an inpatient setting do not apply to the patients in this study (17). In a study of CG-CAHPS score of patients presenting to an outpatient visit for shoulder pain, VAS functional score was the most powerful predictor of satisfaction of patient satisfaction (18). However, the patient group studied was a mix of surgical and non-surgical patients. In a separate study evaluating the effectiveness of an online educational tool on OAS-CAHPS satisfaction after outpatient orthopedic surgery, use of the tool was found to be related to significantly higher satisfaction scores in the “Recovery” domain (7). The present study found significantly lower scores in the same OAS-CAHPS domain. This suggests that the “Recovery” or “Preparation for Discharge and Recovery” domain may be an area for targeted improvement in OAS-CAHPS scores.

There were two methods used to assess CAHPS satisfaction in this study. Average and top-box scoring. The results demonstrated much higher average satisfaction scores when compared to top-box scores. Only the top-box scores are currently used to compare results across institutions (4), however, this study demonstrates that when used as a tool to evaluate performance on a single-system level the difficulty in achieving the top-box score must be taken into account. Given the increasing dependence that hospital systems place on these scores as evaluation metrics, the clinician should be aware of the discrepancy between reported top-box score and average scoring.

The high level of satisfaction of patients after outpatient surgery has been well-established in total knee and shoulder arthroplasty (27,28). However, little has been described on the difference between satisfaction in outpatient surgery at different types of ambulatory surgery centers, an important consideration, as at least 29% of the ambulatory surgery centers are associated with a larger medical campus (29). This study found no differences between satisfaction rates at the two different locations.

Limitations

This study poses limitations that primarily stem from the CAHPS surveys used. First, using the top-box system groups patients who may have been relatively satisfied and very unsatisfied together. This may artificially lower overall CAHPS satisfaction scores and dilute any associations between demographic or pre-operative surgery survey results. In addition, there was likely some recall bias from the difference in response time for the surveys, a factor that has been previously established to cause differences in CAHPS scores (30). The authors of the study attempted to limit the difference in time elapsed and surgery between respondents, however, variations were unavoidable due to the timing of the study coinciding with the novel coronavirus pandemic. Finally, we did not conduct a power analysis prior to arriving at the sample size of 75 patients. The sample size may not have been large enough to capture significance of mild to moderate strength. However, as all of these patients had their procedures performed by a single sports medicine surgeon variability in experience was limited.

Conclusions

Top-box calculation generally yields lower satisfaction scores than use of the average-score method despite being the preferred methodology by the CAHPS. Patients who had higher VAS scores for pain at time of discharge report lower levels of clinician (CG-CAHPS) and ambulatory surgery center (OAS-CAHPS) satisfaction after outpatient shoulder surgery. Overall, patient CAHPS satisfaction seems to be dependent on the processes for delivery of care.

Acknowledgments

We would like to thank the Department of Orthopaedics and Sports Medicine and the University of Cincinnati Medical Center for their support throughout the study. We would also like to thank all of our administrative staff who make everything possible.

Funding: None.

Footnote

Data Sharing Statement: Available at https://asj.amegroups.com/article/view/10.21037/asj-22-11/dss

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://asj.amegroups.com/article/view/10.21037/asj-22-11/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the IRB of the University of Cincinnati FWA #00003152 and informed consent was taken from all the patients (UC IRB Protocol #2019-0673).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Brinker MR, O'Connor DP. Stakeholders in outcome measures: review from a clinical perspective. Clin Orthop Relat Res 2013;471:3426-36. [Crossref] [PubMed]
2. Porter ME. What is value in health care? N Engl J Med 2010;363:2477-81. [Crossref] [PubMed]
3. Stein SM, Shah SS, Carcich A, et al. A Novel Approach to Improving Patient Experience in Orthopedics. Am J Med Qual 2017;32:655-60. [Crossref] [PubMed]
4. CMS. CAHPS Hospital Survey. Accessed August 2, 2020. Available online: https://www.hcahpsonline.org/
5. Centers for Medicare & Medicaid Services (CMS). HHS. Medicare program; hospital inpatient value-based purchasing program. Final rule. Fed Regist 2011;76:26490-547. [PubMed]
6. CAHPS for MIPS Survey | CMS. Accessed August 2, 2020. Available online: https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/MIPS
7. van Eck CF, Toor A, Banffy MB, et al. Web-Based Education Prior to Outpatient Orthopaedic Surgery Enhances Early Patient Satisfaction Scores: A Prospective Randomized Controlled Study. Orthop J Sports Med 2018;6:2325967117751418. [Crossref] [PubMed]
8. Goldfarb CA, Bansal A, Brophy RH. Ambulatory Surgical Centers: A Review of Complications and Adverse Events. J Am Acad Orthop Surg 2017;25:12-22. [Crossref] [PubMed]
9. Colvin AC, Egorova N, Harrison AK, et al. National trends in rotator cuff repair. J Bone Joint Surg Am 2012;94:227-33. [Crossref] [PubMed]
10. Buterbaugh KL, Liu SY, Krajewski A, et al. Safety of Outpatient Shoulder Surgery at a Freestanding Ambulatory Surgery Center in Patients Aged 65 Years and Older: A Review of 640 Cases. J Am Acad Orthop Surg Glob Res Rev 2018;2:e075. [Crossref] [PubMed]
11. Nwankwo CD, Dutton P, Merriman JA, et al. Outpatient Total Shoulder Arthroplasty Does Not Increase the 90-Day Risk of Complications Compared With Inpatient Surgery in Prescreened Patients. Orthopedics 2018;41:e563-8. [Crossref] [PubMed]
12. Eftekhary N, Feng JE, Anoushiravani AA, et al. Hospital Consumer Assessment of Healthcare Providers and Systems: Do Patient Demographics Affect Outcomes in Total Knee Arthroplasty? J Arthroplasty 2019;34:1570-4. [Crossref] [PubMed]
13. Peres-da-Silva A, Kleeman LT, Wellman SS, et al. What Factors Drive Inpatient Satisfaction After Knee Arthroplasty? J Arthroplasty 2017;32:1769-72. [Crossref] [PubMed]
14. Mets EJ, Mercier MR, Hilibrand AS, et al. Patient-related Factors and Perioperative Outcomes Are Associated with Self-Reported Hospital Rating after Spine Surgery. Clin Orthop Relat Res 2020;478:643-52. [Crossref] [PubMed]
15. Yu S, Dundon J, Solovyova O, et al. Can Multimodal Pain Management in TKA Eliminate Patient-controlled Analgesia and Femoral Nerve Blocks? Clin Orthop Relat Res 2018;476:101-9. [Crossref] [PubMed]
16. Boylan MR, Slover JD, Kelly J, et al. Are HCAHPS Scores Higher for Private vs Double-Occupancy Inpatient Rooms in Total Joint Arthroplasty Patients? J Arthroplasty 2019;34:408-11. [Crossref] [PubMed]
17. Matar RN, Shah NS, Vincent JC, et al. Factors that influence inpatient satisfaction after shoulder arthroplasty. J Shoulder Elbow Surg 2021;30:e165-72. [Crossref] [PubMed]
18. Bedeir YH, Grawe BM. Patient factors influencing outpatient satisfaction in patients presenting with shoulder pain. J Shoulder Elbow Surg 2018;27:e367-71. [Crossref] [PubMed]
19. Mancuso CA, Altchek DW, Craig EV, et al. Patients' expectations of shoulder surgery. J Shoulder Elbow Surg 2002;11:541-49. [Crossref] [PubMed]
20. Aggregated Data | The CAHPS® Database. Accessed August 3, 2020. Available online: https://cahpsdatabase.ahrq.gov/cahpsidb/
21. Lee JS, Hobden E, Stiell IG, et al. Clinically important change in the visual analog scale after adequate pain control. Acad Emerg Med 2003;10:1128-30. [Crossref] [PubMed]
22. Abtahi AM, Presson AP, Zhang C, et al. Association Between Orthopaedic Outpatient Satisfaction and Non-Modifiable Patient Factors. J Bone Joint Surg Am 2015;97:1041-8. [Crossref] [PubMed]
23. Rauck RC, Ruzbarsky JJ, Swarup I, et al. Predictors of patient satisfaction after reverse shoulder arthroplasty. J Shoulder Elbow Surg 2020;29:e67-74. [Crossref] [PubMed]
24. Tokish JM, Kissenberth MJ, Tolan SJ, et al. Resilience correlates with outcomes after total shoulder arthroplasty. J Shoulder Elbow Surg 2017;26:752-6. [Crossref] [PubMed]
25. Henn RF 3rd, Kang L, Tashjian RZ, et al. Patients' preoperative expectations predict the outcome of rotator cuff repair. J Bone Joint Surg Am 2007;89:1913-9. [Crossref] [PubMed]
26. Koorevaar RCT, Haanstra T, Van't Riet E, et al. The development of the Patient Expectations of Shoulder Surgery survey. J Shoulder Elbow Surg 2017;26:1701-7. [Crossref] [PubMed]
27. Kelly MP, Calkins TE, Culvern C, et al. Inpatient Versus Outpatient Hip and Knee Arthroplasty: Which Has Higher Patient Satisfaction? J Arthroplasty 2018;33:3402-6. [Crossref] [PubMed]
28. Leroux TS, Zuke WA, Saltzman BM, et al. Safety and patient satisfaction of outpatient shoulder arthroplasty. JSES Open Access 2018;2:13-7.
29. Elhag D, Dexter F, Elhakim M, et al. Many US hospital-affiliated freestanding ambulatory surgery centers are located on hospital campuses, relevant to interpretation of studies involving ambulatory surgery. J Clin Anesth 2018;49:88-91. [Crossref] [PubMed]
30. Hargraves JL, Cosenza C, Elliott MN, et al. The effect of different sampling and recall periods in the CAHPS Clinician & Group (CG-CAHPS) survey. Health Serv Res 2019;54:1036-44. [Crossref] [PubMed]