Patients want a smooth recovery

With six surgeries behind her, Linda was already a hospital veteran when she found herself in the emergency department last year. This time was different, though. Her most recent operation had fixed a major problem — a bowel obstruction — but new pain and vomiting brought her back to the local hospital weeks later. The diagnosis? Clostridium dificile, a hospital-acquired infection.

Surgery solves many health problems and often saves lives. But even successful operations have the risk — however small — of infection, bleeding or other complications that prompt patients like Linda to return to the hospital. Every patient is different, and that makes every surgery different, too. But patients have one thing in common: they want to know that their chances of a full recovery are high, and that the risk of complications is low.

Surgeons and hospital leaders want that as well, for Linda and for all patients. They take different approaches to ensure patient safety: one way involves studying how well patients recover from surgery, gathering the results and comparing them to other facilities. That’s what a group of Ontario surgeons and hospitals are doing as part of an international effort to improve patient care before, during and after an operation.


How does collecting new data help?

Ontario’s operating rooms are busy: surgeons performed over 600,000 scheduled operations [1] in the province in 2016-2017, with C-sections, knee and hip replacements, bone surgery and heart-related surgery among the most common inpatient procedures.[2] All surgical teams routinely track efforts to protect patient safety, including those that prevent complications. For example, recent data[3] show high uptake on steps to avoid surgical site infections: Ontario’s hospitals report 98% compliance in giving hip and knee replacement patients antibiotics before surgery to avert infection (range: 80-100%).

How can more data support better outcomes for patients? Tracking detailed surgical data before, during and after each operation can show hospitals and surgical teams exactly where they can improve patient outcomes.

An international program developed by the American College of Surgeons in 1994 — known as the National Surgical Quality Improvement Program — takes this approach. It collects and compares detailed clinical information from all stages of patient surgery. It looks at 14 specific outcomes — ranging from whether a patient developed an infection or breathing problems to severe outcomes, like death. The program is designed to improve the quality of care before, during and after an operation. It includes outcomes even after a patient leaves the hospital, which is when some complications may develop.

The program’s data do not identify patients to protect their privacy, but are adjusted by age and pre-existing illnesses to make sure that comparisons are relevant. That means that comparisons account for differences in how sick patients are, such that hospitals that treat sicker patients are not unfairly compared to those with healthier patients.

Though the program began in the U.S., it has expanded internationally and now includes 31 Ontario hospitals as members: The Ottawa Hospital was the first to sign on in 2010, followed by Sunnybrook Health Sciences Centre, the University Health Network and Hamilton Health Sciences.

In 2015, the province officially launched a surgical quality improvement initiative. These hospitals voluntarily participate in the Ontario Surgical Quality Improvement Network, which connects hospitals as they collectively work to improve care. The current 31 members span hospitals that treat both adults and children and represent organizations of all sizes — from urban to community and rural hospitals — as well as all regions of the province.

Other provinces in Canada participating in the broader program include British Columbia, Alberta, Saskatchewan, Quebec and Newfoundland. All participating provinces share their data in an international database.

The data sharing allows each of the nearly 700 member hospitals around the world to see how they are doing relative to their peers and where they have room to improve.

The number of Ontario hospitals (Figure 1) in the Ontario Surgical Quality Improvement Network has grown to 31 members, including three pediatric hospitals.

Tracking quality: surgeries included in this report

Network hospitals may track different surgeries through the program. Depending on their priorities, they can opt for an overview of all eligible surgeries or choose specific procedures. Health Quality Ontario included the following surgeries in this report: orthopedic surgery (excluding podiatry), oncology procedures, general and abdominal surgery, gynaecologic surgery, neurosurgery, plastic and reconstructive surgery, head and neck surgery and vascular surgery.

Current Ontario Surgical Quality Improvement Network Hospitals

South West
Grey Bruce Health Services (Owen Sound)
London Health Sciences Centre, University Hospital (London)

Waterloo Wellington
Grand River Hospital, Kitchener Waterloo Campus (Kitchener)
Guelph General Hospital (Guelph)

Hamilton Niagara Haldimand Brant
Hamilton Health Sciences, Hamilton General Hospital (Hamilton)
Hamilton Health Sciences, Juravinski Hospital (Hamilton)
McMaster Children’s Hospital (Hamilton)
Niagara Health System, St. Catharines Site (St. Catharines)

Central West
William Osler Health Centre, Brampton Civic Hospital (Brampton)
William Osler Health Centre, Etobicoke General Hospital (Etobicoke)

Mississauga Halton
Halton Healthcare, Oakville Trafalgar Memorial Hospital (Oakville)

Toronto Central
The Hospital for Sick Children (Toronto)
St. Michael’s Hospital (Toronto)
Sunnybrook Health Sciences Centre (Toronto)
University Health Network, Princess Margaret Hospital (Toronto)
University Health Network, Toronto General Hospital (Toronto)
University Health Network, Toronto Western Hospital (Toronto)

Central
Markham Stouffville Hospital Corporation (Markham)
Markham Stouffville Hospital (Markham)
North York General Hospital (North York)

Central East
Peterborough Regional Health Centre (Peterborough)
The Scarborough Hospital (Scarborough)

Champlain
The Children’s Hospital of Eastern Ontario (Ottawa)
The Ottawa Hospital (Ottawa)
Queensway Carleton Hospital (Ottawa)
Renfrew Victoria Hospital (Renfrew)

North Simcoe Muskoka
Collingwood General and Marine Hospital (Collingwood)
Orillia Soldiers’ Memorial Hospital (Orillia)

North East
Health Sciences North, Ramsey Lake Health Centre (Sudbury)
North Bay Regional Health Centre (North Bay)

North West
Sioux Lookout Meno Ya Win Health Care (Sioux Lookout)
Thunder Bay Regional Health Sciences Centre (Thunder Bay)

Focus on prevention

Linda's Story

Linda was hoping for a smooth recovery. After having emergency surgery to fix a bowel obstruction in 2016, she arranged for a home care nurse to help out. She’d had multiple hernia and bowel operations over the past 14 years and wanted to avoid complications, like infection, this time.

The nurse monitored the surgical incision and changed the dressing, but complications developed and the wound reopened. Linda also developed Clostridium difficile, a common infection.

With her medical history, Linda had a higher risk of common complications, notes Dr. Michael Lisi, chief of staff at Collingwood General and Marine Hospital. Because of this, she represented exactly the kind of patient a surgical quality improvement program aimed to help. Repeat visits and complications are hard on patients like Linda, he adds. “She’s been in and out of hospitals,” he says. “These are the individuals [who] require our attention for prevention.”

When Linda first became Lisi’s patient a few years ago, the hospital had not yet joined the program. At that time, the hospital didn’t have as clear a picture of post-surgical complications, says Lisi, “we just knew that complications happened.”

By the time she needed emergency surgery in 2016, the hospital had adopted some of the program’s best practices to improve recovery and minimize surgical site infections. Since Linda’s surgery was an emergency procedure, she couldn’t be part of the full Enhanced Recovery After Surgery process, which includes specific pre-op education and diet steps for patients to follow. However, she benefited from the hospital’s surgical site infection practices. “She’s a good example of [ the patient experience ] before the program,” he says. “Now that it’s underway, we’re now able to identify areas to focus improvement initiatives.”

After joining the program in 2015, Collingwood General and Marine has reduced its surgical site infection rates from 3.4% to 2.4% between January and July 2017. Lisi notes that the hospital continues to expand its quality improvement strategies. “We’re also working on length of stay and readmission, and have been identifying our pneumonia rates — those are slightly high — and we’re exploring why this is the case.”

Linda didn’t know that the hospital was part of a quality improvement program until later. “I thought it was great,” she says. After a nine-day hospital stay to treat the Clostridium difficile infection, she returned to the wound clinic each week, where specialists took pictures of her incision to track her progress. As she healed, she returned monthly and then less frequently.

Now, she’s fully recovered, and recalls charting her progress by the length of walks she took. First, she and her husband would walk down the street until they reached a small bench a short distance away. Gradually, they went further, eventually making it up and down their town’s main street, and along the waterfront.

How is she feeling now? “Really good. I’m back at work full time. I’m feeling great.” She looks back on her surgical journey with relief that the complications — and the hospital trips — are behind her. “I spent too much time there.”


How are participating Ontario hospitals performing?

The Ontario Surgical Quality Improvement Network focuses on common outcomes, ranging from whether a patient developed an infection or breathing problems to severe outcomes, like death.

Surgical Outcomes Tracked

1. Clostridium difficile colitis: A common hospital-acquired infection that affects the colon

2. Extended ventilation (over 48 hours): Most patients need a ventilator during general surgery, but needing one longer than 48 hours after surgery can increase their risk of pneumonia

3. Unplanned intubations: When patients have trouble breathing, they may need intubation—a tube inserted into their throat—to help

4. Pneumonia: An infection in the lungs that can develop post-surgery

5. Returning to the operating room: Patients may need another surgery to address a complication

6. Blood clots (venous thromboembolism): Patients who have major surgery are often at higher risk for blood clots because they aren’t active, and their blood flows more slowly

7. Hospital readmissions: Depending on the type of surgery, complications like infection or blood clots may mean that patients are readmitted to hospital

8. Heart complications: Major surgery can be stressful on a patient’s heart and can lead to heart attacks or cardiac arrest, depending on their underlying conditions

9. Kidney failure: Kidney injury during surgery can affect its function, or even lead to kidney failure

10. Sepsis: Surgery puts patients at higher risk of infection, which can develop into a widespread blood infection

11. Urinary tract infections: Catheters can increase the risk of infection in the bladder and urinary tract

12. Surgical site infections: These infections can affect the incision or the body part involved in the surgery

Overall Outcomes Tracked

13. Morbidity: Measures the rate at which patients experience the above complications during the 30 days following surgery

14. Mortality: Measures the overall rate of death in the patient population during the 30 days following surgery

Understanding the data

The Ontario Surgical Quality Improvement Network has 31 members, but this report reflects data from the 29 adult hospital members. On the graph (Figure 2), a single blue dot represents one of the Ontario adult hospitals participating in the program, so each organization can see how they rank relative to others on each of the patient outcomes tracked. The further to the left a hospital falls, the better the performance. There is wide variation among participating Ontario hospitals across each patient outcome. On certain outcomes, we can see a cluster of hospitals on the higher-performing end of the spectrum, such as for extended ventilation (over 48 hours). On others, we see several hospitals clustered on the lower-performing end, such as for surgical site infections.

Two additional comparisons, (Figure 3) mortality and morbidity, provide a summary view of each hospital’s surgical performance. Mortality measures deaths related to complications following surgery, and morbidity is a summary of multiple complications, such as infection, kidney failure or sepsis.

Small changes mean big benefits for patients

Dr. Rozario's Story

Dr. Duncan Rozario knows that even minor adjustments in surgical practices can yield big benefits, especially when it comes to better patient recovery. “It may be only a 0.1% change here, a 0.1% change there,” but the total effect of all those small changes adds up to lower surgical complications, he says.

When Oakville Trafalgar Memorial Hospital joined the quality improvement program in 2015, Rozario’s role was to determine where those changes were most needed.

After measuring patient outcomes for six months, the hospital found that its surgical site and urinary tract infection rates ranked poorly. The hospital —  part of Halton Healthcare — was one of the first Ontario hospitals in the program to implement a set of changes meant to prevent infection before, during and after surgery.

Halton Healthcare implemented seven program steps at the Oakville hospital: some involve the patient, having them shower with a hospital-grade antiseptic the night before and the morning of surgery. That practice has had good adoption, notes Rozario. “Most people are happy to do things to improve their surgery outcomes,” he says.

The hospital also refers patients to public health smoking cessation programs well before their surgery date. Smokers have a higher risk of pneumonia, blood clots and wound infections, Rozario says, noting that even the use of a nicotine patch for a month before surgery is far better for patient outcomes. Although those efforts have been slower to catch on, the overall results have been positive: “After six months, our surgical site infection rate dropped by half, [when] we were expecting maybe a 25% improvement,” he says.

Outside the operating room, Rozario points to an overall shift in mindset about general surgical patients — especially those with perforated colons, who historically experience high wound infection rates. “Those high numbers aren’t acceptable anymore,” he says.

He appreciates that the program lets hospitals here compare their progress against so many innovative hospitals in the U.S. “It pushes everyone to do better, here in Ontario,” he notes. “Sometimes you need a protocol and a program and a group behind you to make real change.”


To date, the tracking of outcomes serves as a learning tool for participating hospitals to focus improvement efforts, so the results for individual hospitals are not identified in this report. However, as the program evolves, and more Ontario hospitals have the opportunity to participate, the aim is to publicly report these results by hospital.

To make sense of all of these data, the Ontario surgical quality improvement initiative monitors the outcomes of participating Ontario hospitals as a group compared to all other participating hospitals internationally.

Within the program, hospitals are benchmarked along a spectrum that shows whether they fall into the ‘room for improvement,’ ‘as expected’ or ‘better than expected’ categories based on two things: their ranking among all hospitals in the program and the statistical difference in their performance compared to the larger group of international hospitals.

While Figures 2 and 3 show how individual hospitals rank against all network hospitals, Figure 4 shows how the group of hospitals compare using both the ranking method and the statistical significance.

Looking at the bigger picture, Figure 4 shows performance for Ontario member hospitals across all 14 outcomes. It identifies those outcomes that have all hospitals performing ‘better than expected’ or ‘as expected.’ For example, Clostridium difficile is one of the Ontario group’s best outcomes because all hospitals show ‘better than expected’ or ‘as expected’ performance. If a large proportion of hospitals show ‘room for improvement’ for an outcome, the group performance indicates that this outcome has ‘room for improvement.’

The two summary outcomes, morbidity and mortality, show that the majority of Ontario members are doing ‘better than expected’ or ‘as expected’ for mortality and a high proportion show ‘room for improvement’ for morbidity.

It’s important to acknowledge that international performance isn’t necessarily the absolute or only reference for ‘better.’ There may be room for hospitals to improve even if they’ve achieved ‘better than expected’ or ‘as expected’ performance.

Local leadership

While the American College of Surgeons developed the international quality improvement program, Health Quality Ontario plays a critical role in supporting the program here in Ontario.

In establishing the Ontario Surgical Quality Improvement Network, Health Quality Ontario is working to foster even more collaboration between members of the provincial network. In bringing the individual members together to share data and best practices, the network promotes cooperation and teamwork within the alliance. And, by building on their work— through conferences, workshops, digital tools and mentorship—the hope is that members of the provincial network will see quality improvement results even faster.

Health Quality Ontario also supports complementary programs: Improving and Driving Excellence Across Sectors lays the foundation for better care in partnership with medical schools, while the Institute for Healthcare Improvement Open School shares quality improvement methodology via online learning.

The collective approach means that surgical teams aren’t just working toward better care for their own patients, but for patients across Ontario.

How are participating Ontario hospitals doing?

Outcomes where all 29 hospitals are performing ‘as expected’ or ‘better than expected’:

  • Unplanned intubations
  • Extended ventilation (over 48 hours )
  • Clostridium difficile colitis

Outcomes where the majority of the 29 hospitals are performing ‘as expected’ or ‘better than expected’:

  • Return to the operating room
  • Mortality
  • Blood clots (venous thromboembolism)
  • Heart complications
  • Hospital readmissions
  • Kidney failure
  • Pneumonia
  • Sepsis

Outcomes where the 29 hospitals have the most ‘room for improvement’:

  • Surgical site infections
  • Urinary tract infections
  • Morbidity

Shorter stays, fewer complications

Dr. Singh's Story

Dr. Ravinder Singh believes in borrowing what’s working at other hospitals. So, when North Bay Regional Health Centre joined the quality improvement program, he already had a goal in mind: “I’m always thinking what can we do to make patients feel better, get home faster and stay home,” the hospital’s head of surgery says.

To do that, he looked to the on-call emergency surgeon concept, which has surgeons forego elective procedures like surgery, endoscopy and clinic for a week, dedicating that time to patient rounds in acute care. That concept, first developed at U.S. hospitals, had documented fewer complications and shorter hospital stays there. 

Since adding the on-call concept, the overall stay for general surgery patients admitted through emergency has gone down by two days, he says.  For appendicitis patients, it’s down by more than half a day, and for emergency gall bladder patients, “it’s a full day shorter,” he says. The sickest patients have benefited the most from the on-call surgeons in emergency, he notes, because the extra attention prevents further complications.

The hospital has also rolled out the Enhanced Recovery After Surgery program, which is meant to empower patients with their own care. When Singh learned of Mount Sinai Hospital’s post-surgery patient activity booklet — part of the recovery program, he adopted it for North Bay.  Patients are encouraged to fill out the booklet in detail, answering questions like, ‘Are you doing deep breathing?’ , ‘What did you eat and drink on day 1? Day 2?’ and ‘when did the catheter come out?’

Singh thinks the program is revolutionary in its challenge to surgical status quo. “It was developed to question the dogma around how we treat surgical patients,” he says.

Adding to those efforts, Singh has inspired others to pursue continuous quality improvement and has promoted surgical site infection prevention steps to other hospitals in the program. “It’s all about connecting and learning from each other,” he says. “Big or small, academic or non-academic hospital, we’re all trying to reach the same goals.”


How can better data become better care?

Once hospitals have their results, what does that mean for patients like Linda? Better, more detailed surgical data is often the starting point for efforts to improve care and establish goals for improvement. For instance, if the data showed a need for improvement in surgical site infections, surgical teams could rethink how they design and deliver care to address that complication.

The program and other health care organizations have developed specific steps surgeons, nurses, other care providers and patients can take to improve care and outcomes. These steps may seem small, but they represent two things: a marked effort to improve care; and to increase the knowledge and skills needed to undertake and sustain quality improvement.

One set of steps aimed at reducing patients’ risk of pneumonia after surgery includes special deep breathing exercises before and after surgery and coughing after surgery to clear the lungs and help distribute oxygen. It also calls for specific oral care — brushing teeth and using mouthwash to get rid of mouth bacteria, sitting upright in bed rather than lying flat and encouraging them to get up and stay active.

Another set of care steps aimed at reducing urinary tract infections advises surgical teams to restrict urinary catheter use, if it’s appropriate, and to ensure that catheters are inserted using the proper technique.

Ontario’s member hospitals are expanding their surgical safety efforts in different ways, and building on existing quality improvement measures. For instance, 24 members are now taking part in the Enhanced Recovery After Surgery process, which focuses on best practices for procedures like colon surgery that tend to have more complications. And, some members have already started sharing some of their successes with a broader group of health care organizations.

What are the opportunities for improvement? While some studies have shown that the program can prevent up to 500 complications per year[4], cut down on health care costs and save lives, others have pointed out that data alone won’t ensure gains[5] — only changes in hospital processes and culture will.

What do patients say about their surgery?

Dr. van Heest's Story

In the early days of William Osler Health System’s quality improvement efforts, Dr. Rardi van Heest found that people returned to the emergency department within 30 days of surgery for different reasons.

Pain was a major factor. Patients may not know how to use pain medication properly, a risk to them and their household if they had unused medication lying about, she notes.  Post-surgery patients also visit emergency for non-urgent conditions, including constipation, vomiting, swelling and redness around a surgical incision.

Van Heest and her team thought that if patients knew what to expect when they went home, the visits might decrease. So, they developed a Surgeon Discharge Instruction Sheet, outlining normal symptoms like gas pain, constipation and incisional pain. It also highlighted symptoms that required follow-up with the surgeon’s office, such as fever greater than 38.5 degrees or pain that wasn’t controlled by medication.

Within three months of using the instruction sheets, emergency visits by this group dropped by half. Van Heest is now sharing the sheet with other surgeons in the Ontario surgical quality improvement network so that more hospitals can benefit.

Next, she wanted to get a better sense of the patient experience. The hospital had already successfully reduced its surgical incision infection and urinary tract infection rates using the quality improvement program’s Enhanced Recovery After Surgery steps, and van Heest wanted to build on that. “I want to get a barometer of how we’re doing, right away,” she says. “I don’t see a lot of [research] on how the patients think their surgeries went.”

Many patients at Brampton Civic and Etobicoke General Hospitals are first-generation Canadians, she says, noting that English may not be their first language. This presented a challenge to the hospital’s surgical clinical reviewers, who speak to patients after surgery. She added two questions — related to patients’ overall surgical experience and how they felt it could be better — to the program’s standard questions. After gathering feedback from 400 patients at both hospitals, the results showed general satisfaction with interactions with hospital staff, particularly nurses; and less satisfaction in understanding system processes, wait times or post-surgical pain.

To improve their experience, patients have suggested more access to physiotherapy and follow-up care and greater staff availability for patients in need. “If a patient isn’t happy, regardless of what the documented outcome is, then we haven’t done our job,” she says.


The Road Ahead

At present, about half of all adult surgeries in Ontario are performed in Ontario Surgical Quality Improvement Network hospitals.[6] These hospitals review a representative number of surgeries for the program, yet all patients — regardless of their type of surgery — benefit from the data comparisons, new processes and overall quality improvement efforts.

There are early signs that care is improving: for example, the largest hospitals report progress in their work to reduce surgical site infections, and all of the Ontario members have made reducing all surgery-related infections—pneumonia, urinary tract infections and surgical incision infections— a priority.

Linda is fully recovered now, but other patients like her face complications during or after surgery every day. Ontario’s hospitals need to continue to improve surgical care just as they work to improve patient care in all parts of the health care system.

The participating hospitals’ efforts — and their cooperation with other organizations outside the network — show commitment to making surgery and recovery as safe as possible.

Moving forward, hospitals participating in the Ontario surgical quality improvement initiative will continue to collect and learn from these data, and address important opportunities for improvement. Health Quality Ontario is actively working to increase hospital participation in the program so that even more surgeries will be covered. In the future, as more hospitals have the opportunity to join the program, Health Quality Ontario aims to move towards routine public reporting of individual hospital performance on surgical outcomes.


Methods Notes

The Methods Notes provide a brief description of the data and methods used in this report. For more details, please see the American College of Surgeons National Surgical Quality Improvement Program website: https://www.facs.org/quality-programs/acs-nsqip

Data sources

The data presented in this report were provided by the American College of Surgeons National Surgical Quality Improvement Program, a program that builds a community of practice for surgical teams interested in surgical quality improvement. The program is based in the United States, and has an international membership that currently includes nearly 700 hospitals. A community of 31 hospitals at present across Ontario form the Ontario Surgical Quality Improvement Network.[7,8] Data are presented in this report for the 29 adult hospitals.

Hospitals that participate in the program collect clinical data on randomly selected surgical patients from the pre-operative period through to 30-days post-surgery.[7] Hospitals submit data to the program, which conducts risk-adjusted analysis on a defined set of postsurgical complications for all hospitals and the local network, and reports the data back to the hospitals by means of Semi-Annual Reports. Hospitals can view their data and see comparisons of their performance with other hospitals in their collaborative and to the larger network.[7]

The data used in this report were extracted from the July 2017 National Surgical Quality Improvement Program Ontario Collaborative Semi-annual report.[9]

List of Surgeries

The following is a list of surgical groups included in this report:

  • Orthopaedic Surgery (excluding podiatry)
  • Oncology Procedures
  • General and Abdominal Surgery
  • Gynaecologic Surgery
  • Neurosurgery
  • Plastic and Reconstructive Surgery
  • Head and Neck Surgery
  • Urologic Surgery
  • Vascular Surgery

Analysis

Methodology[8,9]

The program provided hospital performance data for a number of patient complications within 30 days post-surgery: Clostridium difficile colitis, ventilation for over 48 hours, unplanned intubation, pneumonia, return to operating room, venous thromboembolism, readmission, heart complications, renal failure, sepsis, urinary tract infection, and surgical site infection, as well as data for two summary measures, morbidity and mortality. For more detailed definitions on the complications, please see the American College of Surgeons National Surgical Quality Improvement Program website: https://www.facs.org/qualityprograms/acs-nsqip.

The program fits the data for each postsurgical complication to hierarchical logistic regression models that are weighted and risk-adjusted across all hospitals in the full collaborative to produce an odds ratio for each hospital.

Ontario’s performance

This report uses the program methodology that identifies the percentage of hospitals within the Ontario collaborative that fall into each of three program performance categories (‘better than expected,’ ‘as expected,’ ‘needs improvement’) for each post-surgical complication for Figure 4. For each hospital within each indicator, a performance categorization was assigned based on whether the hospital odds ratio a) fell into an extreme decile or b) was an outlier.

  • Extreme decile ranking. All odds ratios of the program hospitals in the international collaborative (including the Ontario collaborative hospitals) are ordered from smallest to largest and assigned to 10 equally sized sequential groups (deciles) representing 10% of all program hospitals. In this manner, we can see how the Ontario hospitals compare to hospitals in the program’s international collaborative. A hospital with an odds ratio in the first decile is within the 10% of hospitals that are the highest performers and a hospital with an odds ratio in the 10th decile is within the 10% of hospitals that are the lowest performers.
  • Outlier. A hospital is determined to be an outlier based on the confidence intervals for the odds ratio. If the confidence interval for a hospital is entirely above the value of 1.0, then the hospital is considered to be a high outlier (a hospital has a higher likelihood of a post-surgical complication). Conversely, if the confidence interval for a hospital is entirely below the value of 1.0, then the hospital is a low outlier (a hospital has a lower likelihood of a post-surgical complication). If the confidence interval contains 1.0, then the hospital is not an outlier (a hospital is doing as expected).

‘Better than expected’ performance is attributed to hospitals that are doing very well in comparison to other program hospitals for a post-surgical complication. This status is assigned to hospitals that had a decile rank of 1 or were low outliers. ‘Room for improvement’ status is assigned to hospitals that are not doing as well as other program hospitals for a postsurgical complication. This status is assigned to hospitals that had a decile rank of 10 or were high outliers.

‘As expected’ status is attributed to hospitals that are doing similarly in comparison to other program hospitals with regard to a postsurgical complication. This status is assigned to hospitals that had a decile rank of 2-9 and were not outliers.

Building off of the NSQIP methodology, we have highlighted the specific post-surgical complications where Ontario hospitals show higher performance or where there is room for improvement. Higher performance post-surgical complications are those that the program identified as having all hospitals performing ‘better than expected’ or ‘as expected’ for those complications. Post-surgical complications with ‘room for improvement’ are those that the program identified as having the largest percentage of hospitals that were in the ‘room for improvement’ category. For other post-surgical complications, the majority of hospitals performed ‘better than expected’ or ‘as expected.’

This report uses the program methodology that reports solely on decile ranking of hospital odds ratios across all hospitals for Figures 2 and 3. Please see the section above for decile ranking methodology. Figures 2 and 3 only display the ranks of Ontario collaborative hospitals, however all program hospitals were used to produce the decile ranking. For more information on the methodology please see the American College of Surgeons National Surgical Quality Improvement Program website: https://www.facs.org/quality-programs/acs-nsqip.


Acknowledgements

Surgical Leadership

Health Quality Ontario would like to acknowledge the contributions of:

Dr. Timothy Jackson BSc, MD, MPH, FRCSC, FACS

Provincial Lead, Ontario Surgical Quality Improvement Network, Health Quality Ontario Department of Surgery, University Health Network

And of the Ontario Surgical Quality Improvement Network Steering Committee:

Lee Fairclough, Dr. Tim Jackson, Dr. David Schramm, Dr. Husein Moloo, Dr. Robin McLeod, Dr. Avery Nathens, Dr. Duncan Rozario, Dr. Claude LaFlamme and Dr. Kyle Kirkham.

Board

Andreas Laupacis
Chair

Shelly Jamieson
Vice-Chair

Tom Closson

Jeremy Grimshaw

Stewart Kennedy

Bernard Leduc

Julie Maciura

Angela Morin

James Morrisey

Camille Orridge

Dorothy Pringle

Rick Vanderlee

Biographies are posted at: http://www.hqontario.ca/About-Us/Our-Mandate-and-Our-People/Board-of-Directors-and-Committees

Management

Dr. Joshua Tepper
President and Chief Executive Officer

Dr. Irfan Dhalla
Vice President, Evidence Development and Standards

Lee Fairclough
Vice President, Quality Improvement

Mark Fam
Vice President, Corporate Services

Anna Greenberg
Vice President, Health System Performance

Jennie Pickard
Director, Strategic Partnerships

Michelle Rossi
Director, Policy and Strategy

Jennifer Schipper
Chief of Communications and Patient Engagement

Dr. Jeffrey Turnbull
Chief, Clinical Quality

Biographies are posted at: http://www.hqontario.ca/About-Us/Our-Mandate-and-Our-People/Executive-Team

Report Development

Health Quality Ontario thanks the patients and providers, Linda, Dr. Duncan Rozario, Dr. Michael Lisi, Dr. Ravinder Singh and Dr. Rardi van Heest for sharing their experiences and insights for the stories and quotes in this report.

This report was developed by a multi-disciplinary team from Health Quality Ontario, including Heather Angus-Lee, Pierrette Price Arsenault, Tricia Beath, Maria Bennell, Lisa Bitonti-Bengert, Susan Brien, Shirley Chen, Jethro Cheng, Naushaba Degani, Gail Dobell, Ryan Emond, Terri Goveia, Louise Grenier, Binil Tahlan, Mina Viscardi-Johnson and Dave Zago.

Health Quality Ontario also acknowledges and thanks the American College of Surgeons National Surgical Quality Improvement Program® who provided clarification on methodology and data for the report, namely Emma Malloy, Gina Pope and Vanessa Thompson.

Parts of this material are based on data and information compiled and provided by CIHI. However, the analyses, conclusions, opinions and statements expressed herein are those of the author, and not necessarily those of CIHI.


References

  1. Ontario Wait Times. October, 2010. Ministry of Health and Long-Term Care. Available from: www.ontariowaittimes.com/surgeryDI/EN/2.
  2. Quick Stats Report 2015-2016. Ottawa (ON):Canadian Institute for Health Information.[cited 2017 Sept 29]
  3. The Self Reporting Initiative, Health Analytics Branch. Ministry of Health and Long-Term Care. [cited 2017 Sept 29]
  4. Hall, BL. Does surgical quality improve in the American College of Surgeons National Quality Improvement Program?: an evaluation of all participating hospitals.Ann Surg 2009 Sept [cited 2017 Sept 29] 250(3)
  5. Berwick, DM. Measuring surgical outcomes for improvement. Was Codman wrong? JAMA 2015 Feb 3 [cited 2017 Sept 29] 313(5)
  6. Your Health System Insight. Discharge Abstract Database 2009/10-2016-17. Canadian Institute for Health Information. [cited 2017 June 12]
  7. Facs.org. Chicago: American College of Surgeons National Surgical Quality Improvement Program [cited 2017 Oct 6] Available from: http://www.facs.org/quality-programs/acs-nsqip/aboutReferences
  8. ACS NSQIP® Semi-annual report. Chicago (IL)American College of Surgeons National Quality Surgical Quality Improvement Program; 2017 July.
  9. NSQIP-Ontario Collaborative® Semi-annual report.Chicago (IL) American College of Surgeons National Quality Surgical Quality Improvement Program; 2017, July.