Clinician and Institution Level Variation:
The Road Ahead
Douglas A Colquhoun, MB ChB, MSc, MPH
Assistant Professor, University of Michigan
Associate Research Director, MPOG
Financial relationships:
- Medscape, Inc – Honorarium (2022)
Research Funding from (PI/Co-PI):
- National Institutes of Health
- Heart Lung and Blood Institute (K08-HL159327)
- Institute on Drug Abuse (R61-DA059168)
- Foundation for Anesthesia Education and Research (2021)
- Merck & Co – Research Funding involving Sugammadex (2019-2021 to U-M)
Disclosures:
Outline:
• Understanding variation in care is foundational to research and quality improvement
practice
• Why structure of healthcare organizations matters in thinking about variation
• MPOG Initiatives to incorporate and understand structure on variation for research and
quality improvement
Thank You!
• Synthesis of work of many across MPOG Missions
• Presenting these ideas and work from across MPOG team
• Feedback from many in the iterative development of this talk
Some (Fictional) Hospitals:
Hospital B Hospital CHospital A Hospital D
Some (Fictional) Hospitals:
Ballachulish
Regional
Coldbackie
District
Ardnamurchan
General
Drumnadrochit
Memorial
Mrs McTavish Needs Surgery:
Ballachulish
Regional
Coldbackie
District
Ardnamurchan
General
Drumnadrochit
Memorial
Mrs McTavish needs
Carpal Tunnel Release
Will she receive
General or Regional
Anesthesia?
5% GA 80% GA10% GA3% GA
Carpal Tunnel Under General – Really?
Harris et al. Hand. 2020
By Center (n = 113)
By Surgeon (n=780)
Rate of General Anesthesia
What is Unexplained Variation in Healthcare?
• Differences in healthcare process,
resources or outcome
• Measured across (examples):
– Time
– Patient groups
– Clinicians
– Hospitals
– Geographic
• “Care that differs in ways that are not a
direct and proportionate response to
available evidence; or to the healthcare
needs and informed choices of patients.”
State Level
Variation in
Neuraxial Use in
Labor
Butwick et al
JAMA Netw
Open 2018
Structure and Variation
• The clinician-patient interaction is informed
by professional training and prior experience
• But also impacted by facilities and resources
available and culture and “normal practice”
• Structures result in differences in patient
care
Structure and Variation
• The clinician-patient interaction is informed
by professional training and prior experience
• But also impacted by facilities and resources
available and culture and “normal practice”
• Structures result in differences in patient
care
• May be described as “levels”
Why Study Unexplained Variation? Quality Improvement
• Programmatic QI is about
understanding and minimizing
unexplained variation
• In areas where there are established
practices or standards
• Emphasizes role of processes,
equipment, training and practice
• Understanding contribution of
structural and systematic features
• Tension between responding to
individual events and looking for
patterns across organizational units
Why Study Unexplained Variation? Research
• Unexplained Variation as a Focus:
– Variation in practice and patient outcome
– Differences in practice across hospitals or
geographical areas
– Differences across clinicians
– Understand questions of equity and efficacy
of care
• Statistical approaches can quantify
contribution to overall variation
These effects are real…
Mathis MR et al
Anesthesiology 2023
McIssac D et al
Anesthesiology 2018
Pennington B et al
Unpublished
Janda AM et al
BJA 2022
Question
Inotrope Use in
Cardiac Surgery
GA vs Neuraxial
for Hip Fracture
TIVA for General
Anesthesia
Benzodiazepines
in Cardiac Surgery
Percent
Variation At
Each Level
Hospital 23% 20% 34% 55%
Clinician 6.8% 20% 18% 15%
Patient or
Unexplained
71% 60% 48% 31%
Why Study Unexplained Variation? Research
• Variation in Practice is Necessary for
Inference:
– Emerges from uncertainty
– Everybody is doing what they believe is best
for individual patient
– Foundational for understanding connection
between an exposure and an outcome
NMB Agent PPCs
Clinician experience
or preference
absent robust data
Accredited Center of
Excellence
Intractable Resource
Constraints
Precision Medicine
Patient’s Informed
Expectations
Examples of Warranted Variation
Patient Level Clinician Level
Hospital Level
Unwarranted Variation Addressed By:
QI Initiative
(individualized feedback)
Practice Guidance
Accreditation Boards /
Specialty Training
QI Initiative
(departmental structure
/ process focus)
Health Policy / Health
Services Research
Research to determine
efficacious clinical care
Inequities / Disparities
Research
Patient Level Clinician Level
Hospital Level
Looking at the Story of Hip Fractures:
Observing Variation: Inference Powered By Variation:
Looking at the Story of Hip Fractures:
STRUCTURES ARE EVERYWHERE AND
EVOLVING
Evolving Healthcare Landscape:
• Hospitals: Evolution and
Development of The Networked
Healthcare System
– Single hospitals expanding as
healthcare system
– Purchases/mergers with other
hospitals in surrounding area
– National chains and cross-market
mergers
• Clinician groups forming across
hospital or health system
boundaries
Current MPOG Community:
• Single Centers becoming
parts of health systems
• Mergers of health systems
participating within MPOG
• Concentration of
Healthcare Provision
• Anesthesiologists working
across multiple MPOG
hospitals
Fictional Examples – Hospitals Merge:
Coldbackie
District
Ballachulish
Regional
Ardnamurchan
General
Drumnadrochit
Memorial
Nevis Health
Will Ardnamurchan start
following Ballaculish’s
Best Practice?
Variation May Exist Within Health Systems:
• Health system networks offer
promise of sharing best practice
• Reality of variable integration across
hospitals within a network
• This can lead to significant variation
between sites
• Looking at USNWR “Honor Roll”
Health Systems:
– Complications: 1.1 – 4.3 x variation
– Mortality: 1.1 – 4.1 x variation
Fictional Examples – Dr Ferguson Gets a New Job:
Ballachulish
Regional
Coldbackie
District
Ardnamurchan
General
Drumnadrochit
Memorial
5% GA 80% GA10% GA3% GA
Will she start offering
Carpal Tunnel Repairs
under Regional?
Fictional Examples – New Anesthesia Group:
Coldbackie
District
Ballachulish
Regional
Ardnamurchan
General
Drumnadrochit
Memorial
Will Glenfinnan
Anesthesia Group
(GAS) mandate
regional anesthetics?
Clinician Changes
• Our structures are becoming more complex
– Clinicians work across hospitals
– Anesthesia groups cover multiple hospitals
– Multiple groups within a single hospital
– One health system might have multiple anesthesia
groups
Clinician Changes
• Our structures are becoming more complex
– Clinicians work across hospitals
– Anesthesia groups cover multiple hospitals
– Multiple groups within a single hospital
– One health system might have multiple anesthesia
groups
• More complex structures are possible:
– Cross classified structures
– Explainable if boundaries of clinicians and hospitals
Why does this matter? (Statistically)
• Two patients being cared for at the same hospital or
clinician are not independent
• Practice (and outcome) is correlated within health systems,
hospitals and clinicians
• Gives falsely precise estimates underestimates differences)
• Ignores opportunities to explore and understand sources
of warranted and unwarranted variation
30 Second Terminology Detour…
Mathis MR et al
Anesthesiology 2023
McIssac D et al
Anesthesiology 2018
Pennington B et al
Unpublished
Janda AM et al
BJA 2022
Question
Inotrope Use in
Cardiac Surgery
GA vs Neuraxial for
Hip Fracture
TIVA for General
Anesthesia
Benzodiazepines in
Cardiac Surgery
Percent
Variation At
Each Level
(ICC)
Hospital 23% 20% 34% 55%
Clinician 6.8% 20% 18% 15%
Patient or
Unexplained
71% 60% 48% 31%
30 Second Terminology Detour…
Mathis MR et al
Anesthesiology 2023
McIssac D et al
Anesthesiology 2018
Pennington B et al
Unpublished
Janda AM et al
BJA 2022
Question
Inotrope Use in
Cardiac Surgery
GA vs Neuraxial for
Hip Fracture
TIVA for General
Anesthesia
Benzodiazepines in
Cardiac Surgery
Measure of Variation ICC MOR ICC MOR ICC MOR ICC MOR
Variation At
Each Level
Hospital 23% 3.55 20% 2.36 34% 2.8 55% 4.19
Clinician 6.8% 1.73 20% 2.36 18% 2.2 15% 2.68
Patient or
Unexplained
71% 60% 48% 31%
Difference between ICC and MOR:
Problems To Solve:
1. Reflect organizational realities in hospitals and
clinician care groups
2. Use this to understand and explain clinician and
hospital level variation
Janda AM et al. BJA 2022
A Note of Caution
Attempts to identify individual patients, clinicians or
hospitals are against both the spirit and the letter of the
MPOG Data Use Agreement
“A Hospital, What is it?”
• Revisited data infrastructure for describing
hospitals and health systems
• Understand that some organizations consider
“hospital” differently
• Developed new location mapping hierarchies
• New hospital definition aligns with American
Hospital Association
– Provides objective reference standard
– Allows connection to this dataset to understand in detail
the available resources
Health
System
Hospital Hospital
OR OR OR OR
Unit
Why: Did Nevis Health Equalize Glucose ASPIRE Performance?
Ballachulish
Regional
Ardnamurchan
General
Nevis Health
Applying this to Clinicians
• Most clinicians in MPOG have an NPI number (National
Provider Identifier)
• Observe the same provider across multiple hospitals over
time with good confidence
• Required to sort out hospital effects from clinician ones
• Description of practice based on observed cases or
potentially joining to external datasets
Ardnamurchan
General
Drumnadrochit
Memorial
Explaining Hospital and Clinician Variation:
• Commonly large amounts of observed
variation attributable to Hospital or Clinician
• Explanatory variables typically focus on
patient or case factors
• Future opportunities exist to test hospital or
clinician related hypotheses:
– Hospital (examples): Bed Size, Case Volume,
presence of specialty services
– Clinician (examples): Practice pattern or
specialization, board certification, length of
practice
Explaining Hospital and Clinician Variation:
Hospital ID Bed Size L&D Volume Trauma
Center
2222 325 1200 Y
Clinician ID Cases Per
Year
C-Sections
Per Year
Board
Certified
3333 600 60 Y
Case ID Patient Age Preop Hb GA
C-Section
5555 34 11.6 Y
MPOG Records May Be Self Descriptive
• MPOG captures every electronic anesthetic record at a given hospital
• This creates a longitudinal and comprehensive picture of practice
• Must be created from the entire MPOG dataset, not just the patients meeting
inclusion/exclusion criteria for a study
• Future work: build a library of these, analogous to our case level phenotypes
Take Away Points:
• Variation is ubiquitous in healthcare and is central to both
research and quality improvement
• Structure of healthcare system must be understood to
ensure correct QI and Research solutions are used
• Healthcare Systems and Hospitals are complex
organizationally but can be reflected in MPOG Data.
• This allows correct attribution of variation by differences
in hospitals and clinicians
• MPOG Data may be self descriptive of practice to calculate
explanatory variables
