Prior to developing an MPOG research project, new users should obtain access to DataDirect for self-serve data access by completing an MPOG DataDirect Attestation.
Additionally, it is highly recommended that researchers review the MPOG user Tips and Tricks video modules, and review examples of previous publications using MPOG data:
- Multicenter analysis of practice patterns regarding benzodiazepine use in cardiac surgery
- Hypoxemia in School-age Children Undergoing One-lung Ventilation
- Assessment of Perioperative Outcomes Among Surgeons Who Operated the Night Before
- Association of Anesthesiologist Staffing Ratio With Surgical Patient Morbidity and Mortality
Members of the Coordinating Center research team are also available to assist with the proposal writing process.
MPOG Coordinating Center members can be found below:
- MPOG Executive Director – Sachin Kheterpal, MD, MBA
- MPOG Research Director – Michael Mathis, MD
- MPOG Associate Research Director – Douglas Colquhoun, MB ChB, MSc, MPH
- MPOG Associate Research Director – Robert Schonberger, MD
- MPOG Lead Research Facilitator – Shelley Vaughn
- MPOG Research Facilitator – Diana O’Dell
- MPOG Research Facilitator – Leanna Delhey
- MPOG Research Assistant – Rebecca Pantis
- MPOG Research Assistant – Rachel Stumpf
Action Items:
- Obtain Access to DataDirect by completing an MPOG DataDirect Attestation.
- Review Tips and Tricks video modules
- Review examples of previous publications using MPOG data
As part of determining project feasibility, the primary author should complete a preliminary review of the MPOG data available and pertinent to the research proposal (via MPOG DataDirect, MPOG Concept Browser, and MPOG Phenotype Browser). This usually includes an initial Data Direct query and/or site-level analysis to ensure that the data quantity and quality are sufficient.
The primary author must download their local institution’s data to gain valuable insight as to the quality of the MPOG data, and they may elect to revise study methodology (i.e. study inclusion/exclusion criteria, analytic plan, exposure variables/covariates/outcomes collected) in order to mitigate obstacles to the proposed research. Although a complete single-center study using local institution data does not need to be performed as a prerequisite to PCRC, such data downloaded should be used to develop the “Preliminary Single Center Data” section of the PCRC proposal (including items such as summary statistics, data missingness/accuracy, etc.). Should difficulties be encountered at this stage, the MPOG Site PI and the MPOG Coordinating Center can be contacted to provide guidance.
Occasionally, the study team may determine that a test data download of their institution’s data is infeasible for the proposed research. Most often, this is due to the study team targeting a cohort of patients primarily at other MPOG institutions. In such cases, the study team must discuss this issue with the MPOG Coordinating Center and provide a brief explanation for not performing the test data download.
As the primary author begins formulating an idea but prior to drafting a proposal, they should discuss with their local institution’s MPOG Site PI:
- Proposed research question / study aims
- Whether the research question is potentially answerable using MPOG data
If the MPOG Site PI anticipates challenges, they should work with the primary author to develop a strategy to mitigate these challenges or develop an alternative research question.
Action Items:
- Query your local data using DataDirect
- Review MPOG concept and phenotype browsers using available tools on the MPOG website
- Discuss project idea with local MPOG site PI and revise study methodology as needed
- Develop Preliminary Single Center Data Section of the PCRC proposal
- It may be helpful to connect with a local statistician to help develop this section
After a discussion with the MPOG Site PI and review of data, the primary author must draft and submit a cover sheet (one-page limit) to the coordinating center briefly outlining the proposed research via the Research Consultation Form. The cover sheet can be found on the first page of the PCRC Proposal Template.
The coversheet and subsequent research proposal should be designed with the intent to publish a single manuscript. Exceptions may be made at the discretion of the PCRC committee at the time of PCRC review. However, in most cases, if multiple manuscripts are intended, each manuscript should be addressed via a separate research proposal and PCRC voting decision.
Submitting the cover sheet allows the MPOG Coordinating Center to maintain records of all projects and to provide preliminary feedback or connect the primary author with other MPOG researchers with overlapping research interests.
Once you submit a coversheet, it will be reviewed by the MPOG Central Research team, performed in batches (with notification of timing) once per month. After the batch review meeting, the MPOG Central Research team will provide emailed instructions on next steps.
Action Items:
- Draft and submit a cover sheet for your project
- Submit a research consultation request to review a coversheet for project feasibility
Once the coversheet has been approved by the MPOG Coordinating Center (i.e., confirmed feasible and not actively pursued by another research team), the primary author should then draft a PCRC proposal that includes the following:
- Data query specification (see example below)
- Each research proposal must be accompanied by a Data Query Specification that identifies MPOG concepts and phenotypes to be analyzed. To develop this specification, it is recommended that the primary author first review the structure of MPOG data, as available within the MPOG Concept Browser and MPOG Phenotype Browser.
- Example Data Query specification: Google Sheets
- Exception: for projects using the Standardized Data File, no query spec is needed
The primary author must identify an individual or a group within their institution to serve as the statistician(s) on the project. The research proposal should include a curated analytic plan that specifies the statistical techniques and methodologies for analyzing big data. These details are covered in the Cover sheet / study protocol template above.
Additionally, the primary author must obtain additional project-specific IRB approval at their institution to allow for analysis of a limited dataset (containing dates of service) for research.
Action Items:
- Draft the full PCRC proposal
- Complete the Data Query Specification
- Identify a local statistician for your project who is familiar with analyzing big data
- Draft IRB application at your local site
After completing all the PCRC proposal materials, the primary author must contact the MPOG Coordinating Center at mpog-research@med.umich.edu, and provide the following attachments:
- Cover sheet (from Step 3) – one page maximum
- Full study protocol including Introduction, Methods, Statistical Analyses, and References
- Data query specification
- PCRC Proposal Submission Checklist
Once the PCRC proposal materials have been received, a member of the MPOG Coordinating Center will confirm receipt and address any unresolved issues.
Upon scheduling a PCRC review date, the MPOG Coordinating Center will liaise with that month’s moderator and your study team.
Action Items:
- Submit PCRC proposal materials to MPOG central
- Complete the PCRC Proposal Checklist
The PCRC meets on the second Monday of each month, from 10:00 a.m. – 11:30 a.m. Eastern Time Zone. Finalized proposals (including any recommended revisions from the MPOG Coordinating Center) must be received four weeks prior to a meeting to be considered for review.
Members of the PCRC will evaluate proposals based on the following questions:
- Is the study as presented in the introductions and methods complete and comprehensive?
- Is the specific study question concisely presented?
- Will the data requested be suitable for answering the study questions?
- Are the proposed statistical techniques appropriate?
- Is the literature review complete and comprehensive?
- Does the study address a clinically significant healthcare issue?
- Revised PCRC presentations only: Does the study team adequately respond to the addressable feedback from the previous PCRC presentation?
The study team can either provide an oral summary of the proposed research (and the Coordinating Center can display the PCRC proposal document on the presentation screen), or a short presentation (e.g. PowerPoint). The presentation should last approximately 10 minutes.
Following the presentation, a PCRC Moderator Committee member (independent from the study team) will provide a short critique followed by a moderated discussion open to all PCRC members.
Following discussion of each PCRC presentation, eligible participating MPOG sites will vote on the presented project. Each MPOG site has one vote, collaboratively made by the site PI Champion, or site faculty designated by the site PI Champion, in discussion with other site faculty (e.g. Surgery Clinical Champion, Anesthesia IT Champion). There are four possible voting decisions: Approve, Approve with Revisions, Revise and Represent, and Reject.
More details on voting decisions can be found here.
Action Items:
- Submit your final proposal at least four weeks prior to PCRC meeting you wish to present at
- Prepare oral summary of the proposal or ~5-8 slides to present the PCRC meeting
- MPOG Research Central will follow-up with next steps depending on the voting outcome at the PCRC meeting
Congratulations! Your proposal has been approved by the MPOG community and you should now have access to the study data. An important initial step to most multicenter research projects is to visualize the MPOG data for the completeness and accuracy of each variable, recognizing the limitations to quality and heterogeneity of real-world, multicenter EHR data. We recommend evaluating each variable by institution over the years of the study. As an illustration of the process, consider this simplified hypothetical scenario as an example of what the first steps might look like.
The study team should consider publicly registering their research protocol, as required by some scientific journals. The goal of an a priori research protocol registration is to improve the openness, integrity, and reproducibility of the research performed. Study registration is ideally performed after an inspection and curation of the data (assessing data quality), but before inferential analyses of the data. Registration can be performed via the Open Science Framework, a scholarly commons for tracking projects through the research cycle. For the majority of MPOG projects, the recommended pathway is the Open-Ended Registration option on the OSF website.
MPOG requires use of a secure computing environment for all projects utilizing multi-center, MPOG data. Options for computing environments include:
- The Secure Enclave Services (SES) previously known as Yottabyte Research Cloud (YBRC) is a HIPAA compliant, Windows-based computing platform that allows for the secure analysis of data for MPOG research projects. This UM-based virtual machine environment is preloaded with many of the common statistical and programming language software. More information on the SES environment is available here. Use of SES is currently free of charge to MPOG researchers.
- ARMIS2 is the HIPAA compliant, Linux-based high-performance computing cluster that provides a secure, scalable, and distributed computing environment for our MPOG researchers that may need a more powerful computing option to complete their analyses. More information on the ARMIS2 environment is available here. Please note that the MPOG coordinating center currently covers the associated charges to use the ARMIS2 services.
Action Items:
- Evaluate the data quality for each variable (e.g. completeness, accuracy, heterogeneity, etc.). Recommend examining by institution, and by year over the study period
- Consider publicly registering your research protocol
- Analyze study data with local statistician in the Secure Enclave Server
For dissemination of study results prior to peer-reviewed manuscript publication, approval from the MPOG Perioperative Clinical Research Committee is required if the study results are made known via a permanent public record.
Prior approval required (permanent public record):
- Professional meeting with public indexing (e.g., ASA, IARS, SCA, SOAP, SPA, etc.)
- Annual MPOG Retreat
- Preprint servers (e.g., medRxiv, JMIR Preprints, F1000 Research, AMRC Open Research, etc.)
- News, Web, or Social media (e.g., news article, interview, website, posting on Twitter/X, Slideshare, etc.)
No prior approval required (recommend transparency as to unpublished nature of results):
- Regional, institutional, or departmental meetings without public indexing
- Grand Rounds presentations (if slides not shared on public record, e.g. SlideShare, etc.)
For forums requiring prior approval, or for situations not described above in which a permanent public record may be uncertain, the study team must contact the MPOG Central Research Facilitation team by filling out this Microsoft Form.