Slide Write a Research Proposal
The research committee coordinates the clinical research efforts of MPOG, by reviewing all submitted research proposals and tracking the progress of ongoing research projects. The committee ensures the appropriateness of the clinical research conducted within MPOG and the use of MPOG resources. Only colleagues from active MPOG sites can submit a research proposal to the PCRC for review and it is the responsibility of the MPOG site's Principal Investigator (PI) to review all proposals before submission to the PCRC.

The following steps outline the process for submitting a research proposal:

Prior to developing an MPOG research project, new users should obtain access to necessary research tools by completing an MPOG DataDirect Security Checklist & Authorization.

Additionally, it is highly recommended for researchers to review the new MPOG user Tips and Tricks video series:

Prior to drafting a proposal, the primary author should discuss with their institution’s MPOG Site PI:

      • Proposed research question / study aims
      • How MPOG data will be used to answer the research question

The MPOG Site PI should provide the primary author with guidance on whether the research question is potentially answerable using MPOG data. If challenges are anticipated, the MPOG Site PI should work with the primary author to develop a strategy to mitigate these challenges, or develop an alternative research question.

After a discussion with the MPOG Site PI and review of MPOG data (as available via MPOG DataDirectMPOG Concept Browser, and MPOG Phenotype Browser), the primary author must submit a cover sheet (limit to one-page) to the coordinating center at mpog-research@med.umich.edu briefly outlining the proposed research.  The cover sheet can be found on the first page of the PCRC Proposal Template (see Step 3 below).

The purpose of submitting the cover sheet at this early stage is for MPOG central research record keeping; in some cases, the MPOG central team may provide preliminary feedback, or connect the primary author with other MPOG researchers with overlapping research interests. 

Completed Concurrently with Steps 4 and 5

After identifying a research question answerable using MPOG data, the primary author must obtain research project-specific IRB approval and draft a PCRC proposal to include the following:

      • Cover sheet – one page maximum
      • Full study protocol including Introduction, Methods, Statistical Analyses, and References
      • Data query specification (see Step Four)
      • PCRC Proposal Submission Checklist

Please note that all MPOG research projects require a separate, research project-specific IRB to allow for analysis and use of a limited dataset (containing dates of service) for research. Language describing the use of the HIPAA-compliant MPOG external statistical server, to potentially be included in the IRB, can be found here.

Additionally, research proposals should be designed with an intent to publish a single manuscript. Exceptions may be made at the discretion of the PCRC committee at the time of PCRC review; however in most cases, if multiple manuscripts are intended, each manuscript should be addressed via a separate research proposal and separate PCRC voting decision.

Templates for drafting the PCRC proposal are available here:

An example of a completed PCRC proposal is available here:

The primary author must identify an individual or a group within their institution to serve as the statistician(s) on the project. The research proposal should include a curated analytic plan that specifies the statistical techniques and methodologies for analyzing big data. These details are covered in the Cover sheet / study protocol template above.

Completed Concurrently with Steps 3 and 5

To accompany each research proposal must be a Data Query Specification, identifying MPOG concepts and phenotypes to be analyzed. To develop this specification, it is recommended that the primary author first review the structure of MPOG data, as available within the MPOG Concept Browser and MPOG Phenotype Browser.

Once familiarized with the content and quality of the data available within MPOG, the primary author must create a Data query specification through MPOG DataDirect. To obtain access to DataDirect, the primary author (and/or analysts) will need to complete the DataDirect Security Checklist and Authorization form and return it to the MPOG Central Team.

For ease of tracking, MPOG DataDirect enables users to save and share MPOG query IDs. To enable access to the necessary data for the proposed research, a Data query specification may be composed of multiple individual MPOG query IDs. At a minimum, the primary author should plan to develop at least two MPOG query IDs:

1. MPOG query for inclusion criteria – in order to generate a cohort of patients/cases to be analyzed, an MPOG query including all concepts/phenotypes, institutions, and date ranges must be developed in a unique MPOG query.

      • Importantly, the research team may refine their queried cohort via additional exclusion criteria, in order to develop a final analytic dataset. This is analogous to the starting population of a CONSORT flow diagram (inclusion criteria), and final population analyzed (exclusion criteria). However, such plans for exclusion criteria must be specified a priori in the PCRC proposal.

2. MPOG query for exposure variables, covariates, outcomes, and exclusion criteria – within the cohort of patients/cases identified from the inclusion criteria (#1 above), all data necessary for executing an analysis of a curated dataset must be separately developed in a unique MPOG query. Often, this will include both:

      • Variables occurring once per case (i.e. single measures); such as patient comorbidities, case type, and postoperative outcomes.
      • Variables occurring multiple times per case (i.e. repeated measures); such as physiologic, medication, and anesthesia provider data.

For more information on how to properly submit MPOG query IDs comprising a full MPOG Data Query Specification, information will soon be added to the MPOG DataDirect User Guide.  Please check back for the updated user guide.

Completed Concurrently with Steps 3 and 4

Based on the Data query specification developed (with both inclusion and exclusion criteria), the primary author must determine the approximate study sample size.

In some instances, sample sizes may be unexpectedly small. In such cases, the primary author will need to inspect each individual inclusion/exclusion criteria, through sequentially adding/removing the criteria within Cohort Identification Mode of MPOG DataDirect, and assessing the impact to final sample size. If continued difficulties are encountered at this stage, the primary author should first contact their institution’s MPOG Site PI for guidance. After a discussion with the institution’s MPOG Site PI, the primary author may consider contacting the coordinating center for additional guidance.

In other instances, sample sizes may be exceptionally large. For very large datasets (>500,000 patients and/or using variables with multiple values per case, e.g. physiologic or medication data), the study team must contact the coordinating center to develop a plan for a computationally feasible Data query specification.

All study materials must be reviewed by the institution’s MPOG Site PI for completeness and appropriateness, including the clinical question, introduction/methods, the literature review, the Data query specification, and the adequacy of sample size for the proposed analyses.

The MPOG Site PI must also confirm that the institutional IRB is complete and appropriate for the study. Please note that all MPOG research projects require a separate IRB to allow for analysis and use of a limited dataset (containing dates of service) for research.

Concurrent with MPOG Site PI review, the primary author must perform a test data download of their institution’s data (access to multicenter data granted only after PCRC proposal approved). After a download of local data as relevant to the proposed research, the primary author will gain valuable insight as to the quality of the MPOG data, and may elect to revise study methodology (i.e. study inclusion/exclusion criteria, analytic plan, exposure variables/covariates/outcomes collected) in order to mitigate obstacles to the proposed research. Although a complete single-center study using local institution data need not be performed as a prerequisite to PCRC, such data downloaded should be used to develop the “Preliminary Single Center Data” section of the PCRC proposal (to include such items as summary statistics, data missingness, etc.). Should difficulties be encountered at this stage, the MPOG Site PI and the MPOG coordinating center can be contacted to provide guidance.

To aid in visualizing data quality, MPOG provides valuable research tools, including:

MPOG Data Explorer (coming soon)

MPOG Case Viewer (coming soon)

As an example of how such tools may be used to gain insight into quality of MPOG data, please review the Example Guide to Visualizing and Inspecting MPOG Data Quality (coming soon).

Uncommonly, the study team may determine that a test data download of their institution’s data is infeasible for the proposed research. Most often, this is due to the study team targeting a cohort of patients primarily at other MPOG institutions. In such cases, the study team must discuss this issue with the MPOG coordinating center (mpog-research@med.umich.edu) and provide a brief explanation for not performing the test data download within the PCRC Proposal Submission Checklist.

After completing all the PCRC proposal materials, the primary author must contact the MPOG Coordinating Center at mpog-research@med.umich.edu, and provide the following materials as e-mail enclosures. 

      • Cover Sheet
      • Full study protocol including Introduction, Methods, Statistical Analyses, and References
      • Response to Reviewers (revised PCRC presentations only)
      • Data Query Specification
      • PCRC Proposal Submission Checklist Document 

Within the Cover Sheet of submission materials, it is recommended the primary author consider individuals eligible for by-line authorship, with attention given to ICMJE requirements for authorship.

Recommended by-line authors to consider:

      1. Members of the primary author’s research team, at the discretion of primary author
      2. PCRC Moderator
      3. Primary author’s site Anesthesia IT champion
      4. Primary author’s site Surgery Clinical Champion or Committed Surgery Faculty designated by Surgery Clinical Champion (for proposals seeking integrated surgical registry data e.g. STS, NSQIP).
      5. MPOG coordinating center faculty or staff who provided major guidance to study design, data element selection, and data quality consideration (if applicable)
      6. Faculty or staff who had notable large scientific contributions during PCRC meeting or Google Doc (determined post-hoc)

ICMJE Requirements for Authorship – ICMJE website – must satisfy ALL four criteria:

        1. Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND
        2. Drafting the work or revising it critically for important intellectual content; AND
        3. Final approval of the version to be published; AND
        4. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Once receiving PCRC proposal materials, a member of the MPOG Coordinating Center will contact the primary author to confirm receipt, address any unresolved issues, and – following addressing of unresolved issues – schedule a PCRC Review date.

Upon scheduling a PCRC review date, a statistician at the MPOG Coordinating center will liaise with the primary author’s statistician / statistical team to provide feedback on proposed statistical methods and quality of MPOG data used for the proposed research.

MPOG Coordinating Center members:

The PCRC meets on the second Monday of each month, from 10:00 a.m. – 11:30 a.m. Eastern Time Zone.  Proposals must be received four weeks prior to a meeting to be considered for review.

Members of the PCRC will evaluate proposals based on the following questions:

      1. Is the study as presented in the introductions and methods complete and comprehensive?
      2. Is the specific study question concisely presented?
      3. Will the data requested be suitable for answering the study questions?
      4. Are the proposed statistical techniques appropriate?
      5. Is the literature review complete and comprehensive?
      6. Does the study address a clinically significant healthcare issue?
      7. Revised PCRC presentations only: Does the study team adequately respond to the addressable feedback from previous PCRC presentation? 

Prior to the PCRC Review, the primary author (or study team presenter) must communicate to the MPOG Coordinating Center the presentation plan. This can simply be an oral summary of the proposed research (and Coordinating Center can display the PCRC proposal document on the presentation screen), or a short presentation (e.g. Powerpoint). The presentation length should target 10 minutes.

Following the presentation, a PCRC Moderator Committee member (independent from the study team) will provide a short critique followed by a moderated discussion open to all PCRC members. An example PCRC schedule is below:

      • PCRC Welcome
        • Address any housekeeping issues – 5 minutes
      • PCRC Presentation #1
        • Primary author / presenting author presents proposal – 10 minutes
        • PCRC Moderator Committee member provides critique – 5 minutes
        • Moderated open discussion – 5-10 minutes
        • Voting
      • PCRC Presentation #2
        • (repeat as above)
      • PCRC Presentation #3
        • (repeat as above)
      • Closing remarks – 2 minutes

The PCRC Moderator Committee is a rotating group of methods and clinical content experts, familiar MPOG research infrastructure, serving to provide focused feedback to the study team. One member from the PCRC Moderator Committee will serve as moderator for all PCRC proposals presented on a specific date. When possible, a PCRC Moderator Committee member will be selected to moderate, as matched by methodologic and clinical content expertise to the proposals presented.

Following discussion of each PCRC presentation, eligible participating MPOG sites will vote on the presented project. Each MPOG site has one vote, collaboratively made by the site PI Champion, or site faculty designated by the site PI Champion, in discussion with other site faculty (e.g. Surgery Clinical Champion, Anesthesia IT Champion). Four voting decisions can be made:

Approve

          • The primary author will update the by-line author list with consideration given to faculty or staff providing notable large scientific contributions in advance of, or during the PCRC meeting (if applicable) with the expectation of by-line authors to continue to adhere to ICMJE requirements for authorship. 

          • Additionally, although no revisions to the proposal are required for proposals with an ‘Approve’ decision, the primary author may decide to make changes to the proposal based upon the PCRC feedback. Once completed, the proposal is registered to the MPOG PCRC website.

Approve with Revisions

          • In addition to actions taken with an ‘Approve’ decision, the primary author is required to revise the PCRC proposal based on PCRC meeting feedback. 
          • The revised PCRC proposal is an updated Google Doc, completed within four weeks of presentation; for revisions extending past four weeks, the study team must provide a written explanation to the MPOG Coordinating Center. 
          • Revisions made to the original proposal should be made transparent (for example, using red font). Additionally, the primary author is required to draft a “Response to Reviewers” document providing point-by-point responses to PCRC meeting feedback, to accompany the revised proposal. 
          • The PCRC Moderator is given 7 days to review the revised PCRC proposal. Although the PCRC Moderator may provide feedback on aspects of the PCRC proposal not discussed during the PCRC meeting, the revised PCRC proposal is only required to address comments provided before the PCRC meeting, and issues discussed during the PCRC meeting.
          • Once all feedback is adequately addressed, as adjudicated by the PCRC Moderator, the revised proposal is formally approved and registered to the MPOG PCRC website.

Revise and re-present PCRC proposal

          • The primary author is required to revise the PCRC proposal based on PCRC meeting feedback. 
          • The primary author will update the by-line author list with consideration given to faculty or staff providing notable large scientific contributions in advance of, or during the PCRC meeting (if applicable) with the expectation of by-line authors to continue to adhere to ICMJE requirements for authorship. 
          • The revised PCRC proposal is scheduled for PCRC re-presentation. Given the potential for extensive revisions, there is no time limit for the primary author to re-present the revised PCRC proposal. 
          • Revisions made to the original PCRC proposal should be made transparent (for example, using red font). Responses to feedback are required to be included in a “Response to Reviewers” document, to be included in the submission materials for PCRC re-presentation.
          • In contrast to an ‘Approve with Revisions’ decision, the PCRC moderator is not required to adjudicate feedback in the Google Doc. Instead, feedback will be addressed and discussed at the future PCRC re-presentation.

Reject

          • Study team may not re-present the proposed work at a future PCRC review.

Prior to performing data analysis, the study team should inspect the quality of multicenter data and perform additional data cleaning/pre-processing as needed for analysis.

To aid in visualizing data quality, MPOG provides valuable research tools, including:

MPOG Data Explorer ** Note: process for linking MPOG DataDirect Queries to Data Explorer currently under construction; check back for more updates.

MPOG Case Viewer (coming soon)

As an example of how such tools may be used to gain insight into quality of MPOG data, please review the Example Guide to Visualizing and Inspecting MPOG Data Quality (coming soon).

Following PCRC approval and prior to multicenter data access, the study team should publicly register their research protocol, as required by scientific journals. The goal of a priori research protocol registration, is to improve the openness, integrity, and reproducibility of the research performed.

Study registration can be performed via the Open Science Framework, a scholarly commons for tracking projects through the research cycle.

Once registered, the study team will be given access to multicenter data via MPOG Central to perform the proposed analyses.

It is the obligation of the primary author to write a single manuscript and abstract for all approved PCRC proposals.

The goal of the research project is to produce high quality original research which is submitted for publication in a peer-reviewed journal. If the primary author does not complete a draft of a manuscript within nine months of receiving the completed statistics on the data, a co-author may request primary authorship and assume the responsibility of authoring the manuscript. At this point the original primary author will assume a co-authorship role.

Immediately prior to manuscript submission at a target journal, the manuscript draft will be sent by the primary author to the MPOG Coordinating Center.  The manuscript will be processed via plagiarism detection software (iThenticate – service freely provided by the MPOG Coordinating Center); this report will be provided to the primary author.  The manuscript draft and the original PCRC proposal will then be circulated to the PCRC review committee to ensure that the data were used in the manner proposed. This review is not to evaluate the results or conclusions, simply to confirm that the data usage is consistent.

Please note: only active sites will have access to PCRC meeting notes and recordings.