Phase II Quick Outline of Work

  • Present progress and respond to Phase I feedback
    • Read and adjust project based on reviewer feedback
    • Create high level overview presentation of project goals and progress
    • Identify project adjustments and concerns for discussion
  • Obtain and explore core analysis dataset
    • Download, extract, and examine the contents of your code dataset
    • Decide on and describe a data selection or exclusion strategy
    • Calculate summary statistics (counts, averages, ranges, percentages, etc) for important variables or subpopulations in your likely analysis
  • Investigate and formalize your analysis plan
    • Update literature reviews with more investigation to appropriate analysis methods and benchmarks
    • Describe individual steps of proposed analysis workflow
    • Identify compute, software, and packages intended to complete analysis
    • Identify project risks and potential alternatives strategies
  • Submit and review Phase II reports

Phase II Submissions and Deadlines [year-long overview]

  • Phase II Progress Presentation/Discussion (18 min present & discuss): 5 pts, May 6 - Jun 14 @ [Zoom]
  • Phase II Report and Proposal (around 4 pg team report): 15pts, Fri, Jun 21 @ [form]
  • Phase II Peer Evaluations (3 report evals): 9pts, Wed, July 3 @ [link tbd]

Progress Presentation Sessions

Phase Objectives

  • PP-1. Present your project importance and progress quickly to a generalized audience
  • PP-2. Prepare responses to written reviews and respond to feedback and questions from a live audience
  • PP-3. Identify project risks and weaknesses and solicit assistance from other scientists/researchers
  • PP-4. Participate in and provide valuable contributions to data-related scientific discussions science

Session Format.

  • Each one-hour Zoom session will focus on three projects with twenty minutes allocated to each project. Course staff and all students and mentors from the three teams will be encouraged to attend, share their video, and engage in the discussion throughout. With a team’s twenty minutes, the following is expected to occur:
    • Team presentation [~ 12 minutes]
      • A quick overview of the project importance, goals, and progress through previous phases [~5 min]
      • Responses based on the most valuable feedback from the previous phases, either answers to important questions asked or significant alterations to project plan based on reviewer comments [~4 min]
      • Plans to complete the objectives for the current phase and updates on progress [~2 min]
      • Possible discussion questions for the faculty audience or peers about project struggles or unknowns
    • Group Discussion [~ 7 mins]
      • Audience questions and feedback and discussion of team concerns
  • The 5pts of this session will be based on student participation in the session throughout the hour.

Updated and Revised Phase II Report

Phase Objectives

  • P2-1. Produce a concise, clear, and well documented written proposal for a data analysis that will be relevant to an important clinical problem and demonstrate that the primary dataset(s) are available and sufficient.

Phase II Report:

  • At the end of Phase II, all teams are expected to revise and update their previous submissions into a more complete report. By the end of Phase II, the report should contain at least the following sections:
    • Project Title & Authors (Revised)
    • Proposal Abstract (New) - short paragraph conveying the high level background, goal, and plan for how your data analysis will address an important clinical or medical issue
    • Introduction -
      • Literature review of clinical relevance of issue (Revised)
      • Background about primary dataset(s) (Revised)
      • Literature review of analysis methods and benchmarks (New) - Evidence from medical literature about the types of analysis methods, their advantages and disadvantages, and any previous benchmark values that have been applied and set in similar prior studies
    • Methods
      • Methods for Basic Dataset Characterization (New)
      • Methods for downloading, extracting, and pre-processing primary dataset
      • Description and figure of data selection inclusion or exclusion steps
    • Results
      • Results for Basic Dataset Characterization (New)
      • Initial description of important summary values observed in selected data
      • Table with summary statistics for key sample populations (cohorts) and important variables related to the proposed data analysis
    • Proposed Analysis Plan (New)
      • Step-by-step description of multi-phase approach to answer clinical question with data analysis of selected data set
      • Indicating choice of analysis methods and evaluation criteria for each analysis step
      • Figure diagram to summarize that high-level analysis approach
      • Description of analysis requirements: computing environments, software applications and packages, other project needs or expenses
      • Discussion on risks and weaknesses of analysis plan (what will be the biggest challenges, where will the most time be spent, what are the greatest uncertainties) and possible approaches or alternatives that minimize those risks
    • References - cited throughout and listed at the end (Revised)
  • The length of this report is expected to be equivalent to around 4 pages of 12 pt single-spaced text, not counting any figures, tables, or bibliography sections. As always, it is expected to conform to a medical journal or technical report style and be submitted as a pdf following the team number and shortened title naming convention, e.g. “Team 01 - Analysis of Breast Cancer Readmission.pdf”. Additional material can be included as a single (or zipped directory) supplementary file.

Report Evaluation:

  • Submissions will be evaluated by faculty and peers on the following:
    • Previous content (clinical review, dataset background) appropriately updated
    • Clear and concise proposal abstract
    • Organization and clarity of the flow of thought, especially with integration of previous and revised materials
    • Tables and figures are numbered with titles and descriptions and cited in the text
    • Correct usage of grammar, punctuation, and spelling
    • Adhering to professional journal/report formatting and style

Basic Dataset Characterization

…more details and evaluation criteria coming soon.

Methods and Benchmarks Literature Review

…more details and evaluation criteria coming soon.

Project Proposal and Analysis Plan

…more details and evaluation criteria coming soon.

Critical Evaluation

Phase Objectives

  • E-1. Read, understand, and think critically about data analysis reports.
  • E-2. Corroborate and assess the soundness of proposed and reported research in domains outside your expertise.
  • E-3. Provide meaningful and professional peer review feedback.

Phase Peer Evaluations:

We will assign every student to review three submitted reports each phase and provide valuable feedback to their peers. The purpose of this exercise is to give reviewers exposure to the efforts and outputs of other teams and exercise the ability to read and think critically about analyses in other domains presented to them and practice communicating their questions or suggestions. For the teams reviewed, this provides additional outside perspectives on the presentation and direction of their project that they have the chance to consider and respond to. We expect peer reviews to contain Meaningful Feedback, defined as

  • advice for fixing content errors (not grammatical errors) in the presentation, organizing the information in different ways to make it easier for the audience to follow, or suggestions for alternative methodology, research questions, or interpretation of findings which may constitute a future improvement to the work.

Some resources for how to perform and write a meaningful review can be found at the paper, How to Review a Clinical Research Paper or the JEE reviewer guidelines