Phase IV Quick Outline of Work

  • Present progress and respond to Phase III feedback
    • Read and adjust project proposal based on reviewer feedback
    • Create high-level overview presentation of project analysis plan and progress
    • Identify project adjustments and concerns for discussion
  • Complete end-to-end data analysis
    • Run all steps of the proposed analysis plan on the selected data at least once
    • Identify potential fine tunings to analysis plan methods or parameters to further refine the models, and improve the methodologies
  • Clinical interpretation of all results
    • Create a cohesive narrative that connects the key findings of the analysis to each other and to the clinical question
    • Explain the key results of the analysis with supporting figures and tables
    • Anchor the key findings to previously known understanding in the literature
    • Articulate the rationale of why the selected statistical or machine learning techniques were the correct ones to employ
  • Submit and review Phase IV reports

Phase IV Submissions and Deadlines

  • [year-long overview]
  • Phase IV Progress Presentation / Discussion (18 min present & discuss): 5 pts, Sep 9 - Nov 1 @ [Zoom]
  • Phase IV Report (around 7 pg team report): 25pts, Fri, Nov 8 @ [link]
  • Phase IV Peer Evaluations (3 report evals): 9pts, Wed, Nov 20 @ [link]

Progress Presentation Sessions

  • Phase Objectives
    • PP-1. Present your project importance and progress quickly to a generalized audience
    • PP-2. Prepare responses to written reviews and respond to feedback and questions from a live audience
    • PP-3. Identify project risks and weaknesses and solicit assistance from other scientists/researchers
    • PP-4. Participate in and provide valuable contributions to data-related scientific discussions science
  • Session Format
    • Each one-hour Zoom session will focus on three projects with twenty minutes allocated to each project. Course staff and all students and mentors from the three teams will be encouraged to attend, share their video, and engage in the discussion throughout. With a team’s twenty minutes, the following is expected to occur:
  • Team presentation [~ 12 minutes]
    • [~2 min] A quick overview of the project’s importance, goals, and progress through previous phases
    • [~3 min] Responses based on the most valuable feedback from the previous phases, either answers to important questions asked or significant alterations to project plan based on reviewer comments
    • [~6 min] Present analysis results, including choice of models, tuning selections, model evaluations, and result interpretation
    • [~1 min] Possible discussion questions for the faculty audience or peers about project struggles or unknowns
  • Group Discussion [~ 7 mins]
    • Audience questions and feedback and discussion of team concerns The 5pts of this session will be based on student participation in the session throughout the hour.
    • 1 pt - Engaging with other teams’ discussion in a meaningful way (speaking or via chat)
    • 4 pt - Participating in own team’s presentation and discussion

End-to-end Analysis Report

  • Phase Objectives
    • P4-1. Describe your complete analysis methods in a concise manner to a general audience
    • P4-2. Identify method revisions, additions, and fine-tunings that should be completed for the final analysis
  • Overview
    • In Phase IV, your report draft should describe your complete analysis at a concise level that is understandable to the general audience of your targeted journal audience with enough detail that would allow other researchers to replicate your results
  • Evaluation of the report for its description of the analysis will be based on if it:
    • Provides descriptions and justifies the use of data source(s)
    • Clearly describes the data extraction, pre-processing, selection, and transformation processes
    • Justifies and clearly describes analysis methods that correspond to presented results
    • Contains enough package, methods, and parameter detail to allow for replication of analysis
    • Indicates additional methodological revisions and fine-tunings planned for final analysis
    • Discusses the potential shortcomings or future refinements of analysis approach

Results Presentation and Clinical Interpretation

  • Phase Objectives
    • P4-3. Identify a potential journal or venue where you could submit your report
    • P4-4. Learn how to build an easy-to-follow narrative around your key results appropriate for your target audience and anchored to your clinical questions
    • P4-5. Support your findings with specific results from the analysis
    • P4-6. Provide context to your findings by relating them to what is known in literature.
  • Overview
    • In Phase IV, you should create a draft presentation of your key findings, building an easy-to-follow narrative that indicates to the reader how the findings relate to your clinical question, the analysis results, and prior knowledge in related literature.
  • Evaluation of the report for its results presentation and clinical interpretation will be based on whether it:
    • Clinical rationale of analysis is understandable to audience of the targeted journal
    • Presents, summarizes, and interprets key results connected to the clinical question
    • Supports key results with tables, figures, and/or significance values
    • Provides clear connections of analysis findings to the known literature

Updated and Revised Phase IV Report

In this phase, your primary objective is to complete the initial draft of your end-to-end data analysis. By the end of this phase, you should have executed all the analyses outlined in your project proposal on your selected data. You should also have identified potential iterations to fine-tune your models, refine hypotheses, improve methodologies, and interpret your results in a meaningful way. Your report should articulate the statistical or machine learning techniques employed, the rationale behind their selection, and how the results contribute to the overall objectives of your research.

A special requirement of this phase is to pick a potential target journal or venue of submission. Please be mindful that the targeted journal should fit the overall goal and scope of your project. If your targeted journal accepts more than one article type, please indicate which article type best aligns with your report. For example, you should not target a journal that only publishes review-type articles. You should find and include the webpage link with the journal’s submission criteria and start formatting your report to match your targeted journal’s style/format.

The report format should include the following sections:

  • Title, Authors, and Affiliation
    • Add a line with the name of a potential target journal for submission and a link to their submission criteria. (New)
  • Project Abstract (Revised)
    • Draft findings: Include the high-level findings that result from your analysis.
  • Introduction and Literature Review (Revised)
  • Analysis Methods
    • Descriptions and figure(s) for data extraction, pre-processing, selection, transformation, and analysis (Revised)
    • Specifics about the data manipulation, statistics, or machine learning packages, models, and parameters used in the analysis (Revised)
  • Comprehensive Data Analysis Results
    • Dataset Summary Statistics: Provide an overview of your dataset contents to provide context (Revised)
    • Key Findings: Highlight the most important outcomes from each step of your analysis and your initial interpretations (New)
    • Analysis Support: Summarize the rationale of applying your models/approach and how they were used and evaluated (New)
    • Model Tuning: Summarize the steps taken or needed to optimize your models, including parameter adjustments and any cross-validation techniques (New)
  • References - cited throughout and listed at the end (Revised)

The length of this report is expected to be equivalent to around 7 pages of 12 pt single-spaced text, not counting any figures, tables, or bibliography sections. It should also only contain the most important figures and tables for understanding the analysis and the results. Additional material from previous phases can be included in a single (or zipped directory) supplementary file.

  • Supplementary Figures, Tables, and Methods (Revised)
    • Figures, tables, and additional methods descriptions/code that are included in the supplement, should have proper numbering and descriptions, and be cited in the main report text.

Report Evaluation: Submissions will be evaluated by faculty and peers on the following:

  • Abstract and literature review is clear, concise, and comprehensive
  • Updates previous phase content (abstract, literature review, methods, dataset description, references, etc)
  • Organizes report with clear flow, integrating materials across phases
  • Figures and tables are numbered, cited, and interpretable by reader
  • Supplementary content is cleanly cited and organized
  • Follows professional journal style with correct tone, spelling, and grammar

Critical Evaluation of Peer Reports

  • Phase Objectives for evaluation
    • E-1. Read, understand, and think critically about the research presented in the reports of your peers
    • E-2. Assess the soundness of proposed and reported research in domains outside your expertise by finding and comparing to external literature sources.
    • E-3: Provide meaningful and professional peer review feedback that resembles a medical journal review process
  • Phase Peer Evaluations: We will assign every student to review three submitted reports each phase and provide valuable feedback to their peers. The purpose of this exercise is to give reviewers exposure to the efforts and outputs of other teams and exercise the ability to read and think critically about analyses in other domains presented to them and practice communicating their questions or suggestions. For the teams reviewed, this provides additional outside perspectives on the presentation and direction of their project that they have the chance to consider and respond to. We expect peer reviews to contain Meaningful Feedback, defined as
    • advice for fixing content errors (not grammatical errors) in the presentation, organizing the information in different ways to make it easier for the audience to follow, or suggestions for alternative methodology, research questions, or interpretation of findings which may constitute a future improvement to the work.
  • Some resources for how to perform and write a meaningful review can be found at the paper, How to Review a Clinical Research Paper or the JEE reviewer guidelines