Objectives

The main objective of this phase to conduct an initial end-to-end analysis of your dataset and to describe the process and preliminary findings in the Phase III report. This will be the first version of the data analysis in you final report.

During this phase your team should:

  • Set up your analysis environments with all necessary software packages and data
  • Transform and partition your dataset to prepare it for analysis
  • Perform all of the proposed data analysis steps on some subset of your data
  • Meet with your guidance mentor(s) to plan, present, and possibly improve the initial analysis results
  • Revise previous sections of your report and add the new methods and results writeups
  • Submit the Phase III report as a team through the [Phase III Report Submission Form]

Timeline and Key Dates

  • [40 Points] Submit the Phase III Report by Nov 11, 2022 at 11:59 pm
    • Phase III Report Submission Link]
    • Contains updated Phase I & II material [5 Points] and the new sections related to data analysis [35 Points]
  • Mentors will submit a evaluation of the submitted report by Dec 9, 2022. [Submission Link]

Phase III Report Requirements

Report Format

Your report should be formatted in a formal paper submission style. Several report templates can be found for different science/medical journals at nextgenediting.com and would make an excellent starting point. NEJM also has several manuscript requirements, which you should read and use as a general guideline for writing your report.

Overall, your combined report at this phase should be submitted as a pdf that is not longer than 7 single-spaced pages (excluding any figures, tables, and references). The name of the submitted pdf should contain a short title of the project followed by the last names of the medical student team members (e.g. “Being President - Washington, Adams.pdf”).

Phase III Report Contents

The Phase III Report mainly consists of your (revised) previous report sections and the new preliminary data analysis writeup. Consider this new component as the first draft of the methods and results section in a journal paper. It should be well-organized and should

  • Describe the tools and the steps you used for your analysis with enough detail that they could be recreated
  • Describe the observations you made from the analysis and
  • Highlight how those observations relate to the clinical problem your project is addressing
  • Give proper references formatted in the style of a journal paper

The methods and results sections should consider and cover answers to questions such as:

  • Training Procedures: How are data samples stored, extracted, transformed, and used in data analysis and model training? What technologies and software are used? How are the training data samples selected? Are there biases in the training data and can they be accounted for?
  • Evaluation Metrics: What metrics are being used to evaluate model performance? What are alternative evaluation metrics and the advantages and shortcomings of the selected ones? Are there biases in the evaluation? How are hyperparameters to the model optimized? How does performance change as these parameters vary? How reproducible is the model performance on a different training set? How does model performance change with more training data and time?
  • Predictive Quality: Which attributes of which data samples are well-predicted by a trained model? Are there common patterns in the samples with incorrect predictions? Is there a confidence measure associated with each prediction?
  • Significant Features: What features of the model are most significant for correct predictions? By what metric/method are these features ranked? What are the assumptions of the feature prioritization method and considerations to keep in mind when interpreting the results?

Phase III Report Update

If you received a low score on your Phase II submission, you have the chance to earn some of those points back with revision in this submission. Just remember, for improvements and revisions to be considered, they should be clearly indicated in the submission form.

Phase III Report Submission

When you submit your Phase III report, you will need to answer additional questions in the Google form that:

  • Reaffirm your team membership and project title
  • Highlight changes made to sections of the report from previous phases
  • Describe your interaction with your mentors

Grading Criteria Considerations (Final Report)

We would like to provide our grading criteria of the final report early. It will be graded in consideration of the following items. This list should give you an idea on what considerations to make while writing up your report.

  • [Content] Content is thorough, accurate, and supported by citation of relevant literature. There is no redundancy or unnecessary minutia. Shows ability to summarize essential points of a large body of information. The report has an appropriate number of words. The targeted page limit is 8 with 12 point font. There is no page limit for the appendix.
  • [Data Analytics] Data collection, pre-processing and analysis are described clearly. Figures/tables are used to support the arguments and conclusion. Additional information should be presented in the appendix.
  • [Reasoning and Clarity of Summary] The topic is introduced, and groundwork is laid as to the direction of the report. Conclusions logically rely on prior information.
  • [Format & Organization] Clear organization. Paragraphs are complete yet concise, with strong topic sentences indicating the focus of each paragraph. Transitions tie sections together, as well as adjacent paragraphs. Formatting consistent throughout.
  • [Audience] Written at the appropriate level for an audience of medical researchers and data scientists, with no or well-defined jargon. Concepts should be explained when necessary in a formal tone.
  • [Clarity of Language and Writing Technique] Writing is crisp, clear, and concise. Use active voice, complete sentences, and grammatically correct subject-verb constructions with no spelling errors.
  • [Citations / References] Cites appropriate references in the body of the paper, using proper format. The bibliography is complete, appropriately formatted, and matches in-text references