Overview

After the project has been completed, each of you will review the projects of 2 of your classmates, similar to peer review of a manuscript submitted for publication.

I have several goals for this project review:

  • You get feedback that helps you improve your project.
  • You get experience with giving and receiving reviewer feedback, like you would when you submit a paper for publication or review someone’s submitted paper.
  • You practice being on the ‘receiving end’ of reproducible research, i.e. you will have to be able to reproduce someone else’s project so you can properly critique it.

Logistics

  • By the deadline specified in the Schedule, everyone has to push their finished project to their project repository.
  • The week following the deadline is devoted to project reviews.
  • Go to the projects Slack channel and find the 2 persons/projects you have been assigned to review. You can find links to project repositories in the Discussion 11 channel.
  • For reviewing, you can either use the fork and pull workflow, or the project owner can make you a collaborator and you can pull/push directly. Fork and pull is maybe a bit safer since there are now 2 reviewers making changes to the repository at the same time. But you can decide on the workflow.
  • Follow the instructions provided in the repository to run all code and reproduce everything. Review the whole project using this review template (right-klick to download).
  • Write up a detailed review using and filling out the template Markdown file. Once you are done, place your completed review document into the repository you are reviewing, and send a pull request (or do a direct push) for each project you review.

As you reproduce the project, you will likely make a lot of changes to the repository (e.g., by re-creating figures/tables). If you and another reviewer do that both, you might run into conflicts when sending a pull request. If you encounter problems like this, you can provide the project owner your completed review in some other manner (e.g. through email or Slack) and they’ll add it to the repository. The important part is that at the end of the review process, there should be (at least) 2 completed review documents in the project repository (either the main folder, or make a reviews subfolder for them.)

Review Assessment Rubric

Ok, now it gets maybe a bit complicated. I will review and assess the quality of your reviews. To that end, I’ll use a fairly simple rubric, similar to the ones for the previous project submissions.

Category Description Score
Sufficient Reviews are complete or fairly complete 3
Somewhat insufficient Reviews are somewhat incomplete, lack useful/detailed feedback 2
Insufficient Only one review was submitted, or submissions were very incomplete 1
Absent No reviews were submitted 0

You get a full week to do those reviews, so I expect thorough and good quality work!

Finals steps

  • Use the peer feedback/reviews from your classmates to further improve your project. You can make any further improvements you want to make.
  • Push your final project to your GitHub repository by the specified deadline for final grading.
  • I will assess your final project using the same categories in the template used during peer review.