Registration and project submissions will be handled by our OpenReview Portal.
We provide a sample Reproducibility Summary template here. The first page of the template is mandatory and you can organize the rest of the report as you deem fit. We have also released a recommended outline of the report which you can use to incorporate your findings.
We will use the same OpenReview Portal to submit your Reproducibility projects. Click on “Add Reproducibility Challenge Report” to submit your work. In the field of “Paper URL”, paste the paper URL of the paper (from the appropriate conference proceedings). In the Abstract, copy paste the Reproducibility Summary from your report.
Please check Important Dates for the deadline for submission of your report. Your report should be of maximum 8 pages excluding references, in NeurIPS style format. You can also submit supplementary materials. Code submission is mandatory along with your supplementary materials.
We will be selecting reproducibility reports which present a clear, well motivated, and thorough reproducibility effort. The report should very briefly discuss the main ideas of the paper and not simply reiterate the main work. Your report should clearly mention the key findings of your effort, and if necessary include discussion / commentary / suggestions to the readers re-implementing this work on its replicability / complexity. Your report must contain the Reproducibility Summary in the first page as defined in the previous section.
Reports will be reviewed on the following criteria: