Registration and project submissions will be handled by our OpenReview Portal.
You can choose to reproduce papers published in the calendar year of 2022 for the following venues: NeurIPS 2022, ICML 2022, ICLR 2022, ACL 2022, EMNLP 2022, CVPR 2022, ECCV 2022, AAAI 2022, IJCAI-ECAI 2022, ACM FAccT 2022, SIGIR 2022, and also for papers published in top ML journals in 2022, including JMLR, TACL and TMLR.
You have to use the Reproducibility Summary template for writing your reports. The submission is strictly double blind, any violation of this will lead to desk reject. The first page of your report must incorporate this summary template, which should consist of the following sections:
You can find the Reproducibility Summary template here. The first page of the template is mandatory and you can organize the rest of the report as you deem fit. We have also released a recommended outline of the report which you can use to incorporate your findings.
We will use the same OpenReview Portal to submit your Reproducibility projects. Click on “Add Reproducibility Challenge Report” to submit your work. In the field of “Paper URL”, paste the paper URL of the paper (from the appropriate conference proceedings). In the Abstract, copy paste the Reproducibility Summary from your report.
Please check Important Dates for the deadline for submission of your report. Your report should be of maximum 9 pages excluding references, in ReScience style format as specified in Reproducibility Summary template. You can also submit supplementary materials. Code submission is mandatory along with your supplementary materials.
We will be selecting reports which present a clear, well motivated, and thorough reproducibility effort. The report should very briefly discuss the main ideas of the paper and not simply reiterate the main work. Your report should clearly mention the key findings of your effort, and if necessary include discussion / commentary / suggestions to the readers re-implementing this work on its replicability / complexity. Your report must contain the Reproducibility Summary in the first page as defined in the previous section.
Reports will be reviewed on the following criteria. You can also refer to the reviewing criteria used in MLRC 2021 for reference.