Shifts Challenge 2022 Banner
Join

Submission instructions

In order to be eligible for the official ranking, any submission must be described in a corresponding detailed technical report with associated reproducible code and models so that they can be locally verified (see Technical Report). This is used to prevent possible cheating using extra data, hand-crafted solutions, etc...

The submissions for both tracks are allowed to teams participating in the challenge through the Grand-Challenge platform. Please ensure you have joined the challenge. Note, for both tracks the evaluation data is hidden. Hence, you will submit algorithm containers with your trained models and evaluation code to get a place on the leaderboard.  You should thus submit a Docker container. 

Track 1

This section details how to make a submission for the MS Lesion Segmentation track.

You must provide a built image containing your method.  Your docker image will be run by the organizers on the private test dataset and the output scores will be evaluated as explained in the Evaluation page.

We provide you with a fully-functional, simple docker example that you can re-use to build your submission according to  our guidelines.

Below there is  a step-by-step guide on how to create your submission. 

  • Start by installing docker with nvidia support as explained here.
  • Please follow the ReadMe for instructions on how to develop baseline models and evaluate them locally.
  • Based on the previous step, you should now have trained models and inference code to evaluate these models. The models and inference code needs to be wrapped up into an algorithm container. We provide a fully functional Algorithm container as a template for constructing your own container.  Please follow the instructions for this example repository to correctly prepare your model and inference code.
  • You should now have an exported .tar.gz file containing your algorithm container. Please click on the Submit button and navigate to the submission page for MS Lesion Segmentation: Phase 1. If this is your first algorithm container, you will have the option on this page to create a new algorithm container. Click on this link to upload your algorithm container - it will take some time (maximum 1 hour) for your algorithm container to be ready. Once your container is ready, navigate back to the submission page to make your submission by selecting your newly uploaded container.
Track 2

This section details how to make a submission for the Power Estimation track.

You must provide a built image containing your method.  Your docker image will be run by the organizers on the private test dataset and the output scores will be evaluated as explained in the Evaluation page.

We provide you with a fully-functional, simple docker example that you can re-use to build your submission according to  our guidelines.

Below there is  a step-by-step guide on how to create your submission. 

  1. Tutorials and code about development and evaluation of models locally can be found here. 
  2. Based on the previous step, you should now have trained models and inference code to evaluate these models. The models and inference code needs to be wrapped up into an algorithm container.  We provide a fully functional Algorithm container as a template for constructing your own container. You will find instructions on how to utilize the template in ReadMe.
  3. You should now have an exported .tar.gz file containing your algorithm container. Please click on the Submit button and navigate to the submission page for Power Estimation: Phase 1. If this is your first algorithm container, you will have the option on this page to create a new algorithm container. Click on this link to upload your algorithm container - it will take some time (maximum 1 hour) for your algorithm container to be ready. Once your container is ready, navigate back to the submission page to make your submission by selecting your newly uploaded container.