Challenge rules¶

To participate on the challenge, participants must accept the following rules:

• In order to participate in the competition, participants must register by accepting these Terms as described on the Competition Website. Competitors who wish to take part in the medical track should also sign the DUA.
• DeepSea, Cambridge University Engineering Department Speech Group, Basel University Hospital, Lausanne University Hospital, University of Applied Sciences of Western Switzerland employees can participate in the competition, but are ineligible for prizes and will be excluded from the final leaderboard.
• Participants can work individually or in teams. It is responsibility of participants to declare their team. Participants are incentivised to form teams, as team are able to make more submissions.
  • Teams can contain up to 5 participants.
  • A participant can only be part of 1 team.
  • Participants part of multiple teams will be disqualified. 
  • If a submission by participant from a team wins one of the ranked places, then the entire team holds that place.
• Participants can work on both competition tracks. Participation in any track is not mutually exclusive with participation the other track.
• Participants should submit docker files containing their solution to be evaluated on development and evaluation data, which will be used to update the respective leader boards. Participants' models should, in addition to making prediction, also yield a measure of uncertainty to accompany each prediction. Details of what to submit and how it should be submitted will be provided on the competition website.
• Participants should download the datasets from either the Data Provider’s website or from the Competition Website, and accept the applicable terms and conditions. The Organizer will specify the links to the data on the Competition Website.
• The participants should build models only on the data provided by the Organizers.
  • Models should only be trained on the training or unsupervised data.
  • Using the development sets for training is forbidden.
  • Use of any external data for any of the two competition tracks is forbidden, unless this data is publicly released and freely shared with all other competitors.
  • Data augmentation is allowed and encouraged.
• The proposed solution should yield predictions within 800ms per 1 input sample. Submitted solutions which break these limitations will not be considered in the final scoring.
• To be eligible for prizes, after the challenge has ended the top-3 participants in each track are required to submit:
  • A detailed technical report on their solution.
  • Their code used to train the model -- code must be reproducible by organizers.
  • The model which corresponds to the submission.
  • Performance on evaluation data will be verified using the provided code, models, and technical report after the end of the competition during a verification stage. Only submissions that are verified to be consistent with submitted scores will eligible for prizes. Organizers will reach out to participants in case difficulties arise.
• Each participant (whether a team or independent ) can have 1 winning solution per track. In other words, a participant cannot simultaneously occupy the 1st and 2nd places, for example.
• During the development phase, we will limit the number of submissions on the development data to 20, per track, per participant.
• During the evaluation phase, we will limit the number of submissions on the evaluation data to 3, per track, per person.
• Organizers have the right to update the rules in unknown situations (e.g., a tie among participants ranking) or lead the competition in a way that best suits the goals.
• Organizers reserves the right to modify the leaderboard at its discretion for reasons of fairness, proper play, and/or compliance with these Official Rules, including, for example, if the Organizers believes that participants or entries shown on the leaderboard do not meet any eligibility or other requirements in these Official Rules.

Computational Limitations We place a hard limitation on the compute which solutions can use. Models must run on, at most,  within 800ms per 1 input sample on the Grand-Challenge backend. Submitted solutions which break these limitations will not be considered in the leaderboard. This is done for several reasons. Firstly, to decrease costs, as every model evaluation on Grand Challenge costs money. Secondly, for realism, as in many practical applications there are significant limitations placed on the compute or memory budgets and run-time of algorithms. Finally, by limiting the inference-time compute, we level the playing field for participants who do not have access to vast amounts of compute.