Algorithm Evaluation Pending for Extended Period

I submitted my result, but it's been in 'provisioned' (pending) mode for over 24 hours. Also happening on task2 where it shows nothing on the status.

Im experiencing the same in task two. Its been ages since it shows: executing algorithm. Pretty sure more than 12h. I was expecting a runtime error. I don’t know if the fact that the website was down yesterday afternoon has something to do

Same here

Is it still blocker for you too?

Yes. I can’t check the status of my submission. Info is just: executing algorithm. And while is in that state I can’t send another submission. I sent an email to the organizers but still no answer yet

Same here😅 Probably will take a few more days

Hi all,

I’m checking this and will let you know in couple of hours. Most likely the evaluation Docker containers aren’t working correctly. In the meantime, please don’t resubmit new Docker images. Sorry for the delayed response, I was out of the office.

Best, Sezgin

Hi Sezgin,

Thanks for the update! I don’t think the issue is with our submission specifically, as we’ve also noticed that even the try-out algorithms have been stuck at the “validating inputs” stage for several days now.

Looking forward to your feedback once you’ve had the chance to check.

Best regards, Daniele

Any news?

Hi Daniele,

The main issue is that our current local infrastructure isn’t sufficient to evaluate every submission. Each submission consists of multiple sub‑functions that are processed sequentially, so when you send a submission it must wait for all the pending sub‑functions of other submissions to finish before its next sub‑function can start, which lengthens the overall evaluation time. Other than that, there are no errors I can fix on the server side. We anticipated this bottleneck and are already migrating the evaluations to an external cluster. Apologies for the delayed response, I’m currently in a very different time zone and in a very busy schedule. By the time I let Ibrahim know that there are inqueries to be answered. I hope this clarifies everything.

Best, Sezgin

Dear Sezgin:

I'm in the same boat. Since submitting my image on July 26th, it's still stuck in the "Import Initialization" and "Inactive" states. I saw you mentioned migrating to an external cluster for verification. I'm wondering how that's progressing and whether we're in a sequential queue. I understand you're busy, but thorough verification before final submission is very helpful for us to evaluate and improve our approach. Thank you again!

Best,Jiawei