Merge develop into master (v1.29)

compute_worker/compute_worker.py

@@ -968,7 +968,7 @@ async def _run_container_engine_cmd(self, container, kind):
                     await websocket.wait_closed()
                 except Exception as e:
                     logger.error(e)
-            client.remove_container(container, force=True)
+            client.remove_container(container, v=True, force=True)
 
             logger.debug(f"Container {container.get('Id')} exited with status code : {str(return_Code['StatusCode'])}")
 
@@ -983,7 +983,7 @@ async def _run_container_engine_cmd(self, container, kind):
         finally:
             try:
                 # Last chance of removing container
-                client.remove_container(container.get("Id"), force=True)
+                client.remove_container(container.get("Id"), v=True, force=True)
             except Exception:
                 pass
 
@@ -1347,7 +1347,7 @@ def start(self):
 
             for container in containers_to_kill:
                 try:
-                    client.remove_container(str(container), force=True)
+                    client.remove_container(str(container), v=True, force=True)
                 except docker.errors.APIError as e:
                     logger.error(e)
                 except Exception as e:
@@ -1398,7 +1398,7 @@ def start(self):
                         containers_to_kill = self.scoring_program_container_name
                     try:
                         client.kill(containers_to_kill)
-                        client.remove_container(containers_to_kill, force=True)
+                        client.remove_container(containers_to_kill, v=True, force=True)
                     except docker.errors.APIError as e:
                         logger.error(e)
                     except Exception as e:

docker-compose.yml

@@ -200,7 +200,7 @@ services:
   #----------------------------------------------------------------------------------------------------
   site_worker:
     # This auto-reloads
-    command: ["watchmedo auto-restart -p '*.py' --recursive -- celery -A celery_config worker -B -Q site-worker -l info -n site-worker@%n --concurrency=2"]
+    command: ["celery -A celery_config worker -B -Q site-worker -l info -n site-worker@%n --concurrency=2"]
     working_dir: /app/src
     container_name: site_worker
     image: django_site-worker

documentation/docs/Organizers/Benchmark_Creation/Leaderboard-Functionality.md

@@ -49,6 +49,7 @@ Computation options are:
   - avg
   - min
   - max
+  - avg_rank
 
 These are applied across the columns specified as `computation_indexes`.
 

documentation/docs/Organizers/Benchmark_Creation/Yaml-Structure.md

@@ -259,7 +259,7 @@ fact_sheet: {
   - Ascending: smaller scores are better
   - Descending: larger scores are better
 - **computation:** computation to be applied *must be accompanied by computation indexes*
-  - computation options: sum, avg, min, max
+  - computation options: sum, avg, min, max, avg_rank
 - **computation_indexes:** an array of indexes of the columns the computation should be applied to
 - **precision:** (*integer, default=2*) to round the score to *precision* number of digits
 - **hidden:** (*boolean, default=False*) to hide/unhide a column on leaderboard

documentation/docs/Project_CodaBench_FAQ.md

@@ -5,8 +5,8 @@
 Codabench benchmarks are aimed at researchers, scientists and other professionals who want to track algorithm performance via benchmarks or have participants participate in a competition to find the best solution to a problem. We run a free public instance at [https://www.codabench.org/](https://www.codabench.org/) and the raw code is on [Github](https://github.com/codalab/codabench).
 
 
-### Can CodaLab competitions be privately hosted?
-Yes, you can host your own CodaLab instance on a private or hosted server (e.g. Azure, GCP or AWS). For more information, see [how to deploy Codabench on your server](Developers_and_Administrators/How-to-deploy-Codabench-on-your-server.md) and [local installation](Developers_and_Administrators/Codabench-Installation.md) guide. However, most benchmark organizers do NOT need to run their own instance. If you run a computationally demanding competition, you can hook up your own [compute workers](Organizers/Running_a_benchmark/Compute-Worker-Management---Setup.md) in the backend very easily. 
+### Can Codabench be privately hosted?
+Yes, you can host your own Codabench instance on a private or hosted server (e.g. Azure, GCP or AWS). For more information, see [how to deploy Codabench on your server](Developers_and_Administrators/How-to-deploy-Codabench-on-your-server.md) and [local installation](Developers_and_Administrators/Codabench-Installation.md) guide. However, most benchmark organizers do NOT need to run their own instance. If you run a computationally demanding competition, you can hook up your own [compute workers](Organizers/Running_a_benchmark/Compute-Worker-Management---Setup.md) in the backend very easily. 
 
 
 ### How to change my username?