Here is the list of things we are currently working on to improve the contributor experience:
- An evaluation mechanism for newly-uploaded heuristics. This will give contributors an immediate feedback on the usefulness of their work in various scenarios. A first version of this system is already available.
- Based on this evaluation, a ranking of the heuristics.
- Display detailed informations when a heuristic crash (or is considered problematic by the platform). This will help you to debug your heuristics.
- Display more detailed experiment results (for instance: a ROC curve, the worst mistakes, ...).
- Add more experiment types (object detection and goal-planning).
- Permanently run public experiments using the public heuristics.
Stay tuned!
