Evaluation Criteria

Submissions to the RuleML Challenge general Track will be evaluated according to the following criteria:

  • clear exposition of the objectives, outcomes, benefits for going beyond the state of the art in the application domain (25%)
  • demos should demonstrate a good level of effectiveness to manage complex rules according to statistical/analytical/benchmarking evaluations (25%)
  • demos have to demonstrate the results with a concrete example balancing conciseness and completeness (25%)
  • demos should preferably (but not necessarily) be embedded into a web-based or distributed environment (15%)
  • demos should pay attention to end-user interactions, providing an adequate and usable interface that favors a concrete usage of the application (10%).

Submissions focused on this year’s special theme (Knowledge-Based Applications combining Rules and Ontologies) should show the potential and benefits of combining rule-based reasoning and taxonomic/frame/description-logic reasoning in a realistic application. Demos submitted to the Special Theme will be evaluated according to the following criteria:

  • papers should clearly expose the objectives, outcomes, benefits for going beyond the state of the art in the application domain (20%)
  • demos should demonstrate a good level of integration or interaction between rules and ontologies (40%)
  • demos should demonstrate their results with a simple but meaningful case study, balancing conciseness and completeness (20%)
  • demos should preferably be deployed in a web-based or distributed environment where multiple nodes exchange information (10%)
  • demos should pay attention to end-user interactions, providing an adequate and usable interface that favors a concrete usage of the application (10%).

Comments are closed.