Collaborative Benchmarking Rule-Reasoners with B-Runner - INRIA - Institut National de Recherche en Informatique et en Automatique
Communication Dans Un Congrès Année : 2024

Collaborative Benchmarking Rule-Reasoners with B-Runner

Résumé

Conducting experimental analysis on rule reasoners is a mainstream task for validating novel algorithms and systems. Nevertheless, providing robust, veriable, and reproducible experiments can still raise a sensible challenge. We propose to demonstrate B-Runner, an open library for collaborative benchmarking focusing on the deployment of articulate tests for knowledge and rule-based systems with low cost and high robustness. B-Runner reduces the benchmarking setup time while guaranteeing experiment repeatability. At the same time, it improves the scrutability of experimental protocols thereby enhancing their robustness as well as fairness of system comparisons. This demonstration proposes to show- case the use of the tool for systematically testing a number of systems as well as to introduce its architecture and its extensibility to novel tools and experimental protocols.

Domaines

Informatique
Fichier principal
Vignette du fichier
brunner_bda.pdf (3.5 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

lirmm-04646842 , version 1 (12-07-2024)
lirmm-04646842 , version 2 (12-08-2024)

Identifiants

  • HAL Id : lirmm-04646842 , version 1

Citer

Federico Ulliana, Pierre Bisquert, Akira Charoensit, Renaud Colin, Florent Tornil, et al.. Collaborative Benchmarking Rule-Reasoners with B-Runner. BDA 2024 - 40e Conférence sur la Gestion de Données – Principes, Technologies et Applications, Oct 2024, Orleans, France. ⟨lirmm-04646842v1⟩
377 Consultations
51 Téléchargements

Partager

More