Photo by Maxime Dufour

Artefact Evaluation

Following ACM’s definition, an artefact is “a digital object that was either created by the authors to be used as part of the study or generated by the experiment itself. For example, artifacts can be software systems, scripts used to run experiments, input datasets, raw data collected in the experiment, or scripts used to analyze results”.

To improve and reward reproducibility and to give more visibility and credit to the effort of tool developers in the DisCoTec community, authors of submitted papers are invited to submit publicly available artefacts (using permanent repositories such as Software Heritage, Zenodo, etc.), which will be associated with their paper for evaluation. Based on the result of the artefact evaluation, one or more badges may be applied to a paper. Specifically, DisCoTec uses the EAPLS badging scheme), which in its own turn is based on and consistent with the ACM initiative.

Artefact submission is

• mandatory for tool papers: the result of the artefact evaluation will be considered in the tool paper’s acceptance decision
• optional for all the other paper categories: the result of the artefact evaluation will not affect the paper’s acceptance decision but may affect the best paper selection

Why Submitting an Artifact

In summary:

• it is good scientific practice and a valuable contribution for the community;
• it is mandatory and affect the acceptance decision for tool papers;
• for any paper, badges may be assigned to acknowledge the availability and functionality of provided artefacts;
• authors of artefacts may be given a session at the conference to present their artefacts;
• authors of valuable artefacts may be invited to submit their artefacts to a special issue.

Artefact Submission Dates

See the Dates page.

Artefact Submission Instructions

Please see the specific page of your target conference.

• COORDINATION Artefact Submission Instructions
• FORTE Artefact Submission Instructions