The following agreements describe the general terms for participation in the SEALS
Evaluation Campaigns and the policies for using the resources and results produced
in these evaluation campaigns.
- Terms of participation
By submitting a tool and/or its results to a SEALS Evaluation Campaign the participants grant their permission for the publication of the tool results on the SEALS web
site and for their use for scientific purposes (e.g., as a basis for experiments).
In return, it is expected that the provenance of these results is correctly and duly
acknowledged.
- Use rights
In order to avoid any inadequate use of the data provided by the SEALS Evaluation
Campaigns, we make clear the following rules of use of these data.
It is the responsibility of the user of the data to ensure that the authors of the results
are properly acknowledged, unless these data are used in an anonymous aggregated
way. In the case of participant results, an appropriate acknowledgement is the mention
of this participant and a citation of a paper from the participants (e.g., the paper
detailing their participation). The specific conditions under which the results have
been produced should not be misrepresented (an explicit link to their source in the
SEALS web site should be made).
These rules apply to any publication mentioning these results. In addition, specific
rules below also apply to particular types of use of the data.
- Rule applying to the non-public use of the data
Anyone can freely use the evaluations, test data and evaluation results for evaluating
and improving their tools and methods.
- Rules applying to evaluation campaign participants
The participants of some evaluation campaign can publish the results as long as they
cite the source of the evaluations and in which evaluation campaign they were obtained.
Participants can compare their results with other published results on the SEALS
web site as long as they also:
- compare with the results of all the participants of the same evaluation scenario;
and
- compare with all the test data of this evaluation scenario.
Of course, participants can mention their participation in the evaluation campaign.
- Rules applying to people who did not participate in an evaluation campaign
People who did not participate in an evaluation campaign can publish their results as
long as they cite the sources of the evaluations and in which evaluation campaign they
were obtained and they need to make clear that they did not participate in the official
evaluation campaign.
They can compare their results with other published results on the SEALS web
site as long as they:
- cite the source of the evaluations and in which evaluation campaign they were
obtained;
- compare with the results of all the participants of the same evaluation scenario;
and
- compare with all the test data of this evaluation scenario.
They cannot pretend having executed the evaluation in the same conditions as the
participants. Furthermore, given that evaluation results change over time, it is not
ethical to compare one tool against old results; one should always make comparisons
with the state of the art.
- Rules applying to other cases
Anyone can mention the evaluations and evaluation campaigns for discussing them.
Any other use of these evaluations and their results is not authorized (you can ask
for permission however to the contact point) and failing to comply to the requirements
above is considered as unethical.
|