The evaluation of semantic search tools is split into two complementary phases. The first addresses the usability of the tool by using a user study in which subjects use your tool to answer a number of questions and then complete a number of questionnaires to record their assessment (the user-in-the-loop phase). The second address the underlying performance of the tool from the perspective of result accuracy and tool performance (the automated phase). 

The Semantic Search Tool Theme is pleased to announce the release of the latest version of the user-in-the-loop experiment software. You can download the software and associated documentation from the Experiment Pack page. Alternatively, take a look at the What do I have to do as a participant? page to get a fuller picture of how to participate.

If you haven't registered your tool for the evaluation but are interested in taking part, there's still time! The first step is to join the SEALS Community after which you can register your tools for the evaluation campaign. Joining the community is easy and poses no obligations; indeed, by being a member of the community you receive the latest information about the evaluation campaign including details of newly published data sets, tips and advice on how to get the most out of your participation and the availability of results and analyses.

More information on the Semantic Search Tool Theme.