***********************************************

Call for Participation

SEALS Evaluation Campaign 2010

Semantic Search Tools Theme

***********************************************

Dear Search Tool Developer,

We are pleased to announce the first International Evaluation Campaign for Semantic Technologies which will take place during Summer 2010. This campaign is organised by the Semantic Evaluation At Large Scale (SEALS) Project. 

We cordially invite you to participate in the Semantic Search Tool theme of this campaign. Participation is open to anyone who is interesting in benchmarking such a tool. Terms and conditions and general information about SEALS can be found on the SEALS Portal at http://www.seals-project.eu/

The goals of the semantic search tool evaluation initiative are to support developers to improve their tools; compare their tools against their competitors and to generally improve the interoperability of semantic technologies. Tools will be evaluated according to a number of different criteria including query expressiveness (the means by which queries are formulated within the tool) and scalability (the ability of the tool to load and query ontologies of increasing size). Given the interactive nature of semantic search tools, a core interest in this evaluation is the usability of a particular tool (effectiveness, efficiency, satisfaction).

Detailed information regarding the Semantic Search Tool Theme can be found on the SEALS Portal at http://www.seals-project.eu/seals-evaluation-campaigns/semantic-search-tools

The SEALS Campaign

***********************************************

The 1st SEALS Evaluation Campaign is open to all and will focus on benchmarking five core technology areas on the basis of a number of criteria such as Interoperability, Scalability, Usability, Conformance to Standards, and Efficiency. Each area's campaign will be largely automated and executed on the SEALS Platform thus reducing the overhead normally associated with such evaluations. 

Why get involved?

***********************************************

Broadly speaking, the benefits are threefold. Firstly, participation in the evaluation campaigns provides you with a respected and reliable means of benchmarking your semantic technologies. It provides an independent mechanism for demonstrating your tool's abilities and performance to potential adopters / customers.

Secondly, since you will have perpetual, free-of-charge access to the SEALS Platform, it gives you the highly valuable benefit of being able to regularly (and confidentially) assess the strengths and weaknesses of your tool relative to your competitors as an integral part of the development cycle.

Thirdly, your participation benefits the wider community since the evaluation campaign results will be used to create 'roadmaps' to assist adopters new to the field to determine which technologies are best suited to their needs thus improving general semantic technology market penetration.

How to get involved

***********************************************

Joining the SEALS Community is easy and poses no obligations. Indeed, by being a member of the community you receive the latest information about the evaluation campaign including details of newly published data sets, tips and advice on how to get the most out of your participation and the availability of results and analyses. Join now by going to http://www.seals-project.eu/join-the-community

Once you have your community login, you will be able to register your tools for the evaluation campaign (http://www.seals-project.eu/registertool). Users with tools registered to the evaluation campaign will receive notifications as further details and services of the SEALS platform become available.

Timeline for the campaign

***********************************************

June Registration opens

July Data, documentation available

July - August Participants upload tool(s)

August - September Evaluation executed

September Evaluation results are analysed (by SEALS)

November Evaluation results are discussed at IWEST 2010 workshop (located at ISWC)

About SEALS

***********************************************

The SEALS Project is developing a reference infrastructure known as the SEALS Platform to facilitate the formal evaluation of semantic technologies. This allows both large-scale evaluation campaigns to be run (such as the one described in this communication) and ad-hoc evaluations by individuals or organizations. 

Find out more

***********************************************

More information about SEALS and the evaluation campaign can be found from the SEALS portal http://www.seals-project.eu

If you would like to contact us directly:

SEALS Coordinator: Prof. Asuncion Gomez-Perez ( This e-mail address is being protected from spambots. You need JavaScript enabled to view it )

Evaluation Campaign Coordinator: Prof. Fabio Ciravegna ( This e-mail address is being protected from spambots. You need JavaScript enabled to view it )

Semantic Search Tool Theme Coordinator: Dr. Stuart Wrigley ( This e-mail address is being protected from spambots. You need JavaScript enabled to view it )