The SEALS project will create a lasting reference infrastructure for semantic technology evaluation (the SEALS Platform) and thus facilitate the continuous evaluation of semantic technologies at a large scale. The SEALS Platform will be an independent, open, scalable, extensible and sustainable infrastructure that will allow the evaluation of semantic technologies by providing an integrated set of evaluation services and test suites.

The SEALS Platform will be used in two public world-wide evaluation campaigns and the results of these evaluation campaigns will be employed in creating semantic technology roadmaps which will identify sets of efficient and compatible tools for developing large-scale semantic applications.

The first evaluation campaign will take place during 2010. It will be launched in February with the publication of the evaluation methodology and test data sets, and the SEALS Platform for executing the evaluations will be released in July. First results will be made known in October.

The semantic technology evaluation services will initially be available for five different types of technologies (ontology engineering tools, storage and reasoning systems, matching tools, semantic search tools, and semantic web service tools) and for different evaluation criteria (interoperability, scalability, etc.).

The provision of this service will benefit tool vendors and tool consumers alike.

Tool consumers will have easy and free access to the evaluation results and semantic technology roadmaps allowing them to efficiently and effectively compare the available technologies and subsequently select suitable tool combinations to fulfil their semantic technology requirements.

From a tool vendor's perspective, there are a number of potential benefits.

The SEALS Platform will enable the establishment and diffusion of best practices in evaluation within the whole community. The semantic technology community will benefit by having access to a set of services that support the whole life-cycle of the evaluation of these technologies.

Vendors will be able to use the standardised evaluation and benchmarking services to monitor internal development of new and existing tools and be able to reliably and consistently compare their own performance with that of the academic community and other commercial vendors.

Furthermore, it is envisaged that adoption of such a comprehensive and open evaluation process will lead to faster maturation of semantic technologies and, thus, increased adoption of semantic technologies within industry.

It is important to emphasise the stability and longevity of such an evaluation platform. SEALS will work towards the creation of an open and sustainable worldwide community which will survive the SEALS project and which is focused on the evaluation and progressive development of semantic technologies. Our long-term goal is that the SEALS Platform be actively used and managed by the semantic community.

SEALS Community members will receive first information about the evaluation campaigns and exclusive access to the SEALS evaluation resources and tools. Join now.

Evaluation campaigns contact: Stuart Wrigley, University of Sheffield   s DOT wrigley AT dcs DOT shef DOT ac DOT uk