Introduction

At the heart of the SEALS initiative is the organisation of two public world-wide evaluation campaigns (one which was conducted in Summer 2010 and the second which will be officially launched at ESWC 2011 with it's main results dissemination event being held at ESWC 2012). The results of these evaluation campaigns will be employed in creating semantic technology roadmaps which will identify sets of efficient and compatible tools for developing large-scale semantic applications. Indeed, the results of the first campaign can be inspected now by selecting one of the campaign areas in the menu to the left. The official SEALS Evaluation Campaign 2010 results dissemination event was part of the IWEST 2010 workshop held at ISWC 2010. 

Tool developers are cordially invited to participate in the next SEALS Evaluation Campaign 2011 in one or more of the five core areas. Participation is open to developers interested in evaluating their tool or to anyone who wants to evaluate a certain tool.

From the tool provider's perspective, the SEALS Evaluation Campaign has been designed to require minimal effort on their part. For the majority of SEALS Evaluation Campaigns, the evaluation process is fully automated from the point at which the tool has been uploaded to the SEALS Platform.

The SEALS Platform, which is coming on stream as we speak and will be fully available for the second campaign, will manage all aspects of the evaluation. Once the evaluation has been completed, the results and analyses will be available from the SEALS Portal.

In order to facilitate interaction between your tool and the SEALS Platform, the SEALS teams from each technology area have defined an easy to use API. Details about each technology area's API is available from their SEALS Evaluation Campaign's Portal page listed on the menu to the left.

Organisation

The organisation and execution of the SEALS campaigns is overseen by two types of committee (note that membership of these committees may change ahead of the second campaign - if you are interested in being involved in any of these, do get in  This e-mail address is being protected from spambots. You need JavaScript enabled to view it ): 

Evaluation Campaign Organizing Committee (E.C.O.C.)

The E.C.O.C. is in charge of the general organization and monitoring of the evaluation campaign across all technology areas. Currently, E.C.O.C. membership is:

Evaluation Campaign Executing Committee (E.C.E.C.)

The five E.C.E.C.s - one per technology area - are in charge of organizing the evaluation scenarios that are performed in the evaluation campaign and of taking them to a successful end. Details of membership can be found on the respective technology area's page (listed on the menu to the left).

If you are interested in being involved in the definition and organisation of future SEALS evaluation campaigns and/or sitting on either the E.C.O.C. or one or more E.C.E.C.s, please get in touch - we are keen to involve members of the community in the SEALS initiative.

Timeline

Workshop

We will hold an evaluation workshop at ESWC 2012 in which we will invite participants to present their systems and evaluation results. 

How do I find out more or get involved?

The easiest way to find out more is to join the SEALS Community. Joining the SEALS Community is easy and poses no obligations. On the other hand, by being a member of the community you will have first-hand information about the semantic technology evaluation campaigns and results. Here you can join the SEALS Community.

To learn more about the future activities of the SEALS community, you can read about the evaluation campaigns for each technology area in their own section of the SEALS portal (see the sub-menu items under Evaluation Campaigns). Each campaign also has its own RSS news feed (see the sub-menu items under News). In addition to this, you can also follow some E.C.E.C.s on twitter - check out their pages for more information.

Download the evaluation campaigns flyer. 

Evaluation campaigns contact: Stuart Wrigley, University of Sheffield   s DOT wrigley AT dcs DOT shef DOT ac DOT uk