Get newsfeed!
You are here:
Home 
News Semantic Search News SEALS Sponsors the 4th International Semantic Search workshop at WWW 2011
News Semantic Search News SEALS Sponsors the 4th International Semantic Search workshop at WWW 2011
| SEALS Sponsors the 4th International Semantic Search workshop at WWW 2011 |
| Monday, 02 May 2011 10:56 |
|
The 4th International Semantic Search workshop took place March 29, 2011 at the International World Wide Web Conference (WWW2011) in Hyderabad, India [1]. This workshop series aims to bring together researchers working at the intersection of Information Retrieval and the Semantic Web, for which this major Web conference provides an ideal background. Submissions at the workshop deal with issues related to searching annotated text collections and searching over structured data, in particular searching RDF data collections. The workshop places a strong emphasis on establishing for these new tasks the kind of rigorous evaluation practices that have become commonplace in other fields of IR. To this end, the workshop started an evaluation campaign in 2010, that continued this year in an expanded form. This evaluation, sponsored by SEALS, featured two tracks. In the Entity Search Track, the task is to rank resources according to the degree to which they are relevant to a particular entity (e.g. "butte montana"). Resources that describe the exact same entity that the user is looking for are considered most relevant, while resources that describe related entities are considered partly relevant.In the List Search Track, the goal is to retrieve resources that match the criteria described by the query. In contrast to the Entity Search Track, there are a set of entities that potentially match the criteria, such as "countries in africa". The participants are given real web search queries and a real dataset collected from the Semantic Web, in which they have to rank resources with respect to their relevance to the queries. This evaluation thus focuses solely on ranking, and not other aspects of retrieval, e.g. effects of result presentation and performance are not part of the competition. The SEALS project sponsored the evaluation of the results submitted by the contestants. The results were evaluated using Amazon's Mechanical Turk system, following careful validation of the evaluation setup. Five and six teams participated in the two tracks respectively, and the results have been posted on the evaluation website [2] hosted by Yahoo! Research. Descriptions of the winning submissions will be also published as part of the workshop proceedings. The result assessments sponsored by SEALS are also made publicly available to the research community, so that researchers can independently verify the results and evaluate new systems other than those that have been submitted. [1] http://km.aifb.kit.edu/ws/semsearch11/ [2] http://semsearch.yahoo.com/ Thanks to Peter Mika for the text. |
