The scalability of ontology engineering environments will be tested against both real world ontologies and generated data of varying sizes and complexity.

The real world ontologies that will be used in the scalability evaluation include well established and used ontologies in the literature. In comparison to the previous evaluation campaign test suites have been extended and are currently aimed to test the performance of ontology tools depending not only on the file size of ontology, but also on the amount of triples. Additionally, real-world ontology test suites are specified in order to analyze the dependence between the ontology complexity and the tools' performance.

Real World Ontology Scalability Test Data Suite(UPDATED!) contains complex ontologies belonging to biological or medical domains.

NCI Ontology Scalability Test Data Suite(NEW!) includes subsets from NCI Thesaurus ontology-like vocabulary.

GALEN is a medical ontology that contains the building blocks for defining procedures - the anatomy, surgical deeds, diseases, and their modifiers used in the definitions of surgical procedures. The ontology for the GALEN CORE model is designed to be re-usable and application independent. GALEN is a relatively famous ontology for DL system developers since it has been proven notoriously difficult to classify. The expressivity of GALEN is SHIF. Although, much more expressive and big ontologies exist, the difficulty with GALEN is that its structure is very complex. Besides the full GALEN, there are several weaker versions of it. In our dataset we have considered several of these versions.

The Foundational Model of Anatomy Ontology (FMA) is an evolving computer based knowledge source for biomedical informatics; it is concerned with the representation of classes or types and relationships necessary for the symbolic representation of the phenotypic structure of the human body in a form that is understandable to humans and is also navigable, parseable and interpretable by machine-based systems. Specifically, the FMA is a domain ontology that represents a coherent body of explicit declarative knowledge about human anatomy.

The OBO Foundry (OBO) is a collaborative experiment involving developers of science-based ontologies who are establishing a set of principles for ontology development with the goal of creating a suite of orthogonal interoperable reference ontologies in the biomedical domain. The OBO Foundry consists of more that 120 ontologies of varying expressive power and size. As an example in our dataset we have included FMA-lite and subsets of NCI-Thesaurus .

To test the scalability of ontology engineering environments against generate synthetic ontologies we provide LUBM syntetic test data generator.

The synthetic data, incroporated into LUBM Generator Scalability Test Suite(EXTENDED!), will be used for evaluation of size or complexity against time or memory consumption.