The conformance evaluation has the goal of evaluating the conformance of semantic technologies with regards to ontology representation languages, that is, to evaluate up to what extent semantic technologies adhere to the specification of ontology representation languages.

 In the evaluation we need an automatic and uniform way of accessing most of the semantic tools, and the operations performed to access such tools must be supported by most of them. Due to the high heterogeneity in semantic tools, ontology management APIs vary from one tool to another. Therefore, the way chosen to automatically access the tools is through the following two operations commonly supported by most semantic tools: to import an ontology from a file (i.e., to load an ontology from a file into the tool internal model), and to export an ontology to a file (i.e., to store an ontology from the tool internal model into a file).

During the evaluation, a common group of tests is executed and each test describes one input ontology that has to be imported by the tool and then exported.

Each test execution comprises two steps. Starting with a file containing an ontology, the execution consists in importing the file with the ontology into the origin tool and then exporting the ontology to another file.

After a test execution, we have two ontologies in the ontology representation language, namely, the original ontology and the final ontology exported by the tool. By comparing these ontologies we can know up to what extent the tool conforms to the ontology language.

The only requirement for performing the evaluation on a tool is that the tool is able of importing and exporting ontologies in the ontology language. Therefore, this evaluation can be performed not only on ontology engineering tools but also on other types of semantic technologies (e.g., ontology repositories).