Unit 12: Ontology Evaluation – Case Study
Overview
As per the course website, "This unit focuses on the end-to-end development of knowledge-based systems using ontology. It discusses the approach to ontology development as well as the methodology. It also discusses the approaches to evaluation based on a case study."
My Reflection
Overall Reflection
In this final unit, we wrapped up the module with a series of readings on the development and evaluation of an ontology for guiding appropriate antibiotic prescriptions and ontology development strategies in industrial contexts, in addition to the final chapter of the text book Handbook of Knowledge Representation on Knowledge Engineering. I also did an additional reading for an an insightful article that I came across on the direction of ontologies, context graphs and semantic layers in 2026. These readings collectively served as a wrap up for my main two questions that kept my mind busy throughout the module: 1) What is the relation between ontologies -knowledge representation in general- and relational databases? 2) Where do ontologies and knowledge representation stand in the era of large language models (LLMs)?
Lastly, the assignment of the week is to submit an individual reflection on the whole module, which I will post to the the main page of the module.
Readings Reflection
The first paper by Bright et al. (2012) provided an example paper of how to present the development and evaluation of an ontology, which made me reflect on my module's final assignment that I submitted in Unit 11. Since this paper did not have the 1000-word limit constraint, it was possible to present the paper in an IMRD format, which served the purpose of paper well. Other than this, I especially liked the paper's emphasis on incorporating domain expert feedback and review in the development and evaluation of the ontology. Had I read this paper before the submission of my final assignment , I would have stated the absence of domain expert feedback as a limitation of my work. I also liked that the paper presented the the alerts generated by their ontology from a user perspective, showing that that the developed ontology's axioms are intact and serves the purpose of knowledge representation and reasoning.
The second paper by Tarasov, Seigerroth and Sandkuhl (2019) focused on ontologies that bridge terminologies of business and IT (BITA), and how industrial contexts shape ontology development strategies for this purpose. The paper introduced a few development methodologies, like Ontology 101, Methontology, Extreme Design (XD) and hybrid approaches. It emphasised the discussion around the applicability of the conceptual models to problem solving systems and the ability to integrate or re-use existing ontologies, and build the new ones for further re-use by others. The paper provided several case studies, like OSTAG for software development, CLICK for research collaboration, SEMCO for automotive systems engineering and SEMA for semantic video object recognition. The paper made me think further about how an ontology can be developed in my own workplace and my team to bridge the gap between business and technical sides.
The book chapter on Knowledge Engineering by Schreiber (2008) provided an insightful overview on Knowledge Engineering and where ontologies fit in it. Regarding ontologies, the chapter emphasised the importance of context, '“Context” is therefore an important notion when reusing an ontology. We cannot expect other people or programs to understand our conceptualization, if we do not explicate what the context of the ontology is.' It also summarised what I previously understood in the module that there are generally three types of ontologies: foundational, domain-specific and task-specific. That chapter added that although foundational ontologies are the ones that are closest to the philosophical notion of ontology, and are recieving a lot of attention, according to the author, most of ontologies are domain specific.
The chapter also introduced me to the concepts of ontology engineering as a discipline, and the ontological commitments as an act that ontology engineers make, even if they are not aware of it. I may describe this act as how strict an ontology claims to represent reality, and which reality it claims to represent. Ontologies can have too many axioms that do not serve its purpose, and therefore is said to be 'over-committed'. Ontology commitments can also be biased, as in asserting that each user must have a first name and a last name, for example, which reflects a Western cultural bias. This last bit on ontological biases made me think about all the systems that we use every day, and how often we feel some discrepancies between them and the reality of the culture that it claims to be part of and represent.
Lastly, the chapter very concisely answered my first main lingering question on the relationship between ontologies and relational databases. In one paragraph, the author said, 'The difference between ontologies and data models does not lie in the language being used: you can define an ontology in a basic ER language (although you will be hampered in what you can say); similarly, you can write a data model with OWL. Writing something in OWL does not make it an ontology! The key difference is not the language the intended use. A data model is a model of the information in some restricted well-delimited application domain, whereas an ontology is intended to provide a set of shared concepts for multiple users and applications. To put it simply: data models live in a relatively small closed world; ontologies are meant for an open, distributed world (hence their importance for the Web).'
Thank you, Schreiber, the author of the chapter, for the answer!
Finally, the additional article that I read and that I want to highlight, by Talisman (2026) provided a connection between semantic layers, ontologies and context graphs, in the new context of AI agents and assistants, and a bit of historical contexts of how semantic layers have been used to solve the problem of discrepancy in metric calculation across databases in the domain of business intelligence, a problem that I can relate to. It further argues that, with the advent of AI, semantic layers are no longer enough, and ontologies are needed as they formally represent knowledge with its underlying concepts, relationships and constraints, and emphasise meaning, which enable reasoning and inference. It also provides a comparison between semantic layers, ontologies and context graphs that can be summarised as follows:
| Feature | Semantic Layers | Ontologies | Context Graphs |
|---|---|---|---|
| Purpose | Metric consistency, BI dashboards | Knowledge representation, reasoning | Capturing decision context & justification |
| Output | Standardised metrics | Classes, properties, relationships | Audit trails, procedural knowledge |
| Strengths | Easy adoption by data teams | Supports inference, disambiguation | Explains “why” actions occurred |
| Limitations | Focused on calculations, not meaning | Requires rare expertise, long investment | Needs deep knowledge elicitation |
Reference List
Bright, T.J. et al. (2012) ‘Development and Evaluation of an Ontology for Guiding Appropriate Antibiotic Prescribing’, Journal of Biomedical Informatics, 45(1), pp. 120–128. Available at: https://doi.org/10.1016/j.jbi.2011.10.001.
Schreiber, G. (2008) ‘Knowledge Engineering’, in F. van Harmelen, V. Lifschitz, and B. Porter (eds) Handbook of Knowledge Representation. United Kingdom: Elsevier B.V., pp. 929–946.
Talisman, J. (2026) ‘Ontologies, Context Graphs, and Semantic Layers: What AI Actually Needs in 2026’, Metadata Weekly, 22 January. Available at: https://metadataweekly.substack.com/p/ontologies-context-graphs-and-semantic (Accessed: 23 January 2026).
Tarasov, V., Seigerroth, U. and Sandkuhl, K. (2019) ‘Ontology Development Strategies in Industrial Contexts’, in Lecture Notes in Business Information Processing. 21st International Conference on Business Information Systems, BIS 2018, Cham: Springer, pp. 156–167. Available at: http://urn.kb.se/resolve?urn=urn:nbn:se:hj:diva-43205 (Accessed: 23 January 2026).