Creating a Semantic Hierarchy of SUN Database Object Labels Using WordNet

Using a semantic hierarchy as a labeling scheme can provide object detection systems with a more robust expert knowledge of the relationships between object classes. This knowledge can be used to improve object class prediction in cases where an object detector encounters an object of a class upon which it was not trained, known as zero-shot object detection or open-set recognition. Datasets which are useful for a particular application, domain, or task may not have their object labels organized into an appropriate semantic hierarchy. For example, the Scene UNderstanding (SUN1) Database has image scenes organized into a hierarchy, but no such organization exists with the object labels. Objects in the images of this dataset were annotated in a crowd-sourced manner which allowed annotators to define the polygons which bound the objects as well as assign the labels. The challenge taken up by the method presented in this paper was to take the original object labels of the SUN Database and create a semantic hierarchy such that each child-parent pair of object classes demonstrates an "IS-A" relationship. By associating common labels within the dataset with the most relevant and fine-grained WordNet2 synonym, this approach resulted in a multi-layered semantic hierarchy for SUN Database object labels. The product is a tree-structured graph where each node is a WordNet synonym of the original label and a node's parent is determined by its WordNet hypernym. Other ontological frameworks, such as Basic Formal Ontology(3) and the Operational Environment Ontology Suite,(4) are also discussed.