Abstract: A huge number of historical documents have been digitized over the last ten years. Browsing into these collections can be done using query by keywords or query by example systems. Going from one kind of query to another raises the problem of the semantic gap. In order to deal with this problem, this paper presents an ontology-based approach to the resolution of the semantic gap problem that uses inference rules with historical images. To do this, historians’ knowledge and knowledge from the document processing domain were modeled using dedicated ontologies. Then, links between the regions of interest from the computer vision algorithms on the one hand, and their meaning on the other hand, were automatically created. These links will subsequently be used to help historians retrieve similar images. Based on the three ontologies defined and combined in this approach, we have defined rules to automatically annotate an image (to define the background for example) or a part of an image (to identify a letter, a body-part, …).
