Semantic query processing and annotation generation for content-based retrieval of histological images

In this paper we present a semantic content representation scheme and the associated techniques for supporting (a) query by image examples or by natural language in a histological image database and (b) automatic annotation generation for images through image semantic analysis. In this research, various types of query are analyzed by either a semantic analyzer or a natural language analyzer to extract high level concepts and histological information, which are subsequently converted into an internal semantic content representation structure code-named `Papillon'. Papillon serves not only as an intermediate representation scheme but also stores the semantic content of the image that will be used to match against the semantic index structure within the image database during query processing. During the image database population phase, all images that are going to be put into the database will go through the same processing so that every image would have its semantic content represented by a Papillon structure. Since the Papillon structure for an image contains high level semantic information of the image, it forms the basis of the technique that automatically generates textual annotation for the input images. Papillon bridges the gap between different media in the database, allows complicated intelligent browsing to be carried out efficiently, and also provides a well-defined semantic content representation scheme for different content processing engines developed for content-based retrieval.