Data-driven manga layout and composition

: models and methods

Abstract

Due to unique storytelling techniques and rich expression styles, Manga, i.e., Japanese Comics, has become one of most popular storytelling mediums across the world, with a growing number of people consuming manga in various forms and creating their own manga-like artworks. However, the manga production process often requires a significant amount of expertise and hands-on experiences, rendering it impossible for nonexperts to express their vision as professional manga artists do. In an attempt to assist novices in comic creation, researchers have developed various rule-based computer algorithms to automate some steps or even the whole process of comic production. Unfortunately, it is still difficult for the algorithms to faithfully reproduce the styles and functional characteristics of manga. This is mainly because existing algorithms lack sophisticated understanding of the complex knowledge and skills behind the production process, which are utilized by professional artists but cannot be fully captured by a fixed set of rules. A large collection of existing artworks created by professional artists essentially encode a wide range of knowledge and skills used by professional artists. This inspires us to look at the problem from a data-driven perspective. Therefore, to develop computational algorithms to assist manga creation, we propose to employ a data-driven strategy instead of using traditional rule-based methods. We aim at empowering the computer with the ability to understand and learn knowledge implicit to the domain expert of professional artists from a corpus of existing manga artworks. This allows the computer to help novices produce professional-looking manga artworks with least amount of efforts. To this end, we present two novel data-driven techniques for manga layout and composition. In both techniques, we propose a series of parametric models that describe various stylistic and functional aspects of manga and can be learned from existing manga pages, and methods based on the models for particular tasks. Manga Layout. Manga layout is a core component in manga production, characterized by its unique styles. We propose an approach to automatically generate a stylistic manga layout from a set of input artworks with user-specified semantics. We first introduce three parametric style models that encode the unique stylistic aspects of manga layouts, including layout structure, panel importance, and panel shape. Next, we propose a two-stage approach to generate a manga layout. Through a user study, we demonstrate that our approach enables novice users to easily and quickly produce higher-quality layouts that exhibit realistic manga styles, when compared to a commercially-available manual layout tool. Manga Element Composition. Picture subjects and text balloons are basic elements in comics, working together to propel the story forward. Japanese comics artists often leverage a carefully designed composition of subjects and balloons (generally referred to as panel elements) to provide a continuous and fluid reading experience. We propose an approach for novices to synthesize a composition of panel elements that can effectively guide the reader's attention to convey the story. Our primary contribution is a probabilistic graphical model that describes the relationships among the artist's guiding path, the panel elements, and the viewer attention, which can be effectively learned from a small set of existing manga pages. We show that the proposed approach can measurably improve the readability, visual appeal, and communication of the story of the resulting pages, as compared to an existing method.

Date of Award	16 Feb 2015
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Antoni Bert CHAN (Supervisor) & Rynson W H LAU (Co-supervisor)