Context-based Sketch Classification

Jianhui Zhang; Yilan Chen; Lei Li; Hongbo Fu; Chiew-Lan Tai

doi:10.1145/3229147.3229154

Context-based Sketch Classification

Jianhui Zhang, Yilan Chen, Lei Li, Hongbo Fu, Chiew-Lan Tai

School of Creative Media

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

13 Citations (Scopus)

293 Downloads (CityUHK Scholars)

Abstract

We present a novel context-based sketch classification framework using relations extracted from scene images. Most of existing methods perform sketch classification by considering individually sketched objects and often fail to identify their correct categories, due to the highly abstract nature of sketches. For a sketched scene containing multiple objects, we propose to classify a sketched object by considering its surrounding context in the scene, which provides vital cues for alleviating its recognition ambiguity. We learn such context knowledge from a database of scene images by summarizing the inter-object relations therein, such as co-occurrence, relative positions and sizes. We show that the context information can be used for both incremental sketch classification and sketch co-classification. Our method outperforms a state-of-the-art single-object classification method, evaluated on a new dataset of sketched scenes.

Original language	English
Title of host publication	Expressive '18 - Proceedings of the Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering
Publisher	Association for Computing Machinery
ISBN (Print)	9781450358927
DOIs	https://doi.org/10.1145/3229147.3229154
Publication status	Published - Aug 2018
Event	2018 Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering (Expressive '18) - University of Victoria, Victoria, Canada Duration: 17 Aug 2018 → 19 Aug 2018 http://expressive.graphics/2018/

Publication series

Name	Proceedings - The Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering, Expressive
Volume	2018

Conference

Conference	2018 Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering (Expressive '18)
Place	Canada
City	Victoria
Period	17/08/18 → 19/08/18
Internet address	http://expressive.graphics/2018/

Bibliographical note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Research Keywords

Co-analysis
Context
Object relations
Sketch classification

Publisher's Copyright Statement

COPYRIGHT TERMS OF DEPOSITED POSTPRINT FILE: © 2018 ACM. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Expressive '18: Proceedings of the Joint Symposium on Computational Aesthetics and Sketch-Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering, http://dx.doi.org/10.1145/3229147.

Access Output

10.1145/3229147.3229154

Post-PrintPost-print, 7.67 MBLicence: Unspecified

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Zhang, J., Chen, Y., Li, L., Fu, H., & Tai, C.-L. (2018). Context-based Sketch Classification. In Expressive '18 - Proceedings of the Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering Article 3 (Proceedings - The Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering, Expressive; Vol. 2018). Association for Computing Machinery. https://doi.org/10.1145/3229147.3229154

Zhang, Jianhui ; Chen, Yilan ; Li, Lei et al. / Context-based Sketch Classification. Expressive '18 - Proceedings of the Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering. Association for Computing Machinery, 2018. (Proceedings - The Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering, Expressive).

@inproceedings{5f4839a22fda43dc9a811f281eee7506,

title = "Context-based Sketch Classification",

abstract = "We present a novel context-based sketch classification framework using relations extracted from scene images. Most of existing methods perform sketch classification by considering individually sketched objects and often fail to identify their correct categories, due to the highly abstract nature of sketches. For a sketched scene containing multiple objects, we propose to classify a sketched object by considering its surrounding context in the scene, which provides vital cues for alleviating its recognition ambiguity. We learn such context knowledge from a database of scene images by summarizing the inter-object relations therein, such as co-occurrence, relative positions and sizes. We show that the context information can be used for both incremental sketch classification and sketch co-classification. Our method outperforms a state-of-the-art single-object classification method, evaluated on a new dataset of sketched scenes.",

keywords = "Co-analysis, Context, Object relations, Sketch classification",

author = "Jianhui Zhang and Yilan Chen and Lei Li and Hongbo Fu and Chiew-Lan Tai",

note = "Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).; 2018 Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering (Expressive '18) ; Conference date: 17-08-2018 Through 19-08-2018",

year = "2018",

month = aug,

doi = "10.1145/3229147.3229154",

language = "English",

isbn = "9781450358927",

series = "Proceedings - The Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering, Expressive",

publisher = "Association for Computing Machinery",

booktitle = "Expressive '18 - Proceedings of the Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering",

address = "United States",

url = "http://expressive.graphics/2018/",

}

Zhang, J, Chen, Y, Li, L, Fu, H & Tai, C-L 2018, Context-based Sketch Classification. in Expressive '18 - Proceedings of the Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering., 3, Proceedings - The Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering, Expressive, vol. 2018, Association for Computing Machinery, 2018 Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering (Expressive '18), Victoria, British Columbia, Canada, 17/08/18. https://doi.org/10.1145/3229147.3229154

Context-based Sketch Classification. / Zhang, Jianhui; Chen, Yilan; Li, Lei et al.
Expressive '18 - Proceedings of the Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering. Association for Computing Machinery, 2018. 3 (Proceedings - The Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering, Expressive; Vol. 2018).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Context-based Sketch Classification

AU - Zhang, Jianhui

AU - Chen, Yilan

AU - Li, Lei

AU - Fu, Hongbo

AU - Tai, Chiew-Lan

N1 - Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

PY - 2018/8

Y1 - 2018/8

N2 - We present a novel context-based sketch classification framework using relations extracted from scene images. Most of existing methods perform sketch classification by considering individually sketched objects and often fail to identify their correct categories, due to the highly abstract nature of sketches. For a sketched scene containing multiple objects, we propose to classify a sketched object by considering its surrounding context in the scene, which provides vital cues for alleviating its recognition ambiguity. We learn such context knowledge from a database of scene images by summarizing the inter-object relations therein, such as co-occurrence, relative positions and sizes. We show that the context information can be used for both incremental sketch classification and sketch co-classification. Our method outperforms a state-of-the-art single-object classification method, evaluated on a new dataset of sketched scenes.

AB - We present a novel context-based sketch classification framework using relations extracted from scene images. Most of existing methods perform sketch classification by considering individually sketched objects and often fail to identify their correct categories, due to the highly abstract nature of sketches. For a sketched scene containing multiple objects, we propose to classify a sketched object by considering its surrounding context in the scene, which provides vital cues for alleviating its recognition ambiguity. We learn such context knowledge from a database of scene images by summarizing the inter-object relations therein, such as co-occurrence, relative positions and sizes. We show that the context information can be used for both incremental sketch classification and sketch co-classification. Our method outperforms a state-of-the-art single-object classification method, evaluated on a new dataset of sketched scenes.

KW - Co-analysis

KW - Context

KW - Object relations

KW - Sketch classification

UR - http://www.scopus.com/inward/record.url?scp=85056142071&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85056142071&origin=recordpage

U2 - 10.1145/3229147.3229154

DO - 10.1145/3229147.3229154

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9781450358927

T3 - Proceedings - The Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering, Expressive

BT - Expressive '18 - Proceedings of the Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering

PB - Association for Computing Machinery

T2 - 2018 Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering (Expressive '18)

Y2 - 17 August 2018 through 19 August 2018

ER -

Zhang J, Chen Y, Li L, Fu H, Tai CL. Context-based Sketch Classification. In Expressive '18 - Proceedings of the Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering. Association for Computing Machinery. 2018. 3. (Proceedings - The Joint Symposium on Computational Aesthetics Sketch-Based Interfaces and Modeling Non-Photorealistic Animation and Rendering, Expressive). doi: 10.1145/3229147.3229154

Context-based Sketch Classification

Abstract

Publication series

Conference

Bibliographical note

Research Keywords

Publisher's Copyright Statement

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

GRF: Support-driven Shape Analysis

GRF: Data-Driven 3D Interpretation of Freehand Drawings

Cite this

Context-based Sketch Classification

Abstract

Publication series

Conference

Bibliographical note

Research Keywords

Publisher's Copyright Statement

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

GRF: Support-driven Shape Analysis

GRF: Data-Driven 3D Interpretation of Freehand Drawings

Cite this