Learning Graph-embedded Key-event Back-tracing for Object Tracking in Event Clouds
Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45) › 32_Refereed conference paper (with host publication) › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Title of host publication | Advances in Neural Information Processing Systems 35 (NeurIPS 2022) |
Editors | S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh |
Publisher | Neural Information Processing Systems Foundation, Inc. |
Pages | 7462-7476 |
Number of pages | 15 |
ISBN (Print) | 9781713871088 |
Publication status | Published - Nov 2022 |
Publication series
Name | Advances in Neural Information Processing Systems |
---|---|
Volume | 35 |
ISSN (Print) | 1049-5258 |
Conference
Title | 36th Conference on Neural Information Processing Systems (NeurIPS 2022) |
---|---|
Location | Hybrid, New Orleans Convention Center |
Place | United States |
City | New Orleans |
Period | 28 November - 9 December 2022 |
Link(s)
Document Link | Links |
---|---|
Link to Scopus | https://www.scopus.com/record/display.uri?eid=2-s2.0-85163171239&origin=recordpage |
Permanent Link | https://scholars.cityu.edu.hk/en/publications/publication(284c0df6-4e03-4df2-b10a-bd7d269e26fd).html |
Abstract
Event data-based object tracking is attracting attention increasingly. Unfortunately, the unusual data structure caused by the unique sensing mechanism poses great challenges in designing downstream algorithms. To tackle such challenges, existing methods usually re-organize raw event data (or event clouds) with the event frame/image representation to adapt to mature RGB data-based tracking paradigms, which compromises the high temporal resolution and sparse characteristics. By contrast, we advocate developing new designs/techniques tailored to the special data structure to realize object tracking. To this end, we make the first attempt to construct a new end-to-end learning-based paradigm that directly consumes event clouds. Specifically, to process a non-uniformly distributed large-scale event cloud efficiently, we propose a simple yet effective density-insensitive downsampling strategy to sample a subset called key-events. Then, we employ a graph-based network to embed the irregular spatio-temporal information of key-events into a high-dimensional feature space, and the resulting embeddings are utilized to predict their target likelihoods via semantic-driven Siamese-matching. Besides, we also propose motion-aware target likelihood prediction, which learns the motion flow to back-trace the potential initial positions of key-events and measures them with the previous proposal. Finally, we obtain the bounding box by adaptively fusing the two intermediate ones separately regressed from the weighted embeddings of key-events by the two types of predicted target likelihoods. Extensive experiments on both synthetic and real event datasets demonstrate the superiority of the proposed framework over state-of-the-art methods in terms of both the tracking accuracy and speed. The code is publicly available at https://github.com/ZHU-Zhiyu/Event-tracking.
Bibliographic Note
Information for this record is supplemented by the author(s) concerned.
Citation Format(s)
Learning Graph-embedded Key-event Back-tracing for Object Tracking in Event Clouds. / ZHU, Zhiyu; HOU, Junhui; LYU, Xianqiang.
Advances in Neural Information Processing Systems 35 (NeurIPS 2022). ed. / S. Koyejo; S. Mohamed; A. Agarwal; D. Belgrave; K. Cho; A. Oh. Neural Information Processing Systems Foundation, Inc., 2022. p. 7462-7476 (Advances in Neural Information Processing Systems; Vol. 35).
Advances in Neural Information Processing Systems 35 (NeurIPS 2022). ed. / S. Koyejo; S. Mohamed; A. Agarwal; D. Belgrave; K. Cho; A. Oh. Neural Information Processing Systems Foundation, Inc., 2022. p. 7462-7476 (Advances in Neural Information Processing Systems; Vol. 35).
Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45) › 32_Refereed conference paper (with host publication) › peer-review