ST-TrackNet: A Multiple-Object Tracking Network Using Spatio-Temporal Information

Sukai Wang; Yuxiang Sun; Zheng Wang; Ming Liu

doi:10.1109/TASE.2022.3216450

ST-TrackNet: A Multiple-Object Tracking Network Using Spatio-Temporal Information

Sukai Wang, Yuxiang Sun, Zheng Wang, Ming Liu^*

^*Corresponding author for this work

Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review

10 Citations (Scopus)

Abstract

Multiple-object tracking (MOT) is a crucial component in autonomous driving systems. However, inaccurate object detection is always the bottleneck for MOT. Most detectors are not designed to take the temporal information across consecutive frames into consideration. To take advantage of such information, we design a novel data representation, the spatio-temporal (ST) map, which collects a batch of detection results spatio-temporally, and we train a novel network, ST-TrackNet, to assign predicted track IDs to each positive detection across a sequence. With our ST map detection fed into the tracker, the correlation of objects between adjacent frames becomes prominent, which improves the performance of the tracker in the data association step. Moreover, the long-term trajectory in a sequence also helps to refine the detection results. We train and evaluate our network on the KITTI dataset, a CARLA simulation dataset, and a dataset recorded in a factory environment. Our approach generally achieves superior performance over the state-of-the-art. © 2022 IEEE.

Original language	English
Pages (from-to)	284-295
Journal	IEEE Transactions on Automation Science and Engineering
Volume	21
Issue number	1
Online published	31 Oct 2022
DOIs	https://doi.org/10.1109/TASE.2022.3216450
Publication status	Published - Jan 2024
Externally published	Yes

Funding

This article was recommended for publication by Associate Editor Y. Pan and Editor C. Seatzu upon evaluation of the reviewers’ comments. This work was supported in part by the Guangdong Basic and Applied Basic Research Foundation under Project 2021B1515120032 and in part by the Zhongshan Science and Technology Bureau Fund under Project 2020AG002. The work of Ming Liu was supported by the Project of Hetao Shenzhen-Hong Kong Science and Technology Innovation Cooperation Zone under Grant HZQBKCZYB-2020083. (

Research Keywords

Autonomous driving
deep learning
Detectors
Feature extraction
Long short term memory
multi-object tracking
Object detection
Point cloud compression
spatio-temporal map
Tracking
Trajectory

Access Output

10.1109/TASE.2022.3216450

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@article{748ba0126c8a4ae9826568e4908b748c,

title = "ST-TrackNet: A Multiple-Object Tracking Network Using Spatio-Temporal Information",

abstract = "Multiple-object tracking (MOT) is a crucial component in autonomous driving systems. However, inaccurate object detection is always the bottleneck for MOT. Most detectors are not designed to take the temporal information across consecutive frames into consideration. To take advantage of such information, we design a novel data representation, the spatio-temporal (ST) map, which collects a batch of detection results spatio-temporally, and we train a novel network, ST-TrackNet, to assign predicted track IDs to each positive detection across a sequence. With our ST map detection fed into the tracker, the correlation of objects between adjacent frames becomes prominent, which improves the performance of the tracker in the data association step. Moreover, the long-term trajectory in a sequence also helps to refine the detection results. We train and evaluate our network on the KITTI dataset, a CARLA simulation dataset, and a dataset recorded in a factory environment. Our approach generally achieves superior performance over the state-of-the-art. {\textcopyright} 2022 IEEE.",

keywords = "Autonomous driving, deep learning, Detectors, Feature extraction, Long short term memory, multi-object tracking, Object detection, Point cloud compression, spatio-temporal map, Tracking, Trajectory",

author = "Sukai Wang and Yuxiang Sun and Zheng Wang and Ming Liu",

year = "2024",

month = jan,

doi = "10.1109/TASE.2022.3216450",

language = "English",

volume = "21",

pages = "284--295",

journal = "IEEE Transactions on Automation Science and Engineering",

issn = "1545-5955",

publisher = "IEEE",

number = "1",

}

TY - JOUR

T1 - ST-TrackNet

T2 - A Multiple-Object Tracking Network Using Spatio-Temporal Information

AU - Wang, Sukai

AU - Sun, Yuxiang

AU - Wang, Zheng

AU - Liu, Ming

PY - 2024/1

Y1 - 2024/1

N2 - Multiple-object tracking (MOT) is a crucial component in autonomous driving systems. However, inaccurate object detection is always the bottleneck for MOT. Most detectors are not designed to take the temporal information across consecutive frames into consideration. To take advantage of such information, we design a novel data representation, the spatio-temporal (ST) map, which collects a batch of detection results spatio-temporally, and we train a novel network, ST-TrackNet, to assign predicted track IDs to each positive detection across a sequence. With our ST map detection fed into the tracker, the correlation of objects between adjacent frames becomes prominent, which improves the performance of the tracker in the data association step. Moreover, the long-term trajectory in a sequence also helps to refine the detection results. We train and evaluate our network on the KITTI dataset, a CARLA simulation dataset, and a dataset recorded in a factory environment. Our approach generally achieves superior performance over the state-of-the-art. © 2022 IEEE.

AB - Multiple-object tracking (MOT) is a crucial component in autonomous driving systems. However, inaccurate object detection is always the bottleneck for MOT. Most detectors are not designed to take the temporal information across consecutive frames into consideration. To take advantage of such information, we design a novel data representation, the spatio-temporal (ST) map, which collects a batch of detection results spatio-temporally, and we train a novel network, ST-TrackNet, to assign predicted track IDs to each positive detection across a sequence. With our ST map detection fed into the tracker, the correlation of objects between adjacent frames becomes prominent, which improves the performance of the tracker in the data association step. Moreover, the long-term trajectory in a sequence also helps to refine the detection results. We train and evaluate our network on the KITTI dataset, a CARLA simulation dataset, and a dataset recorded in a factory environment. Our approach generally achieves superior performance over the state-of-the-art. © 2022 IEEE.

KW - Autonomous driving

KW - deep learning

KW - Detectors

KW - Feature extraction

KW - Long short term memory

KW - multi-object tracking

KW - Object detection

KW - Point cloud compression

KW - spatio-temporal map

KW - Tracking

KW - Trajectory

UR - http://www.scopus.com/inward/record.url?scp=85141602914&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85141602914&origin=recordpage

U2 - 10.1109/TASE.2022.3216450

DO - 10.1109/TASE.2022.3216450

M3 - RGC 21 - Publication in refereed journal

SN - 1545-5955

VL - 21

SP - 284

EP - 295

JO - IEEE Transactions on Automation Science and Engineering

JF - IEEE Transactions on Automation Science and Engineering

IS - 1

ER -

ST-TrackNet: A Multiple-Object Tracking Network Using Spatio-Temporal Information

Abstract

Funding

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this