JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds

Zeyu Hu; Mingmin Zhen; Xuyang Bai; Hongbo Fu; Chiew-lan Tai

doi:10.1007/978-3-030-58565-5_14

JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds

Zeyu Hu^*
, Mingmin Zhen
, Xuyang Bai
, Hongbo Fu
, Chiew-lan Tai

^*Corresponding author for this work

School of Creative Media

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

109 Citations (Scopus)

Abstract

Semantic segmentation and semantic edge detection can be seen as two dual problems with close relationships in computer vision. Despite the fast evolution of learning-based 3D semantic segmentation methods, little attention has been drawn to the learning of 3D semantic edge detectors, even less to a joint learning method for the two tasks. In this paper, we tackle the 3D semantic edge detection task for the first time and present a new two-stream fully-convolutional network that jointly performs the two tasks. In particular, we design a joint refinement module that explicitly wires region information and edge information to improve the performances of both tasks. Further, we propose a novel loss function that encourages the network to produce semantic segmentation results with better boundaries. Extensive evaluations on S3DIS and ScanNet datasets show that our method achieves on par or better performance than the state-of-the-art methods for semantic segmentation and outperforms the baseline methods for semantic edge detection. Code release: https://github.com/hzykent/JSENet.

Original language	English
Title of host publication	Computer Vision – ECCV 2020
Subtitle of host publication	16th European Conference 2020, Proceedings
Editors	Andrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
Publisher	Springer Nature
Pages	222-239
Volume	XX
ISBN (Electronic)	978-3-030-58565-5
ISBN (Print)	978-3-030-58564-8
DOIs	https://doi.org/10.1007/978-3-030-58565-5_14
Publication status	Published - Aug 2020
Event	16th European Conference on Computer Vision (ECCV 2020) - Online, Glasgow, United Kingdom Duration: 23 Aug 2020 → 28 Aug 2020 https://eccv2020.eu/

Publication series

Name	Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)
Volume	12365
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th European Conference on Computer Vision (ECCV 2020)
Abbreviated title	ECCV 2020
Place	United Kingdom
City	Glasgow
Period	23/08/20 → 28/08/20
Internet address	https://eccv2020.eu/

Bibliographical note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Research Keywords

3D point clouds
3D scene understanding
Semantic edge detection
Semantic segmentation

Access Output

10.1007/978-3-030-58565-5_14

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Hu, Z., Zhen, M., Bai, X., Fu, H., & Tai, C. (2020). JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds. In A. Vedaldi, H. Bischof, T. Brox, & J.-M. Frahm (Eds.), Computer Vision – ECCV 2020: 16th European Conference 2020, Proceedings (Vol. XX, pp. 222-239). (Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics); Vol. 12365). Springer Nature. https://doi.org/10.1007/978-3-030-58565-5_14

Hu, Zeyu ; Zhen, Mingmin ; Bai, Xuyang et al. / JSENet : Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds. Computer Vision – ECCV 2020: 16th European Conference 2020, Proceedings. editor / Andrea Vedaldi ; Horst Bischof ; Thomas Brox ; Jan-Michael Frahm. Vol. XX Springer Nature, 2020. pp. 222-239 (Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)).

@inproceedings{e416fd6008a6461d9578f4d8849f6ac7,

title = "JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds",

abstract = "Semantic segmentation and semantic edge detection can be seen as two dual problems with close relationships in computer vision. Despite the fast evolution of learning-based 3D semantic segmentation methods, little attention has been drawn to the learning of 3D semantic edge detectors, even less to a joint learning method for the two tasks. In this paper, we tackle the 3D semantic edge detection task for the first time and present a new two-stream fully-convolutional network that jointly performs the two tasks. In particular, we design a joint refinement module that explicitly wires region information and edge information to improve the performances of both tasks. Further, we propose a novel loss function that encourages the network to produce semantic segmentation results with better boundaries. Extensive evaluations on S3DIS and ScanNet datasets show that our method achieves on par or better performance than the state-of-the-art methods for semantic segmentation and outperforms the baseline methods for semantic edge detection. Code release: https://github.com/hzykent/JSENet.",

keywords = "3D point clouds, 3D scene understanding, Semantic edge detection, Semantic segmentation",

author = "Zeyu Hu and Mingmin Zhen and Xuyang Bai and Hongbo Fu and Chiew-lan Tai",

note = "Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).; 16th European Conference on Computer Vision (ECCV 2020), ECCV 2020 ; Conference date: 23-08-2020 Through 28-08-2020",

year = "2020",

month = aug,

doi = "10.1007/978-3-030-58565-5\_14",

language = "English",

isbn = "978-3-030-58564-8",

volume = "XX",

series = "Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)",

publisher = "Springer Nature",

pages = "222--239",

editor = "Andrea Vedaldi and Horst Bischof and Thomas Brox and Jan-Michael Frahm",

booktitle = "Computer Vision – ECCV 2020",

url = "https://eccv2020.eu/",

}

Hu, Z, Zhen, M, Bai, X, Fu, H & Tai, C 2020, JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds. in A Vedaldi, H Bischof, T Brox & J-M Frahm (eds), Computer Vision – ECCV 2020: 16th European Conference 2020, Proceedings. vol. XX, Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics), vol. 12365, Springer Nature, pp. 222-239, 16th European Conference on Computer Vision (ECCV 2020), Glasgow, United Kingdom, 23/08/20. https://doi.org/10.1007/978-3-030-58565-5_14

JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds. / Hu, Zeyu; Zhen, Mingmin; Bai, Xuyang et al.
Computer Vision – ECCV 2020: 16th European Conference 2020, Proceedings. ed. / Andrea Vedaldi; Horst Bischof; Thomas Brox; Jan-Michael Frahm. Vol. XX Springer Nature, 2020. p. 222-239 (Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics); Vol. 12365).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - JSENet

T2 - 16th European Conference on Computer Vision (ECCV 2020)

AU - Hu, Zeyu

AU - Zhen, Mingmin

AU - Bai, Xuyang

AU - Fu, Hongbo

AU - Tai, Chiew-lan

N1 - Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

PY - 2020/8

Y1 - 2020/8

N2 - Semantic segmentation and semantic edge detection can be seen as two dual problems with close relationships in computer vision. Despite the fast evolution of learning-based 3D semantic segmentation methods, little attention has been drawn to the learning of 3D semantic edge detectors, even less to a joint learning method for the two tasks. In this paper, we tackle the 3D semantic edge detection task for the first time and present a new two-stream fully-convolutional network that jointly performs the two tasks. In particular, we design a joint refinement module that explicitly wires region information and edge information to improve the performances of both tasks. Further, we propose a novel loss function that encourages the network to produce semantic segmentation results with better boundaries. Extensive evaluations on S3DIS and ScanNet datasets show that our method achieves on par or better performance than the state-of-the-art methods for semantic segmentation and outperforms the baseline methods for semantic edge detection. Code release: https://github.com/hzykent/JSENet.

AB - Semantic segmentation and semantic edge detection can be seen as two dual problems with close relationships in computer vision. Despite the fast evolution of learning-based 3D semantic segmentation methods, little attention has been drawn to the learning of 3D semantic edge detectors, even less to a joint learning method for the two tasks. In this paper, we tackle the 3D semantic edge detection task for the first time and present a new two-stream fully-convolutional network that jointly performs the two tasks. In particular, we design a joint refinement module that explicitly wires region information and edge information to improve the performances of both tasks. Further, we propose a novel loss function that encourages the network to produce semantic segmentation results with better boundaries. Extensive evaluations on S3DIS and ScanNet datasets show that our method achieves on par or better performance than the state-of-the-art methods for semantic segmentation and outperforms the baseline methods for semantic edge detection. Code release: https://github.com/hzykent/JSENet.

KW - 3D point clouds

KW - 3D scene understanding

KW - Semantic edge detection

KW - Semantic segmentation

UR - https://www.scopus.com/pages/publications/85097365665

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85097365665&origin=recordpage

U2 - 10.1007/978-3-030-58565-5_14

DO - 10.1007/978-3-030-58565-5_14

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 978-3-030-58564-8

VL - XX

T3 - Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)

SP - 222

EP - 239

BT - Computer Vision – ECCV 2020

A2 - Vedaldi, Andrea

A2 - Bischof, Horst

A2 - Brox, Thomas

A2 - Frahm, Jan-Michael

PB - Springer Nature

Y2 - 23 August 2020 through 28 August 2020

ER -

Hu Z, Zhen M, Bai X, Fu H, Tai C. JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds. In Vedaldi A, Bischof H, Brox T, Frahm JM, editors, Computer Vision – ECCV 2020: 16th European Conference 2020, Proceedings. Vol. XX. Springer Nature. 2020. p. 222-239. (Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)). doi: 10.1007/978-3-030-58565-5_14

JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds

Abstract

Publication series

Conference

Bibliographical note

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this