Comparing Span Extraction Methods for Semantic Role Labeling

Zhisong Zhang; Emma Strubell; Eduard Hovy

doi:10.18653/v1/2021.spnlp-1.8

Comparing Span Extraction Methods for Semantic Role Labeling

Zhisong Zhang
, Emma Strubell
, Eduard Hovy

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

13 Citations (Scopus)

1 Downloads (CityUHK Scholars)

Abstract

In this work, we empirically compare span extraction methods for the task of semantic role labeling (SRL). While recent progress incorporating pre-trained contextualized representations into neural encoders has greatly improved SRL F1 performance on popular benchmarks, the potential costs and benefits of structured decoding in these models have become less clear. With extensive experiments on PropBank SRL datasets, we find that more structured decoding methods outperform BIO-tagging when using static (word type) embeddings across all experimental settings. However, when used in conjunction with pre-trained contextualized word representations, the benefits are diminished. We also experiment in cross-genre and cross-lingual settings and find similar trends. We further perform speed comparisons and provide analysis on the accuracy-efficiency trade-offs among different decoding methods. © 2021 Association for Computational Linguistics.

Original language	English
Title of host publication	SPNLP 2021 - The 5th Workshop on Structured Prediction for NLP, Proceedings of the Workshop
Editors	Zornitsa Kozareva, Sujith Ravi, Andreas Vlachos, Priyanka Agrawal, André Martins
Publisher	Association for Computational Linguistics
Pages	67-77
Number of pages	11
ISBN (Print)	9781954085756
DOIs	https://doi.org/10.18653/v1/2021.spnlp-1.8
Publication status	Published - Aug 2021
Externally published	Yes
Event	5th Workshop on Structured Prediction for NLP (SPNLP 2021) - Virtual, Bangkok, Thailand Duration: 6 Aug 2021 → … https://aclanthology.org/2021.spnlp-1

Publication series

Name	SPNLP - Workshop on Structured Prediction for NLP, Proceedings of the Workshop

Conference

Conference	5th Workshop on Structured Prediction for NLP (SPNLP 2021)
Place	Thailand
City	Bangkok
Period	6/08/21 → …
Internet address	https://aclanthology.org/2021.spnlp-1

Publisher's Copyright Statement

This full text is made available under CC-BY 4.0. https://creativecommons.org/licenses/by/4.0/

Access Output

10.18653/v1/2021.spnlp-1.8

316818309.pdfFinal Published version, 640 KBLicence: CC BY

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Zhang, Z., Strubell, E., & Hovy, E. (2021). Comparing Span Extraction Methods for Semantic Role Labeling. In Z. Kozareva, S. Ravi, A. Vlachos, P. Agrawal, & A. Martins (Eds.), SPNLP 2021 - The 5th Workshop on Structured Prediction for NLP, Proceedings of the Workshop (pp. 67-77). (SPNLP - Workshop on Structured Prediction for NLP, Proceedings of the Workshop). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.spnlp-1.8

Zhang, Zhisong ; Strubell, Emma ; Hovy, Eduard. / Comparing Span Extraction Methods for Semantic Role Labeling. SPNLP 2021 - The 5th Workshop on Structured Prediction for NLP, Proceedings of the Workshop. editor / Zornitsa Kozareva ; Sujith Ravi ; Andreas Vlachos ; Priyanka Agrawal ; André Martins. Association for Computational Linguistics, 2021. pp. 67-77 (SPNLP - Workshop on Structured Prediction for NLP, Proceedings of the Workshop).

@inproceedings{d1b04cde39e74bd6a129af618234a6cd,

title = "Comparing Span Extraction Methods for Semantic Role Labeling",

abstract = "In this work, we empirically compare span extraction methods for the task of semantic role labeling (SRL). While recent progress incorporating pre-trained contextualized representations into neural encoders has greatly improved SRL F1 performance on popular benchmarks, the potential costs and benefits of structured decoding in these models have become less clear. With extensive experiments on PropBank SRL datasets, we find that more structured decoding methods outperform BIO-tagging when using static (word type) embeddings across all experimental settings. However, when used in conjunction with pre-trained contextualized word representations, the benefits are diminished. We also experiment in cross-genre and cross-lingual settings and find similar trends. We further perform speed comparisons and provide analysis on the accuracy-efficiency trade-offs among different decoding methods. {\textcopyright} 2021 Association for Computational Linguistics.",

author = "Zhisong Zhang and Emma Strubell and Eduard Hovy",

year = "2021",

month = aug,

doi = "10.18653/v1/2021.spnlp-1.8",

language = "English",

isbn = "9781954085756",

series = "SPNLP - Workshop on Structured Prediction for NLP, Proceedings of the Workshop",

publisher = "Association for Computational Linguistics",

pages = "67--77",

editor = "Zornitsa Kozareva and Sujith Ravi and Andreas Vlachos and Priyanka Agrawal and Andr{\'e} Martins",

booktitle = "SPNLP 2021 - The 5th Workshop on Structured Prediction for NLP, Proceedings of the Workshop",

address = "United States",

note = "5th Workshop on Structured Prediction for NLP (SPNLP 2021) ; Conference date: 06-08-2021",

url = "https://aclanthology.org/2021.spnlp-1",

}

Zhang, Z, Strubell, E & Hovy, E 2021, Comparing Span Extraction Methods for Semantic Role Labeling. in Z Kozareva, S Ravi, A Vlachos, P Agrawal & A Martins (eds), SPNLP 2021 - The 5th Workshop on Structured Prediction for NLP, Proceedings of the Workshop. SPNLP - Workshop on Structured Prediction for NLP, Proceedings of the Workshop, Association for Computational Linguistics, pp. 67-77, 5th Workshop on Structured Prediction for NLP (SPNLP 2021), Bangkok, Thailand, 6/08/21. https://doi.org/10.18653/v1/2021.spnlp-1.8

Comparing Span Extraction Methods for Semantic Role Labeling. / Zhang, Zhisong; Strubell, Emma; Hovy, Eduard.
SPNLP 2021 - The 5th Workshop on Structured Prediction for NLP, Proceedings of the Workshop. ed. / Zornitsa Kozareva; Sujith Ravi; Andreas Vlachos; Priyanka Agrawal; André Martins. Association for Computational Linguistics, 2021. p. 67-77 (SPNLP - Workshop on Structured Prediction for NLP, Proceedings of the Workshop).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Comparing Span Extraction Methods for Semantic Role Labeling

AU - Zhang, Zhisong

AU - Strubell, Emma

AU - Hovy, Eduard

PY - 2021/8

Y1 - 2021/8

N2 - In this work, we empirically compare span extraction methods for the task of semantic role labeling (SRL). While recent progress incorporating pre-trained contextualized representations into neural encoders has greatly improved SRL F1 performance on popular benchmarks, the potential costs and benefits of structured decoding in these models have become less clear. With extensive experiments on PropBank SRL datasets, we find that more structured decoding methods outperform BIO-tagging when using static (word type) embeddings across all experimental settings. However, when used in conjunction with pre-trained contextualized word representations, the benefits are diminished. We also experiment in cross-genre and cross-lingual settings and find similar trends. We further perform speed comparisons and provide analysis on the accuracy-efficiency trade-offs among different decoding methods. © 2021 Association for Computational Linguistics.

AB - In this work, we empirically compare span extraction methods for the task of semantic role labeling (SRL). While recent progress incorporating pre-trained contextualized representations into neural encoders has greatly improved SRL F1 performance on popular benchmarks, the potential costs and benefits of structured decoding in these models have become less clear. With extensive experiments on PropBank SRL datasets, we find that more structured decoding methods outperform BIO-tagging when using static (word type) embeddings across all experimental settings. However, when used in conjunction with pre-trained contextualized word representations, the benefits are diminished. We also experiment in cross-genre and cross-lingual settings and find similar trends. We further perform speed comparisons and provide analysis on the accuracy-efficiency trade-offs among different decoding methods. © 2021 Association for Computational Linguistics.

UR - https://www.scopus.com/pages/publications/85118021740

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85118021740&origin=recordpage

U2 - 10.18653/v1/2021.spnlp-1.8

DO - 10.18653/v1/2021.spnlp-1.8

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9781954085756

T3 - SPNLP - Workshop on Structured Prediction for NLP, Proceedings of the Workshop

SP - 67

EP - 77

BT - SPNLP 2021 - The 5th Workshop on Structured Prediction for NLP, Proceedings of the Workshop

A2 - Kozareva, Zornitsa

A2 - Ravi, Sujith

A2 - Vlachos, Andreas

A2 - Agrawal, Priyanka

A2 - Martins, André

PB - Association for Computational Linguistics

T2 - 5th Workshop on Structured Prediction for NLP (SPNLP 2021)

Y2 - 6 August 2021

ER -

Zhang Z, Strubell E, Hovy E. Comparing Span Extraction Methods for Semantic Role Labeling. In Kozareva Z, Ravi S, Vlachos A, Agrawal P, Martins A, editors, SPNLP 2021 - The 5th Workshop on Structured Prediction for NLP, Proceedings of the Workshop. Association for Computational Linguistics. 2021. p. 67-77. (SPNLP - Workshop on Structured Prediction for NLP, Proceedings of the Workshop). doi: 10.18653/v1/2021.spnlp-1.8

Comparing Span Extraction Methods for Semantic Role Labeling

Abstract

Publication series

Conference

Publisher's Copyright Statement

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this