Tracking GPTs Third Party Service: Automation, Analysis, and Insights

Chuan Yan; Liuhuo Wan; Bowei Guan; Fengqi Yu; Guangdong Bai

doi:10.1145/3696630.3728711

Tracking GPTs Third Party Service: Automation, Analysis, and Insights

Chuan Yan
, Liuhuo Wan
, Bowei Guan
, Fengqi Yu
, Guangdong Bai^*

^*Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

2 Downloads (CityUHK Scholars)

Abstract

ChatGPT has quickly advanced from simple natural language processing to tackling more sophisticated and specialized tasks. Drawing inspiration from the success of mobile app ecosystems, OpenAI allows developers to create applications that interact with third-party services, known as GPTs. GPTs can choose to leverage third-party services to integrate with specialized APIs for domain-specific applications. However, the way these disclose privacy setting information limits accessibility and analysis, making it challenging to systematically evaluate the data privacy implications of third-party integrate to GPTs. In order to support academic research on the integration of third-party services in GPTs, we introduce GPTs-ThirdSpy, an automated framework designed to extract GPTs’ privacy settings. GPTs-ThirdSpy provides academic researchers with real-time, reliable metadata on third-party services used by GPTs, enabling in-depth analysis of their integration, compliance, and potential security risks. By systematically collecting and structuring this data, GPTs-ThirdSpy facilitates large-scale research on the transparency and regulatory challenges associated with the GPT app ecosystem. © 2025 Copyright held by the owner/author(s).

Original language	English
Title of host publication	FSE Companion 2025 - Companion Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering
Publisher	Association for Computing Machinery
Pages	1602-1606
Number of pages	5
ISBN (Print)	979-8-4007-1276-0
DOIs	https://doi.org/10.1145/3696630.3728711
Publication status	Published - 2025
Externally published	Yes
Event	33rd ACM International Conference on the Foundations of Software Engineering, FSE Companion 2025 - Trondheim, Norway Duration: 23 Jun 2025 → 27 Jun 2025

Publication series

Name	Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering
ISSN (Print)	1539-7521

Conference

Conference	33rd ACM International Conference on the Foundations of Software Engineering, FSE Companion 2025
Place	Norway
City	Trondheim
Period	23/06/25 → 27/06/25

Funding

We thank the reviewers for their insightful comments. This research has been partially supported by Australian Research Council Discovery Projects (DP230101196, DP240103068).

Research Keywords

Large Language Model
Privacy
Testing

Publisher's Copyright Statement

This full text is made available under CC-BY 4.0. https://creativecommons.org/licenses/by/4.0/

Access Output

10.1145/3696630.3728711

382960534.pdfFinal Published version, 4.78 MBLicence: CC BY

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Yan, C., Wan, L., Guan, B., Yu, F., & Bai, G. (2025). Tracking GPTs Third Party Service: Automation, Analysis, and Insights. In FSE Companion 2025 - Companion Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering (pp. 1602-1606). (Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering). Association for Computing Machinery. https://doi.org/10.1145/3696630.3728711

Yan, Chuan ; Wan, Liuhuo ; Guan, Bowei et al. / Tracking GPTs Third Party Service : Automation, Analysis, and Insights. FSE Companion 2025 - Companion Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. Association for Computing Machinery, 2025. pp. 1602-1606 (Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering).

@inproceedings{2280168eb0e44fa0a56baf87881f197a,

title = "Tracking GPTs Third Party Service: Automation, Analysis, and Insights",

abstract = "ChatGPT has quickly advanced from simple natural language processing to tackling more sophisticated and specialized tasks. Drawing inspiration from the success of mobile app ecosystems, OpenAI allows developers to create applications that interact with third-party services, known as GPTs. GPTs can choose to leverage third-party services to integrate with specialized APIs for domain-specific applications. However, the way these disclose privacy setting information limits accessibility and analysis, making it challenging to systematically evaluate the data privacy implications of third-party integrate to GPTs. In order to support academic research on the integration of third-party services in GPTs, we introduce GPTs-ThirdSpy, an automated framework designed to extract GPTs{\textquoteright} privacy settings. GPTs-ThirdSpy provides academic researchers with real-time, reliable metadata on third-party services used by GPTs, enabling in-depth analysis of their integration, compliance, and potential security risks. By systematically collecting and structuring this data, GPTs-ThirdSpy facilitates large-scale research on the transparency and regulatory challenges associated with the GPT app ecosystem. {\textcopyright} 2025 Copyright held by the owner/author(s).",

keywords = "Large Language Model, Privacy, Testing",

author = "Chuan Yan and Liuhuo Wan and Bowei Guan and Fengqi Yu and Guangdong Bai",

year = "2025",

doi = "10.1145/3696630.3728711",

language = "English",

isbn = "979-8-4007-1276-0",

series = "Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering",

publisher = "Association for Computing Machinery",

pages = "1602--1606",

booktitle = "FSE Companion 2025 - Companion Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering",

address = "United States",

note = "33rd ACM International Conference on the Foundations of Software Engineering, FSE Companion 2025 ; Conference date: 23-06-2025 Through 27-06-2025",

}

Yan, C, Wan, L, Guan, B, Yu, F & Bai, G 2025, Tracking GPTs Third Party Service: Automation, Analysis, and Insights. in FSE Companion 2025 - Companion Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering, Association for Computing Machinery, pp. 1602-1606, 33rd ACM International Conference on the Foundations of Software Engineering, FSE Companion 2025, Trondheim, Norway, 23/06/25. https://doi.org/10.1145/3696630.3728711

Tracking GPTs Third Party Service: Automation, Analysis, and Insights. / Yan, Chuan; Wan, Liuhuo; Guan, Bowei et al.
FSE Companion 2025 - Companion Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. Association for Computing Machinery, 2025. p. 1602-1606 (Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Tracking GPTs Third Party Service

T2 - 33rd ACM International Conference on the Foundations of Software Engineering, FSE Companion 2025

AU - Yan, Chuan

AU - Wan, Liuhuo

AU - Guan, Bowei

AU - Yu, Fengqi

AU - Bai, Guangdong

PY - 2025

Y1 - 2025

N2 - ChatGPT has quickly advanced from simple natural language processing to tackling more sophisticated and specialized tasks. Drawing inspiration from the success of mobile app ecosystems, OpenAI allows developers to create applications that interact with third-party services, known as GPTs. GPTs can choose to leverage third-party services to integrate with specialized APIs for domain-specific applications. However, the way these disclose privacy setting information limits accessibility and analysis, making it challenging to systematically evaluate the data privacy implications of third-party integrate to GPTs. In order to support academic research on the integration of third-party services in GPTs, we introduce GPTs-ThirdSpy, an automated framework designed to extract GPTs’ privacy settings. GPTs-ThirdSpy provides academic researchers with real-time, reliable metadata on third-party services used by GPTs, enabling in-depth analysis of their integration, compliance, and potential security risks. By systematically collecting and structuring this data, GPTs-ThirdSpy facilitates large-scale research on the transparency and regulatory challenges associated with the GPT app ecosystem. © 2025 Copyright held by the owner/author(s).

AB - ChatGPT has quickly advanced from simple natural language processing to tackling more sophisticated and specialized tasks. Drawing inspiration from the success of mobile app ecosystems, OpenAI allows developers to create applications that interact with third-party services, known as GPTs. GPTs can choose to leverage third-party services to integrate with specialized APIs for domain-specific applications. However, the way these disclose privacy setting information limits accessibility and analysis, making it challenging to systematically evaluate the data privacy implications of third-party integrate to GPTs. In order to support academic research on the integration of third-party services in GPTs, we introduce GPTs-ThirdSpy, an automated framework designed to extract GPTs’ privacy settings. GPTs-ThirdSpy provides academic researchers with real-time, reliable metadata on third-party services used by GPTs, enabling in-depth analysis of their integration, compliance, and potential security risks. By systematically collecting and structuring this data, GPTs-ThirdSpy facilitates large-scale research on the transparency and regulatory challenges associated with the GPT app ecosystem. © 2025 Copyright held by the owner/author(s).

KW - Large Language Model

KW - Privacy

KW - Testing

UR - https://www.scopus.com/pages/publications/105013968952

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-105013968952&origin=recordpage

U2 - 10.1145/3696630.3728711

DO - 10.1145/3696630.3728711

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 979-8-4007-1276-0

T3 - Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering

SP - 1602

EP - 1606

BT - FSE Companion 2025 - Companion Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering

PB - Association for Computing Machinery

Y2 - 23 June 2025 through 27 June 2025

ER -

Yan C, Wan L, Guan B, Yu F, Bai G. Tracking GPTs Third Party Service: Automation, Analysis, and Insights. In FSE Companion 2025 - Companion Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. Association for Computing Machinery. 2025. p. 1602-1606. (Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering). Epub 2025 Jul 28. doi: 10.1145/3696630.3728711

Tracking GPTs Third Party Service: Automation, Analysis, and Insights

Abstract

Publication series

Conference

Funding

Research Keywords

Publisher's Copyright Statement

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this