Invariant Random Forest: Tree-Based  Model Solution for OOD Generalization

Yufan Liao; Qi Wu; Xing Yan

doi:10.1609/aaai.v38i12.29283

Invariant Random Forest: Tree-Based Model Solution for OOD Generalization

Yufan Liao
, Qi Wu
, Xing Yan^*

^*Corresponding author for this work

Department of Data Science

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

Abstract

Out-Of-Distribution (OOD) generalization is an essential topic in machine learning. However, recent research is only focusing on the corresponding methods for neural networks. This paper introduces a novel and effective solution for OOD generalization of decision tree models, named Invariant Decision Tree (IDT). IDT enforces a penalty term with regard to the unstable/varying behavior of a split across different environments during the growth of the tree. Its ensemble version, the Invariant Random Forest (IRF), is constructed. Our proposed method is motivated by a theoretical result under mild conditions, and validated by numerical tests with both synthetic and real datasets. The superior performance compared to non-OOD tree models implies that considering OOD generalization for tree models is absolutely necessary and should be given more attention. Copyright © 2024, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Original language	English
Title of host publication	Proceedings of the 38th AAAI Conference on Artificial Intelligence
Publisher	AAAI Press
Pages	13772-13781
ISBN (Electronic)	978-1-57735-887-9
DOIs	https://doi.org/10.1609/aaai.v38i12.29283
Publication status	Published - 2024
Event	38th Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence (AAAI-24) - Vancouver Convention Center, Vancouver, Canada Duration: 20 Feb 2024 → 27 Feb 2024 https://aaai.org/aaai-conference/ https://ojs.aaai.org/index.php/AAAI/issue/archive

Publication series

Name	Proceedings of the AAAI Conference on Artificial Intelligence
Number	12
Volume	38
ISSN (Print)	2159-5399
ISSN (Electronic)	2374-3468

Conference

Conference	38th Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence (AAAI-24)
Place	Canada
City	Vancouver
Period	20/02/24 → 27/02/24
Internet address	https://aaai.org/aaai-conference/ https://ojs.aaai.org/index.php/AAAI/issue/archive

Funding

Qi Wu acknowledges the support from The CityU-JD Digits Joint Laboratory in Financial Technology and Engineering; The Hong Kong Research Grants Council [General Research Fund 11219420/9043008 and 11200219/9042900]; and The HK Institute of Data Science. The work described in this paper was partially supported by the InnoHK initiative, the Government of the HKSAR, and the Laboratory for AI-Powered Financial Technologies. Xing Yan acknowledges that this work was supported in part by the Young Scientists Fund of National Natural Science Foundation of China under Grant 62206300, and in part by Public Computing Cloud of Renmin University of China.

RGC Funding Information

RGC-funded

Access Output

10.1609/aaai.v38i12.29283

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{12d19ed7121f47cda5615bc70f10b7ce,

title = "Invariant Random Forest: Tree-Based Model Solution for OOD Generalization",

abstract = "Out-Of-Distribution (OOD) generalization is an essential topic in machine learning. However, recent research is only focusing on the corresponding methods for neural networks. This paper introduces a novel and effective solution for OOD generalization of decision tree models, named Invariant Decision Tree (IDT). IDT enforces a penalty term with regard to the unstable/varying behavior of a split across different environments during the growth of the tree. Its ensemble version, the Invariant Random Forest (IRF), is constructed. Our proposed method is motivated by a theoretical result under mild conditions, and validated by numerical tests with both synthetic and real datasets. The superior performance compared to non-OOD tree models implies that considering OOD generalization for tree models is absolutely necessary and should be given more attention. Copyright {\textcopyright} 2024, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.",

author = "Yufan Liao and Qi Wu and Xing Yan",

year = "2024",

doi = "10.1609/aaai.v38i12.29283",

language = "English",

series = "Proceedings of the AAAI Conference on Artificial Intelligence",

publisher = "AAAI Press",

number = "12",

pages = "13772--13781",

booktitle = "Proceedings of the 38th AAAI Conference on Artificial Intelligence",

note = "38th Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence (AAAI-24) ; Conference date: 20-02-2024 Through 27-02-2024",

url = "https://aaai.org/aaai-conference/, https://ojs.aaai.org/index.php/AAAI/issue/archive",

}

Liao, Y , Wu, Q & Yan, X 2024, Invariant Random Forest: Tree-Based Model Solution for OOD Generalization. in Proceedings of the 38th AAAI Conference on Artificial Intelligence. Proceedings of the AAAI Conference on Artificial Intelligence, no. 12, vol. 38, AAAI Press, pp. 13772-13781, 38th Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence (AAAI-24), Vancouver, British Columbia, Canada, 20/02/24. https://doi.org/10.1609/aaai.v38i12.29283

Invariant Random Forest: Tree-Based Model Solution for OOD Generalization. / Liao, Yufan ; Wu, Qi; Yan, Xing.
Proceedings of the 38th AAAI Conference on Artificial Intelligence. AAAI Press, 2024. p. 13772-13781 (Proceedings of the AAAI Conference on Artificial Intelligence; Vol. 38, No. 12).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Invariant Random Forest

T2 - 38th Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence (AAAI-24)

AU - Liao, Yufan

AU - Wu, Qi

AU - Yan, Xing

PY - 2024

Y1 - 2024

N2 - Out-Of-Distribution (OOD) generalization is an essential topic in machine learning. However, recent research is only focusing on the corresponding methods for neural networks. This paper introduces a novel and effective solution for OOD generalization of decision tree models, named Invariant Decision Tree (IDT). IDT enforces a penalty term with regard to the unstable/varying behavior of a split across different environments during the growth of the tree. Its ensemble version, the Invariant Random Forest (IRF), is constructed. Our proposed method is motivated by a theoretical result under mild conditions, and validated by numerical tests with both synthetic and real datasets. The superior performance compared to non-OOD tree models implies that considering OOD generalization for tree models is absolutely necessary and should be given more attention. Copyright © 2024, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

AB - Out-Of-Distribution (OOD) generalization is an essential topic in machine learning. However, recent research is only focusing on the corresponding methods for neural networks. This paper introduces a novel and effective solution for OOD generalization of decision tree models, named Invariant Decision Tree (IDT). IDT enforces a penalty term with regard to the unstable/varying behavior of a split across different environments during the growth of the tree. Its ensemble version, the Invariant Random Forest (IRF), is constructed. Our proposed method is motivated by a theoretical result under mild conditions, and validated by numerical tests with both synthetic and real datasets. The superior performance compared to non-OOD tree models implies that considering OOD generalization for tree models is absolutely necessary and should be given more attention. Copyright © 2024, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

UR - https://www.scopus.com/pages/publications/85189544733

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85189544733&origin=recordpage

U2 - 10.1609/aaai.v38i12.29283

DO - 10.1609/aaai.v38i12.29283

M3 - RGC 32 - Refereed conference paper (with host publication)

T3 - Proceedings of the AAAI Conference on Artificial Intelligence

SP - 13772

EP - 13781

BT - Proceedings of the 38th AAAI Conference on Artificial Intelligence

PB - AAAI Press

Y2 - 20 February 2024 through 27 February 2024

ER -

Invariant Random Forest: Tree-Based Model Solution for OOD Generalization

Abstract

Publication series

Conference

Funding

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

GRF: Generative Models of Multivariate Dependence for Asset Returns

GRF: Risk-Potential Framework for Dynamic Portfolio Selection

Cite this

Invariant Random Forest: Tree-Based Model Solution for OOD Generalization

Abstract

Publication series

Conference

Funding

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

GRF: Generative Models of Multivariate Dependence for Asset Returns

GRF: Risk-Potential Framework for Dynamic Portfolio Selection

Cite this