Skip to main navigation Skip to search Skip to main content

Timestamp Approximate Nearest Neighbor Search Over High-Dimensional Vector Data

Yuxiang Wang, Ziyuan He, Yongxin Tong, Zimu Zhou, Yiman Zhong

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review

Abstract

Unstructured data, such as images and texts, are increasingly represented as high-dimensional vectors for emerging AI applications like retrieval-augmented generation. A key operation in these applications is querying for vectors that are both semantically similar and temporally relevant. This operation can be formulated as Timestamp Approximate Nearest Neighbor Search (TANNS), where both the vectors and the query incorporate temporal attributes, aiming to retrieve the approximate nearest neighbors valid at the given timestamp. A naive solution is to create separate indexes for each timestamp, which enables accurate and fast searches but incurs high update latency and excessive storage demands. In this paper, we introduce the timestamp graph, a novel structure that supports rapid index updates while minimizing storage costs. Exploiting the temporal locality of changes in valid vectors, our timestamp graph effectively manages a unified index across all historical timestamps, thereby substantially reducing storage overhead. Moreover, we design the historic neighbor tree, which further compresses the space complexity to that of a single-timestamp index. Extensive evaluations on four standard datasets show that our method achieves over 99% accuracy while improving the query efficiency by 4.4× to 138.1× than existing solutions. © 2025 IEEE.
Original languageEnglish
Title of host publicationProceedings - 2025 IEEE 41st International Conference on Data Engineering, ICDE 2025
Place of PublicationLos Alamitos, Calif.
PublisherIEEE Computer Society
Pages3043-3055
Number of pages13
ISBN (Electronic)9798331536039
ISBN (Print)9798331536046
DOIs
Publication statusPublished - 2025
Event41st IEEE International Conference on Data Engineering (ICDE 2025) - Hong Kong SAR, China
Duration: 19 May 202523 May 2025
https://ieee-icde.org/2025
https://ieeexplore.ieee.org/xpl/conhome/11112833/proceeding

Publication series

NameProceedings - International Conference on Data Engineering
ISSN (Print)1084-4627
ISSN (Electronic)2375-0286

Conference

Conference41st IEEE International Conference on Data Engineering (ICDE 2025)
PlaceChina
CityHong Kong SAR
Period19/05/2523/05/25
Internet address

Funding

This work was partially supported by National Key Research and Development Program of China under Grant No. 2023YFF0725103, National Science Foundation of China (NSFC) (Grant Nos. 62425202, U21A20516, 62336003), the Beijing Natural Science Foundation (Z230001), the Fundamental Research Funds for the Central Universities No. JK2024-03, the Didi Collaborative Research Program and the State Key Laboratory of Complex & Critical Software Environment (SKLCCSE). Zimu Zhou's research is supported by Chow Sang Sang Group Research Fund No. 9229139.

Research Keywords

  • approximate nearest neighbor search
  • high-dimensional data

Fingerprint

Dive into the research topics of 'Timestamp Approximate Nearest Neighbor Search Over High-Dimensional Vector Data'. Together they form a unique fingerprint.

Cite this