Joint Optimization of Data Persistence and Access Performance on Non-Volatile Memory

Abstract

Byte-addressable, low-latency non-volatile memory (NVM) bridges the gap between memory and persistent storage, providing new opportunities and posing new challenges for the design of modern storage hierarchies. NVM challenges traditional software abstractions by enabling memory-level performance and direct, fine-grained access to persistent data, especially in consistency mechanisms. Ensuring strong persistence often comes with significant time and space overhead, making it difficult to get the most out of NVM. This thesis investigates how to jointly optimize data persistence and access performance in NVM-based systems by rethinking file systems, user-space buffers, and memory-mapped interfaces.

Specifically, this thesis includes three key contributions: (1) a novel crash consistency mechanism for memory-mapped I/O; (2) a flexible, file-based persistence buffer abstraction optimized for high-persistence workloads; and (3) an efficient DAX-based persistence framework that maximizes the performance of NVM while ensuring persistence semantics.

For the first topic, we propose Multi-Grained Shadow Paging (MGSP), a strong crash consistency mechanism designed for memory-mapped I/O in NVM. MGSP introduces Shadow Logging (SL), which manages persistence efficiently by using redo and undo logs as shadow logs. MGSP further supports general file access by combining multi-granularity updates and locks while significantly reducing runtime overhead, making it well-suited for write-intensive and concurrent workloads.

For the second topic, we propose FlexPB, a persistence user buffer framework that addresses existing solutions' poor flexibility and inefficiency. FlexPB introduces a file-based design and fsync-based persistence semantics to provide an efficient and flexible interface. It also dynamically merges DRAM into the persistence path based on NVM characteristics and workload patterns to select the best solution to optimize performance. FlexPB demonstrates significant adaptability and performance improvements through a case study of SQLite benchmarks in high-persistence environments.

The third topic introduces DAX-Cache, a user-space file system that decouples caching from immediate persistence. At its core is In-Place Logging (IPL), a novel mechanism that enables space-efficient consistency guarantees with minimal runtime overhead. By combining IPL with compact metadata logging, DAX-Cache achieves fsync-based atomic persistence, fully leveraging the performance benefits of NVM while maintaining crash consistency. Evaluations on various workloads demonstrate that DAX-Cache delivers near-optimal performance for DAX-based accesses while significantly reducing time and space costs.

In conclusion, this thesis comprehensively explores system-level techniques for co-optimizing persistence and performance on NVM. The proposed solution enables practical, high-throughput, highly resilient systems based on NVM by revisiting long-standing assumptions in consistency mechanisms, buffer management, and file system design. These findings provide actionable insights and design principles for future software stacks aiming to unleash the full power of persistent memory technologies.

Date of Award	5 Sept 2025
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Nan GUAN (Supervisor), Tei-Wei KUO (External Co-Supervisor) & Jason XUE (External Co-Supervisor)