Multi-layer Compression for Lean Flash Storage

With reduced technology size and an increased number of bits per cell, NAND flash memories have shown significant improvement in storage density. Since 2012, 3D NAND flash memories have been introduced to improve storage density, and they have been deployed in cloud systems, web servers, and mobile devices for a high-performance, high-capacity storage solution. There are several issues for the state-of-the-art high capacity NAND flash memories, including degraded lifetime and access latency. With the increased storage density, the access latencies of read, write, and erase operations are also significantly increased. For example, the write latency is approaching a few milliseconds, and the erase latency is approaching 10 milli-seconds. Regarding lifetime, the most advanced Quadruple Level Cell (QLC) NAND flash memories can only endure less than 500 erase operations [Lee 18] [Jin 16]. Compression is a promising technique to improve not only write performance but also lifespan, as it reduces the total amount of data written to flash memories. Although there have been techniques of data compression on NAND flash storage, few of them consider compressions with joint considerations of data access patterns and the characteristics of different system layers. This project aims to fill this gap and develop a multi-layer compression framework for lean NAND flash storage. Data storage involves multiple layers of system software, including the file system layer, block layer, and controller layer. This project proposes to deploy compression schemes on multiple layers to improve access performance and lifetime for lean flash storage, while minimizing overheads. The following tasks will be conducted: First, a compression aware file system for flash memory will be developed. Data reduction without impacting system performance will be carefully studied. Second, flash storage characteristic such as garbage collection and error correction code will be exploited for deploying compression in the controllers of flash storage. Third, a collaborative cross-layer compression scheme will be analyzed for optimized performance and lifetime improvement, as certain compressions will be better done in certain layers. Finally, the relationship between compression and deduplication will be explored on flash devices for a top-to-bottom lean flash storage. The proposed schemes will be verified on an industry-provided testing platform and evaluated with widely used simulators and popular workloads. The success of this project will enable lean flash storage systems. As data and storage is the key for business innovation of tomorrow, this project will provide a tremendous drive for the innovation industry of Hong Kong.