Modern data centers are experiencing a continuous surge in network communication and computation demands, which places a significant burden on server CPUs due to the stagnation of CPU performance, especially for typical network services like network stacks and network functions. In light of this, SmartNICs have emerged as a promising solution. They offer demonstrated advantages in performance, host CPU saving, and programmability, making them an appealing substrate for serving data center services. However, there remains a gap in effectively harnessing the potential of SmartNICs to achieve high performance and resource efficiency in network communication and computation, necessitating further exploration.

In this dissertation, we identify the crucial requirements involved in offloading network communication and computation onto SmartNICs, and present two novel frameworks that facilitate high-performance and resource-efficient offloading for network stacks and network functions.

First, we observe the communication overhead in containerized clouds is predominantly attributed to the TCP stack within the kernel and the memory copy between the kernel and user spaces, especially for intra-host cases. To mitigate these, we present PipeDevice, a hardware-software co-design framework for low-overhead intra-host container communication. PipeDevice offloads data forwarding entirely onto SmartNICs, which copies application data in hugepages on the host, thereby eliminating CPU overhead from memory copy and TCP processing. PipeDevice also preserves memory isolation and scales well to connections, making it deployable in public clouds.

Second, we identify that the offloading of computation is always confined to serving the hosts within the local scope, and SmartNICs are typically owned by individual application teams. These result in significant inflexibility and inefficiency for SmartNIC management and multiplexing. To address these challenges, We present Meili, a new system that features SmartNIC as a service. Meili organizes SmartNIC resources as a pool and offloads workloads on heterogeneous SmartNIC clusters. It provides heterogeneity-transparent modular programming abstractions that enable users to build common SmartNIC workloads flexibly and transparently, and relies on fine-grained SmartNIC multiplexing to achieve high resource efficiency.

We build prototypes for PipeDevice and Meili, and evaluate them with testbed experiments using typical FPGA and SoC SmartNICs. Specifically, PipeDevice efficiently saves precious host CPU cycles while preserving high performance for intra-host container communication, and Meili significantly improves the SmartNIC resource efficiency and provides high performance for network functions.