Power Governing for Emerging Mobile Artificial Intelligence Devices

Power governing is critical for emerging mobile artificial intelligence (mAI) devices, e.g., Nvidia Jetson series and more. In addition to prevent energy wastes, the significance of power governing for mAI devices is more about reducing heat generation due to theircompact architectural design. It ensures that the device can fully exhibit its inherent computing ability, as its hardware spends huge investments to improve every year, but performance drops rapidly when device overheats. Power governing is also important tothe hardware lifecycle, essential to device and application reliability.Dynamic voltage and frequency scaling (DVFS) is a primary power-governing technology, which adjusts processor’s operating frequency to balance its performance and energy consumption. Existing mAI devices mainly adopt the DVFS solution fromtraditional mobiles. However, due to different workload and hardware features, mAI devices require new efficient DVFS designs. First, mAI devices typically execute heavier tasks concurrently (e.g., AI, graphics and networking tasks in autonomous machines)with highly dynamic workloads. However, traditional mobiles usually have one dominant front-end application with few active background-services, and their DVFS naturally does not need to accommodate massive concurrent heavy tasks. Other platforms likedata centers have concurrent tasks, but their workload is huge, characterized by the task incoming distribution varying in the granularity from minutes to hours. Their DVFS is also unsuitable for mAI devices. Second, in terms of hardware, even for emerging AItasks, mAI GPU and other co-processors can only facilitate CPU processing, as many operations are not supported by them. This further requires effective joint frequency adjustments of multi-processors, but they are mainly adjusted independently on mobiles.In this project, our key insight is that hardware data, widely used in mobile DVFS designs, has great potential to enable capable workload awareness and task concurrency adaptability for power governing, but underexplored. We find that workloadcharacteristics can be described in a hyperspace composed of multiple dimensions derived from these data to form a workload contextual indicator that can profile task dynamics and concurrency. On this basis, we propose a new metric to capture thissophisticated relationship and further design a governor of a novel structure, making joint frequency-adjustments efficient.Overall, the idea of considering workload and hardware features to advance DVFS for emerging mAI devices is of essential novelty. It provides a new power-governing solution, which we find still effective with less concurrent tasks. Therefore, our designhas wider utility that is also backward-compatible with traditional mobile devices