Tianshu: Towards Accurate Measuring, Modeling and Simulation of Deep Neural Networks

To help train DNN models more efficiently, researchers have developed a series of new computation devices and parallel training strategies. Choosing which device and strategy to use among those vast candidates is challenging for developers. Simulation is an effective solution to this question since it does not need to deploy models with different settings practically. However, most existing works still require users to measure models on the target hardware first, which cannot help users choose the proper hardware in advance. This paper presents Tianshu, which adopts the ahead-of-time idea to solve this problem. Tianshu first designs an accurate measuring tool to measure the operation's time-use and store the result in a database, which can be shared among the community. For uncovered cases, Tianshu designs an automatic mechanism to train neural network models that can estimate their execution time. Therefore, Tianshu can support simulation without a target hardware. Finally, Tianshu leverages a discrete event simulator to simulate the DNN model's execution. Evaluation on Azure clusters with 8x V100 GPUs shows that Tianshu achieves 93% average accuracy for six well-known DNN models. Tianshu's operation estimator also achieves 96% average accuracy on 9 typical operations and outperforms existing works. © 2024 The Authors.