This paper presents a theoretical study and finite element simulation for the buckling of a thin film on the compliant substrate. First, we develop a continuum mechanics approach for large deformation buckling analysis based on minimizing the total energy of the film/substrate structures, and considering the precise curvature of the buckled film and the Poisson's ratio of the substrate. The predicting results using this proposed theory agree quite well with previous experimental results. Then, we make a modification for the model to simplify the expressions for the wavelength and amplitude of the buckled geometry. Furthermore, considering a thin Si film on shape memory polymer (SMP) substrate, we investigate the buckling behavior of the thin film through theoretical analysis and finite element method. Through the investigation, it is found that the evolution rate of the buckling geometry of thin film depends on the temperature of the SMP substrate, and the buckling geometry changes faster at a higher temperature. Finally, a programmed method to control the buckling of thin Si film on the SMP substrate is proposed and is realized with finite element simulation in ABAQUS.