Under some transient disturbance, the grid-forming voltage source converter may lose its synchronization with the grid, inducing low-frequency oscillation in the instantaneous power, current and angle. The physical origin of such oscillations is found to be a homoclinic bifurcation in this paper. Before the system runs into a homoclinic bifurcation, a stable equilibrium point and a stable periodic orbit co-exist. When a large disturbance is applied, the system exhibits a periodic orbit, which manifests itself as low-frequency oscillation. Moreover, no periodic orbit exists after the homoclinic bifurcation. On the basis of bifurcation analysis, boundaries of stable operation are derived in the parameter space which serve as practical design guidelines to avoid oscillation.