Energy harvesting is an essential technology for enabling low-power, maintenance-free electronic devices, and thus has attracted a great deal of attention in recent years. A variety of designs and approaches have been proposed to harvest ambient vibration energy, but crucial questions remain regarding figures of merit characterizing the performance of energy harvesters. Of primary importance is the energy conversion efficiency. There are large discrepancies in the definition and tested values of efficiency in the literature. This study is intended to answer the fundamental question for energy harvesters: how to define and calculate the energy conversion efficiency. We first review studies on efficiency and analyze the energy flow in an energy harvesting system. Based on the analysis, we derive an efficiency expression for linear cantilever energy harvesters. The developed efficiency expression transparently and quantitatively reveals the relationship between efficiency and key parameters. Experiments are performed to validate the efficiency expression. Furthermore nonlinear energy harvesters are tested in both on-resonance and off-resonance conditions. Both experimental and theoretical studies manifest that the energy conversion efficiency tends to decrease as the excitation frequency rises and its value is related to the phase difference between excitations and responses. Around resonance states where the phase difference of both linear and nonlinear energy harvesters is about 90 degrees, the efficiency calculation is much simplified.