Developing low-cost and high efficient electrocatalysts for both oxygen and hydrogen evolution reaction in an alkaline electrolyte toward overall water splitting is still a significant challenge. Here, a novel hierarchically heterostructured catalyst composed of ultrasmall Mo2C and metallic Co nanoparticles confined within a carbon layer is produced by a facile phase separation strategy. During thermal reduction of CoMoO4 nanosheets in CO ambient, in-situ generated nanoscale Co and ultrafine Mo2C conformally encapsulated in a conductive carbon layer. In addition, some carbon nanotubes catalyzed by Co nanoparticles vertically grew on its surface, creating 3D interconnected electron channels. More importantly, the integrated C@Mo2C/Co nanosheets assembled into the hierarchical architecture, providing abundant active surface and retaining the structural integrity. Benefiting from such unique structure, the constructed hierarchical heterostructure shows low overpotentials of 280 mV and 145 mV to reach a current density of 10 mA cm−2 for OER and HER in an alkaline electrolyte. Furthermore, the symmetrical electrolyzer assembled with catalyst exhibits a small cell voltage of 1.67 V at 10 mA cm−2 in addition to outstanding durability, demonstrating the great potential as a high efficient bifunctional electrocatalyst for overall water splitting.