Increasing mental disorders have emerged as an urgent public health concern such as autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD). Related mental disorders may share high overlap in clinical symptoms. Therefore, their diagnosis can be challenging to merely rely on the observation of cognitive phenotypes and behavioral manifestations. Unfortunately, there is no additional support of biochemical markers, laboratory tests, or neuroimaging analysis, which can be used as a diagnostic gold standard currently. Over the past decades, resting-state functional magnetic resonance imaging (rs-fMRI) has been considered as one of the most promising modality to capture the intrinsic neural activation patterns between regions in the brain. In this work, we focus on ASD and ADHD due to their high prevalence and relevance with the aim to exploit the multi-task learning (MTL) paradigm for their diagnosis. To the best of our knowledge, this is the first time to make use of the disease-specific heterogeneities for the MTL classification of ASD and ADHD via rs-fMRI signal. We propose a novel graph-based feature selection method to filter out irrelevant functional connectivity features. Then an efficient structure of multi-gate mixture-of-experts (MMoE) is applied to the MTL classification framework. Finally, the experiment results demonstrate that the proposed model can achieve a reliable classification performance in a short term, yielding the mean accuracies of 0.687±0.005 and 0.650±0.014 in ASD and ADHD datasets, respectively. The graph-based feature selection method and MMoE model are demonstrated to make great contribution to performance improvement.