The genus Aeromonas is a common gastrointestinal pathogen associated with human and animal infections. Due to the high level of cross-species similarity, their evolutionary dynamics and genetic diversity are still fragmented. Hereby, we investigated the pan-genomes of 29 Aeromonas species, as well as Aeromonas species in microbial communities, to clarify their evolutionary dynamics and genetic diversity, with special focus on virulence factors and horizontal gene transfer events. Our study revealed an open pan-genome of Aeromonas containing 10,144 gene families. These Aeromonas species exhibited different functional constraints, with the single-copy core genes and most accessory genes experiencing purifying selection. The significant congruence between core genome and pan-genome trees revealed that core genes mainly affected evolutionary divergences of Aeromonas species. Gene gains and losses revealed a high level of genome plasticity, exhibited by hundreds of gene expansions and contractions, horizontally transferred genes, and mobile genetic elements. The selective constraints shaped virulence gene pools of these Aeromonas strains, where genes encoding hemolysin were ubiquitous. Of these strains, Aeromonas aquatica MX16A seemed to be more resistant, as it harbored most resistance genes. Finally, the virulence factors of Aeromonas in microbial communities were quite dynamic in response to environment changes. For example, the virulence diversity of Aeromonas in microbial communities could reach levels that match some of the most virulent Aeromonas species (such as A. hydrophila) in penetrated-air and modified-air packaging. Our work shed some light onto genetic diversity, evolutionary history, and functional features of Aeromonas, which could facilitate the detection and prevention of infections. Importance: Aeromonas has long been known as a gastrointestinal pathogen, yet it has many species whose evolutionary dynamics and genetic diversity had been unclear until now. We have conducted pan-genome analysis for 29 Aeromonas species and revealed a high level of genome plasticity exhibited by hundreds of gene expansions and contractions, horizontally transferred genes, and mobile genetic elements. These species also contained many virulence factors both identified from single isolated species and microbial community. This pan-genome study could elevate the level for detection and prevention of Aeromonas infections.