Cellulose and xylan are two major components of lignocellulosic biomass, which represents a potentially important energy source, as it is abundant and can be converted to methane by microbial action. However, it is recalcitrant to hydrolysis, and the establishment of a complete anaerobic digestion system requires a specific repertoire of microbial functions. In this study, we maintained 2-year enrichment cultures of anaerobic digestion sludge amended with cellulose or xylan to investigate whether a cellulose- or xylan-digesting microbial system could be assembled from sludge previously used to treat neither of them. While efficient methane-producing communities developed under mesophilic (35°C) incubation, they did not under thermophilic (55°C) conditions. Illumina amplicon sequencing results of the archaeal and bacterial 16S rRNA genes revealed that the mature cultures were much lower in richness than the inocula and were dominated by single archaeal (genus Methanobacterium) and bacterial (order Clostridiales) groups, although at finer taxonomic levels the bacteria were differentiated by substrates. Methanogenesis was primarily via the hydrogenotrophic pathway under all conditions, although the identity and growth requirements of syntrophic acetate-oxidizing bacteria were unclear. Incubation conditions (substrate and temperature) had a much greater effect than inoculum source in shaping the mature microbial community, although analysis based on unweighted UniFrac distance found that the inoculum still determined the pool from which microbes could be enriched. Overall, this study confirmed that anaerobic digestion sludge treating nonlignocellulosic material is a potential source of microbial cellulose- and xylan-digesting functions given appropriate enrichment conditions.