边收缩池化的网格变分自编码器

Objective 3D shape datasets have been tremendous facilitated nowadays. Data-driven 3D shape analysis has been an active research topic in computer vision and graphics. Apart from regular works, current data-driven works attempted to generalize deep neural networks from images to 3D shapes, including triangular meshes, point clouds and voxel data. Deep neural networks for triangular meshes have been concentrated. 3D meshes have complicated and irregular inter-connection. Most current works tend to keep mesh connectivity unchanged each layer, thus, losing the capability of increased receptive fields when pooling operations are applied. The variational auto-encoder (VAE) has been widely used in various kinds of generation tasks, including generation, interpolation and exploration on triangular meshes. Based on a fully-connected network, the initial MeshVAE requires mega parameters and its generalization capability is often weak. Although the fully connected layers allow changes of mesh connectivity across layers, due to irregular changes, such approaches cannot be directly generalized to convolutional layers. Some works adopt convolutional layers in the VAE structure. However, such convolution operations cannot change the connectivity of the mesh. Sampling operation is also evolved in convolutional neural networks(CNNs) on meshes, but the mesh sampling strategy does not aggregate the whole local neighborhood information when reducing the quantities of vertices. Hence, it is necessary to design a pooling operation for meshes similar to the pooling for images to reduce the amount of network parameters in order to deal with denser models and enhance the generalization ability of the network. Moreover, the defined pooling can support further convolutions and conduct recovery via a corresponding de-pooling operation. Method A novel mesh pooling operation is illustrated based on edge contraction. The VAE architecture in context of the newly defined pooling operation is built up as well. Mesh simplification is applied to organize a mesh hierarchy with different levels of details, and achieves effective pooling by keeping track of the mapping between coarser and finer meshes. To avoid generating highly irregular triangles in mesh simplification, a modified mesh simplification approach is demonstrated based on the classical edge contraction algorithm. The edge length is an essential indicator for the edge contraction process. So, as one of the criteria, the edge length is incorporated to order pairs of points. The new edge length is added to the original quadric error formulation straightforward. The feature of a new vertex is defined as the average feature of the contracted vertices for average pooling, and alternative pooling operations can be similarly ruled. In the decoding process, the features of the vertices on the simplified mesh are equally assigned to the corresponding contracted vertices on the dense mesh for the inverse operation, de-pooling. The input to the illustrated network is a vertex-based deformation feature representation, which is different from 3D coordinates, encodes deformations defined on vertices in terms of deformation gradients analysis. The demonstrated framework uses a cluster of 3D shapes with the same connectivity to train the network. Such meshes can be easily obtained via consistent re-meshing. The network follows a VAE architecture where pooling operations and graph convolutions are applied. It has qualified generalization capabilities and handles much higher resolution meshes in various applications, such as shape generation and interpolation. Result The framework is tested on four datasets, shape completion and animation of people (SCAPE), Swing, Fat and Hand. The capability of the network is tested to generate unseen shapes, and calculate the average root mean squared (RMS) errors. The network with the proposed pooling and without pooling has been initially compared. The RMS error is lower by an average of 6.92% with pooling, which shows the benefits of our pooling and de-pooling operations. The comparisons between the proposed pooling and other pooling or sampling methods are illustrated. The RMS error of the proposed pooling for unseen data is lower on average by 9.34% compared to initial simplification-based pooling, 9.07% compared to uniform remeshing method, 8.06% compared to graph pooling, and 9.64% compared to mesh sampling, which illustrates this modified simplification algorithm is more effective in terms of pooling and the proposed pooling is superior on multiple datasets, demonstrating its generalization capability. The proposed framework is also compared with related mesh-based auto-encoder architectures. Thanks to spectral graph convolutions and the proposed pooling, the method reduces the reconstruction errors of unseen data consistently, showing superior generalizability. For instance, compared with one work which uses the same per-vertex features, the designed network achieves 29% and 32% lower average RMS reconstruction errors on the SCAPE and Face datasets. MeshCNN is compared and the proposed network achieves better results. Moreover, the capability of our framework is demonstrated in shape generation, shape interpolation and shape embedding. Conclusion A newly defined pooling operation, based on a modified mesh simplification algorithm, is integrated into a mesh variational auto-encoder architecture. Our generative model has its good generalization capability. Compared to the original MeshVAE, our method can generate high quality deformable models with richer details.