While active shape model (ASM) has been increasingly adopted in the medical domain, there are issues that need to be addressed for it to be applicable in practice. Among them, the small sample size problem and how to represent the variation of the clutter of surroundings are two of the challenges. In this paper, to overcome these problems, we propose a novel multi-resolution statistical deformable model and the associated techniques for the reconstruction of soft-tissue organs such as livers. To address the small sample size problem, we define a multi-resolution integrated model for soft-tissue organs called MISTO that is able to capture the most significant deformations from a small training set as well as to generate representative variation modes of the organ shapes. To deal with the complex surroundings of the model surface or landmark points in the underlying medical images during model deformation, we propose to apply multi-resolution appearance models which allows the surrounding visual context of the model surface points to be learnt and characterized automatically from the training samples. By combining the powerful shape models and the resulting context constraints, the object segmentation and reconstruction process can be carried out very robustly. Furthermore, to avoid the local minima during model optimization, we develop an adaptive deformation strategy such that the more stable parts of the surface are moved prior to the rest of the model surface. The experimental and validation results verify that our proposed approaches can be successfully and robustly applied to the reconstruction of the soft-tissue organs such as the human liver. The major contributions of our approaches are that we extend the traditional ASM to address open problems associated with reconstructing significantly deformable three-dimensional anatomies in cluttered surrounding, and we propose effective ways to formulate the perceptual knowledge of the anatomies and make use of it in the process of model construction and deformation for medical reconstruction. © 2008 Elsevier Ltd. All rights reserved.