The memristor was theoretically predicted in 1971, but it garnered nascent research interest due to the recent much-acclaimed discovery of nano-memories by engineers at the Hewlett-Packard (HP) laboratory. Since 2008, advanced nanofabrication technology has shown its distinctive advantages and great potentials in applications of nonvolatile memories, analog processing and neuromorphic computing. As a new category of hardware, the neuromorphic computing system (NCS) has also attracted considerable attention recently. It provides the emulation or simulation of the brain models proposed by neuroscientists using available electronic devices. However, most developed NCSs suffer from limited scale and computational capability, due to the conventional implementation approaches and CMOS-based integration technology. The nanoscale memristor with nonvolatility, programmability and analog processing capability provides a promising opportunity to revolutionize conventional neuromorphic computing and to boost very large scale integration (VLSI) circuit implementation of neuromorphic systems. This thesis studies the memristor and its use in implementing neuromorphic computing systems. Two types of compact memristive neural network models are investigated and developed based on the cellular neural network (CNN). Memristors are exploited to realize the compact synaptic circuits in bridge-like structure or crossbar array, which can efficiently execute the complex weighting (multiplication) processing. Meanwhile, the weighted summation is carried out by a proper design of cell (or, neuron) circuits. More specifically, the first type is developed by incorporating the memristor into conventional CNNs. Firstly, a single-layer memristive CNN (M-CNN) with programmable templates is proposed, by using the memristor to construct microscopic synapses and simple cell circuits. It can perform most classical processing tasks of CNNs. Then, a flexible single-layer memristive CNN (Mt-CNN) with time-variant templates is proposed by further leveraging the high programmability of memristors. It facilitates the hardware realization of real-time template updating needed in dealing with some complex problems. Moreover, a multilayer memristive CNN (Mm-CNN) with programmable templates is constructed. With several state variables associated with multiple dynamics rules in each cell, it is capable of solving complicated problems of multi-processing. The second type of the memristive CNNs is developed by further exploiting another emerging nanometer device, the resonant tunneling diode (RTD) to construct more compact cell circuits. The combination of RTD cells and programmable memristor synaptic circuits makes the hybrid RTD and memristor-based CNN (RTDM- CNN) permit more implementable structure and excellent performance. Both single-layer and multilayer RTD-M-CNN models are presented for different processing purposes. Dynamics and stability of the proposed memristive CNNs are also analyzed for physical design guidelines. Their effectiveness and advantages are validated through illustrative examples and comparisons with conventional CNNs by a number of experimental simulations for image processing. It has been demonstrated that the proposed memristive CNNs are superior to conventional CNNs in terms of compactness, nonvolatility, versatility, flexibility, and are more suitable for VLSI implementation, which is expected to greatly improve the hardware development and practical applications of CNNs.