The security of multimedia data becomes more and more important due to the rapid development of the modern computer, networking and information technologies. Notably, the traditional text encryption schemes fail to protect the multimedia data efficiently because of the special properties of multimedia data. To overcome this difficulty, researchers tried to develop special encryption schemes for multimedia data adopting some related nonlinear theories. However, some new schemes have been found to be insecure from the viewpoint of cryptography, and some general recommendations have been drawn to facilitate the design of more secure multimedia encryption schemes. This thesis is concerned with the security analysis of some multimedia encryption schemes. The security of the schemes against some common attack methods, such as bruteforce attack, known/chosen-plaintext attack and differential attack, is investigated in detail with theoretical analyses and experimental verifications. In addition, some special design defects of the schemes are revealed and discussed. The main contributions of the thesis are summarized as follows: 1. The security problems of a symmetric key block cipher using multiple one-dimensional chaotic maps are cryptanalyzed in detail. Some new findings are: 1) a number of weak keys exists; 2) some important intermediate data of the cipher are not sufficiently random; 3) the whole secret key can be broken by a known-plaintext attack with only 120 consecutive known plain-bytes in one known plaintext; 4) an improved version of the chaotic cipher still suffers from all the same security defects. 2. Recently two encryption schemes were proposed by combining circular bit shift and XOR operations, under the control of a pseudorandom bit sequence (PRBS) generated from a chaotic system. These two schemes are cryptanalyzed together, leading to the following findings: 1) there exist some security defects in both schemes; 2) the underlying chaotic PRBS can be reconstructed as an equivalent key by using only two chosen plaintexts; 3) most elements in the underlying chaotic PRBS can be obtained by a differential known-plaintext attack using only two known plaintexts. Experimental results are given to demonstrate the feasibility of the proposed attack. 3. The security of a recently-proposed chaos-based image encryption scheme is studied in detail. The following problems are found: 1) there exist a number of invalid keys and weak keys, and some keys are partially equivalent for encryption/decryption; 2) given one chosen plain-image, a subkey K10 can be guessed with a smaller computational complexity than that of the simple brute-force attack; 3) with at most 128 chosen plain-images, a chosen-plaintext attack can possibly break the following part of the secret key: {Ki mod 128}10 i=4, which works very well when K10 is not too large; 4) when K10 is relatively small, a known-plaintext attack can be carried out with only one known plain-image to recover some visual information of any other plain-images encrypted by the same key. 4. The security of an image encryption scheme based on the Hill cipher is studied and the following problems are found: 1) there is a simple necessary and sufficient condition that makes a number of secret keys invalid; 2) it is insensitive to the change of the secret key; 3) it is insensitive to the change of the plain-image; 4) it can be broken with only one known/chosen-plaintext; 5) it has some other minor defects. 5. Recently, a new image scrambling (i.e., encryption) scheme without bandwidth expansion was proposed based on two-dimensional discrete prolate spheroidal sequences. A comprehensive cryptanalysis is given on this image scrambling scheme, showing that it is not sufficiently secure against various cryptographical attacks including ciphertextonly attack, known/chosen-plaintext attack and chosen-ciphertext attack. Detailed cryptanalytic results suggest that the image scrambling scheme can only be used to realize perceptual encryption, but not to provide content protection for digital images. 6. Based on cryptanalyses of the aforementioned six encryption schemes, some general recommendations for designing secure multimedia encryption scheme are concluded. Keywords: chaos, cryptanalysis, multimedia encryption, brute-force attack, knownplaintext attack, chosen-plaintext attack, differential attack.