Research and Applications of Phase-based Video Motion Magnification Algorithm

Abstract

Traditionally, machine faults are diagnosed using accelerometers to capture the vibration signals generated by each critical machine component while operating. This is a contact-type measurement method, which may lead to substantial error in the analysis if the accelerometers yield a large mass and damping ratio in the captured vibration. This is particularly true when the monitored target has a small mass. Moreover, if the surface of the target object is hot or inaccessible for mounting accelerometers, it may be difficult to conduct vibration-based fault diagnosis using contact-type measurement methods. Non-contact-type vibration measurement techniques, such as displacement measurements performed using a scanning laser vibrometer, can enhance sensing capacity at high spatial resolutions without the need for installing sensors on the structures or inducing the mass-loading effect. However, the operator must wear protective eye goggles and undergo special training to operate the laser equipment. In addition, the high costs are inconvenient for widespread use. Here, alternative methods using digital video cameras are proposed, which are relatively low-cost and provide measurements at very high spatial resolutions.

Recently, a new remote vibration analysis method named phase-based video magnification (PVM) has been proposed, which uses a video file captured by a high-speed camera to measure the movement or displacement of vibrations generated by the monitored machine. Based on the principle that the phase change in complex steerable pyramid (CSP) decomposition is proportional to the local vibration in an image, the vibration signal of the target object is extracted and amplified. Due to the above advantages, non-contact, inexpensive, phase-based video processing has gained popularity. From the analysis conducted using this approach, acceptably accurate vibration signals can be captured and analyzed for machine fault diagnosis.

In the beginning, this thesis described the inconveniences of traditional structural health monitoring methods, as well as the availability of a variety of non-contact measurement methods. Subsequently, the theory of phase-based video motion processing technology was introduced. The capability of this novel technology in capturing the vibration movement remotely was demonstrated through several experiments. The experimental results verified that the designed novel technology was working as expected. From the observations of results, three novel research achievements were discovered. Their novelties and contributions in capturing vibration movements by using such new sensing method were highlighted as follows:

First, to reduce the computational resources required, a novel, non-contact-type of video-based vibration measurement method is presented. It combines the multiscale pyramid decomposition method and sparse representation technique. This method can model and manipulate the spatiotemporal pixel phases that have encoded the local structural vibrations in the video measurements. Different from existing method which uses Riesz transform to replace CSP and sacrifice the ability in multi orientation analysis, the sparse CSP can keep the multi direction analyze ability when saving calculation time. Hence, it enables the rapid extraction of modal frequencies and high-resolution mode shapes from line-of-sight video-based vibration measurements of the machine. This proposed method is found to be more efficient than conventional vision-based methods. In addition, a vibration modal generated by a rotor system and its characteristics were studied. Instead of using conventional contact-type accelerometers to measure the vibrations generated by the rotary machine, the proposed method can measure vibration without the need to physically mount any device on the target object. This innovative video-based method uses a high-speed camera to capture video files while the monitored machine is operating. The video files are then analyzed using our proposed method, which is based on the framework of phase-based video-motion magnification. The experiments indicate that the results obtained by the proposed method for machine shaft fault diagnosis are comparable to those obtained using a physically mounted accelerometer.

Second, a novel and effective dynamic video processing technology is presented. Existing research results about dynamic motion magnification do not mention the influence of sub-pixel. The judgement of the result is the visual effect, not the accuracy of vibration measurement which could be used for SHM. Besides, most research results do not use quantitative experiments like modal exciter to judge the effectiveness of dynamic motion magnification. Aimed at magnifying subtle changes in the presence of large motions, this new method is designed by combining phase-based video processing technology with motion tracking. Here, the motion tracking algorithm was used to detect and measure the position of an inspected object by using a high-speed, mobile camera. According to the object positions captured in the input video frames, the frames were segmented so that the local video that focused on the inspected object could be extracted. After the captured video frames of the object were stabilized, various scales and directions were compensated to improve the SNR of weak vibration signals according to the translation invariance provided by the steerable pyramid method. Hence, subtle vibration movements of the object were selectively magnified using the steerable pyramid method. The novel scientific contributions of this research work include 1) a new framework designed to capture and analyze the vibration movement of a moving object, 2) steerable filters implemented to compensate for subpixel motion, and 3) a new algorithm developed to improve the SNR of weak vibration signals so that the complete vibrations of moving objects can be measured. This new dynamic measurement system can effectively extract the dynamic vibration of any moving object remotely and without any contact with the target object by using high-speed, moving cameras.

Third, two video motion processing methods that improve the time domain analysis of conventional PVM algorithms are presented. Aimed at reducing the input parameters and automatically separating the modal of the structure in video, one algorithm combined principal component analysis with variational model decomposition. The modal exciter experiment was performed to verify the accuracy of the proposed method. After investigating the temporal filter of the conventional PVM method, it was found that the extracted time-varying vibration signals and noise could be magnified at the same time. Hence, to minimize noise and capture the non-stationary characteristics, a video motion processing method based on time–frequency analysis was proposed and implemented. The experimental results revealed that the proposed method could extract non-stationary signals accurately.

Based on the aforementioned new achievements, the proposed technology proves that it is an effective means to detect vibration movement of even very small object without the need to mount the sensor on the small object. The acquired vibration signals can be used to determine the health of a small or moving structure/machine, with the advantages of no contact and low cost. In future, such non-contact, video-based vibration measurement technology can be further developed to monitor the heart and breathing rates of elderly persons without a sensor placed on the body during sleeping but only through the use of a CCTV camera.

Date of Award	7 Feb 2023
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Min XIE (Supervisor), Wai Tat Peter TSE (Co-supervisor) & Qingbo He (External Supervisor)