CN103297657B - A kind of video stabilizing method based on GPU - Google Patents

The present invention discloses a kind of video stabilizing method based on GPU, including: upload reference picture and pending image to GPU；Reference picture and pending figure are generated gaussian pyramid；Initialize motion vector MV；Choose pending figure gaussian pyramid n-th layer；The n-th layer gaussian pyramid of pending image is carried out geometric transformation, the intermediate images T3 that result is stored on GPU；Calculate the difference of intermediate images T3 and reference picture pyramid n-th layer, the intermediate images T4 that result is stored on GPU；According to intermediate images T4, calculate picture quality controlling elements, and be stored in GPU internal memory；Calculate meansigma methods and the standard variance of intermediate images T4；Peace mean distances all in intermediate images T4 are set to 0 more than the point of three times of variances；Choose the motion vector MV that minimum image quality control factor pair is answered；With gained motion vector MV, pending image is carried out geometric transformation；Image after gained image is stabilized treatment after geometric transformation.

A kind of based on GPU Video stabilizing method

Technical field

The present invention relates to a kind of video stabilizing method based on GPU, belong to computer image processing technology field.

Background technology

The cardinal principle of video stabilization algorithm is: estimates the present frame motion relative to former frame (or reference frame), and obtains describing the transformation matrix of change between two two field pictures by outstanding algorithm；Then, utilize transformation matrix that current frame image is corrected so that video information is the most stable.

Stablize video and can preferably show image detail, thus improve the quality of video monitor；In reality, flating is unfavorable for that human eye, to the analysis of image and observation, the most easily causes the visual fatigue of monitoring personnel, and video stabilization can improve this situation；It addition, the digital video frequency recording equipment that stable image is rear end provides the video source being more beneficial for compression, it is greatly improved compression ratio, flow down in same code and can obtain higher-quality image, in other words under identical image quality, being substantially reduced data traffic, beneficially multiple spot is watched simultaneously.

Existing video stabilization technology mainly realizes on CPU, and due to the restriction of hardware speed, video stabilization technology exists following two solution:

1, for improving speed, choosing less pixel calculates, and this method reduces the quality of video stabilization.

2, for improving precision, select computer or the exploitation specialized hardware of higher configured, it improves cost.

Summary of the invention

Goal of the invention: for problems of the prior art, the present invention provides a kind of video stabilizing method based on GPU.The present invention mainly achieves HD image low cost, high-precision real-time stabilization processes, and can be widely used for such as Aerial photography, vehicle-mounted monitoring, and shipping monitors.

Technical scheme: a kind of video stabilizing method based on GPU, key step is as follows:

Step 1: upload reference picture and pending image to GPU；

Step 2: reference picture and pending figure are generated gaussian pyramid；

Step 3: initializing motion vector MV to be calculated is 0 vector；

Step 4: choose pending figure gaussian pyramid n-th layer；

Step 5: set circulation initial value I=5；

Step 6: the n-th layer gaussian pyramid of pending image is carried out geometric transformation；The intermediate images T3 that geometric transformation result is stored on GPU；

Step 7: calculate the difference of intermediate images T3 and reference picture pyramid n-th layer, the intermediate images T4 that result is stored on GPU；

Step 8: calculate intermediate images T4 quadratic sum a little, and be stored in GPU internal memory；

Step 9: calculate meansigma methods and the standard variance of intermediate images T4；

Step 10: peace mean distances all in intermediate images T4 are set to 0 more than the point of three times of variances；

Step 11: calculate motion vector changes delta MV by method of least square；

Step 12:MV=MV+ Δ MV, and it is stored in GPU internal memory；

Step 13:I=I-1, if I >=0, jump procedure 6；

Step 14: choose the motion vector MV that minimum image quality control factor pair is answered, after being amplified according to pyramid interlayer relation, as processing next layer of initial motion vectors MV of pyramid；

Step 15:n=n-1, if n >=0, go to step 5；

Step 16: pending image is carried out geometric transformation with gained motion vector MV；Image after gained image is stabilized treatment after geometric transformation.

Image geometry transform mainly comprises the following steps (pixel of a GPU thread process target image):

Step 201: target image coordinate corresponding for this thread is transformed to the coordinate of source images；

Step 202: reading the pixel that in source images, in step 201, the coordinate of gained is corresponding, result is stored in target image.

Gaussian pyramid generates and mainly comprises the following steps (block that a GPU thread block processes target image).

Step 301: this group Gaussian transformation desired data is read in shared drive by thread block；

Step 302: Gauss line translation, result is stored in shared drive；

Step 303: each thread synchronization in thread block；

Step 304: Gauss rank transformation, result is stored in shared drive；

Step 305: each thread synchronization in thread block；

Step 306: each thread writes the result into target image.

What meansigma methods and variance calculated mainly comprises the following steps: a GPU thread block processes a block of target image,

Step 401: thread block reads the image block that this thread block is corresponding, is stored in shared drive；

Step 402: calculate summation and the quadratic sum of all pixels of this thread block, and the valid pixel handled by this thread block；

Step 403: step 402 acquired results is stored in the global memory that this thread block is corresponding；

Step 404: read in the result of step 403；

Step 402: calculate and store meansigma methods and the variance of whole image.

The present invention uses technique scheme, and the image processing process having the advantages that all is carried out on GPU, and whole processing procedure need not the participation of CPU, improves the speed that GPU runs, reduces the load of PCIE bus.Mainly achieve HD image low cost, high-precision real-time stabilization processes, and can be widely used for such as Aerial photography, vehicle-mounted monitoring, and shipping monitors.

Detailed description of the invention

Below in conjunction with specific embodiment, it is further elucidated with the present invention, it should be understood that these embodiments are merely to illustrate the present invention rather than limit the scope of the present invention, after having read the present invention, those skilled in the art all fall within the application claims limited range to the amendment of the various equivalent form of values of the present invention.

Video stabilizing method based on GPU, scheme based on CPU+GPU, use OpenCL as main GPU development language.It is mainly characterized in that all of image processing process is all carried out on GPU, and whole processing procedure need not the participation of CPU, improve the speed that GPU runs, reduce the load of PCIE bus.This method includes CPU program and seven GPU kernel programs.Seven GPU kernel programs are respectively:

1. Image geometry transform kernel program K1.

2. gaussian pyramid generates kernel K2.

3. image difference calculates kernel K3.

4. picture quality controlling elements calculate kernel K4.

5. meansigma methods and variance calculate kernel K5.

6. method of least square calculates MV kernel K6.

7.MV chooses kernel K7.

The key step of the method is as follows:

Step 1: upload reference picture and pending image to GPU.

Step 2: reference picture and pending figure are generated gaussian pyramid.

Step 3: initialize motion vector MV.

Step 4: choose pending figure gaussian pyramid n-th layer.

Step 5: set circulation initial value I=5

Step 6: the n-th layer pyramid of pending image is carried out geometric transformation.Conversion uses geometric transformation kernel program K1 and motion vector MV.The intermediate images T3 that result is stored on GPU.

Step 7: use image difference to calculate kernel K3 and calculate the difference of T3 and reference picture pyramid n-th layer, the intermediate images T4 that result is stored on GPU.

Step 8: use picture quality controlling elements to calculate kernel K4, according to T4, calculate a quality control factor, and be stored in GPU internal memory.

Step 9: use meansigma methods and variance to calculate kernel K5, calculate meansigma methods and the standard variance of T4.

Step 10: be set to 0 with meansigma methods distance more than the point of three times of variances all in T4.

Step 11: method of least square calculates MV kernel K6 and calculates motion vector changes delta MV.

Step 12:MV=MV+ Δ MV, and it is stored in GPU internal memory.

Step 13:I=I 1, if I >=0, jump procedure 6.

Step 14: use MV to choose kernel K7, choose the MV that minimum mass controlling elements are corresponding, after being amplified according to pyramid interlayer relation, as processing the pyramid initial MV of next layer.

Step 15:n=n 1, if n >=0, go to step 5.

Step 16: pending image is carried out geometric transformation with gained MV.Image after gained image is stabilized treatment after geometric transformation.

Image geometry transform kernel program K1 mainly comprises the following steps (pixel of a GPU thread process target image):

Step 201: target image coordinate corresponding for this thread is transformed to the coordinate of source images.

Step 202: four that read source images step 1 gained coordinate are faced a little, and the result of bilinear interpolation is stored in target image.

Gaussian pyramid generates mono-GPU thread block of kernel K2(and processes a block of target image).

Step 301: this group Gaussian transformation desired data is read in shared drive by thread block.

Step 302: Gauss line translation, result is stored in shared drive.

Step 303: each thread synchronization in thread block.

Step 304: Gauss rank transformation, result is stored in shared drive.

Step 305: each thread synchronization in thread block.

Step 306: each thread writes the result into target image.

Meansigma methods and variance calculate kernel K5.

This kernel is made up of two sub-kernels.

Mono-GPU thread block of kernel 1(processes a block of target image):

Step 401: thread block reads the image block that this thread block is corresponding, is stored in shared drive.

Step 402: calculate summation and the quadratic sum of this thread block, and the valid pixel handled by this thread block.

Step 403: result is stored in the global memory that this thread block is corresponding.

It is view data that kernel 2(merges image block data.)

Step 501: read in the result of last kernel step 403.

Step 502: calculate and store meansigma methods and the variance of image.