CN112200746A - Dehazing method and device for foggy traffic scene images - Google Patents

本发明公开了一种雾天交通场景图像的去雾方法和设备，方法为：A，针对远景区域、近景区域以及过渡区域不同的雾气浓度，分别计算雾天交通场景图像中对应区域的大气光值；然后在HSI颜色空间利用各通道的传输图和大气光值，根据大气散射模型计算初步去雾的交通场景图像；B，基于预设的I通道阈值，对初步去雾的交通场景图像进行全局亮度提升；预设的I通道阈值是根据初步去雾的交通场景图像中天空区域的I通道像素设置得到；且天空区域是基于暗通道特征和相对能量特征对雾天交通场景图像分割得到；C，对步骤B得到的图像，进行限制对比度自适应直方图均衡化和引导滤波处理，得到最后去雾的交通场景图像。本发明可对交通场景图像快速、有效去雾。

The invention discloses a fog removal method and device for foggy traffic scene images. The method is as follows: A. According to different fog concentrations in the long-range, near-field and transition areas, respectively calculate the atmospheric light of the corresponding areas in the foggy traffic scene image. Then use the transmission map and atmospheric light value of each channel in the HSI color space to calculate the preliminary dehazed traffic scene image according to the atmospheric scattering model; B, based on the preset I channel threshold, the preliminary dehazed traffic scene image is processed. The global brightness is improved; the preset I channel threshold is obtained according to the I channel pixel settings of the sky area in the preliminary dehazing traffic scene image; and the sky area is obtained by segmenting the foggy traffic scene image based on dark channel features and relative energy features; C, perform limited contrast adaptive histogram equalization and guided filtering on the image obtained in step B to obtain the final dehazed traffic scene image. The invention can quickly and effectively dehaze images of traffic scenes.

Defogging method and device for traffic scene image in foggy day

Technical Field

The invention belongs to the field of image information processing, and particularly relates to a defogging method and equipment for a traffic scene image in foggy days.

Background

Fog, a common natural phenomenon, is formed by condensation of water vapor in the air contacting the cooler ground surface, consisting of small water droplets floating in the air. In the field of video monitoring, due to the existence of fog, the visibility of visible objects is reduced, so that the image obtained by a sensor is seriously degraded, and the post-processing of the image is not facilitated, thereby influencing the robustness of visual systems such as target tracking, intelligent transportation, video monitoring, aerial photography and the like. In addition, in intelligent traffic and regional video monitoring, the monitoring effect is seriously influenced by the existence of fog, so that the intelligent traffic system judges vehicle information wrongly, images obtained by regional video monitoring are fuzzy, the former is particularly serious, and the existence of fog often causes traffic accidents or traffic jam, even cancellation of flights. Therefore, the investigation of defogging is necessary.

At present, a great deal of research is carried out by many scholars at home and abroad aiming at the problem of image degradation caused by haze weather. Current defogging methods are largely divided into two main categories: non-model based image defogging algorithms and model based image defogging algorithms.

The image defogging algorithm based on the non-model is to improve the brightness and the contrast through the image enhancement algorithm without considering the physical reason of the degradation of the image in the haze weather, so that the visual effect of the image in the haze weather is improved. The image defogging algorithm based on the non-model is relatively simple in processing and wide in application range, but has the problems of information loss, image distortion and the like, and the essence of the method is to reduce the influence of haze on the image as much as possible, so that the defogging effect is not thorough because the defogging is not carried out fundamentally. Common image defogging algorithms based on non-models include a histogram equalization algorithm, a Retinex algorithm, a wavelet transform algorithm and the like.

The image defogging algorithm based on the model is used for analyzing the degradation reason of the image in the foggy day, further establishing the model to simulate the image degradation process, and then obtaining the fogless image through solving the inverse process. The most common and effective model at present is the atmospheric scattering model proposed by McCartney. Based on this model, various kinds of defogging algorithms have been proposed by many scholars. The algorithm analyzes the foggy image forming mechanism under the atmospheric scattering model and essentially analyzes the image degradation reason, so that the defogging result is more ideal and reliable. The earliest defogging algorithm utilized multiple frames of images in combination with an atmospheric scattering model to solve a model equation to obtain a fog-free image. In recent years, a single image defogging algorithm based on an atmospheric scattering model is widely researched, and the algorithm adds known priori knowledge into the atmospheric scattering model, solves the transmittance and atmospheric light value of an image and then obtains a fog-free image according to an atmospheric scattering model formula.

In the aspect of an image defogging patent, huhaofeng et al (with the patent publication number of CN107966412A) obtain total light intensity and polarization difference by correcting two light intensity maps in an orthogonal polarization state, and then obtain a defogged image by calculating through a differential polarization recovery model, but the requirement on equipment for obtaining the light intensity maps in the orthogonal polarization state twice is high, and the defogged image cannot be popularized in a large scale. Tanghong faithful et al (patent publication No. CN107085830A) estimate the atmospheric transmittance by a two-region filtering method and a propagation filtering method, and then optimize the atmospheric light intensity by an adaptive method to realize the recovery of fog-free images, but this method cannot effectively process fog-day images with low brightness. Zuojon et al (patent publication No. CN110992285A) segment a foggy image into content features and detail features through a hierarchical neural network, and then perform defogging processing respectively, and interact with intermediate results to restore a fogless image, but this method is not suitable for situations with insufficient computing power. The methods are all general image defogging algorithms, are not optimized for traffic scenes, and have respective limitations.

Under the background, it is particularly important to research a defogging method for a traffic scene image with strong robustness and low time complexity.

Disclosure of Invention

The invention provides a defogging method and equipment for a foggy traffic scene image based on sky region segmentation and color space conversion, and solves the technical problems that an existing image defogging method is not used for optimizing a traffic scene, and a low-brightness fog image is poor in defogging effect.

In order to achieve the technical purpose, the invention adopts the following technical scheme:

a defogging method for a traffic scene image in foggy days comprises the following steps:

step A, respectively calculating atmospheric light values of corresponding areas in a foggy day traffic scene image according to different fog concentrations of a distant view area, a near view area and a transition area; then, calculating a traffic scene image subjected to preliminary defogging according to an atmospheric scattering model by utilizing the transmission diagram and the atmospheric light value of each channel in the HSI color space;

b, carrying out global brightness improvement on the preliminarily defogged traffic scene image based on a preset I channel threshold value;

the preset I channel threshold value is obtained according to I channel pixels of a sky area in an HSI color space in a traffic scene image subjected to preliminary defogging; the sky area is obtained by segmenting the foggy day traffic scene image based on the dark channel characteristics and the relative energy characteristics;

and step C, carrying out contrast-limiting adaptive histogram equalization and guide filtering processing on the image obtained in the step B to obtain a finally defogged traffic scene image.

In a more preferred technical solution, the method for calculating the dark channel characteristics includes:

wherein I and j are the row number and column number of the pixel in the image, M is the sum of all the row numbers in the image, I is the image composed of r, g and b channels, c represents any color channel of r, g and b, I^cRepresenting the c color channel component of image I, I^c(I, j) represents the c color channel component of image I at pixel (I, j); f_DC++Representing the dark channel characteristics of image I at pixel (I, j).

In a more preferred technical solution, the method for calculating the relative energy characteristic includes:

In a more preferred technical scheme, the method for segmenting the foggy day traffic scene image to obtain the sky area based on the dark channel characteristic and the relative energy characteristic comprises the following steps:

a1, extracting dark channel characteristics and relative energy characteristics of the foggy day traffic scene image;

a2, roughly dividing the foggy day traffic scene image into a sky area and a non-sky area by adopting a K-means clustering method based on the dark channel characteristics and the relative energy characteristics;

a3, selecting partial pixels with higher confidence level from the roughly divided sky area and non-sky area, and respectively using the partial pixels as positive and negative samples; training a machine model to obtain a sky area fine classifier based on the dark channel characteristics and the relative energy characteristics of the positive and negative samples;

a4, classifying pixels in the foggy day traffic scene image by using a fine sky region classifier based on the dark channel characteristics and the relative energy characteristics, and obtaining a corresponding binary image of fine sky region segmentation according to the classification result.

In a more preferred technical scheme, the method for roughly segmenting the foggy day traffic scene image by adopting the K-means clustering method comprises the following steps:

b1, converting the dark channel characteristics and the relative energy characteristics of the foggy day traffic scene image into a one-dimensional column vector by a two-dimensional matrix, and executing regularization operation;

b2, setting the category number of the K-means clustering to be 2, and clustering pixels in the foggy day traffic scene image based on two column vectors of dark channel characteristics and relative energy characteristics to obtain a clustering label and a clustering center;

b3, determining the categories of all pixels in the foggy day traffic scene image according to the clustering center, thereby determining a sky area and a non-sky area;

b4, converting the clustering label from the one-dimensional column vector into a two-dimensional matrix to obtain a rough segmentation image of the foggy day traffic scene image.

In a more preferred technical solution, the method for selecting positive and negative samples from the roughly divided sky region and the non-sky region includes:

c1, traversing the binary image obtained by roughly dividing the binary image into a sky area and a non-sky area according to a column, and recording the column number if the sky area is on the upper part of the column and the ratio of the sky area to the height of the column is not less than a preset ratio;

c2, extracting the longest subsequence of the sky area from the column number recorded in c1, wherein the pixels with the preset proportion at the upper part are the pixels with higher confidence coefficient of the sky area, and the pixels are used as positive samples;

c3, traversing the binary image obtained by roughly dividing the binary image into a sky area and a non-sky area according to a column, and recording the column number if the non-sky area is at the lower part of the column and the ratio of the height of the column is not less than a preset ratio;

c4, extracting the longest subsequence of the non-sky area from the column number recorded in c3, wherein the pixels with the lower preset proportion are the pixels with higher confidence of the non-sky area, and the pixels are used as negative samples.

In a more preferred technical scheme, the method for setting the preset I channel threshold value is as follows:

calculating the number N of sky area pixels in the traffic scene image subjected to preliminary defogging_sky(ii) a Then, sorting I channel pixel values of the traffic scene images subjected to preliminary defogging from large to small, and taking the Nth pixel value in the sorting_skyThe pixel value of each pixel is used as a preset I channel threshold value T;

and B, performing global brightness improvement on the preliminarily defogged traffic scene image, specifically: and multiplying each pixel value of the I channel of the traffic scene image subjected to preliminary defogging by 1T, and replacing the original I channel pixel value with the obtained result.

In a more preferred technical solution, the dividing method of the distant view area, the close view area and the transition area in step a is as follows: clustering all pixels in the foggy day traffic scene image based on the dark channel characteristics, so as to divide the foggy day traffic scene image into a long-range area, a short-range area and a transition area;

the method for calculating the atmospheric light value of the traffic scene image in the foggy day comprises the following steps: calculating the atmospheric light value of a distant view region by adopting a quadtree decomposition method, calculating the atmospheric light value of a near view region by adopting a statistical method, calculating the atmospheric light value of a transition region by adopting linear interpolation, and then carrying out mean value filtering processing on all the obtained atmospheric light values.

In a more preferred technical solution, the traffic scene image of preliminary defogging is calculated according to the atmospheric scattering model by using the transmission map and the atmospheric light value of each channel in the HSI color space, specifically:

d1, converting the foggy day traffic scene image from RGB color space to HSI color space to obtain brightness component I of the foggy day traffic scene image_JSaturation component S_JAnd a chrominance component H_J；

d2, calculating transmission diagrams of the foggy day traffic scene images in the I channel and the S channel:

wherein T is a transmission diagram of the foggy day traffic scene image in the channel I, T is a transmission diagram of the foggy day traffic scene image in the channel S, and I_IIs the luminance component of the fog-free traffic scene image,

d3, using brightness component I of traffic scene image in foggy day in channel I_JTransmission diagram t and atmospheric light value I_AAnd calculating the brightness component of the traffic scene image subjected to preliminary defogging according to the atmospheric scattering model:

I_J(i,j)＝I_I(i,j)t(i,j)+I_A(1-t(i,j))，

the brightness component I of the fog-free traffic scene image calculated by the formula_INamely the brightness component of the traffic scene image subjected to preliminary defogging;

saturation component S of traffic scene image in foggy day in S channel_JTransmission diagram T and atmospheric light value S_ACalculating the saturation component S of the preliminarily defogged traffic scene image according to the atmospheric scattering model and the definition of the saturation_I：

S_J(i,j)＝S_I(i,j)T(i,j)+S_A(1-T(i,j))，S_A＝0；

Keeping the H channel value of the traffic scene image subjected to preliminary defogging and the fog traffic scene image unchanged, namely H_I＝H_J；

d4, using the obtained traffic scene image preliminarily defogged from each channel component H of HSI color space_I、S_IAnd I_IAnd converting the image into the RGB color space to obtain the traffic scene image subjected to preliminary defogging.

The invention also provides an apparatus comprising a processor and a memory; wherein: the memory is to store computer instructions; the processor is configured to execute the computer instructions stored in the memory, and specifically, to perform the method according to any of the above technical solutions.

Advantageous effects

The invention realizes end-to-end image defogging, brightness improvement and detail enhancement by combining the atmospheric scattering model and the image enhancement technology. The method has the advantages of high operation speed, good effect, no need of manual participation, low cost and strong universality. According to the improved dark channel characteristics and the improved relative energy characteristics, the concept from rough segmentation to fine segmentation is applied to realize accurate segmentation of the sky area; the image defogging based on the physical model is realized by adopting the color space conversion and single-channel image defogging technology; and by adopting the CLAHE and the guided filtering technology, the details are enhanced and the visual effect is improved. The method can be widely applied to defogging of videos and images in traffic scenes.

Drawings

FIG. 1 is an overall flow diagram of a process according to an embodiment of the invention;

FIG. 2 is a diagram illustrating the effect of the steps of sky region segmentation in example 1; wherein, the graph a is the improved dark channel characteristic, the graph b is the improved relative energy characteristic, the graph c is the result of the rough segmentation of the sky area, and the graph d is the result of the fine segmentation of the sky area;

FIG. 3 is a graph showing the effect of the defogging steps based on the atmospheric scattering model in example 1; wherein, the graph a is the atmospheric light value, the graph b is the defogging result of the I channel, the graph c is the defogging result of the S channel, and the graph d is the preliminary defogging result;

FIG. 4 is a graph showing the effects of the steps of the post-defogging treatment in example 1; wherein, the graph a is the result of global brightness improvement, the graph b is the result of CLAHE processing, the graph c is the result of guide filtering processing, and the graph d is the original fog graph;

FIG. 5 is a diagram illustrating the effect of the steps of sky region segmentation in example 2; wherein, the graph a is the improved dark channel characteristic, the graph b is the improved relative energy characteristic, the graph c is the result of the rough segmentation of the sky area, and the graph d is the result of the fine segmentation of the sky area;

FIG. 6 is a graph showing the effect of the defogging steps based on the atmospheric scattering model in example 2; wherein, the graph a is the atmospheric light value, the graph b is the defogging result of the I channel, the graph c is the defogging result of the S channel, and the graph d is the preliminary defogging result;

FIG. 7 is a graph showing the effects of the steps of the post-defogging treatment in example 2; wherein, the graph a is the result of global brightness lifting, the graph b is the result of CLAHE processing, the graph c is the result of guided filtering processing, and the graph d is the original fog graph.

Detailed Description

The following describes embodiments of the present invention in detail, which are developed based on the technical solutions of the present invention, and give detailed implementation manners and specific operation procedures to further explain the technical solutions of the present invention.

The present embodiment provides a defogging method for a foggy traffic scene image based on sky region segmentation and color space conversion, and the overall implementation flow is shown in fig. 1, and includes the following steps:

step A, respectively calculating atmospheric light values of corresponding areas in a foggy day traffic scene image according to different fog concentrations of a distant view area, a near view area and a transition area; and then, calculating a traffic scene image subjected to preliminary defogging according to an atmospheric scattering model by using the transmission map and the atmospheric light value of each channel in the HSI color space. The specific treatment process of the step A is as follows:

1) calculating the value of atmospheric light

The fog concentration is not uniformly distributed in the whole image of the traffic scene in foggy days, the fog concentration at a distant scene is higher, and the fog concentration at a near scene is lower. Therefore, the corresponding atmospheric light value is calculated according to the fog concentrations of different areas, so that the problem of image over-enhancement and the problem of halo artifacts after defogging can be solved. Firstly, dividing a foggy day traffic scene image into a long-range area, a short-range area and a transition area, then calculating an atmospheric light value of the long-range area by using a quadtree decomposition method, calculating an atmospheric light value of the short-range area by using a statistical method, calculating an atmospheric light value of the transition area by using linear interpolation, and finally performing mean filtering to smooth the obtained atmospheric light value. The method comprises the following specific steps:

(i) the improved dark channel characteristics are used as fog concentration, the number of the clustering categories is set to be 3, then clustering operation is carried out to obtain a clustering label and a clustering center, and a distant view area, a near view area and a transition area are determined according to the clustering center;

since the dark channel value is invalid in a white non-sky area, such as a white lane line, which is common in traffic scenes, directly using the dark channel value as a feature affects the accuracy of segmentation. Therefore, the embodiment proposes an improved dark channel feature calculation method in combination with the position features (perpendicular to the ground) of the traffic surveillance camera:

wherein I and j are the row number and column number of the pixel in the image, M is the sum of all the row numbers in the image, I is the image composed of r, g and b channels, c represents any color channel of r, g and b, I^cRepresenting the c color channel component of image I, I^c(I, j) represents the c color channel component of image I at pixel (I, j); f_DC++Representing the dark channel characteristics of image I at pixel (I, j).

(ii) Carrying out quadtree decomposition on the divided distant view areas: setting a stopping condition of the quadtree decomposition, continuously decomposing the subarea with the highest average brightness until the stopping condition is met, wherein the pixel average value of the corresponding subarea when the decomposition is stopped is the atmospheric light value of the long-range view area;

(iii) sorting the pixels of the close-range area from large to small according to the brightness values, and calculating the average value of the pixels with the first 1% brightness, namely the atmospheric light value of the close-range area;

(iv) and traversing the atmospheric light value graph according to columns, performing interpolation processing on a transition region between a distant view region and a close view region, and then performing mean filtering to obtain the smooth atmospheric light value graph of the foggy day traffic scene image.

2) RGB color space to HSI color space

In order to alleviate the problems of color distortion, halo artifacts and the like, the original foggy traffic scene graph is converted from the RGB color space to the HSI color space. And during defogging, keeping the H channel unchanged, and restoring the I channel and the S channel. The calculation formula for the conversion from the RGB color space to the HSI color space is as follows:

3) computing transmission maps for I and S channels

According to the atmospheric scattering model, the I channel satisfies the following equation:

I_J(i,j)＝I_I(i,j)t(i,j)+I_A(1-t(i,j))

wherein I_JIs the brightness component of the image of a traffic scene in foggy weather, I_IIs the luminance component of the image of a fog-free traffic scene, I_AThe atmospheric light value of the foggy day traffic scene image in the I channel, namely the brightness component of the atmospheric light value, and t is a transmission diagram of the foggy day traffic scene image in the I channel.

In this embodiment, a method for calculating an I-channel transmission diagram is provided by taking a defogging method of an infrared image (single-channel image) as a reference, and the method includes:

assuming that the S channel also satisfies the atmospheric scattering model, the following equation is obtained:

S_J(i,j)＝S_I(i,j)T(i,j)+S_A(1-T(i,j))

wherein S is_JIs the saturation component of the image of the traffic scene in foggy weather, S_IIs the saturation component of the fog-free traffic scene image, S_AThe atmospheric light value of the foggy day traffic scene image in the S channel is the saturation component of the atmospheric light value, and T is a transmission diagram of the foggy day traffic scene image in the S channel.

Since the saturation of the atmospheric light value is low, approximately equal to 0, i.e. S_A0, so the above equation can be simplified to:

S_J(i,j)＝S_I(i,j)T(i,j)

according to the definition formula of the saturation, the above formula can be expanded as follows:

min(R_J,G_J,B_J)＝t·min(R_I,G_I,B_I)+(1-t)·min(R_A,G_A,B_A)

since the RGB values of the atmospheric light values are approximately equal, the following approximate relationship can be obtained:

I_A≈min(R_A,G_A,B_A)

and finally, a calculation formula of the S channel transmission diagram is obtained:

4) calculating a luminance component I of a preliminary defogged traffic scene image_ISaturation component S_IAnd chrominance component H_I

(i) Utilizing brightness component I of traffic scene image in fog day in channel I_JTransmission diagram t and atmospheric light value I_AAnd calculating the brightness component of the traffic scene image subjected to preliminary defogging according to the atmospheric scattering model:

I_J(i,j)＝I_I(i,j)t(i,j)+I_A(1-t(i,j))，

the brightness component I of the fog-free traffic scene image calculated by the formula_INamely the brightness component of the traffic scene image subjected to preliminary defogging;

(ii) saturation component S of traffic scene image in foggy day in S channel_JTransmission diagram T and atmospheric light value S_ACalculating the saturation component S of the preliminarily defogged traffic scene image according to the atmospheric scattering model and the definition of the saturation_I：

S_J(i,j)＝S_I(i,j)T(i,j)+S_A(1-T(i,j))，S_A＝0；

(iii) Keeping the H channel value of the traffic scene image subjected to preliminary defogging and the fog traffic scene image unchanged, namely H_I＝H_J。

5) Using the obtained preliminarily defogged traffic scene image to be HSI coloredComponent of each channel of space I_I、S_IAnd H_IAnd converting the image into the RGB color space to obtain the traffic scene image subjected to preliminary defogging. The calculation formula for the conversion from HSI color space to RGB color space is as follows:

RG region (0 ≦ H < 120 °):

G＝3I-(R+B)，

B＝I(1-S)；

GB area (120 ≦ H < 240 °):

H＝H-120°，

R＝I(1-S)，

B＝3I-(R+G)；

BR area (240 ≦ H ≦ 360 °):

H＝H-240°，

R＝3I-(G+B)，

G＝I(1-S)，

b, carrying out global brightness improvement on the preliminarily defogged traffic scene image based on a preset I channel threshold value; the preset I channel threshold value is obtained according to I channel pixels of a sky area in an HSI color space in a traffic scene image subjected to preliminary defogging; and the sky area is obtained by segmenting the foggy day traffic scene image based on the dark channel characteristics and the relative energy characteristics.

The defogged image obtained based on the atmospheric scattering model in the step A has the problems of partial dark area, loss of details and the like, and each channel component I is directly processed_I、S_IAnd H_IThe visual effect of converting back to RGB color space is poor and needs to be adoptedSome post-processing steps are taken to enhance the visual effect. In this embodiment, a suitable threshold is obtained according to an I-channel pixel setting of the sky area in the traffic scene image subjected to preliminary defogging in the HSI color space, and the obtained image in step a is subjected to global brightness enhancement based on the preset I-channel threshold. The specific treatment process is as follows:

1) dark channel characteristic and relative energy characteristic of extracted foggy day traffic scene image

The method for calculating the dark channel characteristics is the same as the improved method for calculating the dark channel characteristics, and comprises the following steps:

wherein I and j are the row number and column number of the pixel in the image, M is the sum of all the row numbers in the image, I is the image composed of r, g and b channels, c represents any color channel of r, g and b, I^cRepresenting the c color channel component of image I, I^c(I, j) represents the c color channel component of image I at pixel (I, j); f_DC++Representing the dark channel characteristics of image I at pixel (I, j).

Due to the fact that the quality of the monitoring video is poor mostly, the sky area contains a lot of noise, the difference of relative energy characteristics of the sky area and the non-sky area is not obvious, and therefore the segmentation accuracy is affected. Therefore, the present embodiment proposes an improved method for calculating the relative energy characteristics by combining the position characteristics (perpendicular to the ground) of the traffic monitoring camera and the brightness information of the image:

2) Roughly dividing a sky area: and roughly dividing the foggy day traffic scene image into a sky area and a non-sky area by adopting a K-means clustering method based on the dark channel characteristics and the relative energy characteristics. The specific process is as follows:

(i) converting dark channel characteristics and relative energy characteristics of the foggy day traffic scene image into a one-dimensional column vector by a two-dimensional matrix, and executing regularization operation;

(ii) setting the category number of the K-means clustering to be 2, and clustering pixels in the foggy day traffic scene image based on two column vectors of dark channel characteristics and relative energy characteristics to obtain a clustering label and a clustering center;

(iii) determining the categories of all pixels in the foggy day traffic scene image according to the clustering center, thereby determining a sky area and a non-sky area;

(iv) and converting the clustering label from the one-dimensional column vector into a two-dimensional matrix to obtain a roughly-segmented image of the foggy day traffic scene image.

3) Fine segmentation of sky regions: selecting partial pixels with higher confidence degrees from the roughly divided sky area and non-sky area, and respectively using the partial pixels as positive and negative samples; and training a machine model to obtain a sky area fine classifier based on the dark channel characteristics and the relative energy characteristics of the positive and negative samples.

The errors of the sky region rough segmentation result are mostly concentrated at the junction of the sky region and the non-sky region, in order to obtain the fine segmentation result of the sky region, the sky region and the non-sky region with higher confidence coefficient are extracted from the rough segmentation result and are respectively used as positive and negative samples, then the dark channel characteristics and the relative energy characteristics of the positive and negative samples are sent to a Support Vector Machine (SVM), an SVM classifier is obtained after training, and the fine segmentation result with higher precision can be obtained by using the classifier. The method comprises the following specific steps:

(i) traversing the binary image obtained by roughly dividing the binary image into a sky area and a non-sky area according to a column, and recording the column number if the sky area is positioned at the upper part of the column and the ratio of the sky area to the height of the column is not less than 10%;

(ii) extracting a subsequence with the longest sky area from the column number recorded in the step (i), wherein 10% of pixels at the upper part of the subsequence are pixels with higher confidence coefficient of the sky area, and taking the pixel as a positive sample;

(iii) traversing the binary image obtained by roughly dividing the binary image into a sky area and a non-sky area according to a column, and recording the column number if the non-sky area is positioned at the lower part of the column and the ratio of the non-sky area to the column height is not less than 10%;

(iv) (iv) extracting the longest subsequence of the non-sky region from the recorded column number in (iii), wherein the lower 10% of pixels are pixels with higher confidence coefficient of the non-sky region, and taking the pixels as negative samples;

(v) training the SVM to obtain a fine classifier capable of distinguishing a sky area from a non-sky area according to the dark channel characteristics and the relative energy characteristics of the positive and negative samples and corresponding positive and negative sample labels;

(vi) and classifying pixels in the foggy day traffic scene image by using a fine sky area classifier based on the dark channel characteristics and the relative energy characteristics, and obtaining a corresponding binary image of fine sky area segmentation according to a classification result.

4) Determining a preset I-channel threshold

Based on the binary image obtained by subdivision in the step 3), dividing the preliminarily defogged traffic scene image into a sky area and a non-sky area, and counting the number N of sky area pixels in the preliminarily defogged traffic scene image_sky(ii) a Then calculating traffic scene images for preliminary defogging, sequencing the I channel pixel values from large to small, and selecting and arrangingN in the sequence_skyThe pixel value of each pixel is used as a preset I-channel threshold value T.

5) For the preliminarily defogged traffic scene image, each pixel value I of the I channel is used_IAll multiplied by 1T to obtain I_I'Using I_I'Replacing the original I-channel pixel value I_IThe overall brightness of the traffic scene image subjected to preliminary defogging is improved.

And step C, carrying out contrast-limiting adaptive histogram equalization and guide filtering processing on the image obtained in the step B to obtain a finally defogged traffic scene image.

1) Limited contrast adaptive histogram equalization

After the overall brightness of the traffic scene image subjected to preliminary defogging is improved in the step B, the visual effect of the area with the larger brightness value is better, and the visual effect of the area with the smaller brightness value is still not ideal. Therefore, this embodiment deals with the I channel component I of the image obtained in step B_I’And executing contrast-limited self-adaptive histogram equalization operation to improve the visual effect of the region with smaller brightness value. In this embodiment, the contrast enhancement limiting parameter is set to 0.001, and the required histogram shape parameter is set to "rayleigh", so that I with better overall visual effect can be obtained_I". Then the H is put_I、S_IAnd I_IThe components are converted back to the RGB color space, and a defogged image with a good visual effect can be obtained.

2) Guided filtering

After the limited contrast self-adaptive histogram equalization operation is performed on the preliminarily defogged traffic scene image obtained in the step B, the visual effect of the image is remarkably improved, but the problem of detail loss still exists. Therefore, the operation of guiding the filtering is further performed on the result of the last step of processing to highlight the detail information of the image. The radius parameter of the local window is set to 15, the regularization parameter is set to 0.004, and the defogged image with enhanced details can be obtained by directly processing in the RGB color space, namely the final defogged traffic scene image.

The invention also provides an apparatus comprising a processor and a memory; wherein: the memory is to store computer instructions; the processor is configured to execute the computer instructions stored in the memory, and in particular, to perform the method of the above-described method embodiment.

Example 1:

respectively extracting improved dark channel characteristics [ shown in figure 2(a) ] and improved relative energy characteristics [ shown in figure 2(b) ] of a foggy day image [ shown in figure 4(d) ]; then, according to the characteristics, obtaining a rough segmentation result [ shown in fig. 2(c) ] of the sky region by using K-means clustering; further, sky regions and non-sky regions with higher confidence are extracted from the rough segmentation result, and a fine segmentation result of the sky region is generated using the classification result [ as shown in fig. 2(d) ].

The atmospheric light values of the distant view region, the near view region and the transition region are calculated respectively, and smoothing processing is performed [ as shown in fig. 3(a) ]](ii) a Then, the transmission maps of the I channel and the S channel are calculated in the HSI color space, and the restoration results of the I channel and the S channel can be obtained according to the atmospheric light value and the transmission maps [ as shown in fig. 3(b) and fig. 3(c) ]](ii) a Finally, H is put_I、S_IAnd I_IThe component is transformed back to RGB color space to obtain the preliminary defogging result [ as shown in FIG. 3(d) ]]。

Calculating a threshold value K, and adding I_IMultiplying by 1/K can increase the global brightness of the image [ as shown in FIG. 4(a) ]](ii) a Then CLAHE is performed to obtain a defogged image with better visual effect [ as shown in FIG. 4(b) ]](ii) a Finally, the guiding filtering is performed to obtain the detail-enhanced defogged image [ as shown in FIG. 4(c) ]]。

Example 2:

respectively extracting improved dark channel characteristics [ shown in figure 5(a) ] and improved relative energy characteristics [ shown in figure 5(b) ] of a foggy day image [ shown in figure 7(d) ]; then, according to the characteristics, obtaining a rough segmentation result [ shown in fig. 5(c) ] of the sky region by using K-means clustering; further, sky regions and non-sky regions with higher confidence are extracted from the rough segmentation result, and a fine segmentation result of the sky region is generated using the classification result [ as shown in fig. 5(d) ].

Respectively calculating a distant view region, a close view region andthe atmospheric light value in the transition region is smoothed as shown in FIG. 6(a)](ii) a Then, the transmission maps of the I channel and the S channel are calculated in the HSI color space, and the restoration results of the I channel and the S channel can be obtained according to the atmospheric light value and the transmission maps [ as shown in fig. 6(b) and fig. 6(c) ]](ii) a Finally, H is put_I、S_IAnd I_IThe component is transformed back to RGB color space to obtain the preliminary defogging result [ as shown in FIG. 6(d) ]]。

Calculating a threshold value K, and adding I_IMultiplying by 1/K can increase the global brightness of the image [ as shown in FIG. 7(a) ]](ii) a Then CLAHE is performed to obtain a defogged image with better visual effect [ as shown in FIG. 7(b) ]](ii) a Finally, the guiding filtering is performed to obtain the detail-enhanced defogged image [ as shown in FIG. 7(c) ]]。

It should be noted that the above disclosure is only specific examples of the present invention, and those skilled in the art can devise various modifications according to the spirit and scope of the present invention.

1.一种雾天交通场景图像的去雾方法，其特征在于，包括以下步骤：1. a dehazing method of foggy traffic scene image, is characterized in that, comprises the following steps: 步骤A，针对远景区域、近景区域以及过渡区域不同的雾气浓度，分别计算雾天交通场景图像中对应区域的大气光值；然后在HSI颜色空间利用各通道的传输图和大气光值，根据大气散射模型计算初步去雾的交通场景图像；In step A, the atmospheric light values of the corresponding areas in the foggy traffic scene image are calculated respectively for different fog concentrations in the long-range, close-range and transitional areas; then the transmission map and atmospheric light values of each channel are used in the HSI color space, The scattering model calculates the initial dehazed traffic scene image; 步骤B，基于预设的I通道阈值，对初步去雾的交通场景图像进行全局亮度提升；Step B, based on the preset I channel threshold, perform global brightness enhancement on the initially dehazed traffic scene image; 其中，预设的I通道阈值，是根据初步去雾的交通场景图像中天空区域在HSI颜色空间的I通道像素设置得到；且，天空区域，是基于暗通道特征和相对能量特征对雾天交通场景图像进行分割得到；Among them, the preset I channel threshold is obtained according to the I channel pixel setting of the sky area in the HSI color space in the preliminary dehazing traffic scene image; and, the sky area is based on the dark channel characteristics and relative energy characteristics. The scene image is segmented to obtain; 步骤C，对步骤B得到的图像，进行限制对比度自适应直方图均衡化和引导滤波处理，得到最后去雾的交通场景图像。In step C, the image obtained in step B is subjected to limited contrast adaptive histogram equalization and guided filtering to obtain a final dehazed traffic scene image. 2.根据权利要求1所述的方法，其特征在于，所述暗通道特征的计算方法为：2. method according to claim 1, is characterized in that, the calculation method of described dark channel characteristic is:

其中，i和j分别是图像中像素的行号和列号，M是图像中所有行号之和，I是由r、g、b通道组成的图像，c表示r、g、b的任一颜色通道，I^c表示图像I的c颜色通道分量，I^c(i,j)表示图像I在像素(i,j)的c颜色通道分量；F_DC++表示图像I在像素(i,j)的暗通道特征。where i and j are the row and column numbers of the pixels in the image, respectively, M is the sum of all row numbers in the image, I is an image composed of r, g, and b channels, and c represents any one of r, g, and b. Color channel, I ^c represents the c color channel component of the image I, I ^c (i, j) represents the c color channel component of the image I at the pixel (i, j); F _DC++ represents the image I at the pixel (i, j) color channel component; Dark channel features. 3.根据权利要求1所述的方法，其特征在于，所述相对能量特征的计算方法为：3. The method according to claim 1, wherein the calculation method of the relative energy characteristic is:

其中，Z(i,j)为有关于高斯函数水平和垂直二阶导数的中间变量，α是Z(i,j)的最大值，k是对比度增益，τ是噪声阈值，

表示卷积运算，g_h和g_v分别是高斯函数的水平和垂直二阶导数；I(i,j)表示像素(i,j)的像素值，R、G、B分别为r、g、b通道分量，F_CE++表示图像I在像素(i,j)的暗通道特征。where Z(i,j) is the intermediate variable related to the horizontal and vertical second derivative of the Gaussian function, α is the maximum value of Z(i,j), k is the contrast gain, τ is the noise threshold,

Represents the convolution operation, g _h and g _v are the horizontal and vertical second-order derivatives of the Gaussian function respectively; I(i, j) represents the pixel value of the pixel (i, j), R, G, B are r, g, The b channel component, F _CE++ represents the dark channel feature of image I at pixel (i, j). 4.根据权利要求1所述的方法，其特征在于，基于暗通道特征和相对能量特征对雾天交通场景图像进行分割得到天空区域的方法为：4. The method according to claim 1, wherein the method for segmenting the foggy traffic scene image based on the dark channel feature and the relative energy feature to obtain the sky area is: a1，提取雾天交通场景图像的暗通道特征和相对能量特征；a1, extract the dark channel feature and relative energy feature of the foggy traffic scene image; a2，基于暗通道特征和相对能量特征，采用K均值聚类方法对雾天交通场景图像粗分割为天空区域和非天空区域；a2, based on the dark channel feature and relative energy feature, the K-means clustering method is used to roughly segment the foggy traffic scene image into sky area and non-sky area; a3，从粗分割的天空区域和非天空区域中选取置信度更高的部分像素，分别作为正负样本；并基于正负样本的暗通道特征和相对能量特征，训练机器模型得到天空区域细分类器；a3, select some pixels with higher confidence from the roughly segmented sky area and non-sky area, as positive and negative samples respectively; and based on the dark channel features and relative energy features of the positive and negative samples, train the machine model to obtain a sub-classification of the sky area device; a4，基于暗通道特征和相对能量特征，使用天空区域细分类器对雾天交通场景图像中的像素进行分类，并根据分类结果得到对应的天空区域细分割的二值图像。a4, based on the dark channel feature and relative energy feature, use the sky region sub-classifier to classify the pixels in the foggy traffic scene image, and obtain the corresponding sky region finely segmented binary image according to the classification result. 5.根据权利要求4所述的方法，其特征在于，采用K均值聚类方法对雾天交通场景图像粗分割的方法为：5. method according to claim 4, is characterized in that, the method that adopts K-means clustering method to rough segmentation of foggy traffic scene image is: b1，将雾天交通场景图像的暗通道特征和相对能量特征由二维矩阵转换为一维列向量，并执行正则化操作；b1, convert the dark channel feature and relative energy feature of the foggy traffic scene image from a two-dimensional matrix to a one-dimensional column vector, and perform a regularization operation; b2，设置K均值聚类的类别数为2，基于暗通道特征和相对能量特征这两个列向量，对雾天交通场景图像中的像素进行聚类，得到聚类标签和聚类中心；b2, set the number of categories of K-means clustering to 2, cluster the pixels in the foggy traffic scene image based on the two column vectors of the dark channel feature and the relative energy feature, and obtain the cluster label and cluster center; b3，根据聚类中心确定雾天交通场景图像中所有像素的类别，从而确定天空区域和非天空区域；b3, determine the category of all pixels in the foggy traffic scene image according to the cluster center, so as to determine the sky area and the non-sky area; b4，将聚类标签从一维列向量转换为二维矩阵，得到雾天交通场景图像的粗分割图像。b4, convert the cluster labels from a one-dimensional column vector to a two-dimensional matrix to obtain a coarsely segmented image of the foggy traffic scene image. 6.根据权利要求4所述的方法，其特征在于，从粗分割的天空区域和非天空区域中选取正负样本的方法为：6. method according to claim 4, is characterized in that, the method that selects positive and negative samples from the sky area of rough segmentation and non-sky area is: c1，将粗分割为天空区域和非天空区域得到的二值图像按列遍历，如果天空区域在该列的上部，且在该列高度的占比不少于预设比例，则记录该列号；c1, traverse the binary image obtained by roughly dividing the sky area and the non-sky area into columns. If the sky area is in the upper part of the column, and the proportion of the height of the column is not less than the preset ratio, record the column number ; c2，从c1记录的列号中提取天空区域最长的子序列，其上部预设比例的像素即为天空区域置信度更高的像素，将其作为正样本；c2, extract the longest subsequence in the sky area from the column number recorded by c1, and the pixels with the preset ratio in the upper part are the pixels with higher confidence in the sky area, which are regarded as positive samples; c3，将粗分割为天空区域和非天空区域得到的二值图像按列遍历，如果非天空区域在该列的下部，且在该列高度的占比不少于预设比例，则记录该列号；c3, traverse the binary image obtained by roughly dividing the sky area and the non-sky area into columns. If the non-sky area is in the lower part of the column, and the proportion of the height of the column is not less than the preset ratio, then record the column. No; c4，从c3记录的列号中提取非天空区域最长的子序列，其下部预设比例的像素即为非天空区域置信度更高的像素，将其作为负样本。c4, extract the longest subsequence in the non-sky area from the column number recorded in c3, and the lower preset ratio of pixels is the pixel with higher confidence in the non-sky area, which is regarded as a negative sample. 7.根据权利要求1所述的方法，其特征在于，预设的I通道阈值的设置方法为：7. method according to claim 1, is characterized in that, the setting method of preset I channel threshold value is: 计算初步去雾的交通场景图像中天空区域像素的数量N_sky；然后对初步去雾的交通场景图像，将其I通道像素值从大到小排序，取排序中第N_sky个像素的像素值作为预设的I通道阈值T；Calculate the number of sky area pixels N _sky in the preliminary dehazed traffic scene image; then sort the I channel pixel values from large to small for the preliminary dehazed traffic scene image, and take the pixel value of the Nth _sky pixel in the sorting as the preset I channel threshold T; 步骤B中所述的对初步去雾的交通场景图像进行全局亮度提升，具体为：对初步去雾的交通场景图像，将其I通道的每个像素值均与1/T相乘，得到的结果替换原来的I通道像素值。The global brightness enhancement of the preliminary dehazed traffic scene image described in step B is specifically: for the preliminary dehazed traffic scene image, each pixel value of the I channel is multiplied by 1/T, and the obtained The result replaces the original I channel pixel value. 8.根据权利要求1所述的方法，其特征在于，步骤A中所述的远景区域、近景区域以及过渡区域，划分方法为：基于暗通道特征对雾天交通场景图像中的所有像素进行聚类，从而将雾天交通场景图像划分为远景区域、近景区域以及过渡区域；8. The method according to claim 1, wherein the far-field area, the near-field area and the transition area described in step A are divided into: clustering all pixels in the foggy traffic scene image based on dark channel features. class, so that the foggy traffic scene image is divided into distant view area, close scene area and transition area; 雾天交通场景图像的大气光值计算方法为：采用四叉树分解法计算远景区域的大气光值，采用统计法计算近景区域的大气光值，采用线性插值计算过渡区域的大气光值，然后对所有得到的大气光值进行均值滤波处理。The calculation method of the atmospheric light value of the foggy traffic scene image is as follows: use the quadtree decomposition method to calculate the atmospheric light value of the distant view area, use the statistical method to calculate the atmospheric light value of the near view area, use the linear interpolation to calculate the atmospheric light value of the transition area, and then Mean filtering is performed on all obtained atmospheric light values. 9.根据权利要求1所述的方法，其特征在于，所述在HSI颜色空间利用各通道的传输图和大气光值，根据大气散射模型计算初步去雾的交通场景图像，具体为：9. The method according to claim 1, characterized in that, using the transmission map of each channel and the atmospheric light value in the HSI color space to calculate the preliminary dehazing traffic scene image according to the atmospheric scattering model, specifically: d1，将雾天交通场景图像从RGB颜色空间转换到HSI颜色空间，得到雾天交通场景图像的亮度分量I_J、饱和度分量S_J以及色度分量H_J；d1, convert the foggy traffic scene image from the RGB color space to the HSI color space to obtain the luminance component I _J , the saturation component S _J and the chrominance component H _J of the foggy traffic scene image; d2，计算雾天交通场景图像在I通道和S通道的传输图：d2, calculate the transmission map of the foggy traffic scene image in the I channel and the S channel:

其中，α为补偿系数，β为大气散射系数，e为自然常数，t为雾天交通场景图像在I通道的传输图，T为雾天交通场景图像在S通道的传输图，I_I是无雾交通场景图像的亮度分量，Among them, α is the compensation coefficient, β is the atmospheric scattering coefficient, e is a natural constant, t is the transmission map of the foggy traffic scene image in the I channel, T is the transmission map of the foggy traffic scene image in the S channel, I _I is no the luminance component of the foggy traffic scene image, d3，利用雾天交通场景图像在I通道的亮度分量I_J、传输图t以及大气光值I_A，根据大气散射模型计算初步去雾的交通场景图像的亮度分量：d3, using the brightness component I _J of the foggy traffic scene image in the I channel, the transmission map t and the atmospheric light value I _A , calculate the brightness component of the initially dehazed traffic scene image according to the atmospheric scattering model: I_J(i,j)＝I_I(i,j)t(i,j)+I_A(1-t(i,j))，I _J (i,j)=I _I (i,j)t(i,j)+I _A (1-t(i,j)), 通过该式计算得到的无雾交通场景图像的亮度分量I_I，即为初步去雾的交通场景图像的亮度分量；The brightness component _II of the fog-free traffic scene image calculated by this formula is the brightness component of the traffic scene image that is initially dehazed; 利用雾天交通场景图像在S通道的饱和度分量S_J、传输图T和大气光值S_A＝0，根据大气散射模型和饱和度的定义，计算初步去雾的交通场景图像的饱和度分量S_I：Using the saturation component S _J of the foggy traffic scene image in the S channel, the transmission map T and the atmospheric light value S _A =0, according to the definition of the atmospheric scattering model and saturation, the saturation component of the preliminary dehazing traffic scene image is calculated. S _I : S_J(i,j)＝S_I(i,j)T(i,j)+S_A(1-T(i,j))，S_A＝0；S _J (i,j)=S _I (i,j)T(i,j)+S _A (1-T(i,j)), S _A =0; 保持初步去雾的交通场景图像与雾天交通场景图像的H通道值不变，即H_I＝H_J；Keep the H channel values of the initially dehazed traffic scene image and the foggy traffic scene image unchanged, that is, H _I =H _J ; d4，利用得到的初步去雾的交通场景图像由HSI颜色空间的各通道分量H_I、S_I和I_I转换回RGB颜色空间，即可得到初步去雾的交通场景图像。d4, using the obtained preliminary dehazed traffic scene image to be converted back to RGB color space by each channel component H _I , S _I and _II of the HSI color space, and then the preliminary dehazed traffic scene image can be obtained. 10.一种设备，其特征在于，包括处理器和存储器；其中：所述存储器用于存储计算机指令；所述处理器用于执行所述存储器存储的计算机指令，具体执行如权利要求1-9任一所述的方法。10. A device, characterized by comprising a processor and a memory; wherein: the memory is used to store computer instructions; the processor is used to execute the computer instructions stored in the memory, specifically performing any one of claims 1-9. a described method.