CN103632153B - Region-based image saliency map extracting method - Google Patents

本发明公开了一种基于区域的图像显著图提取方法，其首先通过计算图像的全局颜色直方图，得到基于全局颜色直方图的图像显著图，然后采用超像素分割技术对图像进行分割，分别计算各个区域的颜色对比度和空间稀疏性，并利用区域之间的相似性进行加权，得到基于区域颜色对比度的图像显著图和基于区域空间稀疏性的图像显著图，最后对基于全局颜色直方图的图像显著图、基于区域颜色对比度的图像显著图和基于区域空间稀疏性的图像显著图进行融合，得到最终的图像显著图，优点是获得的图像显著图能够较好地反映全局和局部区域的显著变化情况，符合图像显著语义的特征。

The invention discloses a region-based image saliency map extraction method, which first obtains the image saliency map based on the global color histogram by calculating the global color histogram of the image, and then uses superpixel segmentation technology to segment the image, and calculates respectively The color contrast and spatial sparsity of each region are weighted by the similarity between regions to obtain the image saliency map based on the regional color contrast and the image saliency map based on the regional spatial sparsity, and finally the image based on the global color histogram The saliency map, the image saliency map based on regional color contrast and the image saliency map based on regional spatial sparsity are fused to obtain the final image saliency map. The advantage is that the obtained image saliency map can better reflect the significant changes in the global and local regions situation, which conforms to the characteristics of image salience semantics.

一种基于区域的图像显著图提取方法A region-based image saliency map extraction method

技术领域technical field

本发明涉及一种图像信号的处理方法，尤其是涉及一种基于区域的图像显著图提取方法。The invention relates to an image signal processing method, in particular to an area-based image saliency map extraction method.

背景技术Background technique

在人类视觉接收与信息处理中，由于大脑资源有限以及外界环境信息重要性区别，在处理过程中人脑对外界环境信息并不是一视同仁的，而是表现出选择特征。人们在观看图像或者视频片段时注意力并非均匀分布到图像的每个区域，而是对某些显著区域关注度更高。如何将视频中视觉注意度高的显著区域检测并提取出来是计算机视觉以及基于内容的视频检索领域的一个重要的研究内容。In human visual reception and information processing, due to limited brain resources and differences in the importance of external environmental information, the human brain does not treat external environmental information equally in the processing process, but shows selective characteristics. When people watch images or video clips, their attention is not evenly distributed to every area of the image, but they pay more attention to certain salient areas. How to detect and extract salient regions with high visual attention in videos is an important research content in the field of computer vision and content-based video retrieval.

现有的显著图模型是一种模拟生物体视觉注意机制的选择性注意模型，其通过计算每个像素点在颜色、亮度、方向方面与周边背景的对比，并将所有像素点的显著值构成一张显著图，然而这类方法并不能很好地提取图像显著图信息，这是因为基于像素的显著特征并不能很好地反映人眼观看时的显著语义特征，而基于区域的显著特征能够有效地提高提取的稳定性和准确性，因此，如何对图像进行区域分割，如何对各个区域的特征进行提取，如何对各个区域的显著特征进行描述，如何度量区域本身的显著度和区域与区域之间的显著度，都是对基于区域的显著图提取中需要研究解决的问题。The existing saliency map model is a selective attention model that simulates the visual attention mechanism of organisms. It calculates the contrast of each pixel with the surrounding background in terms of color, brightness, and direction, and constructs the saliency value of all pixels. A saliency map, however, this type of method cannot extract image saliency map information very well, because the salient features based on pixels cannot well reflect the salient semantic features of human eyes, while salient features based on regions can Effectively improve the stability and accuracy of extraction, therefore, how to segment the image, how to extract the features of each area, how to describe the salient features of each area, how to measure the salience of the area itself and the area and area The saliency between them is a problem that needs to be studied and solved in the region-based saliency map extraction.

发明内容Contents of the invention

本发明所要解决的技术问题是提供一种符合显著语义特征，且有较高提取稳定性和准确性的基于区域的图像显著图提取方法。The technical problem to be solved by the present invention is to provide a region-based image saliency map extraction method that conforms to salient semantic features and has high extraction stability and accuracy.

本发明解决上述技术问题所采用的技术方案为：一种基于区域的图像显著图提取方法，其特征在于包括以下步骤：The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a region-based image saliency map extraction method, which is characterized in that it comprises the following steps:

①将待处理的源图像记为{I_i(x,y)}，其中，i=1,2,3，1≤x≤W，1≤y≤H，W表示{I_i(x,y)}的宽，H表示{I_i(x,y)}的高，I_i(x,y)表示{I_i(x,y)}中坐标位置为(x,y)的像素点的第i个分量的颜色值，第1个分量为R分量、第2个分量为G分量和第3个分量为B分量；①Record the source image to be processed as {I _i (x,y)}, where i=1,2,3, 1≤x≤W, 1≤y≤H, W means {I _i (x,y) )}, H means the height of {I _i (x, y)}, I _i (x, y) means the pixel point whose coordinate position is (x, y) in {I _i (x, y)} The color value of the i component, the first component is the R component, the second component is the G component, and the third component is the B component;

②首先获取{I_i(x,y)}的量化图像及量化图像的全局颜色直方图，然后根据{I_i(x,y)}的量化图像，获取{I_i(x,y)}中的每个像素点的颜色种类，再根据{I_i(x,y)}的量化图像的全局颜色直方图和{I_i(x,y)}中的每个像素点的颜色种类，获取{I_i(x,y)}的基于全局颜色直方图的图像显著图，记为{HS(x,y)}，其中，HS(x,y)表示{HS(x,y)}中坐标位置为(x,y)的像素点的像素值，亦表示{I_i(x,y)}中坐标位置为(x,y)的像素点的基于全局颜色直方图的显著值；② First obtain the quantized image of {I _i (x, y)} and the global color histogram of the quantized image, and then according to the quantized image of { _{I i} (x, y)}, obtain the The color type of each pixel in {I _i (x, y)}, and then according to the global color histogram of the quantized image of {I _i (x, y)} and the color type of each pixel in {I i (x, y)}, get { The image saliency map based on the global color histogram of I _i (x,y)} is denoted as {HS(x,y)}, where HS(x,y) represents the coordinate position in {HS(x,y)} The pixel value of the pixel point (x, y) also represents the salient value based on the global color histogram of the pixel point whose coordinate position is (x, y) in {I _i (x, y)};

③采用超像素分割技术将{I_i(x,y)}分割成M个互不重叠的区域，然后将{I_i(x,y)}重新表示为M个区域的集合，记为{SP_h}，再计算{SP_h}中的各个区域之间的相似性，将{SP_h}中的第p个区域与第q个区域之间的相似性记为Sim(SP_p,SP_q)，其中，M≥1，SP_h表示{SP_h}中的第h个区域，1≤h≤M，1≤p≤M，1≤q≤M，p≠q，SP_p表示{SP_h}中的第p个区域，SP_q表示{SP_h}中的第q个区域；③ Use superpixel segmentation technology to divide {I _i (x, y)} into M non-overlapping regions, and then re-express {I _i (x, y)} as a set of M regions, denoted as {SP _h }, and then calculate the similarity between the regions in {SP _h }, and record the similarity between the p-th region and the q-th region in {SP _h } as Sim(SP _p ,SP _q ) , where, M≥1, SP _h represents the hth region in {SP _h }, 1≤h≤M, 1≤p≤M, 1≤q≤M, p≠q, SP _p represents {SP _h } The p-th area in , SP _q means the q-th area in {SP _h };

④根据{SP_h}中的各个区域之间的相似性，获取{I_i(x,y)}的基于区域颜色对比度的图像显著图，记为{NGC(x,y)}，其中，NGC(x,y)表示{NGC(x,y)}中坐标位置为(x,y)的像素点的像素值；④According to the similarity between each region in {SP _h }, obtain the image saliency map based on regional color contrast of {I _i (x,y)}, denoted as {NGC(x,y)}, where, NGC (x,y) indicates the pixel value of the pixel whose coordinate position is (x,y) in {NGC(x,y)};

⑤根据{SP_h}中的各个区域之间的相似性，获取{I_i(x,y)}的基于区域空间稀疏性的图像显著图，记为{NSS(x,y)}，其中，NSS(x,y)表示{NSS(x,y)}中坐标位置为(x,y)的像素点的像素值；⑤According to the similarity between the regions in {SP _h }, obtain the image saliency map based on the spatial sparsity of {I _i (x,y)}, denoted as {NSS(x,y)}, where, NSS(x,y) indicates the pixel value of the pixel point whose coordinate position is (x,y) in {NSS(x,y)};

⑥对{I_i(x,y)}的基于全局颜色直方图的图像显著图{HS(x,y)}、{I_i(x,y)}的基于区域颜色对比度的图像显著图{NGC(x,y)}及{I_i(x,y)}的基于区域空间稀疏性的图像显著图{NSS(x,y)}进行融合，得到{I_i(x,y)}的最终的图像显著图，记为{Sal(x,y)}，将{Sal(x,y)}中坐标位置为(x,y)的像素点的像素值记为Sal(x,y)，Sal(x,y)=HS(x,y)×NGC(x,y)×NSS(x,y)。⑥ Image saliency map {HS(x, y)} based on global color histogram of {I _i (x, y)}, image saliency map based on regional color contrast of {I _i (x, y)} {NGC (x,y)} and the image saliency map {NSS(x,y)} based on regional spatial sparsity of {I _i (x,y)} are fused to obtain the final {I _i (x,y)} The image saliency map is recorded as {Sal(x,y)}, and the pixel value of the pixel point whose coordinate position is (x,y) in {Sal(x,y)} is recorded as Sal(x,y), and Sal( x,y)=HS(x,y)×NGC(x,y)×NSS(x,y).

所述的步骤②的具体过程为：The concrete process of described step 2. is:

②-1、对{I_i(x,y)}中的每个像素点的各个分量的颜色值分别进行量化，得到{I_i(x,y)}的量化图像，记为{P_i(x,y)}，将{P_i(x,y)}中坐标位置为(x,y)的像素点的第i个分量的颜色值记为P_i(x,y)，其中，符号为向下取整符号；②-1. Quantize the color values of each component of each pixel in {I _i (x, y)} respectively to obtain the quantized image of {I _i (x, y)}, which is denoted as {P _i ( x, y)}, record the color value of the i-th component of the pixel whose coordinate position is (x, y) in {P _i (x, y)} as P _i (x, y), Among them, the symbol is the rounding down symbol;

②-2、计算{P_i(x,y)}的全局颜色直方图，记为{H(k)|0≤k≤4095}，其中，H(k)表示{P_i(x,y)}中属于第k种颜色的所有像素点的个数；②-2. Calculate the global color histogram of {P _i (x,y)}, recorded as {H(k)|0≤k≤4095}, where H(k) means {P _i (x,y) } in the number of all pixels belonging to the kth color;

②-3、根据{P_i(x,y)}中的每个像素点的各个分量的颜色值，计算{I_i(x,y)}中对应像素点的颜色种类，将{I_i(x,y)}中坐标位置为(x,y)的像素点的颜色种类记为k_xy，k_xy=P₃(x,y)×256+P₂(x,y)×16+P₁(x,y)，其中，P₃(x,y)表示{P_i(x,y)}中坐标位置为(x,y)的像素点的第3个分量的颜色值，P₂(x,y)表示{P_i(x,y)}中坐标位置为(x,y)的像素点的第2个分量的颜色值，P₁(x,y)表示{P_i(x,y)}中坐标位置为(x,y)的像素点的第1个分量的颜色值；②-3. According to the color value of each component of each pixel in {P _i (x, y)}, calculate the color type of the corresponding pixel in {I _i (x, y)}, and set {I _i ( The color type of the pixel whose coordinate position is (x,y) in x,y)} is recorded as k _xy , k _xy =P ₃ (x,y)×256+P ₂ (x,y)×16+P ₁ (x,y), where P ₃ (x,y) represents the color value of the third component of the pixel whose coordinate position is (x,y) in {P _i (x,y)}, P ₂ (x ,y) represents the color value of the second component of the pixel whose coordinate position is (x,y) in {P _i (x,y)}, and P ₁ (x,y) represents {P _i (x,y) } in the color value of the first component of the pixel whose coordinate position is (x, y);

②-4、计算{I_i(x,y)}中的每个像素点的基于全局颜色直方图的显著值，将{I_i(x,y)}中坐标位置为(x,y)的像素点的基于全局颜色直方图的显著值记为HS(x,y)， HS ( x , y ) = Σ k = 0 4095 ( H ( k ) × D ( k xy , k ) ) , D ( k xy , k ) = ( p k xy , 1 - p k , 1 ) 2 + ( p k xy , 2 - p k , 2 ) 2 + ( p k xy , 3 - p k , 3 ) 2 , 其中，D(k_xy,k)表示{H(k)|0≤k≤4095}中的第k_xy种颜色与第k种颜色之间的欧氏距离， p k xy , 2 = mod ( k xy / 16 ) , p_k,2=mod(k/16)， 表示{H(k)|0≤k≤4095}中的第k_xy种颜色对应的第1个分量的颜色值，表示{H(k)|0≤k≤4095}中的第k_xy种颜色对应的第2个分量的颜色值，表示{H(k)|0≤k≤4095}中的第k_xy种颜色对应的第3个分量的颜色值，p_k,1表示{H(k)|0≤k≤4095}中的第k种颜色对应的第1个分量的颜色值，p_k,2表示{H(k)|0≤k≤4095}中的第k种颜色对应的第2个分量的颜色值，p_k,3表示{H(k)|0≤k≤4095}中的第k种颜色对应的第3个分量的颜色值，mod()为取余数操作函数；②-4. Calculate the salient value of each pixel in {I _i (x, y)} based on the global color histogram, and set the coordinate position in {I _i (x, y)} as (x, y) The salient value of the pixel based on the global color histogram is recorded as HS(x,y), HS ( x , the y ) = Σ k = 0 4095 ( h ( k ) × D. ( k xy , k ) ) , D. ( k xy , k ) = ( p k xy , 1 - p k , 1 ) 2 + ( p k xy , 2 - p k , 2 ) 2 + ( p k xy , 3 - p k , 3 ) 2 , Among them, D(k _xy ,k) represents the Euclidean distance between the k _xy color and the k color in {H(k)|0≤k≤4095}, p k xy , 2 = mod ( k xy / 16 ) , p _k,2 = mod(k/16), Indicates the color value of the first component corresponding to the k _xy color in {H(k)|0≤k≤4095}, Indicates the color value of the second component corresponding to the k _xy color in {H(k)|0≤k≤4095}, Represents the color value of the third component corresponding to the k _xy color in {H(k)|0≤k≤4095}, p _k,1 represents the first color in {H(k)|0≤k≤4095} The color value of the first component corresponding to k colors, p _k,2 means the color value of the second component corresponding to the k color in {H(k)|0≤k≤4095}, p _k,3 Indicates the color value of the third component corresponding to the kth color in {H(k)|0≤k≤4095}, and mod() is the remainder operation function;

②-5、根据{I_i(x,y)}中的每个像素点的基于全局颜色直方图的显著值，得到{I_i(x,y)}的基于全局颜色直方图的图像显著图，记为{HS(x,y)}。②-5. According to the saliency value based on the global color histogram of each pixel in {I _i (x, y)}, the image saliency map based on the global color histogram of {I _i (x, y)} is obtained , denoted as {HS(x,y)}.

所述的步骤③中{SP_h}中的第p个区域与第q个区域之间的相似性Sim(SP_p,SP_q)的获取过程为：The acquisition process of the similarity Sim(SP _p , SP _q ) between the p-th region and the q-th region in {SP _h } in the step ③ is:

③-1、对{SP_h}中的每个区域中的每个像素点的各个分量的颜色值分别进行量化，得到{SP_h}中的每个区域的量化区域，将{SP_h}中的第h个区域的量化区域记为{P_h,i(x_h,y_h)}，将{P_h,i(x_h,y_h)}中坐标位置为(x_h,y_h)的像素点的第i个分量的颜色值记为P_h,i(x_h,y_h)，假设{P_h,i(x_h,y_h)}中坐标位置为(x_h,y_h)的像素点在{I_i(x,y)}中的坐标位置为(x,y)，则其中，1≤x_h≤W_h,1≤y_h≤H_h，W_h表示{SP_h}中的第h个区域的宽度，H_h表示{SP_h}中的第h个区域的高度，符号为向下取整符号；③-1. Quantize the color values of each component of each pixel in each region in {SP _h }, respectively, to obtain the quantized region of each region in {SP _h }, and convert {SP _h } The quantization area of the h-th area of is recorded as {P _h,i (x _h ,y _h )}, and the coordinate position in {P _h,i (x _h ,y _h )} is (x _h ,y _h ) The color value of the i-th component of the pixel is recorded as P _h,i (x _h ,y _h ), assuming that the coordinate position in {P _h,i (x _h ,y _h )} is (x _h ,y _h ) The coordinate position of the pixel in {I _i (x, y)} is (x, y), then Among them, 1≤x _h ≤W _h , 1≤y _h ≤H _h , W _h represents the width of the h-th region in {SP _h }, H _h represents the height of the h-th region in {SP _h }, symbol is the rounding down symbol;

③-2、计算{SP_h}中的每个区域的量化区域的颜色直方图，将{P_h,i(x_h,y_h)}的颜色直方图记为其中，表示{P_h,i(x_h,y_h)}中属于第k种颜色的所有像素点的个数；③-2. Calculate the color histogram of the quantized region of each region in {SP _h }, and mark the color histogram of {P _h,i (x _h ,y _h )} as in, Indicates the number of all pixels belonging to the kth color in {P _h,i (x _h ,y _h )};

③-3、对{SP_h}中的每个区域的量化区域的颜色直方图进行归一化操作，得到对应的归一化后的颜色直方图，将对进行归一化操作后得到的归一化后的颜色直方图记为 { H ′ SP h ( k ) | 0 ≤ k ≤ 4095 } , H ′ SP h ( k ) = H SP h ( k ) Σ h ′ = 1 M H SP h ′ ( k ) , 其中，表示{SP_h}中的第h个区域的量化区域{P_h,i(x_h,y_h)}中属于第k种颜色的像素点的出现概率，表示{SP_h}中的第h'个区域的量化区域{P_h',i(x_h',y_h')}中属于第k种颜色的所有像素点的个数，1≤x_h'≤W_h',1≤y_h'≤H_h'，W_h'表示{SP_h}中的第h'个区域的宽度，H_h'表示{SP_h}中的第h'个区域的高度，P_h',i(x_h',y_h')表示{P_h',i(x_h',y_h')}中坐标位置为(x_h',y_h')的像素点的第i个分量的颜色值；③-3. Perform a normalization operation on the color histogram of the quantized region of each region in {SP _h } to obtain the corresponding normalized color histogram, which will be The normalized color histogram obtained after the normalization operation is marked as { h ′ SP h ( k ) | 0 ≤ k ≤ 4095 } , h ′ SP h ( k ) = h SP h ( k ) Σ h ′ = 1 m h SP h ′ ( k ) , in, Indicates the occurrence probability of pixels belonging to the k-th color in the quantized area {P _h,i (x _h ,y _h )} of the h-th area in {SP _h }, Indicates the number of all pixels belonging to the k-th color in the quantized area {P _h',i (x _h' ,y _h' )} of the h'th area in {SP _h }, 1≤x _h' ≤W h _' ,1≤y _h' ≤H _h' , W _h' represents the width of the h'th region in {SP _h }, H _h' represents the height of the h'th region in {SP _h } , P _h',i (x _h' ,y _h' ) means the pixel point whose coordinate position is (x _h' ,y _h' ) in {P _h',i (x _h' ,y _h' )} The color value of the i component;

③-4、计算{SP_h}中的第p个区域与第q个区域之间的相似性，记为Sim(SP_p,SP_q)，Sim(SP_p,SP_q)=Sim_c(SP_p,SP_q)×Sim_d(SP_p,SP_q)，Sim_c(SP_p,SP_q)表示{SP_h}中的第p个区域与{SP_h}中的第q区域之间的颜色相似性， Sim c ( SP p , SP q ) = Σ k = 0 4095 min ( H ′ SP p ( k ) , H ′ SP q ( k ) ) , Sim_d(SP_p,SP_q)表示{SP_h}中的第p个区域与{SP_h}中的第q区域之间的空间相似性，其中，SP_p表示{SP_h}中的第p个区域，SP_q表示{SP_h}中的第q个区域，表示{SP_h}中的第p个区域的量化区域{P_p,i(x_p,y_p)}中属于第k种颜色的像素点的出现概率，表示{SP_h}中的第q个区域的量化区域{P_q,i(x_q,y_q)}中属于第k种颜色的像素点的出现概率，1≤x_p≤W_p,1≤y_p≤H_p，W_p表示{SP_h}中的第p个区域的宽度，H_p表示{SP_h}中的第p个区域的高度，P_p,i(x_p,y_p)表示{P_p,i(x_p,y_p)}中坐标位置为(x_p,y_p)的像素点的第i个分量的颜色值，1≤x_q≤W_q,1≤y_q≤H_q，W_q表示{SP_h}中的第q个区域的宽度，H_q表示{SP_h}中的第q个区域的高度，P_q,i(x_q,y_q)表示{P_q,i(x_q,y_q)}中坐标位置为(x_q,y_q)的像素点的第i个分量的颜色值，min()为取最小值函数，表示{SP_h}中的第p个区域中的中心像素点的坐标位置，表示{SP_h}中的第q个区域中的中心像素点的坐标位置，符号“‖‖”为求欧式距离符号。③-4. Calculate the similarity between the p-th region and the q-th region in {SP _h }, denoted as Sim(SP _p ,SP _q ), Sim(SP _p ,SP _q )=Sim _c (SP _p ,SP _q )×Sim _d (SP _p ,SP _q ), Sim _c (SP _p ,SP _q ) represents the color between the p-th region in {SP _h } and the q-th region in {SP _h } similarity, Sim c ( SP p , SP q ) = Σ k = 0 4095 min ( h ′ SP p ( k ) , h ′ SP q ( k ) ) , Sim _d (SP _p ,SP _q ) denotes the spatial similarity between the p-th region in {SP _h } and the q-th region in {SP _h }, where, SP _p represents the p-th region in {SP _h }, SP _q represents the q-th region in {SP _h }, Indicates the occurrence probability of pixels belonging to the k-th color in the quantized area {P _p,i (x _p ,y _p )} of the p-th area in {SP _h }, Indicates the occurrence probability of pixels belonging to the kth color in the quantized area {P _q,i (x _q ,y _q )} of the qth area in {SP _h }, 1≤x _p ≤W _p ,1≤ y _p ≤ H _p , W _p represents the width of the p-th region in {SP _h }, H _p represents the height of the p-th region in {SP _h }, P _p,i (x _p ,y _p ) represents The color value of the i-th component of the pixel whose coordinate position is (x _p , y _p ) in {P _p,i (x _p ,y _p )}, 1≤x _q ≤W _q ,1≤y _q ≤H _q , W _q represents the width of the qth region in {SP _h }, H _q represents the height of the qth region in {SP _h }, P _q,i (x _q ,y _q ) represents {P _{q, i} (x _q ,y _q )} is the color value of the i-th component of the pixel whose coordinate position is (x _q ,y _q ), min() is the minimum value function, Indicates the coordinate position of the central pixel point in the p-th region in {SP _h }, Indicates the coordinate position of the central pixel point in the qth region in {SP _h }, and the symbol "‖‖" is the Euclidean distance symbol.

所述的步骤④的具体过程为：The concrete process of described step 4. is:

④-1、计算{SP_h}中的每个区域的颜色对比度，将{SP_h}中的第h个区域的颜色对比度记为 NGC SP h = Σ q = 1 M W ( SP h , SP q ) × | | m SP h - m SP q | | , 其中，SP_h表示{SP_h}中的第h个区域，SP_q表示{SP_h}中的第q个区域，表示{SP_h}中的第h个区域中包含的像素点的总个数，Sim_d(SP_h,SP_q)表示{SP_h}中的第h个区域与{SP_h}中的第q区域之间的空间相似性， 表示{SP_h}中的第h个区域中的中心像素点的坐标位置，表示{SP_h}中的第q个区域中的中心像素点的坐标位置，符号“‖‖”为求欧式距离符号，表示{SP_h}中的第h个区域的颜色均值向量，表示{SP_h}中的第q个区域的颜色均值向量；④-1. Calculate the color contrast of each region in {SP _h }, and record the color contrast of the hth region in {SP _h } as NGC SP h = Σ q = 1 m W ( SP h , SP q ) × | | m SP h - m SP q | | , where, SP _h represents the h-th region in {SP _h }, SP _q represents the q-th region in {SP _h }, Indicates the total number of pixels contained in the h-th area in {SP _h }, Sim _d (SP _h ,SP _q ) means the h-th area in {SP _h } and the q-th area in {SP _h } the spatial similarity between regions, Indicates the coordinate position of the central pixel point in the h-th region in {SP _h }, Indicates the coordinate position of the central pixel point in the qth region in {SP _h }, the symbol "‖‖" is the Euclidean distance symbol, represents the color mean vector of the h-th region in {SP _h }, Represents the color mean vector of the qth region in {SP _h };

④-2、对{SP_h}中的每个区域的颜色对比度进行归一化操作，得到对应的归一化后的颜色对比度，将对{SP_h}中的第h个区域的颜色对比度进行归一化操作后得到的归一化后的颜色对比度记为 其中，NGC_min表示{SP_h}中的M个区域中最小的颜色对比度，NGC_max表示{SP_h}中的M个区域中最大的颜色对比度；④-2. Perform a normalization operation on the color contrast of each region in {SP _h } to obtain the corresponding normalized color contrast, and then perform a normalization operation on the color contrast of the hth region in {SP _h } The normalized color contrast obtained after the normalization operation is recorded as Among them, NGC _min represents the minimum color contrast among the M regions in {SP _h }, and NGC _max represents the maximum color contrast among the M regions in {SP _h };

④-3、计算{SP_h}中的每个区域的基于颜色对比度的显著值，将{SP_h}中的第h个区域的基于颜色对比度的显著值记为 NGC ′ ′ SP h = Σ q = 1 M ( Sim ( SP h , SP q ) × NGC ′ SP h ) Σ q = 1 M Sim ( SP h , SP q ) , 其中，Sim(SP_h,SP_q)表示{SP_h}中的第h个区域与第q个区域之间的相似性；④-3. Calculate the saliency value based on color contrast of each region in {SP _h }, and record the saliency value based on color contrast of the hth region in {SP _h } as NGC ′ ′ SP h = Σ q = 1 m ( Sim ( SP h , SP q ) × NGC ′ SP h ) Σ q = 1 m Sim ( SP h , SP q ) , Among them, Sim(SP _h , SP _q ) represents the similarity between the h-th region and the q-th region in {SP _h };

④-4、将{SP_h}中的每个区域的基于颜色对比度的显著值作为对应区域中的所有像素点的显著值，从而得到{I_i(x,y)}的基于区域颜色对比度的图像显著图，记为{NGC(x,y)}，其中，NGC(x,y)表示{NGC(x,y)}中坐标位置为(x,y)的像素点的像素值。④-4. Take the saliency value based on color contrast of each region in {SP _h } as the saliency value of all pixels in the corresponding region, so as to obtain the saliency value based on region color contrast of {I _i (x, y)} The image saliency map is denoted as {NGC(x,y)}, where NGC(x,y) represents the pixel value of the pixel at the coordinate position (x,y) in {NGC(x,y)}.

所述的步骤⑤的具体过程为：The concrete process of described step 5. is:

⑤-1、计算{SP_h}中的每个区域的空间稀疏性，将{SP_h}中的第h个区域的空间稀疏性记为 NSS SP h = Σ q = 1 M ( Sim ( SP h , SP q ) × D SP h ) Σ q = 1 M Sim ( SP h , SP q ) , 其中，Sim(SP_h,SP_q)表示{SP_h}中的第h个区域与第q个区域之间的相似性，表示{SP_h}中的第h个区域中的中心像素点与{I_i(x,y)}的中心像素点之间的欧氏距离；⑤-1. Calculate the spatial sparsity of each region in {SP _h }, and record the spatial sparsity of the hth region in {SP _h } as NSS SP h = Σ q = 1 m ( Sim ( SP h , SP q ) × D. SP h ) Σ q = 1 m Sim ( SP h , SP q ) , Among them, Sim(SP _h , SP _q ) represents the similarity between the h-th region and the q-th region in {SP _h }, Indicates the Euclidean distance between the central pixel point in the h-th region in {SP _h } and the central pixel point in {I _i (x,y)};

⑤-2、对{SP_h}中的每个区域的空间稀疏性进行归一化操作，得到对应的归一化后的空间稀疏性，将对{SP_h}中的第h个区域的空间稀疏性进行归一化操作后得到的归一化后的空间稀疏性记为 其中，NSS_min表示{SP_h}中的M个区域中最小的空间稀疏性，NSS_max表示{SP_h}中的M个区域中最大的空间稀疏性；⑤-2. Perform a normalization operation on the spatial sparsity of each region in {SP _h } to obtain the corresponding normalized spatial sparsity, and the space of the hth region in {SP _h } sparsity The normalized spatial sparsity obtained after the normalization operation is denoted as Among them, NSS _min represents the smallest spatial sparsity among the M regions in {SP _h }, and NSS _max represents the largest spatial sparsity among the M regions in {SP _h };

⑤-3、计算{SP_h}中的每个区域的基于空间稀疏性的显著值，将{SP_h}中的第h个区域的基于空间稀疏性的显著值记为 NSS ′ ′ SP h = Σ q = 1 M ( Sim ( SP h , SP q ) × NSS ′ SP h ) Σ q = 1 M Sim ( SP h , SP q ) ; ⑤-3. Calculate the saliency value based on spatial sparsity of each region in {SP _h }, and record the saliency value based on spatial sparsity of the hth region in {SP _h } as NSS ′ ′ SP h = Σ q = 1 m ( Sim ( SP h , SP q ) × NSS ′ SP h ) Σ q = 1 m Sim ( SP h , SP q ) ;

⑤-4、将{SP_h}中的每个区域的基于空间稀疏性的显著值作为对应区域中的所有像素点的显著值，从而得到{I_i(x,y)}的基于区域空间稀疏性的图像显著图，记为{NSS(x,y)}，其中，NSS(x,y)表示{NSS(x,y)}中坐标位置为(x,y)的像素点的像素值。⑤-4. Take the saliency value based on spatial sparsity of each region in {SP _h } as the saliency value of all pixels in the corresponding region, so as to obtain the region-based spatial sparsity of {I _i (x,y)} The characteristic image saliency map is denoted as {NSS(x,y)}, where NSS(x,y) represents the pixel value of the pixel at the coordinate position (x,y) in {NSS(x,y)}.

与现有技术相比，本发明的优点在于：Compared with the prior art, the present invention has the advantages of:

1）本发明方法通过分别计算得到基于全局颜色直方图的图像显著图、基于区域颜色对比度的图像显著图和基于区域空间稀疏性的图像显著图，并最终融合得到图像显著图，所获得的图像显著图能够较好地反映图像的全局和局部区域的显著变化情况，且稳定性和准确性高。1) The method of the present invention obtains the image saliency map based on the global color histogram, the image saliency map based on the regional color contrast and the image saliency map based on the regional spatial sparsity by calculating respectively, and finally fuses the image saliency map, and the obtained image The saliency map can better reflect the significant changes in the global and local regions of the image, and has high stability and accuracy.

2）本发明方法采用超像素分割技术对图像进行分割，并利用直方图特征分别计算各个区域的颜色对比度和空间稀疏性，最终利用区域之间的相似性进行加权，得到最终的基于区域的图像显著图，这样能够提取符合显著语义的特征信息。2) The method of the present invention uses superpixel segmentation technology to segment the image, and uses the histogram feature to calculate the color contrast and spatial sparsity of each region, and finally uses the similarity between regions to weight to obtain the final region-based image Saliency map, which can extract feature information that conforms to saliency semantics.

附图说明Description of drawings

图1为本发明方法的总体实现框图；Fig. 1 is the overall realization block diagram of the inventive method;

图2a为“Image1”的原始图像；Figure 2a is the original image of "Image1";

图2b为“Image1”图像的真实（Ground truth）显著图；Figure 2b is the ground truth saliency map of the "Image1" image;

图2c为“Image1”图像的基于全局颜色直方图的图像显著图；Figure 2c is an image saliency map based on the global color histogram of the image "Image1";

图2d为“Image1”图像的基于区域颜色对比度的图像显著图；Figure 2d is the image saliency map based on the regional color contrast of the "Image1" image;

图2e为“Image1”图像的基于区域空间稀疏性的图像显著图；Fig. 2e is an image saliency map based on regional spatial sparsity of the image "Image1";

图2f为“Image1”图像最终的图像显著图；Figure 2f is the final image saliency map of the "Image1" image;

图3a为“Image2”的原始图像；Figure 3a is the original image of "Image2";

图3b为“Image2”图像的真实（Ground truth）显著图；Figure 3b is the ground truth saliency map of the "Image2" image;

图3c为“Image2”图像的基于全局颜色直方图的图像显著图；Figure 3c is an image saliency map based on the global color histogram of the "Image2" image;

图3d为“Image2”图像的基于区域颜色对比度的图像显著图；Figure 3d is an image saliency map based on the regional color contrast of the "Image2" image;

图3e为“Image2”图像的基于区域空间稀疏性的图像显著图；Fig. 3e is the image saliency map based on the spatial sparsity of the "Image2" image;

图3f为“Image2”图像最终的图像显著图；Figure 3f is the final image saliency map of the "Image2" image;

图4a为“Image3”的原始图像；Figure 4a is the original image of "Image3";

图4b为“Image3”图像的真实（Ground truth）显著图；Figure 4b is the real (Ground truth) saliency map of the "Image3" image;

图4c为“Image3”图像的基于全局颜色直方图的图像显著图；Figure 4c is an image saliency map based on the global color histogram of the "Image3" image;

图4d为“Image3”图像的基于区域颜色对比度的图像显著图；Figure 4d is an image saliency map based on regional color contrast of the "Image3" image;

图4e为“Image3”图像的基于区域空间稀疏性的图像显著图；Fig. 4e is an image saliency map based on regional spatial sparsity of the "Image3" image;

图4f为“Image3”图像最终的图像显著图；Figure 4f is the final image saliency map of the "Image3" image;

图5a为“Image4”的原始图像；Figure 5a is the original image of "Image4";

图5b为“Image4”图像的真实（Ground truth）显著图；Figure 5b is the ground truth saliency map of the "Image4" image;

图5c为“Image4”图像的基于全局颜色直方图的图像显著图；Figure 5c is an image saliency map based on the global color histogram of the "Image4" image;

图5d为“Image4”图像的基于区域颜色对比度的图像显著图；Figure 5d is the image saliency map based on the regional color contrast of the "Image4" image;

图5e为“Image4”图像的基于区域空间稀疏性的图像显著图；Fig. 5e is an image saliency map based on regional spatial sparsity of the "Image4" image;

图5f为“Image4”图像最终的图像显著图；Figure 5f is the final image saliency map of the "Image4" image;

图6a为“Image5”的原始图像；Figure 6a is the original image of "Image5";

图6b为“Image5”图像的真实（Ground truth）显著图；Figure 6b is the ground truth saliency map of the "Image5" image;

图6c为“Image5”图像的基于全局颜色直方图的图像显著图；Figure 6c is an image saliency map based on the global color histogram of the "Image5" image;

图6d为“Image5”图像的基于区域颜色对比度的图像显著图；Figure 6d is an image saliency map based on regional color contrast of the "Image5" image;

图6e为“Image5”图像的基于区域空间稀疏性的图像显著图；Fig. 6e is an image saliency map based on regional spatial sparsity of the "Image5" image;

图6f为“Image5”图像最终的图像显著图。Figure 6f is the final image saliency map of the "Image5" image.

具体实施方式detailed description

以下结合附图实施例对本发明作进一步详细描述。The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

本发明提出的一种基于区域的图像显著图提取方法，其总体实现框图如图1所示，其包括以下步骤：A kind of region-based image saliency map extraction method proposed by the present invention, its overall realization block diagram is as shown in Figure 1, and it comprises the following steps:

①将待处理的源图像记为{I_i(x,y)}，其中，i=1,2,3，1≤x≤W，1≤y≤H，W表示{I_i(x,y)}的宽，H表示{I_i(x,y)}的高，I_i(x,y)表示{I_i(x,y)}中坐标位置为(x,y)的像素点的第i个分量的颜色值，第1个分量为R分量、第2个分量为G分量和第3个分量为B分量。①Record the source image to be processed as {I _i (x,y)}, where i=1,2,3, 1≤x≤W, 1≤y≤H, W means {I _i (x,y) )}, H means the height of {I _i (x, y)}, I _i (x, y) means the pixel point whose coordinate position is (x, y) in {I _i (x, y)} The color value of the i component, the first component is the R component, the second component is the G component, and the third component is the B component.

②如果只考虑局部显著性，则图像中变化较剧烈的边缘或复杂的背景区域显著性较高，而平滑的目标区域内部显著性较低，这样还需要考虑全局显著性，全局显著性是指各像素点相对于全局图像的显著程度，因此本发明首先获取{I_i(x,y)}的量化图像及量化图像的全局颜色直方图，然后根据{I_i(x,y)}的量化图像，获取{I_i(x,y)}中的每个像素点的颜色种类，再根据{I_i(x,y)}的量化图像的全局颜色直方图和{I_i(x,y)}中的每个像素点的颜色种类，获取{I_i(x,y)}的基于全局颜色直方图的图像显著图，记为{HS(x,y)}，其中，HS(x,y)表示{HS(x,y)}中坐标位置为(x,y)的像素点的像素值，亦表示{I_i(x,y)}中坐标位置为(x,y)的像素点的基于全局颜色直方图的显著值。②If only local saliency is considered, then sharply changing edges or complex background areas in the image have higher saliency, while smooth target areas have lower saliency. In this way, global saliency needs to be considered. Global saliency refers to The significant degree of each pixel relative to the global image, so the present invention first obtains the quantized image of {I _i (x, y)} and the global color histogram of the quantized image, and then quantizes according to {I _i (x, y)} Image, get the color type of each pixel in {I _i (x, y)}, and then according to the global color histogram of the quantized image of {I _i (x, y)} and {I _i (x, y) }, get the image saliency map based on the global color histogram of {I _i (x,y)}, denoted as {HS(x,y)}, where, HS(x,y ) represents the pixel value of the pixel whose coordinate position is (x, y) in {HS(x, y)}, and also represents the pixel value of the pixel whose coordinate position is (x, y) in {I _i (x, y)} Salient values based on the global color histogram.

在此具体实施例中，步骤②的具体过程为：In this specific embodiment, the concrete process of step 2. is:

②-1、对{I_i(x,y)}中的每个像素点的各个分量的颜色值分别进行量化，得到{I_i(x,y)}的量化图像，记为{P_i(x,y)}，将{P_i(x,y)}中坐标位置为(x,y)的像素点的第i个分量的颜色值记为P_i(x,y)，其中，符号为向下取整符号。②-1. Quantize the color values of each component of each pixel in {I _i (x, y)} respectively to obtain the quantized image of {I _i (x, y)}, which is denoted as {P _i ( x, y)}, record the color value of the i-th component of the pixel whose coordinate position is (x, y) in {P _i (x, y)} as P _i (x, y), Among them, the symbol is the rounding down symbol.

②-2、计算{P_i(x,y)}的全局颜色直方图，记为{H(k)|0≤k≤4095}，其中，H(k)表示{P_i(x,y)}中属于第k种颜色的所有像素点的个数。②-2. Calculate the global color histogram of {P _i (x,y)}, recorded as {H(k)|0≤k≤4095}, where H(k) means {P _i (x,y) } in the number of all pixels belonging to the kth color.

②-3、根据{P_i(x,y)}中的每个像素点的各个分量的颜色值，计算{I_i(x,y)}中对应像素点的颜色种类，将{I_i(x,y)}中坐标位置为(x,y)的像素点的颜色种类记为k_xy，k_xy=P₃(x,y)×256+P₂(x,y)×16+P₁(x,y)，其中，P₃(x,y)表示{P_i(x,y)}中坐标位置为(x,y)的像素点的第3个分量的颜色值，P₂(x,y)表示{P_i(x,y)}中坐标位置为(x,y)的像素点的第2个分量的颜色值，P₁(x,y)表示{P_i(x,y)}中坐标位置为(x,y)的像素点的第1个分量的颜色值。②-3. According to the color value of each component of each pixel in {P _i (x, y)}, calculate the color type of the corresponding pixel in {I _i (x, y)}, and set {I _i ( The color type of the pixel whose coordinate position is (x,y) in x,y)} is recorded as k _xy , k _xy =P ₃ (x,y)×256+P ₂ (x,y)×16+P ₁ (x,y), where P ₃ (x,y) represents the color value of the third component of the pixel whose coordinate position is (x,y) in {P _i (x,y)}, P ₂ (x ,y) represents the color value of the second component of the pixel whose coordinate position is (x,y) in {P _i (x,y)}, and P ₁ (x,y) represents {P _i (x,y) } in the color value of the first component of the pixel whose coordinate position is (x, y).

②-4、计算{I_i(x,y)}中的每个像素点的基于全局颜色直方图的显著值，将{I_i(x,y)}中坐标位置为(x,y)的像素点的基于全局颜色直方图的显著值记为HS(x,y)， HS ( x , y ) = Σ k = 0 4095 ( H ( k ) × D ( k xy , k ) ) , D ( k xy , k ) = ( p k xy , 1 - p k , 1 ) 2 + ( p k xy , 2 - p k , 2 ) 2 + ( p k xy , 3 - p k , 3 ) 2 , 其中，D(k_xy,k)表示{H(k)|0≤k≤4095}中的第k_xy种颜色与第k种颜色之间的欧氏距离， p k xy , 2 = mod ( k xy / 16 ) , p_k,2=mod(k/16)， 表示{H(k)|0≤k≤4095}中的第k_xy种颜色对应的第1个分量的颜色值，表示{H(k)|0≤k≤4095}中的第k_xy种颜色对应的第2个分量的颜色值，表示{H(k)|0≤k≤4095}中的第k_xy种颜色对应的第3个分量的颜色值，p_k,1表示{H(k)|0≤k≤4095}中的第k种颜色对应的第1个分量的颜色值，p_k,2表示{H(k)|0≤k≤4095}中的第k种颜色对应的第2个分量的颜色值，p_k,3表示{H(k)|0≤k≤4095}中的第k种颜色对应的第3个分量的颜色值，mod()为取余数操作函数。②-4. Calculate the salient value of each pixel in {I _i (x, y)} based on the global color histogram, and set the coordinate position in {I _i (x, y)} as (x, y) The salient value of the pixel based on the global color histogram is recorded as HS(x,y), HS ( x , the y ) = Σ k = 0 4095 ( h ( k ) × D. ( k xy , k ) ) , D. ( k xy , k ) = ( p k xy , 1 - p k , 1 ) 2 + ( p k xy , 2 - p k , 2 ) 2 + ( p k xy , 3 - p k , 3 ) 2 , Among them, D(k _xy ,k) represents the Euclidean distance between the k _xy color and the k color in {H(k)|0≤k≤4095}, p k xy , 2 = mod ( k xy / 16 ) , p _k,2 = mod(k/16), Indicates the color value of the first component corresponding to the k _xy color in {H(k)|0≤k≤4095}, Indicates the color value of the second component corresponding to the k _xy color in {H(k)|0≤k≤4095}, Represents the color value of the third component corresponding to the k _xy color in {H(k)|0≤k≤4095}, p _k,1 represents the first color in {H(k)|0≤k≤4095} The color value of the first component corresponding to k colors, p _k,2 means the color value of the second component corresponding to the k color in {H(k)|0≤k≤4095}, p _k,3 Indicates the color value of the third component corresponding to the kth color in {H(k)|0≤k≤4095}, and mod() is the remainder operation function.

②-5、根据{I_i(x,y)}中的每个像素点的基于全局颜色直方图的显著值，得到{I_i(x,y)}的基于全局颜色直方图的图像显著图，记为{HS(x,y)}。②-5. According to the saliency value based on the global color histogram of each pixel in {I _i (x, y)}, the image saliency map based on the global color histogram of {I _i (x, y)} is obtained , denoted as {HS(x,y)}.

③采用超像素（Superpixel）分割技术将{I_i(x,y)}分割成M个互不重叠的区域，然后将{I_i(x,y)}重新表示为M个区域的集合，记为{SP_h}，再考虑局部显著性，图像中相似的区域之间一般具有较低的显著性，因此本发明计算{SP_h}中的各个区域之间的相似性，将{SP_h}中的第p个区域与第q个区域之间的相似性记为Sim(SP_p,SP_q)，其中，M≥1，SP_h表示{SP_h}中的第h个区域，1≤h≤M，1≤p≤M，1≤q≤M，p≠q，SP_p表示{SP_h}中的第p个区域，SP_q表示{SP_h}中的第q个区域。在本实施例中，取M=200。③ Use superpixel segmentation technology to divide {I _i (x, y)} into M non-overlapping regions, and then re-express {I _i (x, y)} as a set of M regions, record is {SP _h }, and considering the local saliency, similar areas in the image generally have lower saliency, so the present invention calculates the similarity between each area in {SP _h }, and {SP _h } The similarity between the p-th area and the q-th area in is denoted as Sim(SP _p , SP _q ), where M≥1, SP _h represents the h-th area in {SP _h }, 1≤h ≤M, 1≤p≤M, 1≤q≤M, p≠q, SP _p denotes the p-th region in {SP _h }, and SP _q denotes the q-th region in {SP _h }. In this embodiment, take M=200.

在此具体实施例中，步骤③中{SP_h}中的第p个区域与第q个区域之间的相似性Sim(SP_p,SP_q)的获取过程为：In this specific embodiment, the acquisition process of the similarity Sim(SP _p , SP _q ) between the p-th region and the q-th region in {SP _h } in step ③ is:

③-1、对{SP_h}中的每个区域中的每个像素点的各个分量的颜色值分别进行量化，得到{SP_h}中的每个区域的量化区域，将{SP_h}中的第h个区域的量化区域记为{P_h,i(x_h,y_h)}，将{P_h,i(x_h,y_h)}中坐标位置为(x_h,y_h)的像素点的第i个分量的颜色值记为P_h,i(x_h,y_h)，假设{P_h,i(x_h,y_h)}中坐标位置为(x_h,y_h)的像素点在{I_i(x,y)}中的坐标位置为(x,y)，则其中，1≤x_h≤W_h,1≤y_h≤H_h，W_h表示{SP_h}中的第h个区域的宽度，H_h表示{SP_h}中的第h个区域的高度，符号为向下取整符号。③-1. Quantize the color values of each component of each pixel in each region in {SP _h }, respectively, to obtain the quantized region of each region in {SP _h }, and convert {SP _h } The quantization area of the h-th area of is recorded as {P _h,i (x _h ,y _h )}, and the coordinate position in {P _h,i (x _h ,y _h )} is (x _h ,y _h ) The color value of the i-th component of the pixel is recorded as P _h,i (x _h ,y _h ), assuming that the coordinate position in {P _h,i (x _h ,y _h )} is (x _h ,y _h ) The coordinate position of the pixel in {I _i (x, y)} is (x, y), then Among them, 1≤x _h ≤W _h , 1≤y _h ≤H _h , W _h represents the width of the h-th region in {SP _h }, H _h represents the height of the h-th region in {SP _h }, symbol is the rounding down symbol.

③-2、计算{SP_h}中的每个区域的量化区域的颜色直方图，将{P_h,i(x_h,y_h)}的颜色直方图记为其中，表示{P_h,i(x_h,y_h)}中属于第k种颜色的所有像素点的个数。③-2. Calculate the color histogram of the quantized region of each region in {SP _h }, and mark the color histogram of {P _h,i (x _h ,y _h )} as in, Indicates the number of all pixels belonging to the kth color in {P _h,i (x _h ,y _h )}.

③-3、对{SP_h}中的每个区域的量化区域的颜色直方图进行归一化操作，得到对应的归一化后的颜色直方图，将对进行归一化操作后得到的归一化后的颜色直方图记为 { H ′ SP h ( k ) | 0 ≤ k ≤ 4095 } , H ′ SP h ( k ) = H SP h ( k ) Σ h ′ = 1 M H SP h ′ ( k ) , 其中，表示{SP_h}中的第h个区域的量化区域{P_h,i(x_h,y_h)}中属于第k种颜色的像素点的出现概率，表示{SP_h}中的第h'个区域的量化区域{P_h',i(x_h',y_h')}中属于第k种颜色的所有像素点的个数，1≤x_h'≤W_h',1≤y_h'≤H_h'，W_h'表示{SP_h}中的第h'个区域的宽度，H_h'表示{SP_h}中的第h'个区域的高度，P_h',i(x_h',y_h')表示{P_h',i(x_h',y_h')}中坐标位置为(x_h',y_h')的像素点的第i个分量的颜色值。③-3. Perform a normalization operation on the color histogram of the quantized region of each region in {SP _h } to obtain the corresponding normalized color histogram, which will be The normalized color histogram obtained after the normalization operation is marked as { h ′ SP h ( k ) | 0 ≤ k ≤ 4095 } , h ′ SP h ( k ) = h SP h ( k ) Σ h ′ = 1 m h SP h ′ ( k ) , in, Indicates the occurrence probability of pixels belonging to the k-th color in the quantized area {P _h,i (x _h ,y _h )} of the h-th area in {SP _h }, Indicates the number of all pixels belonging to the k-th color in the quantized area {P _h',i (x _h' ,y _h' )} of the h'th area in {SP _h }, 1≤x _h' ≤W h _' ,1≤y _h' ≤H _h' , W _h' represents the width of the h'th region in {SP _h }, H _h' represents the height of the h'th region in {SP _h } , P _h',i (x _h' ,y _h' ) means the pixel point whose coordinate position is (x _h' ,y _h' ) in {P _h',i (x _h' ,y _h' )} Color values for the i components.

③-4、计算{SP_h}中的第p个区域与第q个区域之间的相似性，记为Sim(SP_p,SP_q)，Sim(SP_p,SP_q)=Sim_c(SP_p,SP_q)×Sim_d(SP_p,SP_q)，Sim_c(SP_p,SP_q)表示{SP_h}中的第p个区域与{SP_h}中的第q区域之间的颜色相似性， Sim c ( SP p , SP q ) = Σ k = 0 4095 min ( H ′ SP p ( k ) , H ′ SP q ( k ) ) , Sim_d(SP_p,SP_q)表示{SP_h}中的第p个区域与{SP_h}中的第q区域之间的空间相似性，其中，SP_p表示{SP_h}中的第p个区域，SP_q表示{SP_h}中的第q个区域，表示{SP_h}中的第p个区域的量化区域{P_p,i(x_p,y_p)}中属于第k种颜色的像素点的出现概率，表示{SP_h}中的第q个区域的量化区域{P_q,i(x_q,y_q)}中属于第k种颜色的像素点的出现概率，1≤x_p≤W_p,1≤y_p≤H_p，W_p表示{SP_h}中的第p个区域的宽度，H_p表示{SP_h}中的第p个区域的高度，P_p,i(x_p,y_p)表示{P_p,i(x_p,y_p)}中坐标位置为(x_p,y_p)的像素点的第i个分量的颜色值，1≤x_q≤W_q,1≤y_q≤H_q，W_q表示{SP_h}中的第q个区域的宽度，H_q表示{SP_h}中的第q个区域的高度，P_q,i(x_q,y_q)表示{P_q,i(x_q,y_q)}中坐标位置为(x_q,y_q)的像素点的第i个分量的颜色值，min()为取最小值函数，表示{SP_h}中的第p个区域中的中心像素点的坐标位置，表示{SP_h}中的第q个区域中的中心像素点的坐标位置，符号“‖‖”为求欧式距离符号。③-4. Calculate the similarity between the p-th region and the q-th region in {SP _h }, denoted as Sim(SP _p ,SP _q ), Sim(SP _p ,SP _q )=Sim _c (SP _p ,SP _q )×Sim _d (SP _p ,SP _q ), Sim _c (SP _p ,SP _q ) represents the color between the p-th region in {SP _h } and the q-th region in {SP _h } similarity, Sim c ( SP p , SP q ) = Σ k = 0 4095 min ( h ′ SP p ( k ) , h ′ SP q ( k ) ) , Sim _d (SP _p ,SP _q ) denotes the spatial similarity between the p-th region in {SP _h } and the q-th region in {SP _h }, where, SP _p represents the p-th region in {SP _h }, SP _q represents the q-th region in {SP _h }, Indicates the occurrence probability of pixels belonging to the k-th color in the quantized area {P _p,i (x _p ,y _p )} of the p-th area in {SP _h }, Indicates the occurrence probability of pixels belonging to the kth color in the quantized area {P _q,i (x _q ,y _q )} of the qth area in {SP _h }, 1≤x _p ≤W _p ,1≤ y _p ≤ H _p , W _p represents the width of the p-th region in {SP _h }, H _p represents the height of the p-th region in {SP _h }, P _p,i (x _p ,y _p ) represents The color value of the i-th component of the pixel whose coordinate position is (x _p , y _p ) in {P _p,i (x _p ,y _p )}, 1≤x _q ≤W _q ,1≤y _q ≤H _q , W _q represents the width of the qth region in {SP _h }, H _q represents the height of the qth region in {SP _h }, P _q,i (x _q ,y _q ) represents {P _{q, i} (x _q ,y _q )} is the color value of the i-th component of the pixel whose coordinate position is (x _q ,y _q ), min() is the minimum value function, Indicates the coordinate position of the central pixel point in the p-th region in {SP _h }, Indicates the coordinate position of the central pixel point in the qth region in {SP _h }, and the symbol "‖‖" is the Euclidean distance symbol.

④根据{SP_h}中的各个区域之间的相似性，获取{I_i(x,y)}的基于区域颜色对比度的图像显著图，记为{NGC(x,y)}，其中，NGC(x,y)表示{NGC(x,y)}中坐标位置为(x,y)的像素点的像素值。④According to the similarity between each region in {SP _h }, obtain the image saliency map based on regional color contrast of {I _i (x,y)}, denoted as {NGC(x,y)}, where, NGC (x, y) represents the pixel value of the pixel at the coordinate position (x, y) in {NGC(x, y)}.

在此具体实施例中，步骤④的具体过程为：In this specific embodiment, the concrete process of step 4. is:

④-1、计算{SP_h}中的每个区域的颜色对比度，将{SP_h}中的第h个区域的颜色对比度记为 NGC SP h = Σ q = 1 M W ( SP h , SP q ) × | | m SP h - m SP q | | , 其中，SP_h表示{SP_h}中的第h个区域，SP_q表示{SP_h}中的第q个区域，表示{SP_h}中的第h个区域中包含的像素点的总个数，Sim_d(SP_h,SP_q)表示{SP_h}中的第h个区域与{SP_h}中的第q区域之间的空间相似性， 表示{SP_h}中的第h个区域中的中心像素点的坐标位置，表示{SP_h}中的第q个区域中的中心像素点的坐标位置，符号“‖‖”为求欧式距离符号，表示{SP_h}中的第h个区域的颜色均值向量，即将{SP_h}中的第h个区域中的所有像素点的颜色向量求平均得到表示{SP_h}中的第q个区域的颜色均值向量。④-1. Calculate the color contrast of each region in {SP _h }, and record the color contrast of the hth region in {SP _h } as NGC SP h = Σ q = 1 m W ( SP h , SP q ) × | | m SP h - m SP q | | , where, SP _h represents the h-th region in {SP _h }, SP _q represents the q-th region in {SP _h }, Indicates the total number of pixels contained in the h-th area in {SP _h }, Sim _d (SP _h ,SP _q ) means the h-th area in {SP _h } and the q-th area in {SP _h } the spatial similarity between regions, Indicates the coordinate position of the central pixel point in the h-th region in {SP _h }, Indicates the coordinate position of the central pixel point in the qth region in {SP _h }, the symbol "‖‖" is the Euclidean distance symbol, Indicates the color mean vector of the hth area in {SP _h }, that is, the color vector of all pixels in the hth area in {SP _h } is obtained by averaging Color mean vector representing the qth region in {SP _h }.

④-2、对{SP_h}中的每个区域的颜色对比度进行归一化操作，得到对应的归一化后的颜色对比度，将对{SP_h}中的第h个区域的颜色对比度进行归一化操作后得到的归一化后的颜色对比度记为 其中，NGC_min表示{SP_h}中的M个区域中最小的颜色对比度，NGC_max表示{SP_h}中的M个区域中最大的颜色对比度。④-2. Perform a normalization operation on the color contrast of each region in {SP _h } to obtain the corresponding normalized color contrast, and then perform a normalization operation on the color contrast of the hth region in {SP _h } The normalized color contrast obtained after the normalization operation is recorded as Among them, NGC _min represents the minimum color contrast among the M regions in {SP _h }, and NGC _max represents the maximum color contrast among the M regions in {SP _h }.

④-3、计算{SP_h}中的每个区域的基于颜色对比度的显著值，将{SP_h}中的第h个区域的基于颜色对比度的显著值记为 NGC ′ ′ SP h = Σ q = 1 M ( Sim ( SP h , SP q ) × NGC ′ SP h ) Σ q = 1 M Sim ( SP h , SP q ) , 其中，Sim(SP_h,SP_q)表示{SP_h}中的第h个区域与第q个区域之间的相似性。④-3. Calculate the saliency value based on color contrast of each region in {SP _h }, and record the saliency value based on color contrast of the hth region in {SP _h } as NGC ′ ′ SP h = Σ q = 1 m ( Sim ( SP h , SP q ) × NGC ′ SP h ) Σ q = 1 m Sim ( SP h , SP q ) , Among them, Sim(SP _h , SP _q ) represents the similarity between the h-th region and the q-th region in {SP _h }.

④-4、将{SP_h}中的每个区域的基于颜色对比度的显著值作为对应区域中的所有像素点的显著值，即对于{SP_h}中的第h个区域，将{SP_h}中的第h个区域的基于颜色对比度的显著值作为该区域中的所有像素点的显著值，从而得到{I_i(x,y)}的基于区域颜色对比度的图像显著图，记为{NGC(x,y)}，其中，NGC(x,y)表示{NGC(x,y)}中坐标位置为(x,y)的像素点的像素值。④-4. Take the saliency value based on the color contrast of each region in {SP _h } as the saliency value of all pixels in the corresponding region, that is, for the hth region in {SP _h }, set {SP _h } in the hth region based on color contrast as the salient value of all pixels in the region, so as to obtain the image saliency map based on regional color contrast of {I _i (x, y)}, denoted as { NGC(x,y)}, wherein, NGC(x,y) represents the pixel value of the pixel point whose coordinate position is (x,y) in {NGC(x,y)}.

⑤根据{SP_h}中的各个区域之间的相似性，获取{I_i(x,y)}的基于区域空间稀疏性的图像显著图，记为{NSS(x,y)}，其中，NSS(x,y)表示{NSS(x,y)}中坐标位置为(x,y)的像素点的像素值。⑤According to the similarity between the regions in {SP _h }, obtain the image saliency map based on the spatial sparsity of {I _i (x,y)}, denoted as {NSS(x,y)}, where, NSS(x,y) indicates the pixel value of the pixel point whose coordinate position is (x,y) in {NSS(x,y)}.

在此具体实施例中，步骤⑤的具体过程为：In this specific embodiment, the concrete process of step 5. is:

⑤-1、计算{SP_h}中的每个区域的空间稀疏性，将{SP_h}中的第h个区域的空间稀疏性记为 NSS SP h = Σ q = 1 M ( Sim ( SP h , SP q ) × D SP h ) Σ q = 1 M Sim ( SP h , SP q ) , 其中，Sim(SP_h,SP_q)表示{SP_h}中的第h个区域与第q个区域之间的相似性，表示{SP_h}中的第h个区域中的中心像素点与{I_i(x,y)}的中心像素点之间的欧氏距离。⑤-1. Calculate the spatial sparsity of each region in {SP _h }, and record the spatial sparsity of the hth region in {SP _h } as NSS SP h = Σ q = 1 m ( Sim ( SP h , SP q ) × D. SP h ) Σ q = 1 m Sim ( SP h , SP q ) , Among them, Sim(SP _h , SP _q ) represents the similarity between the h-th region and the q-th region in {SP _h }, Indicates the Euclidean distance between the central pixel point in the h-th region in {SP _h } and the central pixel point in {I _i (x,y)}.

⑤-2、对{SP_h}中的每个区域的空间稀疏性进行归一化操作，得到对应的归一化后的空间稀疏性，将对{SP_h}中的第h个区域的空间稀疏性进行归一化操作后得到的归一化后的空间稀疏性记为 其中，NSS_min表示{SP_h}中的M个区域中最小的空间稀疏性，NSS_max表示{SP_h}中的M个区域中最大的空间稀疏性。⑤-2. Perform a normalization operation on the spatial sparsity of each region in {SP _h } to obtain the corresponding normalized spatial sparsity, and the space of the hth region in {SP _h } sparsity The normalized spatial sparsity obtained after the normalization operation is denoted as Among them, NSS _min represents the smallest spatial sparsity among the M regions in {SP _h }, and NSS _max represents the largest spatial sparsity among the M regions in {SP _h }.

⑤-3、计算{SP_h}中的每个区域的基于空间稀疏性的显著值，将{SP_h}中的第h个区域的基于空间稀疏性的显著值记为 NS S ′ ′ SP h = Σ q = 1 M ( Sim ( SP h , SP q ) × NSS ′ SP h ) Σ q = 1 M Sim ( SP h , SP q ) . ⑤-3. Calculate the saliency value based on spatial sparsity of each region in {SP _h }, and record the saliency value based on spatial sparsity of the hth region in {SP _h } as NS S ′ ′ SP h = Σ q = 1 m ( Sim ( SP h , SP q ) × NSS ′ SP h ) Σ q = 1 m Sim ( SP h , SP q ) .

⑤-4、将{SP_h}中的每个区域的基于空间稀疏性的显著值作为对应区域中的所有像素点的显著值，即对于{SP_h}中的第h个区域，将{SP_h}中的第h个区域的基于空间稀疏性的显著值作为该区域中的所有像素点的显著值，从而得到{I_i(x,y)}的基于区域空间稀疏性的图像显著图，记为{NSS(x,y)}，其中，NSS(x,y)表示{NSS(x,y)}中坐标位置为(x,y)的像素点的像素值。⑤-4. Take the saliency value based on spatial sparsity of each region in {SP _h } as the saliency value of all pixels in the corresponding region, that is, for the hth region in {SP _h }, set {SP The saliency value based on the spatial sparsity of the hth region in _h } is used as the saliency value of all pixels in the region, so as to obtain the image saliency map based on the region spatial sparsity of {I _i (x, y)}, It is recorded as {NSS(x,y)}, where NSS(x,y) represents the pixel value of the pixel point whose coordinate position is (x,y) in {NSS(x,y)}.

⑥对{I_i(x,y)}的基于全局颜色直方图的图像显著图{HS(x,y)}、{I_i(x,y)}的基于区域颜色对比度的图像显著图{NGC(x,y)}及{I_i(x,y)}的基于区域空间稀疏性的图像显著图{NSS(x,y)}进行融合，得到{I_i(x,y)}的最终的图像显著图，记为{Sal(x,y)}，将{Sal(x,y)}中坐标位置为(x,y)的像素点的像素值记为Sal(x,y)，Sal(x,y)=HS(x,y)×NGC(x,y)×NSS(x,y)。⑥ Image saliency map {HS(x, y)} based on global color histogram of {I _i (x, y)}, image saliency map based on regional color contrast of {I _i (x, y)} {NGC (x,y)} and the image saliency map {NSS(x,y)} based on regional spatial sparsity of {I _i (x,y)} are fused to obtain the final {I _i (x,y)} The image saliency map is recorded as {Sal(x,y)}, and the pixel value of the pixel point whose coordinate position is (x,y) in {Sal(x,y)} is recorded as Sal(x,y), and Sal( x,y)=HS(x,y)×NGC(x,y)×NSS(x,y).

以下就利用本发明方法对微软亚洲研究院提供的显著对象图像库MSRA中Image1、Image2、Image3、Image4和Image5五组图像的显著图进行提取。图2a给出了“Image1”的原始图像，图2b给出了“Image1”图像的真实（Ground truth）显著图，图2c给出了“Image1”图像的基于全局颜色直方图的图像显著图、图2d给出了“Image1”图像的基于区域颜色对比度的图像显著图、图2e给出了“Image1”图像的基于区域空间稀疏性的图像显著图、图2f给出了“Image1”图像最终的图像显著图；图3a给出了“Image2”的原始图像，图3b给出了“Image2”图像的真实（Ground truth）显著图，图3c给出了“Image2”图像的基于全局颜色直方图的图像显著图、图3d给出了“Image2”图像的基于区域颜色对比度的图像显著图、图3e给出了“Image2”图像的基于区域空间稀疏性的图像显著图、图3f给出了“Image2”图像最终的图像显著图；图4a给出了“Image3”的原始图像，图4b给出了“Image3”图像的真实（Groundtruth）显著图，图4c给出了“Image3”图像的基于全局颜色直方图的图像显著图、图4d给出了“Image3”图像的基于区域颜色对比度的图像显著图、图4e给出了“Image3”图像的基于区域空间稀疏性的图像显著图、图4f给出了“Image3”图像最终的图像显著图；图5a给出了“Image4”的原始图像，图5b给出了“Image4”图像的真实（Ground truth）显著图，图5c给出了“Image4”图像的基于全局颜色直方图的图像显著图、图5d给出了“Image4”图像的基于区域颜色对比度的图像显著图、图5e给出了“Image4”图像的基于区域空间稀疏性的图像显著图、图5f给出了“Image4”图像最终的图像显著图；图6a给出了“Image5”的原始图像，图6b给出了“Image5”图像的真实（Ground truth）显著图，图6c给出了“Image5”图像的基于全局颜色直方图的图像显著图、图6d给出了“Image5”图像的基于区域颜色对比度的图像显著图、图6e给出了“Image5”图像的基于区域空间稀疏性的图像显著图、图6f给出了“Image5”图像最终的图像显著图。从图2a至图6f可以看出，采用本发明方法得到的图像显著图由于考虑了全局和局部区域的显著变化情况，因此能够很好地符合显著语义的特征。In the following, the method of the present invention is used to extract the saliency maps of Image1, Image2, Image3, Image4 and Image5 in the salient object image library MSRA provided by Microsoft Research Asia. Figure 2a shows the original image of "Image1", Figure 2b shows the real (Ground truth) saliency map of the "Image1" image, and Figure 2c shows the image saliency map of the "Image1" image based on the global color histogram, Figure 2d shows the image saliency map based on the regional color contrast of the "Image1" image, Figure 2e shows the image saliency map based on the regional spatial sparsity of the "Image1" image, and Figure 2f shows the final Image saliency map; Fig. 3a shows the original image of "Image2", Fig. 3b shows the real (Ground truth) saliency map of "Image2" image, and Fig. 3c shows the global color histogram of "Image2" image Image saliency map, Fig. 3d shows the image saliency map of the image "Image2" based on the regional color contrast, Fig. 3e shows the image saliency map of the "Image2" image based on the spatial sparsity of the region, Fig. 3f shows the image saliency map of the "Image2" image The final image saliency map of the "image; Fig. 4a shows the original image of "Image3", Fig. 4b shows the real (Groundtruth) saliency map of "Image3" image, and Fig. 4c shows the global color-based image of "Image3" The image saliency map of the histogram, Figure 4d shows the image saliency map of the "Image3" image based on the regional color contrast, Figure 4e shows the image saliency map of the "Image3" image based on the regional spatial sparsity, and Figure 4f shows Figure 5a shows the original image of "Image4", Figure 5b shows the real (Ground truth) saliency map of "Image4" image, and Figure 5c shows the "Image4" image The image saliency map based on the global color histogram, Figure 5d shows the image saliency map based on the regional color contrast of the "Image4" image, Figure 5e shows the image saliency map based on the regional spatial sparsity of the "Image4" image, Figure 5f shows the final image saliency map of the "Image4" image; Figure 6a shows the original image of "Image5", Figure 6b shows the real (Ground truth) saliency map of the "Image5" image, and Figure 6c shows The image saliency map of the "Image5" image based on the global color histogram, Figure 6d shows the image saliency map of the "Image5" image based on the regional color contrast, and Figure 6e shows the image saliency map of the "Image5" image based on the regional spatial sparsity Image saliency map, Figure 6f shows the final image saliency map of the "Image5" image. It can be seen from Fig. 2a to Fig. 6f that the image saliency map obtained by using the method of the present invention can well conform to the characteristics of saliency semantics because the saliency changes of the global and local regions are considered.