CN115546795A - Automatic reading method of circular pointer instrument based on deep learning - Google Patents

本发明公开了一种基于深度学习的圆形指针式仪表自动读数方法，包括：利用实例分割深度学习模型从场景图像中分割表盘、利用实例分割得到的图像掩膜和超分辨率重建深度学习模型、双边滤波等技术对表盘图像进行图像增强和校正、利用表盘字符信息提取模块获取表盘字符信息、利用关键点检测深度学习模型获取仪表指针信息；最终基于一种改进的角度法计算出圆形指针式仪表图像的读数。本发明具有很好的通用性和准确性，不依赖待检测仪表先验信息，同时也不需要对仪表场景进行限制，为解决仪表智能读数问题提供了一种通用、有效的解决方案。

The invention discloses an automatic reading method of a circular pointer instrument based on deep learning. , bilateral filtering and other technologies to enhance and correct the dial image, use the dial character information extraction module to obtain the dial character information, and use the key point detection deep learning model to obtain the instrument pointer information; finally calculate the circular pointer based on an improved angle method readout of the meter image. The invention has good versatility and accuracy, does not rely on the prior information of the instrument to be tested, and does not need to limit the scene of the instrument, and provides a general and effective solution for solving the problem of intelligent reading of the instrument.

一种基于深度学习的圆形指针式仪表自动读数方法An automatic reading method for circular pointer meters based on deep learning

技术领域technical field

本发明涉及仪表读数的技术领域，尤其是指一种基于深度学习的圆形指针式仪表自动读数方法。The present invention relates to the technical field of meter reading, in particular to a method for automatic reading of circular pointer meters based on deep learning.

背景技术Background technique

仪表是显示数值的仪器的总称，是用于测量生产生活中各种数据重要工具，对于了解环境状态有着重要的作用。虽然目前已经进入了数字化的时代，大量传感器数据可以被直接传输到计算机中，可是在变电站等很多传统生产场景下，由于复杂电磁环境或者设备尚未更新换代，指针式仪表仍然在生产生活中发挥着重要的作用。Meter is the general term for instruments that display numerical values. It is an important tool for measuring various data in production and life, and plays an important role in understanding the state of the environment. Although we have entered the era of digitalization and a large amount of sensor data can be directly transmitted to the computer, in many traditional production scenarios such as substations, due to the complex electromagnetic environment or equipment that has not been updated, pointer instruments still play an important role in production and life. important role.

目前指针式仪表的主要数据获取方法是人工抄录，即生产单位安排专门的工作人员进行巡检，然后阅读各个仪表进行数据查验和记录。这种方式工作内容枯燥单一，在一些巡检路程的较长的作业场景下，对巡检人员的体力要求也比较高。随着科技发展，人工智能技术成为了时代的热潮，利用巡检机器人代替工人进行巡检引起了越来越多研究者的关注。At present, the main data acquisition method of pointer instruments is manual transcription, that is, the production unit arranges special staff to conduct inspections, and then reads each instrument for data inspection and recording. The work content of this method is boring and single, and in some work scenarios with a long inspection distance, the physical requirements for the inspection personnel are also relatively high. With the development of science and technology, artificial intelligence technology has become the craze of the times, and the use of inspection robots to replace workers for inspections has attracted more and more researchers' attention.

仪表自动读数识别技术是巡检机器人仪表读数巡检任务的核心技术之一。目前的仪表自动读数识别技术主要依赖数字图像处理技术和模式识别技术，主要可以分为基于传统图像处理的方法、基于模板匹配的方法和基于深度学习的方法。Zhe三种方法的区别主要在与获取表盘ROI图像的方法不同，传统图像处理方法主要使用Hough圆检测方法获取仪表ROI；基于模板匹配的方法主要利用仪表模板从场景图像中得到对应的仪表表盘区域；基于深度学习的方法主要利用目标检测或者实例分割模型从场景中得到仪表表盘区域的矩形边界框或者分割掩膜。在获取仪表表盘图像之后，利用Hough变换等直线检测算法获取指针的拟合直线，然后基于角度法或者距离法，实现读数计算。这类方法通常是将仪表的刻度信息当作先验信息，而仅仅关注表盘指针信息的提取。提取表盘信息的方法多为传统的图像处理方法，对于算法的参数比较敏感，一旦拍摄的光影条件改变或者仪表的类型改变之后算法往往会失效。由于仪表问题的复杂性在于其读数并不是显示呈现在图像中而是蕴藏在指针和刻度点关系之中，计算机视觉领域的各类深度学习模型目前也无法学习如此复杂的表征，也就无法通过深度学习实现一种端到端的仪表读数识别系统。The instrument automatic reading recognition technology is one of the core technologies of the instrument reading inspection task of the inspection robot. The current automatic instrument reading recognition technology mainly relies on digital image processing technology and pattern recognition technology, which can be mainly divided into methods based on traditional image processing, methods based on template matching and methods based on deep learning. The difference between these three methods is that it is different from the method of obtaining the ROI image of the dial. The traditional image processing method mainly uses the Hough circle detection method to obtain the ROI of the instrument; the method based on template matching mainly uses the instrument template to obtain the corresponding instrument dial area from the scene image. ; The method based on deep learning mainly uses the target detection or instance segmentation model to obtain the rectangular bounding box or segmentation mask of the instrument panel area from the scene. After the instrument panel image is obtained, the straight line detection algorithm such as Hough transform is used to obtain the fitting straight line of the pointer, and then the reading calculation is realized based on the angle method or the distance method. This type of method usually regards the scale information of the instrument as prior information, and only focuses on the extraction of dial pointer information. Most of the methods for extracting dial information are traditional image processing methods, which are sensitive to the parameters of the algorithm. Once the light and shadow conditions of the shooting change or the type of the instrument changes, the algorithm will often fail. Due to the complexity of the instrument problem, its readings are not displayed in the image but hidden in the relationship between the pointer and the scale point. Various deep learning models in the field of computer vision are currently unable to learn such complex representations, and cannot pass Deep learning implements an end-to-end meter reading recognition system.

上述现有算法的种种不足之处导致目前的仪表识别算法缺乏一种高度通用的方法，智能化程度不足，在工程应用中往往需要现场对参数进行调整，或者对应用场景进行种种限制。因此，设计一种尽可能通用了仪表自动读数方法，仍然是一项有待解决的挑战性问题。The various shortcomings of the above-mentioned existing algorithms lead to the lack of a highly versatile method for the current meter recognition algorithm, and the lack of intelligence. In engineering applications, it is often necessary to adjust parameters on site or impose various restrictions on application scenarios. Therefore, designing an automatic meter reading method that is as general as possible is still a challenging problem to be solved.

发明内容Contents of the invention

本发明的目的在于克服现有技术的缺点与不足，提出了一种基于深度学习的圆形指针式仪表自动读数方法，将仪表读数问题拆解为若干模块的子问题，然后用不同的计算机视觉深度学习模型分别解决相应子问题，最终得到仪表的读数结果。本发明具有很好的通用性和准确性，不依赖待检测仪表先验信息，同时也不需要对仪表场景进行限制，为解决仪表智能读数问题提供了一种通用、有效的解决方案。The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and proposes a method for automatic reading of circular pointer meters based on deep learning, which disassembles the problem of meter reading into sub-problems of several modules, and then uses different computer vision The deep learning model solves the corresponding sub-problems respectively, and finally obtains the reading results of the instrument. The invention has good versatility and accuracy, does not rely on the prior information of the instrument to be tested, and does not need to limit the scene of the instrument, and provides a general and effective solution for solving the problem of intelligent reading of the instrument.

为实现上述目的，本发明所提供的技术方案为：一种基于深度学习的圆形指针式仪表自动读数方法，包括以下步骤：In order to achieve the above object, the technical solution provided by the present invention is: a method for automatic reading of a circular pointer instrument based on deep learning, comprising the following steps:

1)通过可见光相机获取巡检现场的图像，称为场景图像；1) Obtain the image of the inspection site through the visible light camera, which is called the scene image;

2)使用一个基于深度学习实例分割技术开发的表盘区域提取模块对场景图像中的仪表表盘区域进行检测，将仪表表盘图像从场景图像中分割提取出来，去除背景部分，得到的仅包含仪表表盘的部分，称为表盘ROI图像；2) Use a dial area extraction module developed based on deep learning instance segmentation technology to detect the instrument dial area in the scene image, segment and extract the instrument dial image from the scene image, remove the background part, and obtain only the instrument dial area part, called dial ROI image;

3)通过综合表盘ROI图像的尺寸和模糊度，得到一个度量仪表可读性的指标，称为仪表可读性指数，根据该指数的计算结果，匹配对应的图像增强模块对表盘ROI图像进行图像增强；3) By synthesizing the size and ambiguity of the dial ROI image, an index for measuring the readability of the meter is obtained, which is called the meter readability index. According to the calculation result of the index, match the corresponding image enhancement module to image the dial ROI image enhanced;

4)使用一个基于图像变换技术开发的表盘倾斜校正模块解决拍摄过程中由于相机非正对仪表所在平面拍摄导致的表盘倾斜问题，经过图像增强和表盘校正之后的表盘图像称为高质量表盘ROI图像；4) Use a dial tilt correction module developed based on image transformation technology to solve the problem of dial tilt caused by the camera not facing the plane where the instrument is located during the shooting process. The dial image after image enhancement and dial correction is called a high-quality dial ROI image ;

5)利用深度学习技术提取仪表表盘信息，具体是分别使用基于深度学习OCR技术开发的表盘字符信息提取模块获取高质量表盘ROI图像的字符信息和基于深度学习关键点检测技术开发的表盘指针信息提取模块获取高质量表盘ROI图像的指针信息；5) Use deep learning technology to extract instrument dial information, specifically use the dial character information extraction module developed based on deep learning OCR technology to obtain high-quality dial ROI image character information and the dial pointer information developed based on deep learning key point detection technology The module obtains the pointer information of the high-quality dial ROI image;

6)使用一个指针刻度区间匹配模块从上述仪表表盘信息的提取结果中获取仪表读数的有效信息，所述仪表读数的有效信息分别指：指针末端点、指针旋转中心点、指针所在刻度区间的上下界刻度数字；6) Use a pointer scale interval matching module to obtain the effective information of the instrument reading from the extraction result of the above-mentioned instrument dial information. The effective information of the instrument reading refers to: the end point of the pointer, the center point of the pointer rotation, and the upper and lower points of the scale interval where the pointer is located. Boundary scale numbers;

7)使用一个仪表读数模块，基于上述仪表读数的有效信息计算仪表读数。7) Using a meter reading module to calculate the meter reading based on the available information of the above meter reading.

进一步，在步骤2)中，所述表盘区域提取模块为一个经过训练的实例分割深度学习模型，通过该模型能够获取仪表表盘在场景图像中的矩形边界框坐标和图像掩膜，进而从场景图像中分割出上述矩形边界框范围内部的子图，并利用所述图像掩膜去除上述子图中表盘区域之外的部分，得到仅包含仪表表盘的部分，称为表盘ROI图像。Further, in step 2), the dial area extraction module is a trained instance segmentation deep learning model, through which the rectangular bounding box coordinates and the image mask of the instrument dial in the scene image can be obtained, and then from the scene image Segment the sub-image within the range of the above-mentioned rectangular bounding box, and use the image mask to remove the part outside the dial area in the above-mentioned sub-image, and obtain the part containing only the instrument dial, which is called the dial ROI image.

进一步，在步骤3)中，所述仪表可读性指数记为F，该指数的计算公式如下：Further, in step 3), the instrument readability index is denoted as F, and the calculation formula of this index is as follows:

式中，var代表图像方差，描述了图像的模糊程度；H和W分别代表图像的高度和宽度，描述了图像的尺寸大小；α和β为两个权重系数，用于调节模糊程度和尺寸大小对仪表可读性指数计算结果的影响权重；In the formula, var represents the variance of the image, which describes the degree of blur of the image; H and W represent the height and width of the image, respectively, which describe the size of the image; α and β are two weight coefficients, which are used to adjust the degree of blur and the size of the image The impact weight on the calculation results of the instrument readability index;

其中，所述图像方差var表示对所有像素的灰度值与图像平均灰度值之差的平方求和，然后除以像素总数的计算结果，具体计算过程包括以下步骤：Wherein, the image variance var represents the sum of the squares of the difference between the gray value of all pixels and the average gray value of the image, and then divides the calculation result by the total number of pixels. The specific calculation process includes the following steps:

3.1)将彩色的表盘ROI图像转化为表盘ROI灰度图像；3.1) Convert the colored dial ROI image into a dial ROI grayscale image;

3.2)使用如下拉普拉斯算子模板对表盘ROI灰度图像进行卷积运算：3.2) Use the following Laplacian template to perform convolution operation on the dial ROI grayscale image:

3.3)使用如下公式计算卷积后表盘ROI灰度图像的方差σ²：3.3) Use the following formula to calculate the variance σ ² of the dial ROI grayscale image after convolution:

式中，x表示图像上一点的像素值，μ表示图像的灰度均值，分子的求和范围包括图像中的所有像素，W*H表示图像的总像素个数。In the formula, x represents the pixel value of a point on the image, μ represents the average gray value of the image, the summation range of the numerator includes all pixels in the image, and W*H represents the total number of pixels in the image.

进一步，在步骤3)中，所述图像增强模块包括一个超分辨率重建深度学习模型和一个双边滤波模块，根据不同的仪表可读性指数计算结果，选择不同图像增强方法对表盘ROI图像进行增强；Further, in step 3), the image enhancement module includes a super-resolution reconstruction deep learning model and a bilateral filter module, and according to different calculation results of the instrument readability index, different image enhancement methods are selected to enhance the dial ROI image ;

根据输入表盘ROI图像的仪表可读性指数F，若F大于预先设定的阈值，则判断表盘ROI图像为“低可读性图像”，利用超分辨率重建深度学习模型对输入图像进行图像超分辨率重建处理；若F小于预先设定的阈值，则判断表盘ROI图像为“非低可读性图像”，利用双边滤波模块对图像进行常规图像增强处理；According to the instrument readability index F of the input dial ROI image, if F is greater than the preset threshold, it is judged that the dial ROI image is a "low readability image", and the input image is super-resolved using a super-resolution reconstruction deep learning model. Resolution reconstruction processing; if F is less than the preset threshold, it is judged that the ROI image of the dial is a "non-low readability image", and the bilateral filter module is used to perform conventional image enhancement processing on the image;

所述双边滤波模块的计算公式为：The calculation formula of the bilateral filtering module is:

式中，f(x,y)为滤波之后像素点(x,y)位置的响应，g(x,y)为像素点(x,y)邻域内各个像素点的值，W(x,y)为各像素点的组合权重系数；该双边滤波模块的作用为在保持边缘清晰的情况下对其余部分进行平滑滤波以去除图像噪声，改善图像质量。In the formula, f(x, y) is the response of the position of the pixel (x, y) after filtering, g(x, y) is the value of each pixel in the neighborhood of the pixel (x, y), W(x, y ) is the combined weight coefficient of each pixel; the function of the bilateral filtering module is to perform smooth filtering on the remaining part while keeping the edge clear to remove image noise and improve image quality.

进一步，在步骤4)中，所述表盘倾斜校正模块具体执行以下操作：Further, in step 4), the dial tilt correction module specifically performs the following operations:

4.1)获取表盘区域提取模块得到的仪表图像掩膜，为仪表轮廓二值掩膜mask图像；4.1) Obtain the instrument image mask obtained by the dial area extraction module, which is the instrument contour binary mask mask image;

4.2)对上述掩膜图像进行椭圆拟合得到表盘ROI图像的轮廓拟合椭圆参数，包括椭圆中心点、椭圆长轴长度、椭圆短轴长度和椭圆长轴与竖直方向的夹角；4.2) Carry out ellipse fitting to above-mentioned mask image and obtain the profile fitting ellipse parameter of dial ROI image, comprise ellipse central point, ellipse major axis length, ellipse minor axis length and the included angle of ellipse major axis and vertical direction;

4.3)获取拟合之后得到的椭圆的长轴和短轴端点的坐标，计算短轴所在直线上到椭圆中心点的距离为长轴长度的点的坐标，称为椭圆短轴的校正期望坐标；4.3) Obtain the coordinates of the major axis and the minor axis endpoint of the ellipse obtained after the fitting, and calculate the coordinates of the point whose distance from the minor axis to the center point of the ellipse is the length of the major axis on the straight line where the minor axis is located, which is called the corrected expected coordinate of the minor axis of the ellipse;

4.4)利用椭圆长轴以及短轴端点的坐标和椭圆长轴端点坐标以及椭圆短轴的校正期望坐标构成4组特征点对，利用这些特征点对计算得到射影变换矩阵，射影变换矩阵的计算如下：4.4) Use the coordinates of the major axis and minor axis endpoints of the ellipse, the coordinates of the major axis endpoints of the ellipse, and the corrected expected coordinates of the minor axis of the ellipse to form 4 sets of feature point pairs, and use these feature point pairs to calculate the projective transformation matrix. The calculation of the projective transformation matrix is as follows :

p2＝H′*p1p2=H'*p1

式中，p1表示变换前原图一个点、p2表示p1对应的特征点，H′为射影变换矩阵，表示了原图中点p1映射到变换后图像中点p2的过程，将上述式子展开得到：In the formula, p1 represents a point in the original image before transformation, p2 represents the feature point corresponding to p1, and H′ is the projective transformation matrix, which represents the process of mapping the point p1 in the original image to the midpoint p2 in the transformed image. Expand the above formula to get :

式中，(x₁,y₁)表示点p1的坐标、(x₂,y₂)表示点p2的坐标，H₁₁～H₃₃均为矩阵H′的参数，由上述矩阵形式能够得到一个方程组；其中，由H₁₁～H₃₃这9个参数利用上述4组特征点对构建方程组即可解得唯一的射影变换矩阵H′；In the formula, (x ₁ , y ₁ ) represents the coordinates of point p1, (x ₂ , y ₂ ) represents the coordinates of point p2, H ₁₁ ~ H ₃₃ are all parameters of matrix H′, and an equation can be obtained from the above matrix form group; among them, the 9 parameters H ₁₁ ~ H ₃₃ use the above 4 groups of feature point pairs to construct a system of equations, which can be solved to obtain the unique projective transformation matrix H';

4.5)利用步骤4.4)中公式得到的射影变换矩阵H′能够对表盘ROI图像进行全局处理，计算每一个校正前表盘ROI图像经过校正之后的像素坐标，从而实现将椭圆形的表盘ROI图像校正为正圆形。4.5) Using the projective transformation matrix H' obtained by the formula in step 4.4), the dial ROI image can be globally processed, and the pixel coordinates of each dial ROI image before correction can be calculated, so as to realize the correction of the elliptical dial ROI image as Perfect circle.

进一步，在步骤5)中，所述表盘字符信息提取模块要提取的表盘字符信息包括每个字符图像的字符文本内容及其对应的位置坐标，获取过程具体包括以下步骤：Further, in step 5), the dial character information to be extracted by the dial character information extraction module includes character text content and corresponding position coordinates of each character image, and the acquisition process specifically includes the following steps:

5.1.1)输入经过图像增强和表盘校正的高质量表盘ROI图像于一个已经过训练的场景文本检测深度学习模型，该模型推理输出该高质量表盘ROI图像的所有字符区域信息，所述字符区域信息为字符的矩形边界框，由矩形边界框的四个顶点坐标描述；5.1.1) Input the high-quality dial ROI image through image enhancement and dial correction to a trained deep learning model for scene text detection, and the model reasoning outputs all character area information of the high-quality dial ROI image, and the character area The information is the rectangular bounding box of the character, described by the coordinates of the four vertices of the rectangular bounding box;

5.1.2)通过字符图像的矩形边界框坐标将所有字符从高质量表盘ROI图像中依次分割下来，得到高质量表盘ROI图像的字符图像序列；5.1.2) All characters are sequentially segmented from the high-quality dial ROI image by the rectangular bounding box coordinates of the character image to obtain a character image sequence of the high-quality dial ROI image;

5.1.3)利用各个字符图像的矩形边界框顶点坐标，计算字符位置坐标，所述字符位置坐标定义为字符矩形边界框顶点的中心坐标，其横纵坐标分别为四个矩形边界框顶点坐标的平均值；5.1.3) Utilize the apex coordinates of the rectangular bounding boxes of each character image to calculate the character position coordinates, the character position coordinates are defined as the center coordinates of the apexes of the character's rectangular bounding boxes, and its horizontal and vertical coordinates are respectively the four coordinates of the apex coordinates of the rectangular bounding boxes. average value;

5.1.4)将所述字符图像序列中的每一个字符图像依次送入一个已经过训练的字符图像文本识别深度学习模型，得到字符图像序列中各个字符图像的文本内容识别结果，将文本内容和对应的位置坐标成对保存，即得到了表盘字符信息。5.1.4) Each character image in the character image sequence is sent into a trained character image text recognition deep learning model to obtain the text content recognition result of each character image in the character image sequence, and the text content and The corresponding position coordinates are stored in pairs, that is, the dial character information is obtained.

进一步，在步骤5)中，所述表盘指针信息提取模块具体执行以下操作：Further, in step 5), the dial pointer information extraction module specifically performs the following operations:

5.2.1)通过目标检测深度学习模型对表盘内部的指针进行检测，得到指针的矩形边界框参数，包括指针的中心点位置坐标和矩形边界框的宽高；5.2.1) Detect the pointer inside the dial through the target detection deep learning model, and obtain the rectangular bounding box parameters of the pointer, including the coordinates of the center point of the pointer and the width and height of the rectangular bounding box;

5.2.2)基于指针的矩形边界框参数对从场景图像中获取指针矩形边界框范围内部的子图，得到的图像称为指针ROI图像；5.2.2) Based on the pointer's rectangular bounding box parameter pair, the subimage inside the pointer's rectangular bounding box range is obtained from the scene image, and the obtained image is called the pointer ROI image;

5.2.3)将指针ROI图像输入一个已经过训练的关键点检测深度学习模型，通过该模型直接推理得到指针旋转中心点和指针末端点位置坐标，上述两点及其位置为提取到的指针信息。5.2.3) Input the pointer ROI image into a trained deep learning model for key point detection, through which the model can be directly inferred to obtain the position coordinates of the pointer rotation center point and the pointer end point, the above two points and their positions are the extracted pointer information .

进一步，在步骤6)中，所述指针刻度区间匹配模块分为刻度数字筛选和匹配指针所在刻度区间这两个阶段进行：Further, in step 6), the pointer scale interval matching module is divided into two stages of scale number screening and matching pointer scale interval:

阶段一、刻度数字筛选：Stage 1. Scale number screening:

6.1.1)定义刻度数字为表盘ROI图像中表示仪表刻度的数字字符，根据表盘信息提取模块得到的字符文本内容首先筛选出所有文本内容为纯数字的字符，作为刻度数字的初步筛选结果，以下简称数字字符；6.1.1) Define the scale number as the digital character representing the meter scale in the dial ROI image. According to the character text content obtained by the dial information extraction module, first filter out all the characters whose text content is pure numbers, as the preliminary screening result of the scale number, as follows Numeric characters for short;

6.1.2)计算上述筛选后数字字符到指针旋转中心点的欧式距离，得到一个距离序列；6.1.2) Calculate the Euclidean distance from the above-mentioned screened digital characters to the pointer rotation center point to obtain a distance sequence;

6.1.3)使用k-means聚类算法对上述距离序列进行聚类，设定聚类的类别数k为3，指针式仪表的刻度数字由于空间分布近似在同一个圆上，故到旋转中心点的距离近似一致，被聚类为同一个类别，将这一类别内的数字字符作为数字字符的有效筛选结果，其余数字字符判定为无效筛选结果，从数字字符中去除；6.1.3) Use the k-means clustering algorithm to cluster the above-mentioned distance series, set the number of clustering categories k to 3, and the scale numbers of the pointer instrument are approximately on the same circle due to the spatial distribution, so they reach the rotation center The distance of the points is approximately the same, and they are clustered into the same category, and the numeric characters in this category are regarded as valid screening results for numeric characters, and the rest of the numeric characters are judged as invalid screening results, and are removed from the numeric characters;

6.1.4)计算剩余各个数字字符的旋转参考角，所述旋转参考角定义为以过旋转中心点向下的直线作参考线，数字字符和指针旋转中心点的连线与从指针旋转中心点出发竖直向下的射线之间的角度；6.1.4) Calculate the rotation reference angle of each remaining digital character, the rotation reference angle is defined as a straight line passing through the rotation center point downward as a reference line, the connection line between the digital character and the pointer rotation center point and the pointer rotation center point The angle between rays starting vertically downward;

6.1.5)利用数字字符文本内容和对应的旋转参考角构成<数字字符文本内容，旋转参考角>键值对，所有剩余数字字符的键值对构成一个键值对序列；6.1.5) Use the numeric character text content and the corresponding rotation reference angle to form a key-value pair of <numerical character text content, rotation reference angle>, and all remaining numeric character key-value pairs form a key-value pair sequence;

6.1.6)以数字字符文本内容作为排序关键字对上述键值对序列按照升序规则排序，检查排序后键值对序列中各个键值对的旋转参考角是否也符合升序规则，若存在键值对旋转参考角不符合升序规则，则这些键值对所对应的数字字符被判定为非刻度的数字字符并从数字字符中去除；6.1.6) Use the numeric character text content as the sorting key to sort the above key-value pair sequence according to the ascending order, check whether the rotation reference angle of each key-value pair in the sorted key-value pair sequence also conforms to the ascending order rule, if there is a key-value If the rotation reference angle does not conform to the ascending order rule, the numeric characters corresponding to these key-value pairs are judged as non-scale numeric characters and removed from the numeric characters;

6.1.7)经过上述筛选后保留下来的字符为最终的刻度数字；6.1.7) The characters retained after the above screening are the final scale numbers;

阶段二，匹配指针所在刻度区间：Phase 2, matching the scale interval where the pointer is located:

6.2.1)计算指针末端点到各个刻度数字的欧式距离，将与指针末端点距离最近的两个刻度数字作为刻度区间的上下界刻度数字；6.2.1) Calculate the Euclidean distance from the end point of the pointer to each scale number, and use the two scale numbers closest to the end point of the pointer as the upper and lower scale numbers of the scale interval;

6.2.2)定义两个点之间的角度为以指针旋转中心点为顶点，两点到指针旋转中心点的连线为两边所构成的锐角的角度；根据上述定义能够计算指针末端点分别到待定刻度区间上下界刻度数字之间的角度以及待定刻度区间上下界刻度数字之间的角度；6.2.2) Define the angle between two points as the vertex with the center point of the pointer rotation as the vertex, and the line connecting the two points to the center point of the pointer rotation is the angle of the acute angle formed by the two sides; according to the above definition, the end points of the pointer can be calculated to The angle between the upper and lower scale numbers of the undetermined scale interval and the angle between the upper and lower scale numbers of the undetermined scale interval;

6.2.3)若末端点到待定刻度区间上下界刻度数字的角度之和近似等于待定刻度区间上下界刻度数字的角度之间的角度，匹配成功；若不相等，则引入剩余数字字符中距离指针末端点最近的数字字符与上述两个刻度数字分别组成待定刻度区间，重复上述匹配过程，直到匹配成功，匹配成功的数字字符中数字字符文本内容较大的字符称为刻度区间上界数字，数字字符文本内容较小的字符称为刻度区间下界数字。6.2.3) If the sum of the angles between the end point and the upper and lower scale numbers of the undetermined scale interval is approximately equal to the angle between the angles between the upper and lower bound scale numbers of the undetermined scale interval, the matching is successful; if they are not equal, introduce the distance pointer in the remaining digital characters The number character closest to the end point and the above two scale numbers respectively form the undetermined scale interval, repeat the above matching process until the matching is successful, the character with the larger text content of the number character among the successfully matched number characters is called the upper boundary number of the scale interval, and the number The character with the smaller character text content is called the lower bound number of the scale interval.

进一步，在步骤6.1.4)中，对任意不共线三点坐标的角度计算，具体如下：Further, in step 6.1.4), the calculation of the angle of any non-collinear three-point coordinates is as follows:

利用勾股定理公式计算两点之间的线段长度：Use the Pythagorean formula to calculate the length of a line segment between two points:

式中，a′表示线段的长度，(x_b,y_b)和(x_c,y_c)分别为线段两端的端点坐标，基于上述公式能够计算出任意不共线三点组成的三角形三条边长度；In the formula, a' represents the length of the line segment, (x _b , y _b ) and (x _c , y _c ) are the endpoint coordinates of the two ends of the line segment respectively, based on the above formula, the three sides of a triangle composed of any three points that are not collinear can be calculated length;

利用解三角形的任意角度计算公式由边长计算角度：Use the arbitrary angle calculation formula for solving triangles to calculate angles from side lengths:

式中，a为待求角所在顶点的对边线段长度，b、c分别为待求角所在顶点的邻边线段长度，A1为待求角的角度。In the formula, a is the length of the line segment opposite to the vertex where the angle to be sought is located, b and c are the lengths of the line segment adjacent to the vertex where the angle is to be sought, respectively, and A1 is the angle of the angle to be sought.

进一步，在步骤7)中，所述仪表读数模块是由角度法改进得到的，公式表达如下：Further, in step 7), the meter reading module is obtained by improving the angle method, and the formula is expressed as follows:

式中，Reading表示最终得到的仪表读数；max_scale、min_scale分别表示指刻度区间上界数字和刻度区间下界数字的数字字符文本内容，匹配确定上下界刻度数字之后能够直接获得；ang_end表示指针旋转中心点和刻度区间下界数字之间的角度；ang_interval表示刻度区间上界数字和刻度区间下界数字之间的角度，在确定指针旋转中心点、指针末端点、刻度区间上界数字和刻度区间下界数字位置坐标的情况下能够计算得到ang_end和ang_int erval的角度值，计算方法与步骤6.1.4)中对任意不共线三点坐标的角度计算方法一致；In the formula, Reading represents the final reading of the instrument; max_scale and min_scale respectively represent the text content of the digital characters referring to the upper boundary number of the scale interval and the lower boundary number of the scale interval, which can be directly obtained after matching the upper and lower boundary numbers; ang_end represents the center point of the pointer rotation and the angle between the lower bound number of the scale interval; ang_interval indicates the angle between the upper bound number of the scale interval and the lower bound number of the scale interval, when determining the position coordinates of the pointer rotation center point, the end point of the pointer, the upper bound number of the scale interval and the lower bound number of the scale interval In the case of , the angle values of ang_end and ang_interval can be calculated, and the calculation method is consistent with the angle calculation method for any non-collinear three-point coordinates in step 6.1.4);

上述公式计算得到的结果即为圆形指针式仪表自动读数识别的最终结果。The result calculated by the above formula is the final result of the automatic reading recognition of the circular pointer instrument.

本发明与现有技术相比，具有如下优点与有益效果：Compared with the prior art, the present invention has the following advantages and beneficial effects:

1、本发明方法具有很强的通用性，能够在不依赖先验知识，即不限场景，不限具体仪表型号的情况下，对所有单读数系统的圆形仪表进行自动读数识别，通用性上显著好于现有主流方法。1. The method of the present invention has strong versatility, and can automatically recognize the readings of all circular instruments with a single-reading system without relying on prior knowledge, that is, without limiting the scene or the specific instrument model. significantly outperforms existing mainstream methods.

2、本发明方法具有模块化的特点，每个阶段之间的方法相对独立，可以分别进行优化而不影响整体方法的实现，这样在不同领域出现了性能更优的深度学习模型时，可以快速将业界前沿的算法应用到系统之中，保持性能的领先水平。2. The method of the present invention has the characteristics of modularization. The methods between each stage are relatively independent, and can be optimized separately without affecting the realization of the overall method. In this way, when a deep learning model with better performance appears in different fields, it can quickly Apply the industry's cutting-edge algorithms to the system to maintain the leading level of performance.

3、本发明方法设计了具有一定适应性的图像增强模块，对于场景图像中出现的仪表模糊、仪表分辨率过低的问题能够直接进行处理，可以在一定程度上克服工程现场带来的干扰因素，增强了方法的实用性。3. The method of the present invention designs an image enhancement module with certain adaptability, which can directly deal with the problems of instrument blurring and low instrument resolution in the scene image, and can overcome the interference factors brought by the engineering site to a certain extent , which enhances the practicability of the method.

4、本发明方法在分析仪表表盘信息的时候引入了基于OCR技术的表盘字符信息提取模块，基于这一模块可以进一步获取仪表类型的相关信息，从而实现仪表类型识别，仪表读数单位自动识别等功能，具备更好的进一步开发潜力。4. The method of the present invention introduces a dial character information extraction module based on OCR technology when analyzing the meter dial information. Based on this module, the relevant information of the meter type can be further obtained, thereby realizing functions such as meter type identification and meter reading unit automatic recognition. , with better potential for further development.

附图说明Description of drawings

图1为本发明方法的总体流程图。Fig. 1 is the overall flowchart of the method of the present invention.

图2为高质量表盘ROI图像提取流程图。Figure 2 is a flow chart of high-quality dial ROI image extraction.

图3为表盘校正辅助示意图。Fig. 3 is an auxiliary schematic diagram of dial correction.

图4为表盘有效信息提取流程图。Fig. 4 is a flow chart of extracting effective information on the dial.

图5为旋转参考角实例示意图。Fig. 5 is a schematic diagram of an example of a rotating reference angle.

图6为三角形计算公式辅助示意图。Fig. 6 is an auxiliary schematic diagram of a triangle calculation formula.

图7为仪表读数计算实例示意图。Figure 7 is a schematic diagram of an example of meter reading calculation.

具体实施方式detailed description

下面结合实施例及附图对本发明作进一步详细的描述，但本发明的实施方式不限于此。The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

如图1所示，巡检机器人通过可见光相机获取待检测场景图像(即巡检现场的图像)并输入表盘区域提取模块，以获取表盘ROI图像，然后根据表盘ROI图像的仪表可读性指数计算结果确定对应使用的图像增强模块，表盘ROI图像经过图像增强后进行表盘校正得到高质量表盘ROI图像。将高质量表盘ROI图像输入表盘字符信息提取模块和表盘指针信息提取模块分别得到仪表表盘的字符和指针有效信息，经过指针刻度区间匹配模块找到指针所在刻度区间，然后利用仪表读数模块计算最终的仪表读数。下面结合更加详细的流程图和示意图对上述模块进一步详细介绍。As shown in Figure 1, the inspection robot obtains the image of the scene to be detected (that is, the image of the inspection site) through the visible light camera and inputs it into the dial area extraction module to obtain the dial ROI image, and then calculates the instrument readability index based on the dial ROI image As a result, the corresponding image enhancement module was determined, and the ROI image of the dial was enhanced and corrected to obtain a high-quality dial ROI image. Input the high-quality dial ROI image into the dial character information extraction module and the dial pointer information extraction module to obtain the effective information of the instrument dial characters and pointers respectively, find the scale interval where the pointer is located through the pointer scale interval matching module, and then use the meter reading module to calculate the final meter reading. The above modules will be further described in detail below in combination with more detailed flowcharts and schematic diagrams.

如图2所示，输入的场景图像首先被送入一个基于深度学习实例分割技术开发的表盘区域提取模块，所示表盘区域提取模块为一个已训练的实例分割深度学习模型，具体为Mask RCNN模型，通过该模型能够获取仪表表盘在场景图像中的矩形边界框坐标和仪表图像掩膜，从场景图像中分割出上述矩形边界框范围内部的子图，并利用对应的仪表图像掩膜去除上述子图中表盘区域之外的部分，得到仅包含仪表表盘的部分，称为表盘ROI图像。As shown in Figure 2, the input scene image is first sent to a dial area extraction module developed based on deep learning instance segmentation technology. The dial area extraction module shown is a trained instance segmentation deep learning model, specifically the Mask RCNN model , through this model, the rectangular bounding box coordinates of the instrument panel in the scene image and the instrument image mask can be obtained, and the sub-images inside the above-mentioned rectangular bounding box range can be segmented from the scene image, and the above sub-images can be removed by using the corresponding instrument image mask. The part outside the dial area in the figure, the part containing only the dial is obtained, which is called the dial ROI image.

接着，利用拉普拉斯算子计算该表盘ROI图像的图像方差作为图像模糊程度的评价依据，上述图像方差的计算过程如下：Next, use the Laplacian operator to calculate the image variance of the ROI image of the dial as the basis for evaluating the degree of image blur. The calculation process of the above image variance is as follows:

1)将彩色的表盘ROI图像转化为表盘ROI灰度图像；1) Convert the colored dial ROI image into a dial ROI grayscale image;

2)使用如下拉普拉斯算子模板对表盘ROI灰度图像进行卷积运算：2) Use the following Laplacian operator template to perform convolution operation on the dial ROI grayscale image:

3)使用如下公式计算卷积后表盘ROI灰度图像的方差var(即表盘ROI图像的图像方差)，将该指标作为模糊度的判断指标：3) Use the following formula to calculate the variance var of the dial ROI grayscale image after convolution (that is, the image variance of the dial ROI image), and use this index as a judgment index of blur:

式中，x表示图像上一点的像素值，μ表示图像的灰度均值，式子右边分子部分的求和范围包括图像中的所有像素，H和W分别代表图像的高度和宽度，W*H表示图像的总像素个数。In the formula, x represents the pixel value of a point on the image, μ represents the average gray value of the image, the summation range of the molecular part on the right side of the formula includes all pixels in the image, H and W represent the height and width of the image respectively, W*H Indicates the total number of pixels in the image.

结合表盘ROI图像的图像方差和表盘ROI图像的尺寸提出“仪表可读性指数”这一度量指标，记作F，其计算公式如下：Combining the image variance of the dial ROI image and the size of the dial ROI image, the measurement index "Instrument Readability Index" is proposed, denoted as F, and its calculation formula is as follows:

式中，var代表得到的图像方差；α和β为两个权重系数，用于调节模糊程度和尺寸大小在仪表可读性指数计算中的权重，β设定为仪表表盘图像数据集中图像的尺寸平均值，α默认状态设定为1，后续测试过程中可以根据具体需要在工程实践中对这两权重参数进行调整以获得最优效果。In the formula, var represents the variance of the obtained image; α and β are two weight coefficients, which are used to adjust the weight of the degree of blur and size in the calculation of the instrument readability index, and β is set as the size of the image in the instrument dial image dataset The average value and α are set to 1 by default, and these two weight parameters can be adjusted in engineering practice according to specific needs in the subsequent testing process to obtain the optimal effect.

所述图像增强模块包括一个超分辨率重建深度学习模型和一个双边滤波模块，根据不同的仪表可读性指数计算结果，选择不同图像增强方法对表盘ROI图像进行增强。The image enhancement module includes a super-resolution reconstruction deep learning model and a bilateral filter module, and selects different image enhancement methods to enhance the ROI image of the dial according to different calculation results of the instrument readability index.

当所述仪表可读性指标的计算结果大于设定阈值时，将该图像判定为“低可读性图像”进一步将该表盘ROI图像输入超分辨率重建深度学习模型(RealSR模型)中，使用超分辨率重建对高度模糊的图像进行增强。反之，若上述表盘ROI图像的仪表可读性指标被判定为“非低可读性图像”，则将该图像输入双边滤波模块，对图像进行常规图像增强处理，上述双边滤波模块的计算公式为：When the calculation result of the instrument readability index is greater than the set threshold, the image is judged as a "low readability image" and the dial ROI image is further input into the super-resolution reconstruction deep learning model (RealSR model), using Super-resolution reconstruction enhances highly blurred images. Conversely, if the instrument readability index of the above-mentioned dial ROI image is judged to be "not a low-readability image", then the image is input into the bilateral filter module, and the image is subjected to conventional image enhancement processing. The calculation formula of the above-mentioned bilateral filter module is :

式中，f(x,y)为滤波之后像素点(x,y)位置的响应，g(x,y)为像素点(x,y)邻域内各个像素点的值，W(x,y)为各像素点的组合权重系数。该双边滤波模块的主要作用为在保持边缘清晰的情况下对其余部分进行平滑滤波以去除图像噪声，改善图像质量。In the formula, f(x, y) is the response of the position of the pixel (x, y) after filtering, g(x, y) is the value of each pixel in the neighborhood of the pixel (x, y), W(x, y ) is the combined weight coefficient of each pixel. The main function of the bilateral filtering module is to smooth and filter the remaining part while keeping the edge clear to remove image noise and improve image quality.

经过图像增强之后利用实例分割深度学习模型得到的仪表图像掩膜来进行图像校正，以解决巡检机器人由于拍摄视角未正对仪表表盘带来的失真问题。如图3所示为表盘校正辅助示意图。倾斜视角下拍摄得到的是一个近似椭圆形的仪表轮廓，对该仪表图像掩膜进行轮廓检测和椭圆拟合之后，可以得到如图3中实线所示的椭圆ABCD。在目前的主流计算机视觉算法库中所述拟合椭圆由椭圆中心点、椭圆长轴长度、椭圆短轴长度和椭圆长轴与竖直方向的夹角描述，基于上述信息结合几何知识可以得到该椭圆的四个长短轴交点：A、B、C、D的位置坐标。然后可以由几何知识求得仪表短轴所在直线上距离圆心距离为长轴长度的两点B'、D'位置坐标，将A和C点位置不变重新记作A'和C'点，得到一个圆A'B'C'D'作为校正后的期望表盘轮廓(如图3中虚线圆所示)，A'B'C'D'四点的位置坐标即为椭圆期望校正坐标。After image enhancement, the instrument image mask obtained by the instance segmentation deep learning model is used for image correction to solve the distortion problem caused by the inspection robot's shooting angle not facing the instrument dial. Figure 3 is a schematic diagram of the auxiliary dial calibration. An approximately elliptical profile of the instrument is obtained by shooting at an oblique angle of view. After contour detection and ellipse fitting are performed on the instrument image mask, an ellipse ABCD as shown by the solid line in Figure 3 can be obtained. In the current mainstream computer vision algorithm library, the fitting ellipse is described by the center point of the ellipse, the length of the major axis of the ellipse, the length of the minor axis of the ellipse, and the angle between the major axis of the ellipse and the vertical direction. Based on the above information combined with geometric knowledge, the The intersection points of the four long and short axes of the ellipse: the position coordinates of A, B, C, and D. Then, the position coordinates of the two points B' and D' on the straight line where the short axis of the instrument is located and the distance from the center of the circle to the length of the long axis can be obtained from geometric knowledge, and the positions of points A and C remain unchanged as points A' and C' to obtain A circle A'B'C'D' is used as the corrected expected dial outline (as shown by the dotted circle in Figure 3), and the position coordinates of the four points of A'B'C'D' are the expected corrected coordinates of the ellipse.

利用ABCD和A'B'C'D'构成的4组特征点对，可以求得射影变换矩阵，进而利用射影变换完成仪表的校正工作。所述射影变换矩阵的计算公式如下：The projective transformation matrix can be obtained by using four groups of characteristic point pairs composed of ABCD and A'B'C'D', and then the calibration of the instrument can be completed by projective transformation. The calculation formula of the projective transformation matrix is as follows:

p2＝H′*p1p2=H'*p1

式中，p1表示变换前原图一个点、p2表示p1对应的特征点，H′为射影变换矩阵，表示了原图中点p1映射到变换后图像中点p2的过程，将上述式子展开得到：In the formula, p1 represents a point in the original image before transformation, p2 represents the feature point corresponding to p1, and H′ is the projective transformation matrix, which represents the process of mapping the point p1 in the original image to the midpoint p2 in the transformed image. Expand the above formula to get :

式中，(x₁,y₁)表示点p1的坐标、(x₂,y₂)表示点p2的坐标，H₁₁～H₃₃均为矩阵H′的参数，由上述矩阵形式能够得到一个方程组；其中，由H₁₁～H₃₃这9个参数，利用上述的AA'、BB'、CC'、DD'这4组特征点对可以联立构建四组方程组，即可解得唯一的射影变换矩阵H′；In the formula, (x ₁ , y ₁ ) represents the coordinates of point p1, (x ₂ , y ₂ ) represents the coordinates of point p2, H ₁₁ ~ H ₃₃ are all parameters of matrix H′, and an equation can be obtained from the above matrix form group; among them, from the nine parameters H ₁₁ ~ H ₃₃ , using the above four groups of feature point pairs AA', BB', CC', DD' can simultaneously construct four sets of equations, and the unique Projective transformation matrix H';

利用射影变换矩阵即可实现对整幅图像的射影变换，最终实现表盘校正的任务。经过校正之后，得到了较为清晰且正对相机视角的表盘ROI图像，称为高质量表盘ROI图像。The projective transformation of the entire image can be realized by using the projective transformation matrix, and finally the task of dial correction can be realized. After correction, a relatively clear ROI image of the dial facing the camera is obtained, which is called a high-quality dial ROI image.

如图4所示为表盘有效信息提取流程图，图中左右两条分支分别对应表盘字符信息提取模块和表盘指针信息提取模块的具体流程，分支合并之后则进入了指针刻度区间匹配模块。Figure 4 shows the flow chart of effective information extraction on the dial. The left and right branches in the figure correspond to the specific processes of the dial character information extraction module and the dial pointer information extraction module. After the branches are merged, they enter the pointer scale interval matching module.

表盘字符信息提取模块主要包括文字检测、文字识别两部分。高质量表盘ROI图像首先输入经过已训练的场景文本检测深度学习模型(DB模型)，该模型以矩形边界框四点坐标的形式输出高质量表盘ROI图像内的所有字符区域信息，所述字符区域信息为字符的矩形边界框，由矩形边界框的四个顶点坐标描述，将矩形边界框范围内的子图分割下来得到高质量表盘ROI的字符图像序列。利用所述字符图像序列中各个字符图像的矩形边界框顶点坐标，计算每个字符位置坐标，所述字符位置坐标定义为字符矩形边界框顶点的中心坐标，其横纵坐标分别为四个矩形边界框顶点坐标的平均值。接着，将这些字符图像序列中的字符图像依次送入已训练的字符图像文字识别深度学习模型(CRNN模型)，得到每个字符图像的文本识别结果，即字符图像所对应字符文本内容。将识别得到字符文本内容和对应的位置坐标利用字典等数据结构成对保存，这样就完成了高质量表盘ROI图像内所有字符信息的提取工作，下文中“字符”均指字符图像及其识别结果，包括字符的文本内容和位置坐标两个属性。The dial character information extraction module mainly includes two parts: text detection and text recognition. The high-quality dial ROI image is first input into the trained scene text detection deep learning model (DB model), and the model outputs all character area information in the high-quality dial ROI image in the form of four-point coordinates of a rectangular bounding box. The character area The information is a rectangular bounding box of characters, described by the coordinates of four vertices of the rectangular bounding box, and the sub-images within the range of the rectangular bounding box are divided to obtain a character image sequence of high-quality dial ROI. Utilize the rectangular bounding box vertex coordinates of each character image in the character image sequence to calculate the position coordinates of each character, the character position coordinates are defined as the center coordinates of the character rectangular bounding box vertices, and its horizontal and vertical coordinates are respectively four rectangular boundaries Average of box vertex coordinates. Then, the character images in these character image sequences are sequentially sent to the trained character image text recognition deep learning model (CRNN model) to obtain the text recognition result of each character image, that is, the character text content corresponding to the character image. The recognized character text content and the corresponding position coordinates are stored in pairs using a dictionary and other data structures, thus completing the extraction of all character information in the high-quality dial ROI image. Hereinafter, "character" refers to the character image and its recognition result , including the text content and position coordinates of the character.

表盘指针信息提取模块主要包括一个目标检测深度学习模型(FasterRCNN模型)和一个关键点检测深度学习模型(HRNet模型)。高质量表盘ROI图像首先输入训练好的目标检测深度学习模型中，得到指针的矩形边界框参数，将矩形边界框范围内的子图分割下来，得到指针ROI图像并输入关键点检测深度学习模型中，关键点检测深度学习模型直接预测推理得到指针关键点及相应坐标，所述指针关键点包括指针旋转中心点和指针末端点，这两关键点及相应坐标就是所需指针有效信息，两关键点的连线反映指针所在的直线。The dial pointer information extraction module mainly includes a target detection deep learning model (FasterRCNN model) and a key point detection deep learning model (HRNet model). The high-quality dial ROI image is first input into the trained target detection deep learning model to obtain the pointer's rectangular bounding box parameters, and the sub-images within the rectangular bounding box are divided to obtain the pointer ROI image and input into the key point detection deep learning model , the key point detection deep learning model directly predicts and infers the key points of the pointer and the corresponding coordinates. The key points of the pointer include the center point of the pointer rotation and the end point of the pointer. These two key points and the corresponding coordinates are the effective information of the required pointer. The two key points The connecting line of reflects the straight line where the pointer is.

指针刻度区间匹配模块主要包括刻度数字筛选和匹配指针刻度所在区间两个阶段。The pointer scale interval matching module mainly includes two stages: scale number screening and matching the interval where the pointer scale is located.

刻度数字筛选阶段主要基于上述表盘字符信息提取模块得到的字符和表盘指针信息提取模块得到的指针旋转中心点坐标筛选刻度数字，主要目的是从字符中分离出刻度数字，同时要避免将其它字符识别被误判为刻度数字或者非刻度的数字字符被筛选为刻度数字的情况，整个筛选过程分三轮进行。The scale number screening stage is mainly based on the characters obtained by the above-mentioned dial character information extraction module and the coordinates of the pointer rotation center point obtained by the dial pointer information extraction module. The main purpose is to separate the scale numbers from the characters and avoid identifying other characters. In the case of being misjudged as a scale number or a non-scale number character is screened as a scale number, the whole screening process is divided into three rounds.

第一轮筛选根据字符图像文本识别结果得到的字符文本内容，筛选出所有字符文本内容为纯数字的字符，作为刻度数字的初步筛选结果，以下简称为数字字符。The first round of screening is based on the character text content obtained from the character image text recognition results, and all characters whose text content is pure numbers are screened out, as the preliminary screening results of scale numbers, hereinafter referred to as digital characters.

第二轮筛选基于指针式仪表各个刻度数字大体分布在以指针旋转中心为圆心的一个圆上的特性，利用各个数字字符到指针旋转中心点之间的距离，去除空间分布异常的数字字符。The second round of screening is based on the characteristic that each scale number of the pointer instrument is roughly distributed on a circle centered on the pointer rotation center, and the number characters with abnormal spatial distribution are removed by using the distance between each digital character and the pointer rotation center point.

首先计算各个数字字符到指针旋转中心点的欧式距离，计算公式如下：First calculate the Euclidean distance from each numeric character to the center point of pointer rotation, the calculation formula is as follows:

计算完毕后得到一个各个数字字符的距离序列。After the calculation is completed, a distance sequence of each numeric character is obtained.

使用k-means算法对上述数字字符距离序列中的距离进行聚类，将上述距离聚类为三个簇。聚类之后数字字符数最多的簇作为数字字符的有效筛选结果，其他两个簇则代表了距离指针旋转中心点过近和距离指针旋转中心点过远的数字字符，均为距离序列中的离群点，判定为无效筛选结果，从数字字符中去除。The k-means algorithm is used to cluster the distances in the above-mentioned numeric character distance sequence, and the above-mentioned distances are clustered into three clusters. After clustering, the cluster with the largest number of numeric characters is the effective screening result of numeric characters, and the other two clusters represent the numeric characters that are too close to the center of pointer rotation and too far from the center of pointer rotation, both of which are distances in the distance sequence. Cluster points, which are judged as invalid screening results, are removed from the numeric characters.

第三轮筛选基于指针式仪表各个刻度数字随着指针旋转方向不断增大的特性，计算各个数字字符所对应的参考夹角，结合指针旋转方向上的刻度数字文本内容进行排序，去除不符合上述规律的数字字符。The third round of screening is based on the characteristic that each scale number of the pointer instrument increases continuously with the direction of pointer rotation, calculates the reference angle corresponding to each number character, and sorts according to the text content of the scale number in the direction of pointer rotation, and removes those that do not meet the above requirements. Regular numeric characters.

由图5所示的旋转参考角实例示意图，选用过旋转中心点竖直向下的射线作参考线，定义数字字符X和指针旋转中心点O连线与参考线之间的夹角为数字字符X所对应的旋转参考角，所述参考线为从指针旋转中心点出发，竖直向下的射线。在具体计算过程中可以选择O点下方射线上任一点作为参考点Y，只要保证Y的横坐标和O点一致，纵坐标位于O点的下方即可，利用OXY三个点坐标计算以O点为顶点的角度值，即为旋转参考角。From the schematic diagram of the example of the rotation reference angle shown in Figure 5, the ray passing through the rotation center point vertically downward is selected as the reference line, and the angle between the line connecting the digital character X and the pointer rotation center point O and the reference line is defined as the digital character The rotation reference angle corresponding to X, and the reference line is a vertical downward ray starting from the rotation center point of the pointer. In the specific calculation process, you can choose any point on the ray below point O as the reference point Y, as long as the abscissa of Y is consistent with point O, and the ordinate is below point O, use the coordinates of OXY three points to calculate point O The angle value of the vertex is the rotation reference angle.

参考图6所示的三角形计算公式辅助示意图，基于三个点计算角度值的方法如下：Referring to the auxiliary schematic diagram of the triangle calculation formula shown in Figure 6, the method of calculating the angle value based on three points is as follows:

利用勾股定理公式能够计算任意两点之间的线段长度，以图6中a边长度的计算为例进行说明，计算公式如下：The length of the line segment between any two points can be calculated by using the formula of the Pythagorean theorem. Take the calculation of the length of side a in Figure 6 as an example to illustrate. The calculation formula is as follows:

式中，a表示边的长度，B1(x_b,y_b)和C1(x_c,y_c)分别为a线段两端的端点坐标，基于上述公式可以计算出图6中另外两边的长度，分别记为b,c；In the formula, a represents the length of the side, and B1(x _b , y _b ) and C1(x _c , y _c ) are the endpoint coordinates of the two ends of the line segment a, respectively. Based on the above formula, the lengths of the other two sides in Figure 6 can be calculated, respectively denoted as b,c;

利用解三角形的任意角度计算公式由边长计算角度，以图6中A1角的计算为例进行说明，计算公式如下：Use the arbitrary angle calculation formula for solving triangles to calculate the angle from the side length. Take the calculation of the A1 angle in Figure 6 as an example to illustrate. The calculation formula is as follows:

式中，a为待求角A1所在顶点的对边线段长度，b、c分别为待求角A1所在顶点的邻边线段长度，A1为待求角的角度。In the formula, a is the length of the line segment opposite to the vertex where the angle A1 is to be found, b and c are the lengths of the line segment adjacent to the vertex where the angle A1 is to be found, respectively, and A1 is the angle of the angle to be found.

根据上述方法可以计算所有数字字符对应的旋转参考角。对于指针中心点右侧的数字字符，按照指针式仪表各个刻度数字随着指针旋转方向不断增大的特性应该得到一个大于180度的角，由上述公式直接计算出角A1之后，(360-A1)才是真正的旋转参考角。The rotation reference angles corresponding to all numeric characters can be calculated according to the above method. For the digital characters on the right side of the center point of the pointer, an angle greater than 180 degrees should be obtained according to the characteristic that each scale number of the pointer instrument increases with the direction of rotation of the pointer. After the angle A1 is directly calculated by the above formula, (360-A1 ) is the real rotation reference angle.

构建键值对<数字字符文本内容，旋转参考角>，得到一个键值对序列，以数字字符文本内容作为排序关键字对键值对序列进行升序排序，检查排序后各个键值对的旋转参考角关键字是否也符合升序，若键值对序列中存在旋转参考角不符合升序，则这些键值对所对应的数字字符为异常数字字符，将其判定为非刻度的数字字符被并去除。Construct a key-value pair <numeric character text content, rotation reference angle> to obtain a key-value pair sequence, use the numeric character text content as a sort key to sort the key-value pair sequence in ascending order, and check the rotation reference of each key-value pair after sorting Whether the angle keyword also conforms to the ascending order. If there is a rotation reference angle in the key-value pair sequence that does not conform to the ascending order, the numeric characters corresponding to these key-value pairs are abnormal numeric characters, and the numeric characters that are determined to be non-scale are removed.

经过上述步骤筛选之后保留下来的数字字符为最终刻度数字识别结果，以下简称为刻度数字。The digit characters retained after the above steps of screening are the final scale digit recognition results, hereinafter referred to as scale digits.

指针末端点所在区间匹配部分依据指针末端点到刻度数字的距离和角度关系对刻度数字坐标进行匹配，确定指针末端点所在的刻度区间。匹配的原则是：以刻度数字到指针末端点的直线距离作为匹配指标，在保证末端点位于刻度区间上下界数字之间的情况下取距离指针末端点直线距离最近的刻度数字为匹配结果。具体匹配流程包括：The interval matching part where the pointer end point is located matches the coordinates of the scale number according to the distance and angle relationship between the pointer end point and the scale number, and determines the scale interval where the pointer end point is located. The matching principle is: take the linear distance from the scale number to the end point of the pointer as the matching index, and take the scale number closest to the end point of the pointer as the matching result under the condition that the end point is between the upper and lower bounds of the scale interval. The specific matching process includes:

1)计算指针末端点到各个刻度数字的欧式距离，将距离最小的两个刻度数字作为待定刻度区间的上下界刻度数字；1) Calculate the Euclidean distance from the end point of the pointer to each scale number, and use the two scale numbers with the smallest distance as the upper and lower scale numbers of the undetermined scale interval;

2)为便于说明，定义两点之间角度的概念，在本专利中，两个点之间的角度均指以指针旋转中心点为顶点，两点到指针旋转中心点的线段为两边所构成的锐角的角度。分别计算指针末端点和两个待定刻度数字之间的角度，以及两个刻度数字之间的角度，角度计算方法与指针刻度区间匹配模块部分所述的基于三个点计算角度值的方法完全一致；2) For the convenience of explanation, the concept of the angle between two points is defined. In this patent, the angle between two points refers to the center point of the pointer rotation as the vertex, and the line segment from the two points to the center point of the pointer rotation is formed by two sides. The angle of the acute angle. Calculate the angle between the end point of the pointer and the two undetermined scale numbers, and the angle between the two scale numbers. The angle calculation method is exactly the same as the method of calculating the angle value based on three points described in the pointer scale interval matching module. ;

3)若指针末端点到两个刻度数字的角度之和等于两个刻度数字之间的角度，则说明指针确实位于在两个刻度数字之间，匹配成功。若不相等，则引入剩余数字字符中距离指针末端点最近的刻度数字与上述两个数字刻度分别组成待定刻度区间，重复上述匹配过程，直到匹配成功，匹配成功的数字字符中数字字符文本内容较大的字符称为刻度区间上界数字，数字字符文本内容较小的字符称为刻度区间下界数字。3) If the sum of the angles from the end point of the pointer to the two scale numbers is equal to the angle between the two scale numbers, it means that the pointer is indeed located between the two scale numbers, and the match is successful. If they are not equal, introduce the scale number closest to the end point of the pointer among the remaining numeric characters and the above two numeric scales to form the undetermined scale interval, repeat the above matching process until the matching is successful, and the text content of the numeric characters in the successfully matched numeric characters is smaller. The larger character is called the upper bound number of the scale interval, and the character with smaller text content of the numeric character is called the lower bound number of the scale interval.

经过匹配之后得到了仪表读数的有效信息：指针末端点、指针旋转中心点、指针所在刻度区间上下界数字。After matching, the effective information of the meter reading is obtained: the end point of the pointer, the center point of the pointer rotation, and the upper and lower bounds of the scale interval where the pointer is located.

如图7所示为仪表读数计算实例示意图，图中F点代表仪表的刻度区间下界数字、G点代表仪表的刻度区间上界数字、O'点代表指针旋转中心点，E点代表指针末端点。As shown in Figure 7, it is a schematic diagram of an example of meter reading calculation. Point F in the figure represents the lower boundary number of the scale interval of the meter, point G represents the upper boundary number of the scale interval of the meter, point O' represents the center point of pointer rotation, and point E represents the end point of the pointer .

所述仪表读数模块是由角度法改进得到，即通过计算指针的旋转角度和量程的总角度进行对比来得到具体的读数，公式表达如下：The meter reading module is improved by the angle method, that is, the specific reading is obtained by comparing the rotation angle of the pointer with the total angle of the range, and the formula is expressed as follows:

式中，Reading表示最终得到的仪表读数；max_scale、min_scale分别表示指刻度区间上界数字和刻度区间下界数字的数字字符文本内容，分别对应识别得到的F点数字字符文本内容“20”、G点数字字符文本内容“40”；ang_end表示指针旋转中心点和刻度区间下界数字之间的角度，在示意图中对应∠FO'E；ang_interval表示刻度区间上界数字和刻度区间下界数字之间的角度，在示意图中对应角∠FO'G；In the formula, Reading represents the final reading of the instrument; max_scale and min_scale respectively represent the text content of the digital characters referring to the upper boundary number of the scale interval and the lower boundary number of the scale interval, corresponding to the recognized text content of the digital characters "20" at point F and point G respectively. The text content of numeric characters is "40"; ang_end indicates the angle between the center point of the pointer rotation and the lower boundary number of the scale interval, which corresponds to ∠FO'E in the schematic diagram; ang_interval indicates the angle between the upper boundary number of the scale interval and the lower boundary number of the scale interval, In the schematic diagram, the corresponding angle ∠FO'G;

最终得到本实例中仪表的计算读数如下：Finally, the calculated reading of the instrument in this example is as follows:

上述实施例为本发明较佳的实施方式，但本发明的实施方式并不受上述实施例的限制，其他的任何未背离本发明的精神实质与原理下所作的改变、修饰、替代、组合、简化，均应为等效的置换方式，都包含在本发明的保护范围之内。The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

1.一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于，包括以下步骤：1. A circular pointer instrument automatic reading method based on deep learning, characterized in that, comprising the following steps: 1)通过可见光相机获取巡检现场的图像，称为场景图像；1) Obtain the image of the inspection site through the visible light camera, which is called the scene image; 2)使用一个基于深度学习实例分割技术开发的表盘区域提取模块对场景图像中的仪表表盘区域进行检测，将仪表表盘图像从场景图像中分割提取出来，去除背景部分，得到的仅包含仪表表盘的部分，称为表盘ROI图像；2) Use a dial area extraction module developed based on deep learning instance segmentation technology to detect the instrument dial area in the scene image, segment and extract the instrument dial image from the scene image, remove the background part, and obtain only the instrument dial area part, called dial ROI image; 3)通过综合表盘ROI图像的尺寸和模糊度，得到一个度量仪表可读性的指标，称为仪表可读性指数，根据该指数的计算结果，匹配对应的图像增强模块对表盘ROI图像进行图像增强；3) By synthesizing the size and ambiguity of the dial ROI image, an index for measuring the readability of the meter is obtained, which is called the meter readability index. According to the calculation result of the index, match the corresponding image enhancement module to image the dial ROI image enhanced; 4)使用一个基于图像变换技术开发的表盘倾斜校正模块解决拍摄过程中由于相机非正对仪表所在平面拍摄导致的表盘倾斜问题，经过图像增强和表盘校正之后的表盘图像称为高质量表盘ROI图像；4) Use a dial tilt correction module developed based on image transformation technology to solve the problem of dial tilt caused by the camera not facing the plane where the instrument is located during the shooting process. The dial image after image enhancement and dial correction is called a high-quality dial ROI image ; 5)利用深度学习技术提取仪表表盘信息，具体是分别使用基于深度学习OCR技术开发的表盘字符信息提取模块获取高质量表盘ROI图像的字符信息和基于深度学习关键点检测技术开发的表盘指针信息提取模块获取高质量表盘ROI图像的指针信息；5) Use deep learning technology to extract instrument dial information, specifically use the dial character information extraction module developed based on deep learning OCR technology to obtain high-quality dial ROI image character information and the dial pointer information developed based on deep learning key point detection technology The module obtains the pointer information of the high-quality dial ROI image; 6)使用一个指针刻度区间匹配模块从上述仪表表盘信息的提取结果中获取仪表读数的有效信息，所述仪表读数的有效信息分别指：指针末端点、指针旋转中心点、指针所在刻度区间的上下界刻度数字；6) Use a pointer scale interval matching module to obtain the effective information of the instrument reading from the extraction result of the above-mentioned instrument dial information. The effective information of the instrument reading refers to: the end point of the pointer, the center point of the pointer rotation, and the upper and lower points of the scale interval where the pointer is located. Boundary scale numbers; 7)使用一个仪表读数模块，基于上述仪表读数的有效信息计算仪表读数。7) Using a meter reading module to calculate the meter reading based on the available information of the above meter reading. 2.根据权利要求1所述的一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于：在步骤2)中，所述表盘区域提取模块为一个经过训练的实例分割深度学习模型，通过该模型能够获取仪表表盘在场景图像中的矩形边界框坐标和图像掩膜，进而从场景图像中分割出上述矩形边界框范围内部的子图，并利用所述图像掩膜去除上述子图中表盘区域之外的部分，得到仅包含仪表表盘的部分，称为表盘ROI图像。2. a kind of circular pointer instrument automatic reading method based on deep learning according to claim 1, is characterized in that: in step 2) in, described dial region extracting module is a trained instance segmentation deep learning model , through this model, the coordinates of the rectangular bounding box of the instrument panel in the scene image and the image mask can be obtained, and then the sub-image inside the range of the above-mentioned rectangular bounding box can be segmented from the scene image, and the above-mentioned sub-image can be removed by using the image mask The part outside the middle dial area, the part containing only the instrument dial is obtained, which is called the dial ROI image. 3.根据权利要求2所述的一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于：在步骤3)中，所述仪表可读性指数记为F，该指数的计算公式如下：3. A kind of circular pointer instrument automatic reading method based on deep learning according to claim 2, characterized in that: in step 3), the instrument readability index is denoted as F, and the calculation formula of this index as follows:

式中，var代表图像方差，描述了图像的模糊程度；H和W分别代表图像的高度和宽度，描述了图像的尺寸大小；α和β为两个权重系数，用于调节模糊程度和尺寸大小对仪表可读性指数计算结果的影响权重；In the formula, var represents the variance of the image, which describes the degree of blur of the image; H and W represent the height and width of the image, respectively, which describe the size of the image; α and β are two weight coefficients, which are used to adjust the degree of blur and the size of the image The impact weight on the calculation result of instrument readability index; 其中，所述图像方差var表示对所有像素的灰度值与图像平均灰度值之差的平方求和，然后除以像素总数的计算结果，具体计算过程包括以下步骤：Wherein, the image variance var represents the sum of the squares of the difference between the gray value of all pixels and the average gray value of the image, and then divides the calculation result by the total number of pixels. The specific calculation process includes the following steps: 3.1)将彩色的表盘ROI图像转化为表盘ROI灰度图像；3.1) Convert the colored dial ROI image into a dial ROI grayscale image; 3.2)使用如下拉普拉斯算子模板对表盘ROI灰度图像进行卷积运算：3.2) Use the following Laplacian template to perform convolution operation on the dial ROI grayscale image:

3.3)使用如下公式计算卷积后表盘ROI灰度图像的方差σ²：3.3) Use the following formula to calculate the variance σ ² of the dial ROI grayscale image after convolution:

式中，x表示图像上一点的像素值，μ表示图像的灰度均值，分子的求和范围包括图像中的所有像素，W*H表示图像的总像素个数。In the formula, x represents the pixel value of a point on the image, μ represents the average gray value of the image, the summation range of the numerator includes all pixels in the image, and W*H represents the total number of pixels in the image. 4.根据权利要求3所述的一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于：在步骤3)中，所述图像增强模块包括一个超分辨率重建深度学习模型和一个双边滤波模块，根据不同的仪表可读性指数计算结果，选择不同图像增强方法对表盘ROI图像进行增强；4. a kind of circular pointer instrument automatic reading method based on deep learning according to claim 3, is characterized in that: in step 3) in, described image enhancement module comprises a super-resolution reconstruction deep learning model and a The bilateral filter module selects different image enhancement methods to enhance the ROI image of the dial according to the calculation results of different instrument readability indexes; 根据输入表盘ROI图像的仪表可读性指数F，若F大于预先设定的阈值，则判断表盘ROI图像为“低可读性图像”，利用超分辨率重建深度学习模型对输入图像进行图像超分辨率重建处理；若F小于预先设定的阈值，则判断表盘ROI图像为“非低可读性图像”，利用双边滤波模块对图像进行图像增强处理；According to the instrument readability index F of the input dial ROI image, if F is greater than the preset threshold, it is judged that the dial ROI image is a "low readability image", and the input image is super-resolved using a super-resolution reconstruction deep learning model. Resolution reconstruction processing; if F is less than the preset threshold, it is judged that the ROI image of the dial is a "non-low readability image", and the bilateral filter module is used to perform image enhancement processing on the image; 所述双边滤波模块的计算公式为：The calculation formula of the bilateral filtering module is:

式中，f(x,y)为滤波之后像素点(x,y)位置的响应，g(x,y)为像素点(x,y)邻域内各个像素点的值，W(x,y)为各像素点的组合权重系数；该双边滤波模块的作用为在保持边缘清晰的情况下对其余部分进行平滑滤波以去除图像噪声，改善图像质量。In the formula, f(x, y) is the response of the position of the pixel (x, y) after filtering, g(x, y) is the value of each pixel in the neighborhood of the pixel (x, y), W(x, y ) is the combined weight coefficient of each pixel; the function of the bilateral filtering module is to perform smooth filtering on the remaining part while keeping the edge clear to remove image noise and improve image quality. 5.根据权利要求4所述的一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于：在步骤4)中，所述表盘倾斜校正模块具体执行以下操作：5. A kind of method for automatic reading of a circular pointer instrument based on deep learning according to claim 4, characterized in that: in step 4), the dial tilt correction module specifically performs the following operations: 4.1)获取表盘区域提取模块得到的仪表图像掩膜，为仪表轮廓二值掩膜mask图像；4.1) Obtain the instrument image mask obtained by the dial area extraction module, which is the instrument contour binary mask mask image; 4.2)对上述掩膜图像进行椭圆拟合得到表盘ROI图像的轮廓拟合椭圆参数，包括椭圆中心点、椭圆长轴长度、椭圆短轴长度和椭圆长轴与竖直方向的夹角；4.2) Carry out ellipse fitting to above-mentioned mask image and obtain the profile fitting ellipse parameter of dial ROI image, comprise ellipse central point, ellipse major axis length, ellipse minor axis length and the included angle of ellipse major axis and vertical direction; 4.3)获取拟合之后得到的椭圆的长轴和短轴端点的坐标，计算短轴所在直线上到椭圆中心点的距离为长轴长度的点的坐标，称为椭圆短轴的校正期望坐标；4.3) Obtain the coordinates of the major axis and the minor axis endpoint of the ellipse obtained after the fitting, and calculate the coordinates of the point whose distance from the minor axis to the center point of the ellipse is the length of the major axis on the straight line where the minor axis is located, which is called the corrected expected coordinate of the minor axis of the ellipse; 4.4)利用椭圆长轴以及短轴端点的坐标和椭圆长轴端点坐标以及椭圆短轴的校正期望坐标构成4组特征点对，利用这些特征点对计算得到射影变换矩阵，射影变换矩阵的计算如下：4.4) Use the coordinates of the major axis and minor axis endpoints of the ellipse, the coordinates of the major axis endpoints of the ellipse, and the corrected expected coordinates of the minor axis of the ellipse to form 4 sets of feature point pairs, and use these feature point pairs to calculate the projective transformation matrix. The calculation of the projective transformation matrix is as follows : p2＝H'*p1p2=H'*p1 式中，p1表示变换前原图一个点、p2表示p1对应的特征点，H′为射影变换矩阵，表示了原图中点p1映射到变换后图像中点p2的过程，将上述式子展开得到：In the formula, p1 represents a point in the original image before transformation, p2 represents the feature point corresponding to p1, and H′ is the projective transformation matrix, which represents the process of mapping the point p1 in the original image to the midpoint p2 in the transformed image. Expand the above formula to get :

式中，(x₁,y₁)表示点p1的坐标、(x₂,y₂)表示点p2的坐标，H₁₁～H₃₃均为矩阵H′的参数，由上述矩阵形式能够得到一个方程组；其中，由H₁₁～H₃₃这9个参数利用上述4组特征点对构建方程组即可解得唯一的射影变换矩阵H′；In the formula, (x ₁ , y ₁ ) represents the coordinates of point p1, (x ₂ , y ₂ ) represents the coordinates of point p2, H ₁₁ ~ H ₃₃ are all parameters of matrix H′, and an equation can be obtained from the above matrix form group; among them, the 9 parameters H ₁₁ ~ H ₃₃ use the above 4 groups of feature point pairs to construct a system of equations, which can be solved to obtain the unique projective transformation matrix H'; 4.5)利用步骤4.4)中公式得到的射影变换矩阵H′能够对表盘ROI图像进行全局处理，计算每一个校正前表盘ROI图像经过校正之后的像素坐标，从而实现将椭圆形的表盘ROI图像校正为正圆形。4.5) Using the projective transformation matrix H' obtained by the formula in step 4.4), the dial ROI image can be globally processed, and the pixel coordinates of each dial ROI image before correction can be calculated, so as to realize the correction of the elliptical dial ROI image as Perfect circle. 6.根据权利要求5所述的一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于：在步骤5)中，所述表盘字符信息提取模块要提取的表盘字符信息包括每个字符图像的字符文本内容及其对应的位置坐标，获取过程具体包括以下步骤：6. a kind of circular pointer instrument automatic reading method based on deep learning according to claim 5, is characterized in that: in step 5), the dial character information that described dial character information extraction module will extract includes each The character text content of the character image and its corresponding position coordinates, the acquisition process specifically includes the following steps: 5.1.1)输入经过图像增强和表盘校正的高质量表盘ROI图像于一个已经过训练的场景文本检测深度学习模型，该模型推理输出该高质量表盘ROI图像的所有字符区域信息，所述字符区域信息为字符的矩形边界框，由矩形边界框的四个顶点坐标描述；5.1.1) Input the high-quality dial ROI image through image enhancement and dial correction to a trained deep learning model for scene text detection, and the model reasoning outputs all character area information of the high-quality dial ROI image, and the character area The information is the rectangular bounding box of the character, described by the coordinates of the four vertices of the rectangular bounding box; 5.1.2)通过字符图像的矩形边界框坐标将所有字符从高质量表盘ROI图像中依次分割下来，得到高质量表盘ROI图像的字符图像序列；5.1.2) All characters are sequentially segmented from the high-quality dial ROI image by the rectangular bounding box coordinates of the character image to obtain a character image sequence of the high-quality dial ROI image; 5.1.3)利用各个字符图像的矩形边界框顶点坐标，计算字符位置坐标，所述字符位置坐标定义为字符矩形边界框顶点的中心坐标，其横纵坐标分别为四个矩形边界框顶点坐标的平均值；5.1.3) Utilize the apex coordinates of the rectangular bounding boxes of each character image to calculate the character position coordinates, the character position coordinates are defined as the center coordinates of the apexes of the character's rectangular bounding boxes, and its horizontal and vertical coordinates are respectively the four coordinates of the apex coordinates of the rectangular bounding boxes. average value; 5.1.4)将所述字符图像序列中的每一个字符图像依次送入一个已经过训练的字符图像文本识别深度学习模型，得到字符图像序列中各个字符图像的文本内容识别结果，将文本内容和对应的位置坐标成对保存，即得到了表盘字符信息。5.1.4) Each character image in the character image sequence is sent into a trained character image text recognition deep learning model to obtain the text content recognition result of each character image in the character image sequence, and the text content and The corresponding position coordinates are stored in pairs, that is, the dial character information is obtained. 7.根据权利要求6所述的一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于：在步骤5)中，所述表盘指针信息提取模块具体执行以下操作：7. a kind of circular pointer instrument automatic reading method based on deep learning according to claim 6, is characterized in that: in step 5), described dial pointer information extraction module specifically performs the following operations: 5.2.1)通过目标检测深度学习模型对表盘内部的指针进行检测，得到指针的矩形边界框参数，包括指针的中心点位置坐标和矩形边界框的宽高；5.2.1) Detect the pointer inside the dial through the target detection deep learning model, and obtain the rectangular bounding box parameters of the pointer, including the coordinates of the center point of the pointer and the width and height of the rectangular bounding box; 5.2.2)基于指针的矩形边界框参数对从场景图像中获取指针矩形边界框范围内部的子图，得到的图像称为指针ROI图像；5.2.2) Based on the pointer's rectangular bounding box parameter pair, the subimage inside the pointer's rectangular bounding box range is obtained from the scene image, and the obtained image is called the pointer ROI image; 5.2.3)将指针ROI图像输入一个已经过训练的关键点检测深度学习模型，通过该模型直接推理得到指针旋转中心点和指针末端点位置坐标，上述两点及其位置为提取到的指针信息。5.2.3) Input the pointer ROI image into a trained deep learning model for key point detection, through which the model can be directly inferred to obtain the position coordinates of the pointer rotation center point and the pointer end point, the above two points and their positions are the extracted pointer information . 8.根据权利要求7所述的一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于：在步骤6)中，所述指针刻度区间匹配模块分为刻度数字筛选和匹配指针所在刻度区间这两个阶段进行：8. A method for automatic reading of circular pointer meters based on deep learning according to claim 7, characterized in that: in step 6), the pointer scale interval matching module is divided into scale digital screening and matching pointer where Tick intervals are performed in two phases: 阶段一、刻度数字筛选：Stage 1. Scale number screening: 6.1.1)定义刻度数字为表盘ROI图像中表示仪表刻度的数字字符，根据表盘信息提取模块得到的字符文本内容首先筛选出所有文本内容为纯数字的字符，作为刻度数字的初步筛选结果，以下简称数字字符；6.1.1) Define the scale number as the digital character representing the meter scale in the dial ROI image. According to the character text content obtained by the dial information extraction module, first filter out all the characters whose text content is pure numbers, as the preliminary screening result of the scale number, as follows Numeric characters for short; 6.1.2)计算上述筛选后数字字符到指针旋转中心点的欧式距离，得到一个距离序列；6.1.2) Calculate the Euclidean distance from the above-mentioned screened digital characters to the pointer rotation center point to obtain a distance sequence; 6.1.3)使用k-means聚类算法对上述距离序列进行聚类，设定聚类的类别数k为3，指针式仪表的刻度数字由于空间分布近似在同一个圆上，故到旋转中心点的距离近似一致，被聚类为同一个类别，将这一类别内的数字字符作为数字字符的有效筛选结果，其余数字字符判定为无效筛选结果，从数字字符中去除；6.1.3) Use the k-means clustering algorithm to cluster the above-mentioned distance series, set the number of clustering categories k to 3, and the scale numbers of the pointer instrument are approximately on the same circle due to the spatial distribution, so they reach the rotation center The distance of the points is approximately the same, and they are clustered into the same category, and the numeric characters in this category are regarded as valid screening results for numeric characters, and the rest of the numeric characters are judged as invalid screening results, and are removed from the numeric characters; 6.1.4)计算剩余各个数字字符的旋转参考角，所述旋转参考角定义为以过旋转中心点向下的直线作参考线，数字字符和指针旋转中心点的连线与从指针旋转中心点出发竖直向下的射线之间的角度；6.1.4) Calculate the rotation reference angle of each remaining digital character, the rotation reference angle is defined as a straight line passing through the rotation center point downward as a reference line, the connection line between the digital character and the pointer rotation center point and the pointer rotation center point The angle between rays starting vertically downward; 6.1.5)利用数字字符文本内容和对应的旋转参考角构成<数字字符文本内容，旋转参考角>键值对，所有剩余数字字符的键值对构成一个键值对序列；6.1.5) Use the numeric character text content and the corresponding rotation reference angle to form a key-value pair of <numerical character text content, rotation reference angle>, and all remaining numeric character key-value pairs form a key-value pair sequence; 6.1.6)以数字字符文本内容作为排序关键字对上述键值对序列按照升序规则排序，检查排序后键值对序列中各个键值对的旋转参考角是否也符合升序规则，若存在键值对旋转参考角不符合升序规则，则这些键值对所对应的数字字符被判定为非刻度的数字字符并从数字字符中去除；6.1.6) Use the numeric character text content as the sorting key to sort the above key-value pair sequence according to the ascending order, check whether the rotation reference angle of each key-value pair in the sorted key-value pair sequence also conforms to the ascending order rule, if there is a key-value If the rotation reference angle does not conform to the ascending order rule, the numeric characters corresponding to these key-value pairs are judged as non-scale numeric characters and removed from the numeric characters; 6.1.7)经过上述筛选后保留下来的字符为最终的刻度数字；6.1.7) The characters retained after the above screening are the final scale numbers; 阶段二，匹配指针所在刻度区间：Phase 2, matching the scale interval where the pointer is located: 6.2.1)计算指针末端点到各个刻度数字的欧式距离，将与指针末端点距离最近的两个刻度数字作为刻度区间的上下界刻度数字；6.2.1) Calculate the Euclidean distance from the end point of the pointer to each scale number, and use the two scale numbers closest to the end point of the pointer as the upper and lower scale numbers of the scale interval; 6.2.2)定义两个点之间的角度为以指针旋转中心点为顶点，两点到指针旋转中心点的连线为两边所构成的锐角的角度；根据上述定义能够计算指针末端点分别到待定刻度区间上下界刻度数字之间的角度以及待定刻度区间上下界刻度数字之间的角度；6.2.2) Define the angle between two points as the vertex with the center point of the pointer rotation as the vertex, and the line connecting the two points to the center point of the pointer rotation is the angle of the acute angle formed by the two sides; according to the above definition, the end points of the pointer can be calculated to The angle between the upper and lower scale numbers of the undetermined scale interval and the angle between the upper and lower scale numbers of the undetermined scale interval; 6.2.3)若末端点到待定刻度区间上下界刻度数字的角度之和近似等于待定刻度区间上下界刻度数字的角度之间的角度，匹配成功；若不相等，则引入剩余数字字符中距离指针末端点最近的数字字符与上述两个刻度数字分别组成待定刻度区间，重复上述匹配过程，直到匹配成功，匹配成功的数字字符中数字字符文本内容较大的字符称为刻度区间上界数字，数字字符文本内容较小的字符称为刻度区间下界数字。6.2.3) If the sum of the angles between the end point and the upper and lower scale numbers of the undetermined scale interval is approximately equal to the angle between the angles between the upper and lower bound scale numbers of the undetermined scale interval, the matching is successful; if they are not equal, introduce the distance pointer in the remaining digital characters The number character closest to the end point and the above two scale numbers respectively form the undetermined scale interval, repeat the above matching process until the matching is successful, the character with the larger text content of the number character among the successfully matched number characters is called the upper boundary number of the scale interval, and the number The character with the smaller character text content is called the lower bound number of the scale interval. 9.根据权利要求8所述的一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于：在步骤6.1.4)中，对任意不共线三点坐标的角度计算，具体如下：9. A method for automatic reading of a circular pointer instrument based on deep learning according to claim 8, characterized in that: in step 6.1.4), the calculation of the angle of any non-collinear three-point coordinates is as follows : 利用勾股定理公式计算两点之间的线段长度：Use the Pythagorean formula to calculate the length of a line segment between two points:

式中，a′表示线段的长度，(x_b,y_b)和(x_c,y_c)分别为线段两端的端点坐标，基于上述公式能够计算出任意不共线三点组成的三角形三条边长度；In the formula, a' represents the length of the line segment, (x _b , y _b ) and (x _c , y _c ) are the endpoint coordinates of the two ends of the line segment respectively, based on the above formula, the three sides of a triangle composed of any three points that are not collinear can be calculated length; 利用解三角形的任意角度计算公式由边长计算角度：Use the arbitrary angle calculation formula for solving triangles to calculate angles from side lengths:

式中，a为待求角所在顶点的对边线段长度，b、c分别为待求角所在顶点的邻边线段长度，A1为待求角的角度。In the formula, a is the length of the line segment opposite to the vertex where the angle to be sought is located, b and c are the lengths of the line segment adjacent to the vertex where the angle is to be sought, respectively, and A1 is the angle of the angle to be sought. 10.根据权利要求9所述的一种基于深度学习的圆形指针式仪表自动读数方法，其特征在于：在步骤7)中，所述仪表读数模块是由角度法改进得到，公式表达如下：10. A kind of method for automatic reading of a circular pointer meter based on deep learning according to claim 9, characterized in that: in step 7), the meter reading module is improved by the angle method, and the formula is expressed as follows:

式中，Reading表示最终得到的仪表读数；max_scale、min_scale分别表示指刻度区间上界数字和刻度区间下界数字的数字字符文本内容，匹配确定上下界刻度数字之后能够直接获得；ang_end表示指针旋转中心点和刻度区间下界数字之间的角度；ang_interval表示刻度区间上界数字和刻度区间下界数字之间的角度，在确定指针旋转中心点、指针末端点、刻度区间上界数字和刻度区间下界数字位置坐标的情况下能够计算得到ang_end和ang_interval的角度值，计算方法与步骤6.1.4)中对任意不共线三点坐标的角度计算方法一致；In the formula, Reading represents the final reading of the instrument; max_scale and min_scale respectively represent the text content of the digital characters referring to the upper boundary number of the scale interval and the lower boundary number of the scale interval, which can be directly obtained after matching the upper and lower boundary numbers; ang_end represents the center point of the pointer rotation and the angle between the lower bound number of the scale interval; ang_interval indicates the angle between the upper bound number of the scale interval and the lower bound number of the scale interval, when determining the position coordinates of the pointer rotation center point, the end point of the pointer, the upper bound number of the scale interval and the lower bound number of the scale interval The angle value of ang_end and ang_interval can be calculated under the condition of , and the calculation method is consistent with the angle calculation method for any non-collinear three-point coordinates in step 6.1.4); 上述公式计算得到的结果即为圆形指针式仪表自动读数识别的最终结果。The result calculated by the above formula is the final result of the automatic reading recognition of the circular pointer instrument.