CN110147807A - A kind of ship intelligent recognition tracking - Google Patents

本发明提出了一种船舶智能识别跟踪方法，基于计算机视觉的深度学习算法，改进了传统深度学习中的基础分类网络结构和目标的多尺度预测方法，利用Darknet网络和YOLOv3算法结合的方式实现船舶的跟踪并实时检测识别船舶类型。该方法引入残差网络的思想，采用全卷积结构，增加网络深度，提高了数据特征学习能力。利用YOLOv3算法通过卷积核的方式实现特征图之间的局部特征交互,进行目标的匹配定位，在此基础上将目标区域预测和类别预测整合于单个神经网络模型中，从而实现图像的全局信息进行目标识别。实验结果表明，提出的算法与传统方法相比，不仅具有更好的实时性、准确性，对各种环境变化具有较好的鲁棒性。

The invention proposes a ship intelligent identification and tracking method, which is based on the deep learning algorithm of computer vision, improves the basic classification network structure and the multi-scale prediction method of the target in the traditional deep learning, and realizes the ship by combining the Darknet network and the YOLOv3 algorithm. Tracking and real-time detection to identify ship types. This method introduces the idea of residual network, adopts full convolution structure, increases network depth, and improves data feature learning ability. The YOLOv3 algorithm is used to realize the local feature interaction between the feature maps through the convolution kernel, and the matching and positioning of the target is carried out. On this basis, the target area prediction and category prediction are integrated into a single neural network model, so as to realize the global information of the image. for target recognition. Experimental results show that compared with traditional methods, the proposed algorithm not only has better real-time performance and accuracy, but also has better robustness to various environmental changes.

一种船舶智能识别跟踪方法A method for intelligent identification and tracking of ships

技术领域：Technical field:

本发明涉及水面船舶目标的跟踪和检测领域，具体地说，是一种 船舶智能识别跟踪方法。The invention relates to the field of tracking and detection of surface ship targets, in particular to a method for intelligent identification and tracking of ships.

背景技术：Background technique:

船舶跟踪与识别作为智能船舶视觉感知的一个基本任务，目前传 统的跟踪方法是通过AIS系统和雷达技术结合的手段来进行船舶的跟 踪，而面向于智能船舶，不能将跟踪的目标船舶视为质点。因此，结 合神经网络和卡尔曼滤波算法的优势，构建一种船舶动态跟踪模型。 通过对海事监控船舶视频的背景像素进行建模，构建特征约束方程来 校验特征像素的运动状态，估计船舶运动参数来获得船舶运动位置参 数，最后结合主动轮廓模型算法找到船舶轮廓来确定跟踪的船舶位置。Ship tracking and recognition is a basic task of visual perception of intelligent ships. The current traditional tracking method is to track ships through the combination of AIS system and radar technology. For intelligent ships, the target ship cannot be regarded as a particle . Therefore, combining the advantages of neural network and Kalman filter algorithm, a ship dynamic tracking model is constructed. By modeling the background pixels of the maritime surveillance ship video, constructing a feature constraint equation to verify the motion state of the feature pixels, estimating the ship motion parameters to obtain the ship motion position parameters, and finally combining the active contour model algorithm to find the ship contour to determine the tracking position ship position.

对于船舶类型的识别传统方法是通过AIS数据和合成孔径雷 (SAR)对数据进行人工分析出船舶的类型。基于深度学习理论的方 法是以端到端的卷积神经网络对象检测模型，采用选择性搜索方法在 输入图像上选择若干候选包围盒,对每个包围盒利用卷积神经网络提 取特征,输入到为每个类训练好的支持向量机分类器,通过ROI Pooling 网络层把不同大小的输入映射到一个固定尺度的特征向量，对每个区 域都提取一个固定维度的特征表示，再通过正常的softmax分类函数 得到包围盒属于每个类的分数，最后使用非极大值抑制方法舍弃部分 重复包围盒,得到类型识别结果。考虑到水上交通中船舶成像尺寸的变化，光照变化、船舶成像视角变化和交叉会遇局面中的船舶成像重 叠以及人工参与等问题，严重影响了船舶跟踪识别率。The traditional method for identifying the type of ship is to manually analyze the data through AIS data and synthetic aperture radar (SAR) to find out the type of ship. The method based on deep learning theory is based on the end-to-end convolutional neural network object detection model, using the selective search method to select several candidate bounding boxes on the input image, and using the convolutional neural network to extract features for each bounding box, input to The support vector machine classifier trained for each class maps inputs of different sizes to a fixed-scale feature vector through the ROI Pooling network layer, extracts a fixed-dimensional feature representation for each region, and then classifies it through the normal softmax The function obtains the score of the bounding box belonging to each class, and finally uses the non-maximum value suppression method to discard some repeated bounding boxes to obtain the type recognition result. Considering the change of ship imaging size in water traffic, the change of illumination, the change of ship imaging angle of view, the overlap of ship imaging in cross-meeting situations, and human participation, etc., seriously affect the ship tracking recognition rate.

发明内容：Invention content:

针对以上问题，该发明提出一种船舶智能识别跟踪方法，基于Darknet 网络和YOLOv3算法，能够更加准确、实时进行目标的跟踪和识别。In view of the above problems, this invention proposes a ship intelligent identification and tracking method, based on the Darknet network and YOLOv3 algorithm, which can track and identify targets more accurately and in real time.

为了实现上述目的，本发明提出一种船舶智能识别跟踪算法。本 发明采取的技术方案是：利用Darknet基础网络和训练网络对船舶样 本数据进行特征提取学习，得到物体的特征图模型，通过YOLOv3算 法进行特征图之间的局部特征交互，从而匹配定位，在此基础上构建 分类算法，实时识别船舶类型，该方法包括以下步骤：In order to achieve the above object, the present invention proposes a ship intelligent identification and tracking algorithm. The technical solution adopted by the present invention is: use the Darknet basic network and the training network to perform feature extraction and learning on the ship sample data, obtain the feature map model of the object, and perform local feature interaction between the feature maps through the YOLOv3 algorithm, so as to match and locate. On the basis of constructing a classification algorithm to identify the type of ship in real time, the method includes the following steps:

步骤一：收集不同类型船舶图片，作为船舶图像原始数据，进行标签 预处理，为后续的识别跟踪模型做初始化，步骤一包括数据预处理， 具体实施步骤如下：Step 1: Collect pictures of different types of ships as the original data of ship images, perform label preprocessing, and initialize the subsequent identification and tracking model. Step 1 includes data preprocessing. The specific implementation steps are as follows:

(一)数据预处理：(1) Data preprocessing:

(1)下载Pascal voc2007标准化数据集，清空其原有数据，保留 JPEGImages文件夹，Annotations文件夹和ImageSets文件夹；(1) Download the Pascal voc2007 standardized data set, clear its original data, and keep the JPEGImages folder, Annotations folder and ImageSets folder;

(2)将收集到的不同类型的船舶原始图像数据存放于JPEGImages 文件夹中，包括训练图片和测试图片，其中训练图片和测试图片数量 比列为8：2；(2) Store the collected original image data of different types of ships in the JPEGImages folder, including training pictures and testing pictures, wherein the ratio of the number of training pictures and testing pictures is 8:2;

(3)通过labelImg标记工具，生成模型可读的Xaml文件存放在 Annotations文件夹中，每一个Xaml文件都对应于JPEGImages文件 夹中的一张图片；(3) Through the labelImg marking tool, the Xaml files readable by the generated model are stored in the Annotations folder, and each Xaml file corresponds to a picture in the JPEGImages folder;

(3)在ImageSets文件夹下建立Main文件夹，存放的是每一种船舶 图片类型对应的图像数据信息，包括训练数据集，检测数据集，验证 数据集，训练和验证数据集；(3) Establish the Main folder under the ImageSets folder, which stores the image data information corresponding to each ship image type, including training data sets, detection data sets, verification data sets, training and verification data sets;

步骤二：构建深层网络模型，对输入的船舶图像样本数据进行卷积操 作提取相应特征，进行组合学习，得到物体的特征图模型，在此基础 上添加特征交互层，分为三个尺度，每个尺度内，通过卷积核的方式 实现特征图局部的特征交互，步骤二包括船舶特征提取网络结构和特 征交互层结构，具体实施步骤如下：Step 2: Build a deep network model, perform convolution operation on the input ship image sample data to extract corresponding features, and perform combined learning to obtain the feature map model of the object. On this basis, add a feature interaction layer, divided into three scales, each Within a scale, the local feature interaction of the feature map is realized through the convolution kernel. Step 2 includes the ship feature extraction network structure and feature interaction layer structure. The specific implementation steps are as follows:

(一)船舶特征提取网络：(1) Ship feature extraction network:

(1)输入预处理好的船舶图片，利用高分辨率分类器提高图像的分 辨率，进行规范化处理；(1) Input the pre-processed ship picture, use the high-resolution classifier to improve the resolution of the image, and perform standardized processing;

(2)通过32层卷积核，每个卷积核大小为3*3，步伐为1进行卷积 操作，获得特征映射矩阵；(2) Through 32 layers of convolution kernels, the size of each convolution kernel is 3*3, and the step is 1 to perform convolution operations to obtain the feature map matrix;

(3)通过卷积层的船舶特征图矩阵提取 出高度抽象的船舶特征，其中是第r层卷积网络的第n个输出的特 征映射，函数f表示第r层卷积神经网络神经元的激活函数，是 第r-1个网络层的第m个输入的船舶特征映射，是第n个网络输 出层的船舶特征映射和第m个输入特征映射的连接权重，参数是第r层卷积神经网络的第n个特征映射的偏置量；(3) The ship feature map matrix through the convolutional layer Extract highly abstract ship features, where is the feature map of the nth output of the rth layer convolutional network, and the function f represents the activation function of the rth layer convolutional neural network neuron, is the ship feature map of the mth input of the r-1th network layer, is the connection weight of the ship feature map of the nth network output layer and the mth input feature map, and the parameter is the offset of the nth feature map of the rth convolutional neural network;

(4)在每一个卷积层后添加归一化层，通过函数进行 批量标准化处理，将卷积层输出的矩阵数据分布归一化为均值为0, 方差为1的分布，其中x^k表示输入数据的第k维，E[x^k]表示该维的 平均值，√Var[x^k]表示标准差；(4) Add a normalization layer after each convolutional layer, through the function Batch normalization is performed, and the matrix data distribution output by the convolutional layer is normalized to a distribution with a mean value of 0 and a variance of 1, where x ^k represents the kth dimension of the input data, and E[x ^k ] represents the average value of this dimension , √Var[x ^k ] represents the standard deviation;

(5)引入修正线性单元g(x)＝Max(0,x_r)作为激活函数，对于输入 该层的数据进行单侧抑制，把归一化层输出的数据作为激活函数的输 入数据，当输入数据x_r>0时，梯度恒为1，当x_r<0时，该层的输出 为0；(5) Introduce a modified linear unit g(x)=Max(0,x _r ) as the activation function, perform unilateral suppression on the data input to this layer, and use the data output from the normalization layer as the input data of the activation function, when When the input data x _r >0, the gradient is always 1, and when x _r <0, the output of this layer is 0;

(6)循环步骤(2)-(5)，构建层数为53层的基础网络结构；(6) Steps (2)-(5) are circulated to build a basic network structure with 53 layers;

(7)在此基础网络结构上，通过残差函数F(x)＝H(x)-x₁来构建新的 网络结构，当基础网络中卷积层的输入与输出的维度一样时，采用跨 层跳跃连接方式，在卷积层后添加残差层，改变其原有基础网络结构， 将深层神经网络结构的逐层训练改为逐阶段训练，把网络结构分为若 干个子段，每个小段包含比较浅的网络层数，每一个小段学习总差的 一部分，最终达到总体较小的损失，其中H(x)是输入到求和后的网络 映射函数，F(x)是求和前网络映射函数，x₁是该卷积层的输入数据， 当F(x)＝0，就构成了一个恒等映射H(x)＝x₁；(7) On this basic network structure, a new network structure is constructed by the residual function F(x)=H(x)-x _1. When the input and output dimensions of the convolutional layer in the basic network are the same, use The cross-layer jump connection method adds a residual layer after the convolutional layer, changes its original basic network structure, changes the layer-by-layer training of the deep neural network structure to stage-by-stage training, and divides the network structure into several sub-sections, each The small segment contains a relatively shallow number of network layers, and each small segment learns a part of the total difference, and finally achieves a small overall loss, where H(x) is the network mapping function input to the summation, and F(x) is the before summation Network mapping function, x ₁ is the input data of the convolutional layer, when F(x)=0, an identity mapping H(x)=x ₁ is formed;

(二)特征交互层结构：(2) Feature interaction layer structure:

(1)在步骤二中构建的网路结构后添加特征交互层，分为三个不同 尺度大小的交互层，采用多个尺度融合的方式进行船舶特征交互，3 种交互层如下：(1) Add a feature interaction layer after the network structure constructed in step 2, which is divided into three interaction layers of different scales, and use multiple scale fusion methods for ship feature interaction. The three interaction layers are as follows:

(a)小尺度特征交互层：在网络结构之后添加七层卷积层进行卷积 操作，把卷积后的特征图信息给下一个特征交互层；(a) Small-scale feature interaction layer: add seven convolutional layers after the network structure for convolution operation, and give the convolutional feature map information to the next feature interaction layer;

(b)中尺度特征交互层：把上一层的特征图进行上采样操作使之扩 大两倍，再与基础网络结构中具有相同维度大小的特征图相加,再次 通过卷积后输出特征图信息；(b) Mesoscale feature interaction layer: The feature map of the previous layer is upsampled to double it, and then added to the feature map with the same dimension size in the basic network structure, and the feature map is output after convolution again information;

(c)大尺度特征交互层：把中尺度特征交互层的特征图进行上采样 操作使之扩大两倍，再与基础网络结构中具有相同维度大小的特征图 相加，通过卷积后输出特征图信息；(c) Large-scale feature interaction layer: The feature map of the medium-scale feature interaction layer is upsampled to make it twice larger, and then added to the feature map with the same dimension size in the basic network structure, and the feature is output after convolution map information;

(2)最后，通过损失函数来衡量特征图模型的性能，当损失函数的 值越接近0，该模型性能就越稳定，采用均方和误差作为损失函数， 由坐标误差、IOU误差和分类误差三部分组成，其表达式为：(2) Finally, the performance of the feature map model is measured by the loss function. When the value of the loss function is closer to 0, the performance of the model is more stable. The mean square sum error is used as the loss function, and the coordinate error, IOU error and classification error are used. It consists of three parts, and its expression is:

其中，前面两行表示坐标误差，第一行是包围盒中心坐标的预测，第 二行为宽和高的预测，第三、四行表示包围盒的置信度损失，第五行 表示预测类别的误差，符号为预测值,无帽子的为训练 标记值，表示判断物体落入网格i的第j个包围盒内，如果某个单 元格中没有目标,则不对分类误差进行反向传播，当包围盒中的物体 与真实框中具有最高IOU的一个进行坐标误差的反向传播,其余不进 行；Among them, the first two lines represent coordinate errors, the first line is the prediction of the center coordinates of the bounding box, the second line is the prediction of width and height, the third and fourth lines represent the confidence loss of the bounding box, and the fifth line represents the error of the predicted category. symbol is the predicted value, and the unhatted one is the training tag value, Indicates that it is judged that the object falls into the jth bounding box of grid i. If there is no target in a certain cell, the classification error will not be backpropagated. When the object in the bounding box is compared with the one with the highest IOU in the real box Coordinate error backpropagation, the rest will not be carried out;

步骤三：通过特征提取网络对输入待检测的船舶图片提取特征，得到 一定尺寸的特征图，然后将输入图像分成相应大小的网格，通过数据 标准化处理以及维度聚类、细粒度特征操作，网格直接预测出的包围 盒与真实边框中目标物体的中心坐标进行匹配定位，在此基础上添加 多标签多分类的逻辑回归层，对每个类别做二分类从而实现对目标物 体进行分类识别，步骤三包括坐标预测，匹配定位和分类识别，具体 实施步骤如下：Step 3: Use the feature extraction network to extract features from the input image of the ship to be detected, obtain a feature map of a certain size, and then divide the input image into grids of corresponding size, through data standardization processing, dimension clustering, and fine-grained feature operations, the network The bounding box directly predicted by the grid is matched with the center coordinates of the target object in the real frame. On this basis, a multi-label and multi-classification logistic regression layer is added to perform two classifications for each category to realize the classification and recognition of the target object. Step 3 includes coordinate prediction, matching positioning and classification recognition. The specific implementation steps are as follows:

(一)坐标预测，匹配定位：(1) Coordinate prediction, matching positioning:

(1)对于输入待检测的船舶图片，通过特征提取网络的卷积层降采 样处理，得到大小为13*13，通道数为3的卷积特征图，然后将图 像分割成相应大小的网格；(1) For the input image of the ship to be detected, the convolutional feature map with a size of 13*13 and a channel number of 3 is obtained through downsampling processing of the convolutional layer of the feature extraction network, and then the image is divided into grids of corresponding sizes ;

(2)通过锚点操作，使用3种尺度和3种不同长宽比例的窗口尺寸 在13*13的卷积特征图上进行滑窗操作，以当前滑动窗口中心为中 心映射到原图的一个区域，该区域的中心对应一个尺度和长宽比，每 一个中心可以预测9种不同大小的先验框；(2) Through the anchor point operation, use 3 scales and 3 window sizes with different aspect ratios to perform sliding window operations on the 13*13 convolution feature map, and map to one of the original images centered on the center of the current sliding window Area, the center of the area corresponds to a scale and aspect ratio, and each center can predict 9 different sizes of prior frames;

(3)采用IOU得分评判标准，定义新的距离公式d(box,centroid)＝ 1-IOU(box,centroid)，改进K-means聚类方法自动找到更好的先 验框宽高维度，其中box是预测先验框的坐标，centroid是聚类所有 簇的中心；(3) Using the IOU score evaluation standard, define a new distance formula d(box, centroid) = 1-IOU(box, centroid), improve the K-means clustering method to automatically find a better prior box width and height dimensions, where box is the coordinates of the predicted prior box, and centroid is the center of all clusters;

(4)按照下列算法进行先验框聚类：(4) Perform prior frame clustering according to the following algorithm:

(a)从输入的数据集合中随机选择一个点作为第一个聚类中心；(a) Randomly select a point from the input data set as the first cluster center;

(b)对于每个点，我们都计算其和最近的一个种子点的距离，记作 D(x)；(b) For each point, we calculate the distance to the nearest seed point, denoted as D(x);

(c)选择一个新的数据点作为新的聚类中心，选择的原则是为D(x) 数值较大的点，被选取作为聚类中心的概率较大；(c) Select a new data point as the new cluster center. The principle of selection is that the point with a larger value of D(x) has a higher probability of being selected as the cluster center;

(d)重复(b)和(c)直到k个聚类中心被选出来；(d) Repeat (b) and (c) until k cluster centers are selected;

(e)利用这k个初始的聚类中心来运行k-means算法；(e) use the k initial cluster centers to run the k-means algorithm;

(5)特征图上的每个网格预测的先验框包含5个预测值，分别为tx, ty,tw,th,to，其中前四个是坐标，to是置信度，由实际预测的 t_x,t_y,tw,th得到b_x,b_y,b_w,b_h的过程表示为：(5) The prior frame of each grid prediction on the feature map contains 5 predicted values, namely tx, ty, tw, th, to, where the first four are coordinates, and to is the confidence level, which is determined by the actual prediction The process of obtaining b _x , b _y , b _w , b _h from t _x , t _y , tw, th is expressed as:

b_x＝σ(t_x)+c_x b _x =σ(t _x )+c _x

b_y＝σ(t_y)+c_y b _y =σ(t _y )+c _y

b_w＝Pwe^tw b _w = Pwe ^tw

b_h＝Phe^th b _h = Phe ^th

Pr(object)*IOU(b,centroid)＝σ(t₀)Pr(object)*IOU(b,centroid)=σ(t ₀ )

其中，c_x，c_y为边框的中心坐标所在的网格距离左上角第一个网格的 个数，t_x,t_y为预测的边框的中心点坐标，σ函数为logistic函数，将坐 标归一化到0-1之间，最终得到的b_x,b_y为归一化后的相对于网格位置 的值，tw,th为预测的边框的宽和高，Pw,Ph为候选框的宽和高，最 终得到的b_w,b_h为归一化后相对于候选框位置的值；Among them, c _x , _cy are the number of grids where the center coordinates of the border are located from the first grid in the upper left corner, t _x , t _y are the coordinates of the center point of the predicted border, and the σ function is a logistic function. Normalized to between 0-1, the final b _x and b _y are the normalized values relative to the grid position, tw, th are the width and height of the predicted border, and Pw, Ph are candidate boxes Width and height, the final b _w , b _h is the normalized value relative to the position of the candidate frame;

(6)通过平方和距离误差损失函数来衡量船舶坐标的预测值与实际 值之间的差异，当船舶样本个数为n时，此时的损失函数表示为：(6) The difference between the predicted value and the actual value of the ship coordinates is measured by the square sum distance error loss function. When the number of ship samples is n, the loss function at this time is expressed as:

其中，Y-f(x)表示的是残差，整个式子表示的是残差的平方和，求 解的最小化目标函数值就是坐标值的相似性，且函数值越小，差异性 越好；Among them, Y-f(x) represents the residual, and the whole formula represents the sum of the squares of the residual, and the minimized objective function value is the similarity of coordinate values, and the smaller the function value, the better the difference;

(7)按照下列步骤定位坐标进行匹配跟踪船舶：(7) Follow the steps below to locate coordinates to match and track the ship:

(e)通过对于输入待检测的船舶图片，进行特征图网格划分；(e) Carry out feature map grid division by inputting the image of the ship to be detected;

(f)每一个网格会预测3个候选框，每一个候选框都会预测一个物 体的坐标值，通过步骤(6)的损失函数代价值小于阈值0.1，进 行下一步操作；(f) Each grid will predict 3 candidate frames, and each candidate frame will predict the coordinate value of an object, and the cost value of the loss function through step (6) is less than the threshold value 0.1, and the next step is performed;

(g)通过步骤(5)操作，对图片中的船舶进行位置定位；(g) by step (5) operation, the ship in the picture is positioned;

(h)确定其船舶位置后，用边界框标记出船舶，通过船舶特征匹配 和实时定位坐标进行跟踪船舶；(h) After determining the position of its ship, mark the ship with a bounding box, and track the ship through ship feature matching and real-time positioning coordinates;

(二)分类识别(2) Classification identification

(1)基于步骤一中的特征交互层结构，利用锚点的设计方式使用聚 类操作得到9个聚类中心,将其按照大小均分给3种尺度：(1) Based on the feature interaction layer structure in step 1, use the anchor point design method to use clustering operations to obtain 9 cluster centers, and divide them into 3 scales according to their size:

(a)尺度1：从特征提取网络结构获取的大小为13*13，通道为1024 的特征图进行卷积操作，不改变特征图大小，通道数最后减少为75；(a) Scale 1: The feature map with a size of 13*13 and 1024 channels obtained from the feature extraction network structure is used for convolution operation without changing the size of the feature map, and the number of channels is finally reduced to 75;

(b)尺度2：将上一层的特征图进行卷积操作，生成13*13、256通 道的特征图，然后进行上采样，生成26*26、256通道的特征图，同 时与基础网络结构层的26*26、512通道的特征图进行合并，再进行 卷积操作；(b) Scale 2: Convolute the feature map of the previous layer to generate a feature map of 13*13 and 256 channels, and then perform upsampling to generate a feature map of 26*26 and 256 channels, and at the same time integrate with the basic network structure The feature maps of the 26*26 and 512 channels of the layer are merged, and then the convolution operation is performed;

(c)尺度3：与尺度2类似,使用了32*32大小的特征图进行融合；(c) Scale 3: Similar to scale 2, a feature map of size 32*32 is used for fusion;

(2)将特征交互层处理后的特征图采用多标签分类操作，在网络结 构上添加了多标签多分类的逻辑回归层，用逻辑回归层来对每个类别 做二分类；(2) The feature map processed by the feature interaction layer is operated by multi-label classification, and a multi-label and multi-classification logistic regression layer is added to the network structure, and the logistic regression layer is used to perform two classifications for each category;

(3)通过交叉熵代价函数，衡量逻辑回归层的预测值与实际值之间 的差异，当函数值越小，说明预测值越接近真实值，其表达式为：(3) Measure the difference between the predicted value and the actual value of the logistic regression layer through the cross-entropy cost function. When the function value is smaller, it means that the predicted value is closer to the real value. The expression is:

其中，x表示船舶数据样本，n表示数据样本的总数；Among them, x represents the ship data sample, and n represents the total number of data samples;

(4)通过步骤三的(1)和(2)操作，对于得到的特征图进行等尺 寸比例的划分网格，每个网格都预测C个船舶类型概率，表示一个 网格在包含船舶目标的条件下属于某种船舶类型的概率，其表达式为：(4) Through the operations of (1) and (2) in step 3, the obtained feature map is divided into grids of equal size and proportion, and each grid predicts C ship type probabilities, which means that a grid contains ship targets The probability of belonging to a certain ship type under the condition of , its expression is:

其中Pr(Class_t|Object)表示目标的类别概率，表示预测框与 真实框交叉的面积，Pr(Class_t)表示类别概率，Pr(Object)是目标存 在的概率；where Pr(Class _t |Object) represents the category probability of the target, Represents the intersection area of the predicted frame and the real frame, Pr(Class _t ) represents the category probability, and Pr(Object) is the probability of the existence of the target;

(5)按照下列算法进行船舶类型分类：(5) Carry out ship type classification according to the following algorithm:

(a)在预测的船舶类别中，将得分少于阈值0.2的设置为0，然后再 按得分从高到低排序；(a) among the predicted ship categories, set the score less than the threshold 0.2 to 0, and then sort by the score from high to low;

(b)用非极大值抑制算法计算边界框的IOU值，当IOU大于0.5， 该边界框重复率较大，该得分设为0，去掉重复率较大的边界框，如 果不大于0.5，则不改；(b) Use the non-maximum value suppression algorithm to calculate the IOU value of the bounding box. When the IOU is greater than 0.5, the bounding box has a large repetition rate, and the score is set to 0. Remove the bounding box with a large repetition rate. If it is not greater than 0.5, do not change;

(c)再选择剩下得分里面最大的边界框，重复步骤(b)直到最后；(c) Then select the largest bounding box in the remaining score, and repeat step (b) until the end;

(d)最后保留的边界框得分如果大于0，那么船舶类型的就是这个 得分所对应的类别；(d) If the score of the last reserved bounding box is greater than 0, then the ship type is the category corresponding to this score;

(6)在输出层加入sigmoid函数把船舶类型预测的数值 作为函数的输入数值，经sigmoid函数后，其数值约束在0到1的范 围内，如果输出值大于设定阈值0.75，就识别出船舶类型，并在边界 框左上方标记出该船舶类别名称。(6) Add the sigmoid function to the output layer The predicted value of the ship type is used as the input value of the function. After the sigmoid function, the value is constrained within the range of 0 to 1. If the output value is greater than the set threshold of 0.75, the ship type is identified and marked on the upper left of the bounding box State the name of the ship class.

附图说明：Description of drawings:

为了更清楚地说明本发明技术方案，下面将对描述中所需要使用 的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一 个实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前 提下，还可以根据这些附图获得其他的附图：In order to illustrate the technical solution of the present invention more clearly, the accompanying drawings that need to be used in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are an embodiment of the present invention. For those of ordinary skill in the art, In other words, on the premise of no creative work, other drawings can also be obtained from these drawings:

图1是本发明一种船舶智能识别跟踪方法的流程图；Fig. 1 is the flow chart of a kind of ship intelligent identification tracking method of the present invention;

图2是本发明一种船舶智能识别跟踪方法的深层网络结构的过程图；Fig. 2 is the process diagram of the deep network structure of a kind of ship intelligent identification tracking method of the present invention;

图3是本发明一种船舶智能识别跟踪方法的目标位置预测和分类识 别的过程图。Fig. 3 is the process chart of the target position prediction and classification recognition of a kind of ship intelligent identification tracking method of the present invention.

具体实施方式：Detailed ways:

为了更好地理解本发明的技术特征、目的和效果，下面结合附图 对本发明进行更为详细地描述。应当理解，此处所描述的具体实施例 仅仅用以解释本发明，并不用于限定本发明专利。需要说明的是，这 些附图中均采用非常简化的形式且均使用非精准的比率，仅用于方便、 清晰地辅助说明本发明专利。In order to better understand the technical features, purposes and effects of the present invention, the present invention will be described in more detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, and are not intended to limit the scope of the present invention. It should be noted that these drawings all adopt very simplified forms and use imprecise ratios, which are only used to facilitate and clearly assist in explaining the patent of the present invention.

本发明提出一种船舶智能识别跟踪方法，适用于对监控视频的图 像帧中，正常成像大小的船舶图像的识别和跟踪。本发明是通过网络 的方式收集的不同类型船舶图片，以2万张图片，7种船舶类型为样 本数据。本发明中正常成像大小的船舶图像是指：船舶成像尺寸不小 于该帧图像实际尺寸的0.10％，且成像长度或宽度不小于13个像素。 本发明的船舶监控视频来源于船上摄像头所采集的监控数据。本研究 的实验平台是Windows 10操作系统，16G RAM，CPU处理器的主频 是3.2GHz，GPU为GTX 1050Ti，仿真平台是Pycharm(2018版)。The invention proposes a ship intelligent identification and tracking method, which is applicable to the identification and tracking of ship images of normal imaging size in the image frame of the surveillance video. The present invention is the different types of ship pictures collected by the mode of network, with 20,000 pictures, 7 kinds of ship types are sample data. In the present invention, a ship image of normal imaging size means: the ship image size is not less than 0.10% of the actual size of the frame image, and the imaging length or width is not less than 13 pixels. The ship monitoring video of the present invention comes from the monitoring data collected by the camera on the ship. The experimental platform of this research is Windows 10 operating system, 16G RAM, the main frequency of the CPU processor is 3.2GHz, the GPU is GTX 1050Ti, and the simulation platform is Pycharm (version 2018).

如图1所示，根据本发明的智能船舶识别跟踪方法具体流程为：As shown in Figure 1, the specific process of the intelligent ship identification and tracking method according to the present invention is:

步骤一：本次实验是通过网络的方式，收集2万张不同类型船舶 图片，按照船舶前景占图片背景低于90％的比例从中筛选出7种船舶 类型，共8千张图片作为船舶图像原始数据，进行标签预处理，为后 续的识别跟踪模型做初始化；Step 1: In this experiment, 20,000 pictures of different types of ships were collected through the network, and 7 types of ships were selected according to the ratio of the foreground of the ship to the background of the picture, and a total of 8,000 pictures were used as the original data of the ship image , perform label preprocessing, and initialize for the subsequent identification and tracking model;

步骤二：构建深层网络模型，对输入的船舶图像样本数据进行卷 积操作提取相应特征，进行组合学习，得到物体的特征图模型，在此 基础上添加特征交互层，分为三个尺度，每个尺度内，通过卷积核的 方式实现特征图局部的特征交互，进行特征融合；Step 2: Build a deep network model, perform convolution operation on the input ship image sample data to extract corresponding features, and perform combined learning to obtain the feature map model of the object. On this basis, add a feature interaction layer, divided into three scales, each Within a scale, the local feature interaction of the feature map is realized through the convolution kernel, and the feature fusion is performed;

步骤三：通过特征提取网络对输入待检测的船舶图片提取特征， 得到一定尺寸的特征图，然后将输入图像分成相应大小的网格，通过 数据标准化处理以及维度聚类、细粒度特征操作，网格直接预测出的 包围盒与真实边框中目标物体的中心坐标进行匹配定位，在此基础上 添加多标签多分类的逻辑回归层，对每个类别做二分类从而实现对目 标物体进行分类识别，步骤三包括坐标预测，匹配定位和分类识别。Step 3: Extract features from the input image of the ship to be detected through the feature extraction network to obtain a feature map of a certain size, and then divide the input image into grids of corresponding size, through data standardization processing, dimension clustering, and fine-grained feature operations, the network The bounding box directly predicted by the grid is matched with the center coordinates of the target object in the real frame. On this basis, a multi-label and multi-classification logistic regression layer is added to perform two classifications for each category to realize the classification and recognition of the target object. Step three includes coordinate prediction, matching location and classification recognition.

步骤一具体流程为：The specific process of step one is:

步骤一：通过从网络收集2万张不同类型船舶图片，按照船舶前 景占图片背景低于90％的比例从中筛选出7种船舶类型，分别包括集 装箱船、油轮、化学品船、LNG船、杂货船、散货船和其他船舶，其 中集装箱船图片为2300张，油轮图片1420张，化学品船1240，LNG船图片1250张，杂货船图片2750张，散货船图片2060张， 其他船舶的图片1500张，共12520张图片作为船舶图像原始数据， 进行标签预处理，为后续的识别跟踪模型做初始化；步骤一包括数据 预处理，具体实施步骤如下：Step 1: Collect 20,000 pictures of different types of ships from the Internet, and select 7 types of ships according to the ratio of the foreground of the ship to the background of the picture, including container ships, oil tankers, chemical tankers, LNG ships, and general cargo. Ships, bulk carriers and other ships, including 2,300 pictures of container ships, 1,420 pictures of oil tankers, 1,240 pictures of chemical tankers, 1,250 pictures of LNG ships, 2,750 pictures of general cargo ships, 2,060 pictures of bulk carriers, and pictures of other ships 1500 pictures, a total of 12520 pictures are used as the original data of the ship image, and the label preprocessing is performed to initialize the subsequent identification and tracking model; step 1 includes data preprocessing, and the specific implementation steps are as follows:

(一)数据预处理：(1) Data preprocessing:

(1)下载Pascalvoc2007标准化数据集，清空其原有数据，保留JPEGImages文件夹，Annotations文件夹和ImageSets文件夹；(1) Download the Pascalvoc2007 standardized data set, clear its original data, and keep the JPEGImages folder, Annotations folder and ImageSets folder;

(2)将收集到的不同类型的船舶原始图像数据存放于JPEGImages文 件夹中，包括训练图片和测试图片，其中训练图片和测试图片的比例 为8：2；(2) Store the original image data of different types of ships collected in the JPEGImages folder, including training pictures and testing pictures, wherein the ratio of training pictures and testing pictures is 8:2;

(3)通过labelImg标记工具，生成模型可读的Xaml文件存放在 Annotations文件夹中，每一个Xaml文件都对应于JPEGImages文件 夹中的一张图片；(3) Through the labelImg marking tool, the Xaml files readable by the generated model are stored in the Annotations folder, and each Xaml file corresponds to a picture in the JPEGImages folder;

(4)在ImageSets文件夹下建立Main文件夹，存放的是每一种船舶 图片类型对应的图像数据信息，包括训练数据集，检测数据集，验证 数据集，训练和验证数据集；(4) Create a Main folder under the ImageSets folder, which stores image data information corresponding to each ship image type, including training data sets, detection data sets, verification data sets, training and verification data sets;

(5)修改配置参数如下：(5) Modify the configuration parameters as follows:

(a)打开cfg文件；(a) Open the cfg file;

(b)根据公式：3*(5+len(classes))修改卷积核的数目；其中 classes表示识别的船舶种类；(b) Modify the number of convolution kernels according to the formula: 3*(5+len(classes)); where classes represent the types of ships identified;

(6)修改random参数，原来是1，显存小改为0；(6) Modify the random parameter, the original value is 1, and the memory size is changed to 0;

如图2所示，步骤二具体流程为：As shown in Figure 2, the specific process of Step 2 is as follows:

步骤二：构建深层网络模型，对输入的船舶图像样本数据进行卷 积操作提取相应特征，进行组合学习，得到物体的特征图模型，在此 基础上添加特征交互层，分为三个尺度，每个尺度内，通过卷积核的 方式实现特征图局部的特征交互，步骤二包括船舶特征提取网络结构 和特征交互层结构，具体实施步骤如下：Step 2: Build a deep network model, perform convolution operation on the input ship image sample data to extract corresponding features, and perform combined learning to obtain the feature map model of the object. On this basis, add a feature interaction layer, divided into three scales, each Within a scale, the local feature interaction of the feature map is realized through the convolution kernel. Step 2 includes the ship feature extraction network structure and feature interaction layer structure. The specific implementation steps are as follows:

(一)船舶特征提取网络：(1) Ship feature extraction network:

(1)输入预处理好的船舶图片，利用高分辨率分类器提高图像的分 辨率，进行规范化处理；(1) Input the pre-processed ship picture, use the high-resolution classifier to improve the resolution of the image, and perform standardized processing;

(2)通过32层卷积核，每个卷积核大小为3*3，步伐为1进行卷积 操作，获得特征映射矩阵；(2) Through 32 layers of convolution kernels, the size of each convolution kernel is 3*3, and the step is 1 to perform convolution operations to obtain the feature map matrix;

(3)通过卷积层的船舶特征图矩阵提取 出高度抽象的船舶特征，其中是第r层卷积网络的第n个输出的特 征映射，函数f表示第r层卷积神经网络神经元的激活函数，是 第r-1个网络层的第m个输入的船舶特征映射，是第n个网络输 出层的船舶特征映射和第m个输入特征映射的连接权重，参数是 第r层卷积神经网络的第n个特征映射的偏置量；(3) The ship feature map matrix through the convolutional layer Extract highly abstract ship features, where is the feature map of the nth output of the rth layer convolutional network, and the function f represents the activation function of the rth layer convolutional neural network neuron, is the ship feature map of the mth input of the r-1th network layer, is the connection weight of the ship feature map of the nth network output layer and the mth input feature map, and the parameter is the offset of the nth feature map of the rth convolutional neural network;

(4)在每一个卷积层后添加归一化层，通过函数进行 批量标准化处理，将卷积层输出的矩阵数据分布归一化为均值为0, 方差为1的分布，其中x^k表示输入数据的第k维，E[x^k]表示该维的 平均值，√Var[x^k]表示标准差；(4) Add a normalization layer after each convolutional layer, through the function Batch normalization is performed, and the matrix data distribution output by the convolutional layer is normalized to a distribution with a mean value of 0 and a variance of 1, where x ^k represents the kth dimension of the input data, and E[x ^k ] represents the average value of this dimension , √Var[x ^k ] represents the standard deviation;

(5)引入修正线性单元g(x)＝Max(0,x_r)作为激活函数，对于输入 该层的数据进行单侧抑制，把归一化层输出的数据作为激活函数的输 入数据，当输入数据x_r>0时，梯度恒为1，当x_r<0时，该层的输出 为0；(5) Introduce the corrected linear unit g(x)=Max(0,x _r ) as the activation function, perform unilateral suppression on the data input to this layer, and use the data output from the normalization layer as the input data of the activation function, when When the input data x _r >0, the gradient is always 1, and when x _r <0, the output of this layer is 0;

(6)循环步骤(2)-(5)，构建层数为53层的基础网络结构；(6) Steps (2)-(5) are circulated to build a basic network structure with 53 layers;

(7)在此基础网络结构上，通过残差函数F(x)＝H(x)-x₁来构建新的 网络结构，当基础网络中卷积层的输入与输出的维度一样时，采用跨 层跳跃连接方式，在卷积层后添加残差层，改变其原有基础网络结构， 将深层神经网络结构的逐层训练改为逐阶段训练，把网络结构分为若 干个子段，每个小段包含比较浅的网络层数，每一个小段学习总差的 一部分，最终达到总体较小的损失，其中H(x)是输入到求和后的网络 映射函数，F(x)是求和前网络映射函数，x₁是该卷积层的输入数据， 当F(x)＝0，就构成了一个恒等映射H(x)＝x₁；(7) On this basic network structure, a new network structure is constructed by the residual function F(x)=H(x)-x _1. When the input and output dimensions of the convolutional layer in the basic network are the same, use The cross-layer jump connection method adds a residual layer after the convolutional layer, changes its original basic network structure, changes the layer-by-layer training of the deep neural network structure to stage-by-stage training, and divides the network structure into several sub-sections, each The small segment contains a relatively shallow number of network layers, and each small segment learns a part of the total difference, and finally achieves a small overall loss, where H(x) is the network mapping function input to the summation, and F(x) is the before summation Network mapping function, x ₁ is the input data of the convolutional layer, when F(x)=0, an identity mapping H(x)=x ₁ is formed;

(二)特征交互层结构：(2) Feature interaction layer structure:

(1)在步骤二中构建的网路结构后添加特征交互层，分为三个不同 尺度大小的交互层，采用多个尺度融合的方式进行船舶特征交互，3 种交互层如下：(1) Add a feature interaction layer after the network structure constructed in step 2, which is divided into three interaction layers of different scales, and use multiple scale fusion methods for ship feature interaction. The three interaction layers are as follows:

(a)小尺度特征交互层：在网络结构之后添加七层卷积层进行卷积 操作，把卷积后的特征图信息给下一个特征交互层；(a) Small-scale feature interaction layer: add seven convolutional layers after the network structure for convolution operation, and give the convolutional feature map information to the next feature interaction layer;

(b)中尺度特征交互层：把上一层的特征图进行上采样操作使之扩 大两倍，再与基础网络结构中具有相同维度大小的特征图相加,再次 通过卷积后输出特征图信息；(b) Mesoscale feature interaction layer: The feature map of the previous layer is upsampled to double it, and then added to the feature map with the same dimension size in the basic network structure, and the feature map is output after convolution again information;

(c)大尺度特征交互层：把中尺度特征交互层的特征图进行上采样 操作使之扩大两倍，再与基础网络结构中具有相同维度大小的特征图 相加，通过卷积后输出特征图信息；(c) Large-scale feature interaction layer: The feature map of the medium-scale feature interaction layer is upsampled to make it twice larger, and then added to the feature map with the same dimension size in the basic network structure, and the feature is output after convolution map information;

(2)最后，通过损失函数来衡量特征图模型的性能，当损失函数的 值越接近0，该模型性能就越稳定，采用均方和误差作为损失函数， 由坐标误差、IOU误差和分类误差三部分组成，其表达式为：(2) Finally, the performance of the feature map model is measured by the loss function. When the value of the loss function is closer to 0, the performance of the model is more stable. The mean square sum error is used as the loss function, and the coordinate error, IOU error and classification error are used. It consists of three parts, and its expression is:

其中，前面两行表示坐标误差，第一行是包围盒中心坐标的预测，第 二行为宽和高的预测，第三、四行表示包围盒的置信度损失，第五行 表示预测类别的误差，符号为预测值,无帽子的为训练 标记值，表示判断物体落入网格i的第j个包围盒内，如果某个单 元格中没有目标,则不对分类误差进行反向传播，当包围盒中的物体 与真实框中具有最高IOU的一个进行坐标误差的反向传播,其余不进 行。Among them, the first two lines represent coordinate errors, the first line is the prediction of the center coordinates of the bounding box, the second line is the prediction of width and height, the third and fourth lines represent the confidence loss of the bounding box, and the fifth line represents the error of the predicted category. symbol is the predicted value, and the unhatted one is the training tag value, Indicates that it is judged that the object falls into the jth bounding box of grid i. If there is no target in a certain cell, the classification error will not be backpropagated. When the object in the bounding box is compared with the one with the highest IOU in the real box Coordinate error backpropagation, the rest are not carried out.

如图3所示，步骤三具体流程为：As shown in Figure 3, the specific process of Step 3 is as follows:

步骤三：通过特征提取网络对输入待检测的船舶图片提取特征， 得到一定尺寸的特征图，然后将输入图像分成相应大小的网格，通过 数据标准化处理以及维度聚类、细粒度特征操作，网格直接预测出的 包围盒与真实边框中目标物体的中心坐标进行匹配定位，在此基础上 添加多标签多分类的逻辑回归层，对每个类别做二分类从而实现对目 标物体进行分类识别，步骤三包括坐标预测，匹配定位和分类识别， 具体实施步骤如下：Step 3: Extract features from the input image of the ship to be detected through the feature extraction network to obtain a feature map of a certain size, and then divide the input image into grids of corresponding size, through data standardization processing, dimension clustering, and fine-grained feature operations, the network The bounding box directly predicted by the grid is matched with the center coordinates of the target object in the real frame. On this basis, a multi-label and multi-classification logistic regression layer is added to perform two classifications for each category to realize the classification and recognition of the target object. Step three includes coordinate prediction, matching positioning and classification recognition, and the specific implementation steps are as follows:

(一)坐标预测，匹配定位：(1) Coordinate prediction, matching positioning:

(1)对于输入待检测的船舶图片，通过特征提取网络的卷积层降采 样处理，得到大小为13*13，通道数为3的卷积特征图，然后将图 像分割成相应大小的网格；(1) For the input image of the ship to be detected, the convolutional feature map with a size of 13*13 and a channel number of 3 is obtained through downsampling processing of the convolutional layer of the feature extraction network, and then the image is divided into grids of corresponding sizes ;

(2)通过锚点操作，使用3种尺度和3种不同长宽比例的窗口尺寸 在13*13的卷积特征图上进行滑窗操作，以当前滑动窗口中心为中 心映射到原图的一个区域，该区域的中心对应一个尺度和长宽比，每 一个中心可以预测9种不同大小的先验框；(2) Through the anchor point operation, use 3 scales and 3 window sizes with different aspect ratios to perform sliding window operations on the 13*13 convolution feature map, and map to one of the original images centered on the center of the current sliding window Area, the center of the area corresponds to a scale and aspect ratio, and each center can predict 9 different sizes of prior frames;

(3)采用IOU得分评判标准，定义新的距离公式d(box,centroid)＝ 1-IOU(box,centroid)，改进K-means聚类方法自动找到更好的先 验框宽高维度，其中box是预测先验框的坐标，centroid是聚类所有 簇的中心；(3) Using the IOU score evaluation standard, define a new distance formula d(box, centroid) = 1-IOU(box, centroid), improve the K-means clustering method to automatically find a better prior box width and height dimensions, where box is the coordinates of the predicted prior box, and centroid is the center of all clusters;

(4)按照下列算法进行先验框聚类：(4) Perform prior frame clustering according to the following algorithm:

(a)从输入的数据集合中随机选择一个点作为第一个聚类中心；(a) Randomly select a point from the input data set as the first cluster center;

(b)对于每个点，我们都计算其和最近的一个种子点的距离，记作 D(x)；(b) For each point, we calculate the distance to the nearest seed point, denoted as D(x);

(c)选择一个新的数据点作为新的聚类中心，选择的原则是为D(x) 数值较大的点，被选取作为聚类中心的概率较大；(c) Select a new data point as the new cluster center. The principle of selection is that the point with a larger value of D(x) has a higher probability of being selected as the cluster center;

(d)重复(b)和(c)直到k个聚类中心被选出来；(d) Repeat (b) and (c) until k cluster centers are selected;

(e)利用这k个初始的聚类中心来运行k-means算法；(e) use the k initial cluster centers to run the k-means algorithm;

(5)特征图上的每个网格预测的先验框包含5个预测值，分别为tx, ty,tw,th,to，其中前四个是坐标，to是置信度，由实际预测的 t_x,t_y,tw,th得到b_x,b_y,b_w,b_h的过程表示为：(5) The prior frame of each grid prediction on the feature map contains 5 predicted values, namely tx, ty, tw, th, to, where the first four are coordinates, and to is the confidence level, which is determined by the actual prediction The process of obtaining b _x , b _y , b _w , b _h from t _x , t _y , tw, th is expressed as:

b_x＝σ(t_x)+c_x b _x =σ(t _x )+c _x

b_y＝σ(t_y)+c_y b _y =σ(t _y )+c _y

b_w＝Pwe^tw b _w = Pwe ^tw

b_h＝Phe^th b _h = Phe ^th

Pr(object)*IOU(b,centroid)＝σ(t₀)Pr(object)*IOU(b,centroid)=σ(t ₀ )

其中，c_x，c_y为边框的中心坐标所在的网格距离左上角第一个网格的 个数，t_x,t_y为预测的边框的中心点坐标，σ函数为logistic函数，将坐 标归一化到0-1之间，最终得到的b_x,b_y为归一化后的相对于网格位置 的值，tw,th为预测的边框的宽和高，Pw,Ph为候选框的宽和高，最 终得到的b_w,b_h为归一化后相对于候选框位置的值；Among them, c _x , _cy are the number of grids where the center coordinates of the border are located from the first grid in the upper left corner, t _x , t _y are the coordinates of the center point of the predicted border, and the σ function is a logistic function. Normalized to between 0-1, the final b _x and b _y are the normalized values relative to the grid position, tw, th are the width and height of the predicted border, and Pw, Ph are candidate boxes Width and height, the final b _w , b _h is the normalized value relative to the position of the candidate frame;

(6)通过平方和距离误差损失函数来衡量船舶坐标的预测值与实际 值之间的差异，当船舶样本个数为n时，此时的损失函数表示为：(6) The difference between the predicted value and the actual value of the ship coordinates is measured by the square sum distance error loss function. When the number of ship samples is n, the loss function at this time is expressed as:

其中，Y-f(x)表示的是残差，整个式子表示的是残差的平方和，求 解的最小化目标函数值就是坐标值的相似性，且函数值越小，差异性 越好；Among them, Y-f(x) represents the residual, and the whole formula represents the sum of the squares of the residual, and the minimized objective function value is the similarity of coordinate values, and the smaller the function value, the better the difference;

(7)按照下列步骤定位坐标进行匹配跟踪船舶：(7) Follow the steps below to locate coordinates to match and track the ship:

(i)通过对于输入待检测的船舶图片，进行特征图网格划分；(i) Carry out grid division of the feature map by inputting the image of the ship to be detected;

(j)每一个网格会预测3个候选框，每一个候选框都会预测一个物 体的坐标值，通过步骤(6)的损失函数代价值小于阈值0.1，进 行下一步操作；(j) Each grid will predict 3 candidate boxes, and each candidate box will predict the coordinate value of an object, and the cost value of the loss function in step (6) is less than the threshold value 0.1, and then proceed to the next step;

(k)通过步骤(5)操作，对图片中的船舶进行位置定位；(k) by step (5) operation, the ship in the picture is positioned;

(l)确定其船舶位置后，用边界框标记出船舶，通过船舶特征匹配 和实时定位坐标进行跟踪船舶；(l) After determining its ship's position, mark the ship with a bounding box, and track the ship through ship feature matching and real-time positioning coordinates;

(二)分类识别(2) Classification identification

(1)基于步骤一中的特征交互层结构，利用锚点的设计方式使用聚 类操作得到9个聚类中心,将其按照大小均分给3种尺度：(1) Based on the feature interaction layer structure in step 1, use the anchor point design method to use clustering operations to obtain 9 cluster centers, and divide them into 3 scales according to their size:

(a)尺度1：从特征提取网络结构获取的大小为13*13，通道为1024 的特征图进行卷积操作，不改变特征图大小，通道数最后减少为75；(a) Scale 1: The feature map with a size of 13*13 and 1024 channels obtained from the feature extraction network structure is used for convolution operation without changing the size of the feature map, and the number of channels is finally reduced to 75;

(b)尺度2：将上一层的特征图进行卷积操作，生成13*13、256通 道的特征图，然后进行上采样，生成26*26、256通道的特征图，同 时与基础网络结构层的26*26、512通道的特征图进行合并，再进行 卷积操作；(b) Scale 2: Convolute the feature map of the previous layer to generate a feature map of 13*13 and 256 channels, and then perform upsampling to generate a feature map of 26*26 and 256 channels, and at the same time integrate with the basic network structure The feature maps of the 26*26 and 512 channels of the layer are merged, and then the convolution operation is performed;

(c)尺度3：与尺度2类似,使用了32*32大小的特征图进行融合；(c) Scale 3: Similar to scale 2, a feature map of size 32*32 is used for fusion;

(2)将特征交互层处理后的特征图采用多标签分类操作，在网络结 构上添加了多标签多分类的逻辑回归层，用逻辑回归层来对每个类别 做二分类；(2) The feature map processed by the feature interaction layer is operated by multi-label classification, and a multi-label and multi-classification logistic regression layer is added to the network structure, and the logistic regression layer is used to perform two classifications for each category;

(3)通过交叉熵代价函数，衡量逻辑回归层的预测值与实际值之间 的差异，当函数值越小，说明预测值越接近真实值，其表达式为：(3) Measure the difference between the predicted value and the actual value of the logistic regression layer through the cross-entropy cost function. When the function value is smaller, it means that the predicted value is closer to the real value. The expression is:

其中，x表示船舶数据样本，n表示数据样本的总数；Among them, x represents the ship data sample, and n represents the total number of data samples;

(4)通过步骤三的(1)和(2)操作，对于得到的特征图进行等尺 寸比例的划分网格，每个网格都预测C个船舶类型概率，表示一个 网格在包含船舶目标的条件下属于某种船舶类型的概率，其表达式为：(4) Through the operations of (1) and (2) in step 3, the obtained feature map is divided into grids of equal size and proportion, and each grid predicts C ship type probabilities, which means that a grid contains ship targets The probability of belonging to a certain ship type under the condition of , its expression is:

其中Pr(Class_t|Object)表示目标的类别概率，表示预测框与 真实框交叉的面积，Pr(Class_t)表示类别概率，Pr(Object)是目标存 在的概率；where Pr(Class _t |Object) represents the category probability of the target, Represents the intersection area of the predicted frame and the real frame, Pr(Class _t ) represents the category probability, and Pr(Object) is the probability of the existence of the target;

(5)按照下列算法进行船舶类型分类：(5) Carry out ship type classification according to the following algorithm:

(a)在预测的船舶类别中，将得分少于阈值0.2的设置为0，然后再 按得分从高到低排序；(a) among the predicted ship categories, set the score less than the threshold 0.2 to 0, and then sort by the score from high to low;

(b)用非极大值抑制算法计算边界框的IOU值，当IOU大于0.5， 该边界框重复率较大，该得分设为0，去掉重复率较大的边界框，如 果不大于0.5，则不改；(b) Use the non-maximum value suppression algorithm to calculate the IOU value of the bounding box. When the IOU is greater than 0.5, the bounding box has a large repetition rate, and the score is set to 0. Remove the bounding box with a large repetition rate. If it is not greater than 0.5, do not change;

(c)再选择剩下得分里面最大的边界框，重复步骤(b)直到最后；(c) Then select the largest bounding box in the remaining score, and repeat step (b) until the end;

(d)最后保留的边界框得分如果大于0，那么船舶类型的就是这个 得分所对应的类别；(d) If the score of the last reserved bounding box is greater than 0, then the ship type is the category corresponding to this score;

(6)在输出层加入sigmoid函数把船舶类型预测的数值 作为函数的输入数值，经sigmoid函数后，其数值约束在0到1的范 围内，如果输出值大于设定阈值0.75，就识别出船舶类型，并在边界 框左上方标记出该船舶类别名称。(6) Add the sigmoid function to the output layer The predicted value of the ship type is used as the input value of the function. After the sigmoid function, the value is constrained within the range of 0 to 1. If the output value is greater than the set threshold of 0.75, the ship type is identified and marked on the upper left of the bounding box State the name of the ship class.