CN109034045A - A kind of leucocyte automatic identifying method based on convolutional neural networks - Google Patents

本发明公开了一种基于卷积神经网络的白细胞自动识别方法，属于使用深度学习的机器视觉方案对医学显微镜图像进行自动识别。本发明首先人工对细胞数据集进行标记，做出一个标准的数据集，再使用迁移学习的方法，将VGG‑Net的模型和参数迁移到WBC‑Net卷积神经网络中，通过提取其中效果最好的特征层作为特征参数训练集成分类器，完成对白细胞的识别功能。

The invention discloses a method for automatic identification of white blood cells based on a convolutional neural network, which belongs to a machine vision scheme using deep learning for automatic identification of medical microscope images. The present invention first manually marks the cell data set, makes a standard data set, and then uses the method of migration learning to migrate the model and parameters of VGG-Net to the WBC-Net convolutional neural network, and extracts the most effective A good feature layer is used as a feature parameter to train an integrated classifier to complete the recognition function of white blood cells.

一种基于卷积神经网络的白细胞自动识别方法A method for automatic identification of white blood cells based on convolutional neural network

技术领域technical field

本发明涉及利用计算机视觉方案对医学显微镜图像进行自动识别分类领域，特别是一种基于卷积神经网络的白细胞自动识别方法。The invention relates to the field of automatic identification and classification of medical microscope images by using a computer vision scheme, in particular to a method for automatic identification of white blood cells based on a convolutional neural network.

背景技术Background technique

随着计算机视觉和人工智能的飞速发展，图像处理技术在疾病诊断等医疗方面有了跨越性的进步，在精确度和处理速度上都远远超越了传统的人工处理。白细胞的检测对于很多疾病的诊断分析都有着重大的意义，其中，外周血白细胞的检测可以帮助病理学家诊断诸如白血病和其他血液疾病；通过评估骨髓涂片中的白细胞检测的信息，可以用来诊断淋巴瘤、骨髓瘤、骨髓增殖性肿瘤和贫血以及化疗后的后续护理等目的。急性淋巴细胞白血病是一种严重的血液疾病，诊断非常困难，主要检查在于研究由于癌症而导致畸形的白细胞。因此，图像处理技术的深入研究，将有助于推动医疗技术的发展。With the rapid development of computer vision and artificial intelligence, image processing technology has made leaps and bounds in medical treatment such as disease diagnosis, and has far surpassed traditional manual processing in terms of accuracy and processing speed. The detection of leukocytes is of great significance for the diagnosis and analysis of many diseases. Among them, the detection of peripheral blood leukocytes can help pathologists diagnose leukemia and other blood diseases; by evaluating the information of leukocyte detection in bone marrow smears, it can be used to Diagnosis of lymphoma, myeloma, myeloproliferative neoplasms and anemia and follow-up care after chemotherapy. Acute lymphoblastic leukemia is a serious blood disorder that can be difficult to diagnose, and the main tests are to study the abnormal white blood cells caused by cancer. Therefore, the in-depth study of image processing technology will help to promote the development of medical technology.

一般来说，白细胞的自动检测技术包括图像采集、图像预处理及白细胞检测、白细胞图像的分割、特征提取和分类器设计五个方面。很多研究致力于白细胞图像的分割，比如聚类，阈值法，边缘检测，区域增长，分水岭，颜色和支持向量机等方法，然而由于细胞之间彼此接触，背景与细胞之间差异性小，导致白细胞分割精度难以提高。在特征提取阶段，通过提取圆度、核质比、颜色和形态、几何、纹理特征以及灰度共生矩阵(GLCM)等特征，对某些特定的数据会取得巨大的成功，但这些低级特征对新的数据新的特征的适应性低，因为大多数的手工特征不能简单地应用于新条件。深度学习的思想是寻求更深层次的特征，通过一些非线性的模型把原始数据转变成更高层次、更抽象的表达。深度学习在图像处理中比较成功的应用是使用卷积神经网络的体系结构，CNN能更好和快速准确的理解分析图像场景中形成的区域目标，通过权值共享和池化层的设置，使网络训练的参数大量减少，精简了网络模型，同时提高了训练的效率。Generally speaking, the automatic detection technology of white blood cells includes five aspects: image acquisition, image preprocessing and white blood cell detection, segmentation of white blood cell images, feature extraction and classifier design. Many studies have been devoted to the segmentation of white blood cell images, such as clustering, thresholding, edge detection, region growing, watershed, color and support vector machines. However, due to the contact between cells, the difference between the background and the cells is small, resulting in It is difficult to improve the accuracy of leukocyte segmentation. In the feature extraction stage, by extracting features such as roundness, nuclear-to-cytoplasmic ratio, color and shape, geometry, texture features, and gray level co-occurrence matrix (GLCM), great success will be achieved for some specific data, but these low-level features are of great importance to The adaptability of new features to new data is low, since most handcrafted features cannot be simply applied to new conditions. The idea of deep learning is to seek deeper features and convert raw data into higher-level and more abstract expressions through some nonlinear models. The more successful application of deep learning in image processing is to use the convolutional neural network architecture. CNN can better and quickly and accurately understand and analyze the regional targets formed in the image scene. Through weight sharing and pooling layer settings, the The parameters of network training are greatly reduced, the network model is simplified, and the efficiency of training is improved at the same time.

由于卷积神经网络在准确率和训练效率方面都取得了显著的提高，所以有很多研究者对基于卷积神经网络的白细胞的识别检测做了很多工作，提出了许多实用可行的方法措施，主要包括：Since the convolutional neural network has achieved significant improvements in accuracy and training efficiency, many researchers have done a lot of work on the recognition and detection of white blood cells based on convolutional neural networks, and proposed many practical and feasible methods. include:

(1)专利《一种基于卷积神经网络的白带中白细胞自动识别方法》(中国专利公开号CN 106897682 A)提出了一种基于卷积神经网络的白带中白细胞自动识别的方法，首先经过人工处理，得到白细胞的分割图像，运用最近邻插值算法对分割图像进行缩放处理，通过九层的卷积神经网络对分割图像进行训练和测试，当满足方差代价函数时，则完成训练，白细胞的识别框架就确定了，可以通过输入分割图片，进行白细胞的识别。(1) The patent "A Method for Automatic Identification of Leucorrhea in Leucorrhea Based on Convolutional Neural Network" (Chinese Patent Publication No. CN 106897682 A) proposes a method for automatic identification of leucorrhea in leucorrhea based on convolutional neural network. Processing to obtain the segmented image of white blood cells, use the nearest neighbor interpolation algorithm to scale the segmented image, train and test the segmented image through the nine-layer convolutional neural network, when the variance cost function is satisfied, the training is completed, and the identification of white blood cells The framework is determined, and white blood cells can be identified by inputting the segmented picture.

缺点：对于图片的处理仍然需要人工分割，白细胞的识别精度可能由于分割的误差而有很大的区别；卷积神经网络的设置中缺少对网络过拟合的控制，可能由于数据特征的相关性造成网络的过拟合。Disadvantages: Manual segmentation is still required for image processing, and the recognition accuracy of white blood cells may vary greatly due to segmentation errors; the convolutional neural network setting lacks control over network overfitting, which may be due to the correlation of data features lead to overfitting of the network.

(2)专利《一种基于nu-支持向量机的白细胞分类方法》(中国专利公开号 CN107730499 A)采用一种基于nu-支持向量机的白细胞分类方法，首先对彩色血液显微镜图像进行中值滤波，然后把图像映射到HLS彩色空间，再使用nu-支持向量机把色调图像分割，使用模糊细胞神经网络(Fuzzy Cellular Neural Network——FCNN)从粗分割图像中检测出白细胞区域图像，通过聚类分析法确定阈值，结合阈值分割和二值形态学方法进行分割，得到细胞核局部图像、细胞浆局部图像和背景图像，从细胞核局部图像和细胞浆局部图像中抽取具有代表性的四十七个特征，使用这些特征向量，利用nu-支持向量机完成对白细胞的识别与分类。(2) The patent "A Leukocyte Classification Method Based on nu-Support Vector Machine" (Chinese Patent Publication No. CN107730499 A) adopts a leukocyte classification method based on nu-SVM, and first performs median filtering on color blood microscope images , and then map the image to the HLS color space, and then use the nu-support vector machine to segment the tone image, use the fuzzy cellular neural network (Fuzzy Cellular Neural Network - FCNN) to detect the leukocyte area image from the rough segmented image, and cluster The threshold value is determined by the analysis method, combined with the threshold segmentation and binary morphology method for segmentation, and the partial image of the nucleus, the partial image of the cytoplasm and the background image are obtained, and representative forty-seven features are extracted from the partial image of the nucleus and the partial image of the cytoplasm , using these feature vectors, use nu-support vector machine to complete the identification and classification of white blood cells.

缺点：将细胞图像转换了彩色空间，可能会丢失一些有意义的颜色特征；细胞分割有误差，对之后的特征提取有很大的影响；由于分割提取的是细胞核和细胞浆的局部特征，没有考虑全局特征，在识别和分类的时候有局限性，对新的白细胞图片的适应性比较弱。Disadvantages: Converting the cell image to the color space may lose some meaningful color features; there is an error in the cell segmentation, which has a great impact on the subsequent feature extraction; since the segmentation extracts the local features of the nucleus and cytoplasm, there is no Considering global features, there are limitations in recognition and classification, and the adaptability to new white blood cell pictures is relatively weak.

(3)专利《一种基于深度学习的白细胞五分类方法》(中国专利公开号CN106248559 A)采用一种基于深度学习的白细胞五分类的方法，首先通过对RGB 空间进行转换，得到的粗分割图像进行形态学变化得到完整的细胞核图，进而由细胞核定位出白细胞的图像。对检测的白细胞提取其纹理特征(共生LBP直方图特征)，据此分出嗜碱性粒细胞、嗜酸性粒细胞和其他三类细胞：中性粒细胞、淋巴细胞、单核细胞。使用卷积神经网络自动提取其它三类细胞的特征，通过随机森林进行三分类。(3) The patent "A Method for Five Classifications of White Blood Cells Based on Deep Learning" (Chinese Patent Publication No. CN106248559 A) adopts a method of five classifications of white blood cells based on deep learning. First, the rough segmentation image is obtained by converting the RGB space Perform morphological changes to obtain a complete nucleus map, and then locate the image of white blood cells from the nucleus. The texture feature (symbiotic LBP histogram feature) was extracted from the detected leukocytes, and basophils, eosinophils, and other three types of cells were separated accordingly: neutrophils, lymphocytes, and monocytes. Use the convolutional neural network to automatically extract the features of the other three types of cells, and perform three classifications through the random forest.

缺点：仅通过纹理特征及SVM分类器并不能准确的区分嗜酸性粒细胞和其它三类细胞，而使用神经网络提取的特征是分类错误之后的特征，即将部分嗜酸性粒细胞作为其它三类细胞一并进行了特征的提取，使得用随机森林对其它三类细胞分类的准确率比较低。Disadvantages: Only texture features and SVM classifiers cannot accurately distinguish eosinophils from the other three types of cells, and the features extracted by neural networks are features after classification errors, that is, some eosinophils are regarded as the other three types of cells The features are extracted together, so that the accuracy of random forest classification for the other three types of cells is relatively low.

发明内容Contents of the invention

本发明所要解决的技术问题是，针对现有技术不足，提供一种基于卷积神经网络的白细胞自动识别方法，The technical problem to be solved by the present invention is to provide a method for automatic recognition of leukocytes based on convolutional neural networks,

为解决上述技术问题，本发明所采用的技术方案是：一种基于卷积神经网络的白细胞自动识别方法，该方法包括如下步骤：In order to solve the above-mentioned technical problems, the technical solution adopted in the present invention is: a method for automatic identification of white blood cells based on convolutional neural network, the method includes the following steps:

步骤1：数据集的处理和准备，使用ImageNet数据集用于预训练网络，对包含有白细胞的显微镜图片进行单个细胞提取并对其做分类标注，获得带有标签的标准数据集用于之后对微调卷积神经网络的训练和测试；Step 1: Processing and preparation of the data set, using the ImageNet data set for pre-training the network, extracting individual cells from microscope pictures containing white blood cells and classifying and labeling them, and obtaining a standard data set with labels for later analysis Fine-tuning training and testing of convolutional neural networks;

1)随机提取多组以细胞核心为粗略中心的NxN尺寸的样本块，避免了在确定细胞核心位置时的误差导致提取整个细胞时造成的分割误差，同时，提取的多组样本块可以有效实现对数据集的增强；1) Randomly extract multiple sets of sample blocks of NxN size with the cell core as the rough center, avoiding the segmentation error caused by the error in determining the position of the cell core and the extraction of the entire cell. At the same time, the extracted multiple sets of sample blocks can be effectively realized Enhancements to datasets;

2)由有经验的专业人员对步骤1-1中的每个样本块标注类别标签，准确地分出异常白细胞和正常白细胞；2) Label each sample block in step 1-1 by an experienced professional to accurately separate abnormal white blood cells from normal white blood cells;

步骤2：将步骤1得到的白细胞数据集随机地按适当的比例分成训练集和测试集，训练集用于对卷积神经网络的微调训练过程，测试集用于检验整个算法的效率和参数权重的更新；Step 2: Randomly divide the white blood cell data set obtained in step 1 into a training set and a test set in an appropriate proportion. The training set is used to fine-tune the training process of the convolutional neural network, and the test set is used to test the efficiency and parameter weight of the entire algorithm. update;

步骤3：对步骤2得到的训练集进行图像增强操作，具体的，对图片采用关于垂直方向的镜面随机反射操作，以及在[-30,30]像素范围内随机地左右平移和上下平移操作(测试集不做图像增强操作)；Step 3: Perform image enhancement operations on the training set obtained in step 2. Specifically, use mirror random reflection operations on the vertical direction on the pictures, and randomly translate left and right and up and down within the range of [-30,30] pixels ( The test set does not perform image enhancement operations);

步骤4：卷积神经网络的结构设置，首先对卷积神经网络进行预训练，使用ImageNet数据集训练VGG-Net卷积神经网络，然后采用迁移学习的方法，将预训练后的部分模型和权重参数转移到需要微调的WBC-Net上，使用步骤3中获得的增强训练数据集进行微调操作；Step 4: The structure setting of the convolutional neural network, first pre-training the convolutional neural network, using the ImageNet dataset to train the VGG-Net convolutional neural network, and then adopting the method of migration learning, the pre-trained part of the model and weight The parameters are transferred to the WBC-Net that needs to be fine-tuned, and the fine-tuning operation is performed using the enhanced training data set obtained in step 3;

1)步骤4中的预训练VGG-Net卷积神经网络为16层，分别是输入层I1，卷积层C1，池化层P1，卷积层C2，池化层P2，卷积层C3，池化层P3，卷积层C4，池化层P4，卷积层C5，池化层P5，全连接层FC1，全连接层FC2，全连接层FC3，Softmax层S1，输出层O1；1) The pre-trained VGG-Net convolutional neural network in step 4 has 16 layers, which are input layer I1, convolutional layer C1, pooling layer P1, convolutional layer C2, pooling layer P2, convolutional layer C3, Pooling layer P3, convolutional layer C4, pooling layer P4, convolutional layer C5, pooling layer P5, fully connected layer FC1, fully connected layer FC2, fully connected layer FC3, Softmax layer S1, output layer O1;

2)步骤4中的微调训练WBC-Net卷积神经网络设计为14层，1-13层和步骤4-1中的预训练卷积神经网络VGG-Net的1-13层是相同的，为进行转移学习和权值共享操作提供了网络结构基础，14层为全连接特征提取层FCL，15层为 Softmax层S2，16层为分类输出层CO1；2) The fine-tuning training WBC-Net convolutional neural network in step 4 is designed to be 14 layers, and the 1-13 layers of the pre-trained convolutional neural network VGG-Net in step 4-1 are the same, for The transfer learning and weight sharing operations provide the basis of the network structure. The 14th layer is the fully connected feature extraction layer FCL, the 15th layer is the Softmax layer S2, and the 16th layer is the classification output layer CO1;

步骤5：卷积神经网络结构的参数设置，VGG-Net和WBC-Net的1-13层的结构参数设置相同，卷积层的设置分别是卷积核的大小为3x3，padding设置为1，步长为1；每一次卷积操作之后进行一次激活函数处理，这里选择ReLU激活函数，不仅能够避免梯度消失，同时使网络具有稀疏性，减少参数的相关性，在一定程度上减少过拟合的问题；池化层采用2x2的最大池化层，步长设置为2；全连接层FC1、FC2之后分别进行一次ReLU激活处理，同时设置dropout率为0.5，防止网络过拟合，增强网络的训练效率；VGG-Net的全连接层FC3的输出是1000 维的特征输出，WBC-Net卷积神经网络的全连接特征提取层FCL的输出设置为2；Step 5: Parameter setting of the convolutional neural network structure. The structural parameter settings of the 1-13 layers of VGG-Net and WBC-Net are the same. The setting of the convolution layer is that the size of the convolution kernel is 3x3, and the padding is set to 1. The step size is 1; after each convolution operation, an activation function is processed. The ReLU activation function is selected here, which can not only avoid the disappearance of the gradient, but also make the network sparse, reduce the correlation of parameters, and reduce overfitting to a certain extent. The pooling layer adopts a 2x2 maximum pooling layer, and the step size is set to 2; after the fully connected layers FC1 and FC2, a ReLU activation process is performed respectively, and the dropout rate is set to 0.5 at the same time to prevent network overfitting and enhance network performance. Training efficiency; the output of the fully connected layer FC3 of VGG-Net is a 1000-dimensional feature output, and the output of the fully connected feature extraction layer FCL of the WBC-Net convolutional neural network is set to 2;

步骤6：由步骤4的卷积神经网络的结构设置和步骤5的卷积神经网络结构的参数设置使用ImageNet数据集对VGG-Net进行训练，训练完成后，提取1-13 层的网络结构模型和训练参数用于之后迁移学习的权值共享，至此，整个预训练阶段结束；Step 6: Use the ImageNet data set to train the VGG-Net by setting the structure of the convolutional neural network in step 4 and the parameter setting of the convolutional neural network in step 5. After the training is completed, extract the network structure model of 1-13 layers It is shared with the weights of the training parameters for subsequent transfer learning, so far, the entire pre-training phase is over;

步骤7：卷积神经网络的微调过程，步骤6得到的预训练网络参数作为微调卷积神经网络的初始权值，使用步骤3得到的细胞增强训练集对WBC-Net卷积神经网络进行微调；输入尺寸大小为224x224的细胞图片，经过第一组卷积池化操作得到64张112x112的特征图，经过第二组卷积池化操作得到128张56x56的特征图，经过第三组、第四组、第五组卷积池化操作后最终得到512张7x7的特征图，接着由三层全连接层对512张特征图进行特征融合，通过Softmax层计算出每个类别的概率，最终由输出层输出得到两行一列的分类结果，从输入层到输出层的完整运算过程为一次向前传播；Step 7: The fine-tuning process of the convolutional neural network, the pre-trained network parameters obtained in step 6 are used as the initial weights of the fine-tuned convolutional neural network, and the WBC-Net convolutional neural network is fine-tuned using the cell enhancement training set obtained in step 3; Input a cell picture with a size of 224x224, get 64 feature maps of 112x112 after the first set of convolution pooling operations, and get 128 feature maps of 56x56 after the second set of convolution pooling operations, after the third set, the fourth set After the first group and the fifth group of convolution pooling operations, 512 7x7 feature maps are finally obtained, and then the 512 feature maps are fused by the three-layer fully connected layer, and the probability of each category is calculated through the Softmax layer, and finally output by The output of the layer obtains the classification results of two rows and one column, and the complete operation process from the input layer to the output layer is a forward propagation;

步骤8：在微调卷积神经网络的过程中，采用监督学习的方法，在经过步骤 7的一次向前传播后，使用交叉熵损失函数计算出一次向前传播的分类误差，然后选用随机渐变梯度下降的策略迭代地减小损失函数值，更新每层网络的参数值，为了加快网络的收敛速度，设置动量为0.9，初始学习率为1.0e-4， MiniBatchSize设置为10，经过一次权重参数的更新为一次后反馈传播；Step 8: In the process of fine-tuning the convolutional neural network, the method of supervised learning is adopted. After a forward propagation in step 7, the cross-entropy loss function is used to calculate the classification error of a forward propagation, and then the random gradient gradient is selected. The descending strategy iteratively reduces the value of the loss function and updates the parameter values of each layer of the network. In order to speed up the convergence of the network, the momentum is set to 0.9, the initial learning rate is 1.0e-4, and the MiniBatchSize is set to 10. Feedback propagation after updating to once;

步骤9：使用白细胞数据集微调WBC-Net卷积神经网络，经过多次步骤7的向前传播和步骤8的后反馈传播更新网络结构参数，当达到设置的epoch时，卷积神经网络收敛，结束训练，至此，WBC-Net模型已经训练完成；Step 9: Use the white blood cell data set to fine-tune the WBC-Net convolutional neural network, and update the network structure parameters after multiple forward propagation in step 7 and post-feedback propagation in step 8. When the set epoch is reached, the convolutional neural network converges. End the training, so far, the WBC-Net model has been trained;

步骤10：从步骤9中的已经训练好的WBC-Net卷积神经网络模型中提取FCL 层的权重参数作为分类器的特征输入，训练一组精确度高、鲁棒性强、性能稳定的集成学习分类器，对白细胞做最终的分类；Step 10: Extract the weight parameters of the FCL layer from the trained WBC-Net convolutional neural network model in step 9 as the feature input of the classifier, and train a set of integrated Learn the classifier and make the final classification of white blood cells;

步骤11：集成学习分类器的设置，使用N个决策树桩分类器作为基分类器进行集成训练，训练数据的初始权重分布是均匀分布的；Step 11: The setting of the ensemble learning classifier, using N decision tree stump classifiers as the base classifier for ensemble training, the initial weight distribution of the training data is evenly distributed;

1)由初始权值的训练数据训练基分类器G(x)，由分类误差率确定基分类器的系数；1) Train the base classifier G(x) by the training data of the initial weight, and determine the coefficient of the base classifier by the classification error rate;

2)每次迭代训练一个基分类器，训练完成后更新一次训练数据集的权值分布，并用更新后的训练集训练下一个基分类器；2) Train a base classifier for each iteration, update the weight distribution of the training data set once after the training is completed, and use the updated training set to train the next base classifier;

3)训练后的N个基分类器线性加法组合成最终的白细胞分类器，在分类器的训练过程中使用五倍交叉验证法对其进行验证评估，至此，分类器的设计训练已经完成；3) The linear addition of the N base classifiers after training is combined into the final leukocyte classifier, and the five-fold cross-validation method is used to verify and evaluate it during the training process of the classifier. So far, the design and training of the classifier has been completed;

步骤12：算法测试阶段，把步骤3得到的测试集(包含正常白细胞和异常白细胞)输入步骤9中训练好的WBC-Net卷积神经网络，将提取的特征矩阵作为特征输入到骤11训练好的集成分类器进行分类；Step 12: Algorithm testing phase, input the test set (including normal white blood cells and abnormal white blood cells) obtained in step 3 into the WBC-Net convolutional neural network trained in step 9, and input the extracted feature matrix as a feature to step 11 for training The ensemble classifier for classification;

步骤13：验证算法的性能，通过步骤12得出的预测值与真实值对比，生成混淆矩阵，对混淆矩阵求均值，得到本发明的白细胞分类检测算法的准确率为 98.72％；Step 13: verify the performance of the algorithm, compare the predicted value obtained in step 12 with the real value, generate a confusion matrix, calculate the mean value of the confusion matrix, and obtain an accuracy rate of 98.72% for the white blood cell classification detection algorithm of the present invention;

步骤14：通过以上对VGG-Net卷积神经网络进行微调得到WBC-Net卷积神经网络，提取卷积神经网络层中的深度特征进行集成分类器的训练和测试，最终得到了准确率较高、鲁棒性较强、稳定性较高的白细胞自动检测算法；Step 14: Through the fine-tuning of the VGG-Net convolutional neural network above, the WBC-Net convolutional neural network is obtained, and the deep features in the convolutional neural network layer are extracted for training and testing of the integrated classifier, and finally a high accuracy rate is obtained. , A white blood cell automatic detection algorithm with strong robustness and high stability;

步骤15：把无标签的细胞图片作为输入，通过WBC-Net卷积神经网络的特征提取，再利用集成分类器进行分类，输出正常白细胞或者异常白细胞类别。Step 15: Take the unlabeled cell picture as input, extract the features through the WBC-Net convolutional neural network, and then use the integrated classifier to classify, and output the normal white blood cell or abnormal white blood cell category.

与现有技术相比，本发明所具有的有益效果为：Compared with prior art, the beneficial effect that the present invention has is:

1、本发明通过对白细胞数据集的增强操作，使得卷积神经网络能提取到更丰富的特征参数；1. The present invention enables the convolutional neural network to extract more abundant feature parameters through the enhanced operation of the white blood cell data set;

2、本发明通过使用了迁移学习的思路，用微调的方法来更新卷积神经网络的参数，避免了数据集的数量偏小的弊端，使得小数据集也能训练得到很好的分类效果；2. By using the idea of transfer learning, the present invention updates the parameters of the convolutional neural network with a fine-tuning method, avoiding the disadvantages of small data sets, so that small data sets can also be trained to obtain good classification effects;

3、本发明使用集成分类器替换了VGG-Net中原有的Softmax分类层，使分类准确率有了很大的提升。3. The present invention uses an integrated classifier to replace the original Softmax classification layer in VGG-Net, which greatly improves the classification accuracy.

附图说明Description of drawings

图1为本发明方法流程图；Fig. 1 is a flow chart of the method of the present invention;

图2为本发明整体网络结构框图。FIG. 2 is a block diagram of the overall network structure of the present invention.

具体实施方式Detailed ways

如图1所示，本发明的总体步骤如下：As shown in Figure 1, the overall steps of the present invention are as follows:

步骤1：数据集的处理和准备，使用ImageNet数据集用于预训练网络，对包含有白细胞的显微镜图片进行单个细胞提取并对其做分类标注，获得带有标签的标准数据集用于之后对微调卷积神经网络的训练和测试；Step 1: Processing and preparation of the data set, using the ImageNet data set for pre-training the network, extracting individual cells from microscope pictures containing white blood cells and classifying and labeling them, and obtaining a standard data set with labels for later analysis Fine-tuning training and testing of convolutional neural networks;

1)随机提取多组以细胞核心为粗略中心的257x257尺寸的样本块，避免了在确定细胞核心位置时的误差导致提取整个细胞时造成的分割误差，同时，提取的多组样本块可以有效实现对数据集的增强；1) Randomly extract multiple groups of sample blocks of 257x257 size with the cell core as the rough center, which avoids the segmentation error caused by the error in determining the position of the cell core and the extraction of the entire cell. At the same time, the extracted multiple groups of sample blocks can be effectively realized Enhancements to datasets;

2)由有经验的专业人员对步骤1-1中的每个样本块标注类别标签，准确地分出异常白细胞和正常白细胞；2) Label each sample block in step 1-1 by an experienced professional to accurately separate abnormal white blood cells from normal white blood cells;

步骤2：将步骤1得到的白细胞数据集随机地按7:3的比例分成训练集和测试集，训练集用于对卷积神经网络的微调训练过程，测试集用于检验整个算法的效率和参数权重的更新；Step 2: The white blood cell data set obtained in step 1 is randomly divided into a training set and a test set in a ratio of 7:3. The training set is used to fine-tune the training process of the convolutional neural network, and the test set is used to test the efficiency and efficiency of the entire algorithm. update of parameter weights;

步骤3：对步骤2得到的训练集进行图像增强操作，具体的，对图片采用关于垂直方向的镜面随机反射操作，以及在[-30,30]像素范围内均匀的左右平移和上下平移操作(测试集不做图像增强操作)；Step 3: Perform image enhancement operations on the training set obtained in step 2. Specifically, use mirror random reflection operations in the vertical direction on the pictures, and uniform left-right translation and up-down translation operations within the range of [-30,30] pixels ( The test set does not perform image enhancement operations);

步骤4：卷积神经网络的结构设置如图2所示，首先对卷积神经网络进行预训练，使用ImageNet数据集训练VGG-Net卷积神经网络，然后采用迁移学习的方法，将预训练后的部分模型和权重参数转移到需要微调的WBC-Net上，使用步骤3中获得的增强训练数据集进行微调操作；Step 4: The structural settings of the convolutional neural network are shown in Figure 2. First, pre-train the convolutional neural network, use the ImageNet dataset to train the VGG-Net convolutional neural network, and then adopt the transfer learning method to transfer the pre-trained Transfer part of the model and weight parameters to the WBC-Net that needs to be fine-tuned, and use the enhanced training data set obtained in step 3 to perform fine-tuning operations;

1)步骤4中的预训练VGG-Net卷积神经网络为16层，分别是输入层I1，卷积层C1，池化层P1，卷积层C2，池化层P2，卷积层C3，池化层P3，卷积层 C4，池化层P4，卷积层C5，池化层P5，全连接层FC1，全连接层FC2，全连接层FC3，Softmax层S1，输出层O1；1) The pre-trained VGG-Net convolutional neural network in step 4 has 16 layers, which are input layer I1, convolutional layer C1, pooling layer P1, convolutional layer C2, pooling layer P2, convolutional layer C3, Pooling layer P3, convolutional layer C4, pooling layer P4, convolutional layer C5, pooling layer P5, fully connected layer FC1, fully connected layer FC2, fully connected layer FC3, Softmax layer S1, output layer O1;

2)步骤4中的微调训练WBC-Net卷积神经网络设计为14层，1-13层和步骤4-1中的预训练卷积神经网络VGG-Net的1-13层是相同的，为进行转移学习和权值共享操作提供了网络结构基础，14层为全连接特征提取层FCL，15层为 Softmax层S2，16层为分类输出层CO1；2) The fine-tuning training WBC-Net convolutional neural network in step 4 is designed to be 14 layers, and the 1-13 layers of the pre-trained convolutional neural network VGG-Net in step 4-1 are the same, for The transfer learning and weight sharing operations provide the basis of the network structure. The 14th layer is the fully connected feature extraction layer FCL, the 15th layer is the Softmax layer S2, and the 16th layer is the classification output layer CO1;

步骤5：卷积神经网络结构的参数设置，VGG-Net和WBC-Net的1-13层的结构参数设置相同，卷积层的设置分别是卷积核的大小为3x3，padding设置为1，步长为1；每一次卷积操作之后进行一次激活函数处理，这里选择ReLU激活函数，激活函数ReLU的公式为Step 5: Parameter setting of the convolutional neural network structure. The structural parameter settings of the 1-13 layers of VGG-Net and WBC-Net are the same. The setting of the convolution layer is that the size of the convolution kernel is 3x3, and the padding is set to 1. The step size is 1; after each convolution operation, an activation function is processed. Here, the ReLU activation function is selected. The formula of the activation function ReLU is

f(x)＝max(0，x) (1)f(x)=max(0,x) (1)

从公式可以看出，激活操作不仅能够避免梯度消失，同时使网络具有稀疏性，减少参数的相关性，在一定程度上减少过拟合的问题；池化层采用2x2的最大池化层，步长设置为2；全连接层FC1、FC2之后分别进行一次ReLU激活处理，同时设置dropout率为0.5，防止网络过拟合，增强网络的训练效率；VGG-Net的全连接层FC3的输出是1000维的特征输出，WBC-Net卷积神经网络的全连接特征提取层FCL的输出设置为2；It can be seen from the formula that the activation operation can not only avoid the disappearance of the gradient, but also make the network sparse, reduce the correlation of parameters, and reduce the problem of overfitting to a certain extent; the pooling layer adopts a 2x2 maximum pooling layer, and the step The length is set to 2; after the fully connected layers FC1 and FC2, a ReLU activation process is performed respectively, and the dropout rate is set to 0.5 at the same time to prevent the network from overfitting and enhance the training efficiency of the network; the output of the fully connected layer FC3 of VGG-Net is 1000 Dimensional feature output, the output of the fully connected feature extraction layer FCL of the WBC-Net convolutional neural network is set to 2;

步骤6：由步骤4的卷积神经网络的结构设置和步骤5的卷积神经网络结构的参数设置使用ImageNet数据集对VGG-Net进行训练，训练完成后，提取1-13 层的网络结构模型和训练参数用于之后迁移学习的权值共享，至此，整个预训练阶段结束；Step 6: Use the ImageNet data set to train the VGG-Net by setting the structure of the convolutional neural network in step 4 and the parameter setting of the convolutional neural network in step 5. After the training is completed, extract the network structure model of 1-13 layers It is shared with the weights of the training parameters for subsequent transfer learning, so far, the entire pre-training phase is over;

步骤7：卷积神经网络的微调过程，步骤6得到的预训练网络参数作为微调卷积神经网络的初始权值，使用步骤3得到的细胞增强训练集对WBC-Net卷积神经网络进行微调；输入尺寸大小为224x224的细胞图片，经过第一组卷积池化操作得到64张112x112的特征图，经过第二组卷积池化操作得到128张56x56的特征图，经过第三组、第四组、第五组卷积池化操作后最终得到512张7x7的特征图，接着由三层全连接层对512张特征图进行特征融合，通过Softmax层计算出每个类别的概率，假设S_j表示第j个类别的概率，a_j表示第j类的向量值(本发明中j＝1,2)，T为分类的类别数，概率计算公式为Step 7: The fine-tuning process of the convolutional neural network, the pre-trained network parameters obtained in step 6 are used as the initial weights of the fine-tuned convolutional neural network, and the WBC-Net convolutional neural network is fine-tuned using the cell enhancement training set obtained in step 3; Input a cell picture with a size of 224x224, get 64 feature maps of 112x112 after the first set of convolution pooling operations, and get 128 feature maps of 56x56 after the second set of convolution pooling operations, after the third set, the fourth set After the convolution pooling operation of the first and fifth groups, 512 7x7 feature maps are finally obtained, and then the 512 feature maps are fused by the three-layer fully connected layer, and the probability of each category is calculated through the Softmax layer, assuming S _j Represent the probability of the j category, a _j represent the vector value of the j category (j=1,2 in the present invention), T is the category number of classification, and the probability calculation formula is

最终由输出层输出得到两行一列的分类结果，从输入层到输出层的完整运算过程为一次向前传播；Finally, the classification results of two rows and one column are output by the output layer, and the complete operation process from the input layer to the output layer is a forward propagation;

步骤8：在微调卷积神经网络的过程中，采用监督学习的方法，在经过步骤 7的一次向前传播后，使用交叉熵损失函数计算出一次向前传播的分类误差，y_i表示真实的分类结果，S_j为步骤7中Softmax层算出的每一类的概率，则交叉熵损失函数公式为Step 8: In the process of fine-tuning the convolutional neural network, the method of supervised learning is adopted. After a forward propagation in step 7, the cross-entropy loss function is used to calculate the classification error of a forward propagation, and y _i represents the real Classification results, S _j is the probability of each class calculated by the Softmax layer in step 7, then the formula of the cross entropy loss function is

C＝-∑_iy_ilnS_i (3)C＝-∑ _i y _i lnS _i (3)

然后选用随机渐变梯度下降的策略迭代地减小损失函数值，更新每层网络的参数值，为了加快网络的收敛速度，设置动量为0.9，初始学习率为1.0e-4， MiniBatchSize设置为10，经过一次权重参数的更新为一次后反馈传播；Then use the strategy of random gradient descent to iteratively reduce the loss function value, and update the parameter value of each layer of the network. In order to speed up the convergence speed of the network, set the momentum to 0.9, the initial learning rate to 1.0e-4, and the MiniBatchSize to 10. After a weight parameter update is a post-feedback propagation;

步骤9：使用白细胞数据集微调WBC-Net卷积神经网络，经过多次步骤7的向前传播和步骤8的后反馈传播更新网络结构参数，当达到设置的epoch时，卷积神经网络收敛，结束训练，至此，WBC-Net模型已经训练完成；Step 9: Use the white blood cell data set to fine-tune the WBC-Net convolutional neural network, and update the network structure parameters after multiple forward propagation in step 7 and post-feedback propagation in step 8. When the set epoch is reached, the convolutional neural network converges. End the training, so far, the WBC-Net model has been trained;

步骤10：从步骤9中的已经训练好的WBC-Net卷积神经网络模型中提取FCL 层的权重参数作为分类器的特征输入，训练一组精确度高、鲁棒性强、性能稳定的集成学习分类器，对白细胞做最终的分类；Step 10: Extract the weight parameters of the FCL layer from the trained WBC-Net convolutional neural network model in step 9 as the feature input of the classifier, and train a set of integrated Learn the classifier and make the final classification of white blood cells;

步骤11：集成学习分类器的设置，使用八个决策树桩分类器作为基分类器进行集成训练，假设训练数据集Step 11: The setting of the ensemble learning classifier, using eight decision tree stump classifiers as the base classifier for ensemble training, assuming the training data set

T＝{(x₁，y₁)，(x₂，y₂)，...，(x_N，y_N)},y_i∈{-1，+1}，T={(x ₁ , y ₁ ), (x ₂ , y ₂ ), ..., (x _N , y _N )}, y _i ∈ {-1, +1},

训练数据的初始权重分布是均匀分布的；The initial weight distribution of the training data is uniformly distributed;

1)基分类器G(x)在训练数据集上的分类误差率确定基分类器的系数，分类误差率为1) The classification error rate of the base classifier G(x) on the training data set determines the coefficient of the base classifier, and the classification error rate is

基分类器的系数为The coefficients of the base classifier are

2)每次迭代训练一个基分类器，训练完成后更新一次训练数据集的权值分布，并用更新后的训练集训练下一个基分类器；2) Train a base classifier for each iteration, update the weight distribution of the training data set once after the training is completed, and use the updated training set to train the next base classifier;

3)八个基分类器线性加法组合成最终的白细胞分类器，在分类器的训练过程中使用五倍交叉验证法对其进行验证评估，至此，分类器的设计训练已经完成；3) The linear addition of eight base classifiers is combined into the final white blood cell classifier, and the five-fold cross-validation method is used to verify and evaluate it during the training process of the classifier. So far, the design and training of the classifier has been completed;

步骤12：算法测试阶段，把步骤3得到的测试集(包含正常白细胞和异常白细胞)输入步骤9中训练好的WBC-Net卷积神经网络，将提取的特征矩阵作为特征输入到步骤11训练好的集成分类器进行分类；Step 12: Algorithm testing phase, input the test set (including normal white blood cells and abnormal white blood cells) obtained in step 3 into the WBC-Net convolutional neural network trained in step 9, and input the extracted feature matrix as a feature to step 11 for training The ensemble classifier for classification;

步骤13：验证算法的性能，通过步骤12得出的预测值与真实值对比，生成混淆矩阵，对混淆矩阵求均值，得到本发明的白细胞分类检测算法的准确率为 98.72％；Step 13: verify the performance of the algorithm, compare the predicted value obtained in step 12 with the real value, generate a confusion matrix, calculate the mean value of the confusion matrix, and obtain an accuracy rate of 98.72% for the white blood cell classification detection algorithm of the present invention;

步骤14：通过以上对VGG-Net卷积神经网络进行微调得到WBC-Net卷积神经网络，提取卷积神经网络层中的深度特征进行集成分类器的训练和测试，最终得到了准确率较高、鲁棒性较强、稳定性较高的白细胞自动检测算法；Step 14: Through the fine-tuning of the VGG-Net convolutional neural network above, the WBC-Net convolutional neural network is obtained, and the deep features in the convolutional neural network layer are extracted for training and testing of the integrated classifier, and finally a high accuracy rate is obtained. , A white blood cell automatic detection algorithm with strong robustness and high stability;

步骤15：把无标签的细胞图片作为输入，通过WBC-Net卷积神经网络的特征提取，再利用集成分类器进行分类，输出正常白细胞或者异常白细胞类别。Step 15: Take the unlabeled cell picture as input, extract the features through the WBC-Net convolutional neural network, and then use the integrated classifier to classify, and output the normal white blood cell or abnormal white blood cell category.