CN108376400A - A kind of bone marrow cell automatic classification method - Google Patents

The invention discloses a kind of bone marrow cell automatic classification methods, include the following steps：(1) the bone marrow cell pre-detection based on saturation degree；(2) the bone marrow cell detection based on rarefaction representation；(3) morphologic multi-angle bone marrow cell is based on to divide；(4) bone marrwo cell sorting based on deep learning.The present invention can obtain accurate bone marrow cell test position, divide image and classification results, and whole process is participated in without artificial, be truly realized full-automatic bone marrow cell detection, segmentation and classification, be conducive to the medical treatments such as follow-up Diagnosis of Acute Leukemia.

一种骨髓细胞自动分类方法A method for automatic classification of bone marrow cells

技术领域technical field

本发明涉及医学图像处理的技术领域，特别涉及一种骨髓细胞自动分类方法。The invention relates to the technical field of medical image processing, in particular to an automatic classification method for bone marrow cells.

背景技术Background technique

白血病是起源于造血系统的恶性肿瘤，其特点是骨髓中产生和积聚大量幼稚和异常的白细胞，并浸润其他器官，导致正常造血功能的抑制和衰竭。依据白血病细胞的分化程度和自然病程的长短，可分为急性和慢性两大类。我国白血病类型分布急性多于慢性，以髓细胞白血病多见，其发病率在各种肿瘤中排第六位。急性白血病(AL)患者常伴有贫血、出血、发热、感染以及浸润等临床症状，若不进行及时有效的特殊治疗，平均生存期只有三个月左右，短则甚至在诊断数天后死亡，严重威胁着患者的身体健康和日常生活。急性白血病诊断及分型的准确与否直接关系治疗方案的选择及预后提示。细胞形态学是急性白血病诊断中应用最多、最广泛、最直接和最经济的一种重要诊断手段，是形态学、免疫学、细胞遗传学、分子生物学(MICM)分型诊断的重要组成部分。形态学方法主要是将患者骨髓涂片和血涂片分别进行瑞氏-吉姆萨染色分析，并进一步予以其他细胞化学染色，按照FAB(French、American、Britain)标准对急性白血病类型进行判定。在实际操作中，该方法仍采用人工操作方法，检验工作量大，可重复性差，不仅耗时耗力，医师连续工作易因疲劳或粗心引发错误识别，影响病情诊断，而且对形态描述缺乏客观的定量标准。同时，其诊断水平一定程度上取决于医生的经验。因此，借助医学图像处理技术客观定量的提取和分析骨髓细胞，开发一个自动细胞分类及分析系统对提高白血病诊断的整体水平具有重要意义。Leukemia is a malignant tumor originating from the hematopoietic system, characterized by the production and accumulation of a large number of immature and abnormal white blood cells in the bone marrow, which infiltrate other organs, leading to the suppression and failure of normal hematopoietic function. According to the degree of differentiation of leukemia cells and the length of the natural course, it can be divided into two categories: acute and chronic. The distribution of leukemia types in my country is more acute than chronic, and myeloid leukemia is more common, and its incidence rate ranks sixth among various tumors. Patients with acute leukemia (AL) are often accompanied by clinical symptoms such as anemia, hemorrhage, fever, infection, and infiltration. If no timely and effective special treatment is given, the average survival period is only about three months, and the short period may even die a few days after diagnosis. Threat to the patient's health and daily life. The accuracy of the diagnosis and typing of acute leukemia is directly related to the choice of treatment plan and prognosis. Cytomorphology is the most widely used, most direct and economical important diagnostic method in the diagnosis of acute leukemia, and is an important part of morphological, immunological, cytogenetic, molecular biology (MICM) typing diagnosis . The morphological method is mainly to perform Wright-Giemsa staining analysis on the patient's bone marrow smear and blood smear, and further perform other cytochemical staining, and determine the type of acute leukemia according to the FAB (French, American, British) standard. In actual operation, this method still adopts manual operation method, which has a large inspection workload and poor repeatability. It is not only time-consuming and labor-intensive, but the continuous work of doctors is likely to cause misidentification due to fatigue or carelessness, which affects the diagnosis of the disease, and lacks objective description of the morphology. quantitative standard. At the same time, its diagnostic level depends to a certain extent on the doctor's experience. Therefore, it is of great significance to improve the overall level of leukemia diagnosis by using medical image processing technology to objectively and quantitatively extract and analyze bone marrow cells, and to develop an automatic cell classification and analysis system.

依靠图像处理和模式识别技术，对白细胞进行形态学分析及识别的研究比较多，但目前在市场上，尚没有自动化血细胞形态学分析与识别仪器应用于临床试验。这是由于目前存在的白细胞分割或识别算法存在很多的不足，诸如不能很好的解决复杂的白细胞粘连问题，白细胞分割和识别精度低或算法鲁棒性尚不能满意等。Relying on image processing and pattern recognition technology, there are many studies on the morphological analysis and identification of white blood cells, but currently in the market, there is no automated blood cell morphological analysis and identification equipment used in clinical trials. This is because there are many deficiencies in the current white blood cell segmentation or identification algorithms, such as the inability to solve complex white blood cell adhesion problems, low accuracy of white blood cell segmentation and identification, or unsatisfactory algorithm robustness, etc.

由于涂片制备、染色条件、图像获取设备等的差异，不同来源得到的细胞图像通常比较复杂；目标和背景的颜色、纹理等特征常常变化、特征缺失和特征混淆等情形普遍。现有研究中，几乎所有白细胞自动分割算法均假设图像采样、细胞染色条件良好，图像一致性能够得到保障；但是这样的假设在实际中并不能很好保证，相关算法存在很大的局限性，尚不能满足临床医学检验的实际应用需求。同时，骨髓中包含不同生长阶段的各种细胞，细胞的形态、纹理和着色情况等随种类及核、浆的生长或病变程度而变化；胞浆颜色会受到背景光照的很大影响，有时同背景非常相似，但有时呈现颗粒性，细胞有单核与多核之分，同一个多核细胞的核区有时是相互分离的等等。骨髓涂片中还常常伴随各种细胞团聚现象，细胞间相互重叠、粘连。这些问题都成为了骨髓细胞分析中的主要难题。Due to differences in smear preparation, staining conditions, and image acquisition equipment, cell images obtained from different sources are usually complex; features such as color and texture of the target and background often change, feature loss, and feature confusion are common. In existing studies, almost all white blood cell automatic segmentation algorithms assume that image sampling and cell staining conditions are good, and image consistency can be guaranteed; however, such assumptions cannot be well guaranteed in practice, and related algorithms have great limitations. Still can't satisfy the actual application demand of clinical medical examination. At the same time, bone marrow contains various cells in different growth stages, and the shape, texture, and coloring of the cells vary with the type and the growth or lesion degree of the nucleus and plasma; the color of the cytoplasm is greatly affected by the background light, sometimes at the same time. The background is very similar, but sometimes granular, cells can be divided into mononucleated and multinucleated, and the nuclear regions of the same multinucleated cell are sometimes separated from each other, etc. Bone marrow smears are often accompanied by various cell aggregates, overlapping and adhesion between cells. These issues have become the main problems in the analysis of bone marrow cells.

常用的白细胞识别方法主要涉及特征提取和分类器的选择。首先，白细胞的特征提取主要是依据检验科医生的识别经验，对识别经验进行描述，将其转化为一种适合计算机分析的描述子，然而有些描述子并不能很好的表达医生的识别经验；即使可以表达，但是由于白细胞之间的相似性也不能有效地对白细胞进行分类。同时，血涂片染色偏深或者偏浅都会一定程度上影响分类结果。这些因素都要求特征提取方法和分类方法有更强的适应性和表达能力。其次，现阶段这些识别方法主要针对五类成熟白细胞的分类，训练和测试样本量都很小，实验中效果比较理想；但是，在临床应用中，骨髓细胞除了这五类成熟白细胞外，还有很多其他类别的细胞；即使是白细胞也含有不成熟阶段各种类型的细胞。仅仅对五类成熟白细胞进行分类对于急性白血病诊断是远远不够的。Commonly used leukocyte identification methods mainly involve feature extraction and classifier selection. First of all, the feature extraction of leukocytes is mainly based on the recognition experience of laboratory doctors, describes the recognition experience, and converts it into a descriptor suitable for computer analysis. However, some descriptors cannot express the doctor's recognition experience well; Even if expressed, leukocytes cannot be efficiently classified due to the similarity between leukocytes. At the same time, darker or lighter blood smear staining will affect the classification results to a certain extent. These factors require feature extraction methods and classification methods to have stronger adaptability and expressive ability. Secondly, at this stage, these recognition methods are mainly aimed at the classification of the five types of mature white blood cells. The training and test samples are very small, and the effect in the experiment is relatively ideal; however, in clinical applications, bone marrow cells have Many other types of cells; even white blood cells contain various types of cells in immature stages. It is far from enough to classify the five types of mature white blood cells for the diagnosis of acute leukemia.

发明内容Contents of the invention

为了克服现有技术的上述缺点与不足，本发明的目的在于提供一种骨髓细胞自动分类方法，能够自动检测、分割骨髓中不同阶段的细胞并对这些细胞进行分类，而且算法鲁棒，准确有效，适应于各种复杂细胞图片，满足实际应用需求。In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, the object of the present invention is to provide an automatic classification method for bone marrow cells, which can automatically detect and segment cells of different stages in the bone marrow and classify these cells, and the algorithm is robust, accurate and effective , adapted to a variety of complex cell images, to meet the needs of practical applications.

本发明的目的通过以下技术方案实现：The object of the present invention is achieved through the following technical solutions:

一种骨髓细胞自动分类方法，包括以下步骤：A method for automatically classifying bone marrow cells, comprising the following steps:

(1)基于饱和度的骨髓细胞预检测：将原始图像由RGB彩色空间转为HSV 空间，利用otus阈值对S通道进行二值化，同时，根据骨髓细胞饱和度先验知识，当阈值小于区间[70，75]的任意一值时，将其设为区域[80，87]中的一值，否则保持不变，得到骨髓细胞预检测位置的二值化图像；(1) Pre-detection of bone marrow cells based on saturation: convert the original image from RGB color space to HSV space, and use the otus threshold to binarize the S channel. At the same time, according to the prior knowledge of bone marrow cell saturation, when the threshold is less than the interval [70, 75], set it as a value in the region [80, 87], otherwise keep it unchanged, and obtain the binarized image of the pre-detected position of bone marrow cells;

(2)基于稀疏表示的骨髓细胞检测：(2) Bone marrow cell detection based on sparse representation:

(2-1)超像素分割：用SLIC算法将步骤(1)得到的二值化图像进行超像素分割；用SLIC算法对原始图像进行超像素分割；(2-1) Superpixel segmentation: use SLIC algorithm to carry out superpixel segmentation to the binarized image obtained in step (1); use SLIC algorithm to carry out superpixel segmentation to original image;

(2-2)超像素特征提取：用一个向量v来表示原始图像中的每个超像素，即 v＝{F1，F2，F3，F4}，其中，F1是每个超像素的平均亮度、F2是每个超像素在洋红色至绿色色彩空间的均值、F3是每个超像素在黄色至蓝色色彩空间的均值和F4 代表每个超像素在S通道上的饱和度分布；(2-2) Superpixel feature extraction: use a vector v to represent each superpixel in the original image, that is, v={F1, F2, F3, F4}, where F1 is the average brightness of each superpixel, F2 is the mean value of each superpixel in the magenta to green color space, F3 is the mean value of each superpixel in the yellow to blue color space and F4 represents the saturation distribution of each superpixel on the S channel;

(2-3)构建背景字典：(2-3) Build a background dictionary:

(2-3-1)备选背景区域选择：遍历(2-1)得到的二值化图像超像素，找到当前超像素平均亮度为0且邻接超像素平均亮度也为0的超像素点，作为备选背景超像素；(2-3-1) Alternative background area selection: Traverse the superpixels of the binarized image obtained in (2-1), find the superpixels whose average brightness of the current superpixel is 0 and the average brightness of adjacent superpixels is also 0, as an alternative background superpixel;

(2-3-2)背景区域选择：根据(2-3-1)得到的备选背景超像素点的坐标得到(2-1)中原始图像对应的超像素点，当备选背景超像素区域覆盖原始图像中对应超像素区域的一半以上，则设原始图像对应的超像素点为背景区域，否则视为前景区域；以最终背景超像素的特征作为列向量组合成背景字典矩阵D，即 D＝[v₁，v₂，...，v_m]，其中m为背景超像素个数；(2-3-2) Background area selection: According to the coordinates of the candidate background superpixel obtained in (2-3-1), the superpixel corresponding to the original image in (2-1) is obtained, when the candidate background superpixel If the area covers more than half of the corresponding superpixel area in the original image, the superpixel point corresponding to the original image is set as the background area, otherwise it is regarded as the foreground area; the characteristics of the final background superpixel are used as column vectors to form a background dictionary matrix D, namely D=[v ₁ , v ₂ ,...,v _m ], where m is the number of background superpixels;

(2-4)骨髓细胞检测：(2-4) Bone marrow cell detection:

(2-4-1)按照稀疏分解公式，计算原始图像中每个超像素在背景字典下的稀疏系数，如下式所示：(2-4-1) According to the sparse decomposition formula, calculate the sparse coefficient of each superpixel in the original image under the background dictionary, as shown in the following formula:

其中b_j是求得的稀疏系数，j∈[1，2，...，n]，n为原始图像中超像素个数；λ为正则系数；Where b _j is the obtained sparse coefficient, j ∈ [1, 2, ..., n], n is the number of superpixels in the original image; λ is the regular coefficient;

(2-4-2)利用求得的稀疏系数对原超像素v_j进行重建，得到稀疏重建后的残差ε_j为，如下式：(2-4-2) Use the obtained sparse coefficients to reconstruct the original superpixel v _j , and obtain the residual ε _j after sparse reconstruction as follows:

(2-4-3)将计算得到的残差作为原始图像超像素的显著度，得到骨髓细胞的检测图image_huidu；(2-4-3) using the calculated residual as the saliency of the original image superpixels to obtain the detection map image _huidu of bone marrow cells;

(3)基于形态学的多角度骨髓细胞分割和计数：(3) Multi-angle bone marrow cell segmentation and counting based on morphology:

(3-1)分类简单细胞图像和复杂细胞图像：利用otus阈值对(2-4-3)得到的检测图image_huidu进行二值化，得到图像image_erzhi；当骨髓细胞的面积占总图像面积的40％～45％以上，即为复杂细胞图像；否则为简单细胞图像；(3-1) Classify simple cell images and complex cell images: Use the otus threshold to binarize the detection image image _huidu obtained in (2-4-3) to obtain the image image _erzhi ; when the area of bone marrow cells accounts for the total image area More than 40% to 45% of , it is a complex cell image; otherwise, it is a simple cell image;

(3-2)细胞全局分割：(3-2) Cell global segmentation:

(3-2-1)遍历(3-1)中二值图像image_erzhi的连通区域，找到每个连通区域的最大最小坐标，以此为矩形框图的坐标，对原始图像、(2-4-3)得到的检测图 image_huidu和(3-1)得到的二值图像image_erzhi分别进行裁剪，从而得到骨髓细胞对应的分割图crop1_image、分割灰度图crop1_huidu和分割二值图crop1_erzhi；(3-2-1) Traverse the connected regions of the binary image image _erzhi in (3-1), find the maximum and minimum coordinates of each connected region, and use this as the coordinates of the rectangular frame, for the original image, (2-4- 3) The obtained detection image image _huidu and the binary image image _erzhi obtained in (3-1) are respectively cropped to obtain the segmentation image crop1 _image corresponding to bone marrow cells, the segmentation grayscale image crop1 _huidu and the segmentation binary image crop1 _erzhi ;

(3-2-2)剔除(3-2-1)分割图中的不完整细胞和由于涂片制备、染色条件和手工操作导致的干扰背景；(3-2-2) Eliminate the incomplete cells in the (3-2-1) segmentation map and the interference background caused by smear preparation, staining conditions and manual operations;

定义area_crop1为分割图crop1_image的面积；ratio0_crop1为分割图 crop1_image宽高比，ratio0_crop1∈(0，1]，ratio1_crop1为分割图crop1_image饱和度在区间[102，255]的占比；Define area _crop1 as the area of the crop1 _image ; ratio0 _crop1 is the aspect ratio of the crop1 _image , ratio0 _crop1 ∈ (0, 1], ratio1 _crop1 is the proportion of the saturation of the crop1 _image in the interval [102, 255] ;

1)分割图crop1_image位于原始图像边界时，其满足area_crop1∈ (1000，3000]且ratio0_crop1，ratio1_crop1≥0.45，或者area_crop1∈(3000，∞) 且ratio1_crop1≥0.45则保留；2)分割图crop1_image位于原始图像非边界时，其满足ratio1_crop1≥0.45则保留；1) When the segmentation image crop1 _image is located at the boundary of the original image, it satisfies area _crop1 ∈ (1000, 3000] and ratio0 _crop1 , ratio1 _crop1 ≥ 0.45, or area _crop1 ∈ (3000, ∞) and ratio1 _crop1 ≥ 0.45 is reserved; 2) When the segmentation map crop1 _image is located on the non-boundary of the original image, if it satisfies ratio1 _crop1 ≥ 0.45, it will be retained;

(3-3)细胞局部再分割：(3-3) Partial subdivision of cells:

(3-3-1)对(3-2-2)保留下来的分割灰度图crop1_huidu进行阈值区间为[6，10]的二值化处理，经过边长区间为[2，4]的正方形结构体形态学变换后，定义 round_rate_crop1为细胞曲圆率；当分割图crop1_image满足area_crop1＞35000，或者area_crop1∈(17000，35000]且round_rate_crop1＜0.46时，则判定为待分割的多细胞区域crop1_multi，否则为单细胞区域crop1_single；(3-3-1) Perform binarization processing on the segmented grayscale image crop1 _huidu retained in (3-2-2) with a threshold value interval of [6, 10], and after the edge length interval is [2, 4] After the morphological transformation of the square structure, define round_rate _crop1 as the cell curvature; when the segmentation image crop1 _image satisfies area _crop1 > 35000, or area _crop1 ∈ (17000, 35000] and round_rate _crop1 < 0.46, it is determined to be segmented Multi-cell region crop1 _multi , otherwise single-cell region crop1 _single ;

(3-3-2)利用otus阈值对(3-3-1)中待分割的多细胞区域crop1_multi对应的分割灰度图crop1_huidu进行二值化，当阈值大于区间[112，117]之间的值时，设阈值为[220，240]中的值；图像为简单图像时，设圆形结构体的半径为1；图像为复杂图像时，设圆形结构体的半径为3；经过形态学变换，遍历每个面积大于 1200～1500的连通区域，找到每个连通区域的最大最小坐标，并以此为矩形框图的坐标，对多细胞分割图、多细胞灰度图和多细胞二值图分别进行裁剪，从而得到对应的分割图crop2_image、分割灰度图crop2_huidu和分割二值图crop2_erzhi；(3-3-2) Use the otus threshold to binarize the segmented grayscale image crop1 _huidu corresponding to the multicellular region crop1 _multi to be segmented in (3-3-1), when the threshold is greater than the interval [112, 117] When the value between , the threshold is set to the value in [220, 240]; when the image is a simple image, the radius of the circular structure is set to 1; when the image is a complex image, the radius of the circular structure is set to 3; Morphological transformation, traversing each connected region with an area greater than 1200-1500, finding the maximum and minimum coordinates of each connected region, and using this as the coordinates of the rectangular frame diagram, multi-cell segmentation map, multi-cell grayscale map and multi-cell binary The value map is cropped separately to obtain the corresponding segmentation image crop2 _image , segmented grayscale image crop2 _huidu and segmented binary image crop2 _erzhi ;

(3-3-3)当(3-3-2)分割图crop2_image位于对应的(3-3-1)多细胞区域 crop1_multi的边界时，分割图crop2_image面积低于区间[14000，16000]的值且宽高比低于区间[0.5，0.55]的值时剔除，否则保留该分割图crop2_image；(3-3-3) When the (3-3-2) segmentation image crop2 _image is located at the boundary of the corresponding (3-3-1) multicellular region crop1 _multi , the area of the segmentation image crop2 _image is lower than the interval [14000, 16000 ] and the aspect ratio is lower than the value of the interval [0.5, 0.55], otherwise keep the segmentation map crop2 _image ;

(3-4)细胞S通道再分割：(3-4) Cell S channel subdivision:

(3-4-1)对(3-3-3)保留的分割图对应的灰度图crop2_huidu进行阈值区间为 [6，10]的二值化处理，经过边长区间为[2，4]的正方形结构体形态学变换后，定义 area_crop2是分割图crop2_image的面积；round_rate_crop2为细胞曲圆率； ratio0_crop2为细胞所占面积比；(3-4-1) Perform binarization processing with a threshold interval of [6, 10] on the grayscale image crop2 _huidu corresponding to the segmented image retained in (3-3-3), after the side length interval is [2, 4 ] After the morphological transformation of the square structure, define area _crop2 as the area of the crop2 _image ; round_rate _crop2 is the cell curvature; ratio0 _crop2 is the area ratio of the cell;

1)area_crop2＞27500；2)area_crop2∈(19000，27500]且 round_rate_crop2＜0.56，或者area_crop2∈(19000，27500]且 round_rate_crop2≥0.56但ratio0_crop2＜0.5，则判定为待分割的多细胞区域 crop2_multi，否则为单细胞区域crop2_single；并将(3-3-2)中单细胞区域的矩形框图扩大1.1～1.2倍重新对crop1_multi裁剪得到crop2_new_single；1) area _crop2 >27500; 2) area _crop2 ∈ (19000, 27500] and round_rate _crop2 < 0.56, or area _crop2 ∈ (19000, 27500] and round_rate _crop2 ≥ 0.56 but ratio0 _crop2 < 0.5, then it is determined that there are many crops to be divided The cell area crop2 _multi , otherwise it is the single-cell area crop2 _single ; and the rectangular frame diagram of the single-cell area in (3-3-2) is enlarged by 1.1 to 1.2 times to re-crop crop1 _multi to obtain crop2_new _single ;

(3-4-2)提取(3-4-1)的多细胞区域crop2_multi的S通道图像，并进行再分割，得到分割图crop3_image、分割灰度图crop3_huidu和分割二值图crop3_erzhi；(3-4-2) Extract the S-channel image of the multi-cellular area crop2 _multi in (3-4-1), and then segment it again to obtain the segmented image crop3 _image , the segmented grayscale image crop3 _huidu and the segmented binary image crop3 _erzhi ;

(3-5)细胞H通道再分割：(3-5) Cell H channel subdivision:

当图像为简单图像时，将(3-4-2)中的矩形框图扩大1.2～1.3倍，重新对crop2_multi裁剪得到单细胞区域crop3_new_image；When the image is a simple image, expand the rectangular frame in (3-4-2) by 1.2 to 1.3 times, and re-crop the crop2 _multi to obtain the single-cell region crop3_new _image ;

当图像为复杂图像时，进行细胞H通道再分割：When the image is a complex image, the cell H channel is re-segmented:

(3-5-1)剔除(3-4-2)分割图中不完整细胞：当(3-4-2)分割图crop3_image位于 (3-4-1)待分割多细胞区域crop2_multi的边界时，分割图crop3_image面积低于区间 [14000，16000]的值且宽高比低于区间[0.5，0.55]的值时剔除，否则保留该分割图 crop3_image；(3-5-1) Eliminate incomplete cells in the (3-4-2) segmentation map: when the (3-4-2) segmentation map crop3 _image is located in the (3-4-1) multi-cell region crop2 _multi to be segmented When bordering, if the crop3 _image area of the segmentation map is lower than the value of the interval [14000, 16000] and the aspect ratio is lower than the value of the interval [0.5, 0.55], it will be removed, otherwise the segmentation image crop3 _image will be kept;

(3-5-2)对(3-5-1)保留下来的分割图对应的灰度图crop3_huidu进行阈值区间为[6，10]的二值化处理，经过边长区间为[2，4]的正方形结构体形态学变换后，定义area_crop3是分割图crop3_image的面积；round_rate_crop3为细胞曲圆率； ratio0_crop3为细胞所占面积比；(3-5-2) Perform binarization processing with a threshold interval of [6, 10] on the grayscale image crop3 _huidu corresponding to the segmented image retained in (3-5-1), after the side length interval is [2, 4] After the morphological transformation of the square structure, define area _crop3 as the area of the crop3 _image ; round_rate _crop3 is the cell curvature; ratio0 _crop3 is the area ratio of the cell;

1)area_crop3＞27500；2)area_crop3∈(19000，27500]且 round_rate_crop3＜0.56，或者area_crop3∈(19000，27500]且round_rate_crop3≥ 0.56但ratio0_crop3＜0.5，则判定为待分割的多细胞区域crop3_multi，否则为单细胞区域crop3_single，并将(3-4-2)中的矩形框图扩大1.2～1.3倍，重新对crop2_multi裁剪得到crop3_new_single；1) area _crop3 >27500; 2) area _crop3 ∈ (19000, 27500] and round_rate _crop3 < 0.56, or area _crop3 ∈ (19000, 27500] and round_rate _crop3 ≥ 0.56 but ratio0 _crop3 < 0.5, then it is determined that there are many crops to be divided The cell area crop3 _multi , otherwise it is the single-cell area crop3 _single , and the rectangular frame in (3-4-2) is enlarged by 1.2 to 1.3 times, and crop2 _multi is re-cropped to obtain crop3_new _single ;

(3-5-3)提取(3-5-2)多细胞区域crop3_multi的H通道图像，并进行再分割，得到分割图crop4_image、分割灰度图crop4_huidu和分割二值图crop4_erzhi；(3-5-3) extracting (3-5-2) the H-channel image of crop3 _multi in the multicellular region, and performing re-segmentation to obtain the segmented image crop4 _image , the segmented grayscale image crop4 _huidu and the segmented binary image crop4 _erzhi ;

(3-5-4)剔除(3-5-3)分割图crop4_image中不完整细胞；(3-5-4) Remove incomplete cells in the (3-5-3) segmentation image crop4 _image ;

(3-6)汇总原始图像上分割出来的染色单细胞：图像为简单图像时，汇总(3-3-1)单细胞区域crop1_single、(3-4-1)单细胞区域crop2_new_single和(3-5)单细胞区域crop3_new_image，作为原始图像上待分类的染色单细胞；图像为复杂图像时，汇总(3-3-1)单细胞区域crop1_single、(3-4-1)单细胞区域crop2_new_single、 (3-5-2)单细胞区域crop3_new_single和(3-5-4)单细胞区域crop4_new_image，作为原始图像上待分类的染色单细胞；(3-6) Summarize the stained single cells segmented on the original image: when the image is a simple image, summarize (3-3-1) single cell region crop1 _single , (3-4-1) single cell region crop2_new _single and ( 3-5) single cell area crop3_new _image , as the dyed single cell to be classified on the original image; when the image is a complex image, summarize (3-3-1) single cell area crop1 _single , (3-4-1) single cell Region crop2_new _single , (3-5-2) single cell region crop3_new _single and (3-5-4) single cell region crop4_new _image , as the stained single cell to be classified on the original image;

(4)基于深度学习的骨髓细胞分类：(4) Bone marrow cell classification based on deep learning:

(4-1)三路并行网络结构设计，网络结构主要包括四个部分：数据预处理模块、RGB彩色空间模块CNN-RGB、HSV彩色空间模块CNN-HSV和领域学知识细胞核模块CNN-NUCLEUS；(4-1) Three-way parallel network structure design, the network structure mainly includes four parts: data preprocessing module, RGB color space module CNN-RGB, HSV color space module CNN-HSV and domain knowledge cell nucleus module CNN-NUCLEUS;

其中，CNN-RGB、CNN-HSV和CNN-NUCLEUS网络均为单路卷积神经网络，使用Alex网络的前六层为单路基准网络；CNN-RGB、CNN-HSV和 CNN-NUCLEUS网络通过一个全连接层实现并行；Among them, CNN-RGB, CNN-HSV, and CNN-NUCLEUS networks are all single-channel convolutional neural networks, and the first six layers of the Alex network are single-channel benchmark networks; CNN-RGB, CNN-HSV, and CNN-NUCLEUS networks pass a The fully connected layer realizes parallelism;

(4-2)输入预处理与数据集平衡化处理：(4-2) Input preprocessing and data set balance processing:

(4-2-1)输入预处理：(4-2-1) Input preprocessing:

首先对包含单一骨髓细胞的带分类标签的原始数据图库根据需要进行分类，作为原始细胞数据集；First, classify the original data library with classification labels containing single bone marrow cells as required, and use it as the original cell data set;

原始细胞数据集作为网络CNN-RGB的输入，对原始细胞数据集的图像进行彩色空间变换，转换为HSV，作为网络CNN-HSV的输入；同时利用细胞染色的饱和度差异提取细胞的细胞核，作为网络CNN-NUCLEUS的输入；The original cell data set is used as the input of the network CNN-RGB, and the image of the original cell data set is transformed into color space and converted to HSV, which is used as the input of the network CNN-HSV; at the same time, the cell nucleus is extracted by using the saturation difference of the cell staining as The input of the network CNN-NUCLEUS;

(4-2-2)细胞图像数据集平衡化处理：(4-2-2) Cell image dataset balance processing:

通过对图像旋转和镜像方式，对原始细胞数据集中所有类别图像数据进行增加样本处理，以满足深度网络对数据量的需求；并对图像数量少的某些类别进行更多的图像旋转操作增加样本，达到与多数类样本数量平衡；Through image rotation and mirroring, increase sample processing for all categories of image data in the original cell data set to meet the data volume requirements of the deep network; and perform more image rotation operations for certain categories with a small number of images to increase samples , to achieve a balance with the number of samples in the majority class;

(4-3)三路并行网络模型训练：(4-3) Three-way parallel network model training:

网络模型中的并行结构使用Alex网络中的参数作为初始化权值；使用(4-2-2) 处理后的细胞图像数据集对深度卷积神经网络模型进行有监督的训练The parallel structure in the network model uses the parameters in the Alex network as initialization weights; use the (4-2-2) processed cell image dataset for supervised training of the deep convolutional neural network model

(4-4)对(3-6)待分类的染色单细胞使用已训练好的分类模型进行分类。(4-4) Use the trained classification model to classify the stained single cells to be classified in (3-6).

步骤(3-2-2)中，area_crop1、ratio0_crop1、ratio1_crop1、sign_crop1(i，j)的计算具体如下：In step (3-2-2), the calculation of area _crop1 , ratio0 _crop1 , ratio1 _crop1 , sign _crop1 (i, j) is as follows:

area_crop1＝width_crop1*height_crop1 area _crop1 = width _crop1 * height _crop1

ratio0_crop1＝min(width_crop1，height_crop1)/max(width_crop1，height_crop1)ratio0 _crop1 = min(width _crop1 , height _crop1 )/max(width _crop1 , height _crop1 )

其中，width_crop1，height_crop1分别为分割图crop1_image的宽度和高度，S_crop1是分割图crop1_image提取其S通道的图像， i∈[1，2，...，width_crop1]，j∈[1，2，...，height_crop1]。Among them, width _crop1 and height _crop1 are the width and height of crop1 _image respectively, S _crop1 is the image of S channel extracted from crop1 _image , i∈[1, 2,..., width _crop1 ], j∈[ 1, 2, ..., height _crop1 ].

步骤(3-3-1)中，round_rate_crop1的计算如下：In step (3-3-1), the round_rate _crop1 is calculated as follows:

round_rate_crop1＝4*π*S_round_crop1/C-round_crop1 ² round_rate _crop1 = 4*π*S_round _crop1 /C-round _crop1 ²

其中，S_round_crop1为分割图crop1_image中细胞所占面积，C_round_crop1为分割图crop1_image中细胞的周长。Among them, S_round _crop1 is the area occupied by the cells in the crop1 _image , and C_round _crop1 is the perimeter of the cells in the crop1 _image .

步骤(3-4-1)中，area_crop2、round_rate_crop2、ratio0_crop2的计算如下：In step (3-4-1), area _crop2 , round_rate _crop2 , ratio0 _crop2 are calculated as follows:

area_crop2＝width_crop2*height_crop2 area _crop2 = width _crop2 * height _crop2

round_rate_crop2＝4*π*S_round_crop2/C_round_crop2 ² round_rate _crop2 = 4*π*S_round _crop2 /C_round _crop2 ²

ratio0_crop2＝S_round_crop2/area_crop2 ratio0 _crop2 = S_round _crop2 /area _crop2

其中，width_crop2，height_crop2分别为分割图crop2_image的宽度和高度， S_round_crop2为分割图crop2_image中细胞所占面积，C_round_crop2为分割图 crop2_image中细胞周长。Among them, width _crop2 and height _crop2 are the width and height of the crop2 _image , respectively, S_round _crop2 is the area occupied by the cells in the crop2 _image , and C_round _crop2 is the perimeter of the cells in the crop2 _image .

步骤(3-4-2)所述提取(3-4-1)的多细胞区域crop2_multi的S通道图像，并进行再分割，得到分割图crop3_image、分割灰度图crop3_huidu和分割二值图 crop3_erzhi，具体为：Step (3-4-2) extracts the S-channel image of the multi-cellular region crop2 _multi in (3-4-1), and performs re-segmentation to obtain the segmented image crop3 _image , the segmented grayscale image crop3 _huidu , and the segmented binary value Figure crop3 _erzhi , specifically:

细胞为简单图像时，设阈值为区间[125，130]中的一值对S通道进行二值化；经过半径为1的圆形结构体形态学变换二次，遍历每个面积大于1200～1500的连通区域，找到每个连通区域的最大最小坐标，并以此作为矩形框图的坐标，对多细胞分割图crop2_multi、多细胞灰度图和多细胞二值图分别进行裁剪，从而得到对应的分割图crop3_image、分割灰度图crop3_huidu和分割二值图crop3_erzhi；When the cell is a simple image, set the threshold value to a value in the interval [125, 130] to binarize the S channel; after two morphological transformations of a circular structure with a radius of 1, traverse each area greater than 1200 to 1500 Connected areas, find the maximum and minimum coordinates of each connected area, and use them as the coordinates of the rectangular frame to crop the multi-cell segmentation map crop2 _multi , multi-cell grayscale image and multi-cell binary image respectively, so as to obtain the corresponding Segmentation image crop3 _image , segmentation grayscale image crop3 _huidu and segmentation binary image crop3 _erzhi ;

细胞为复杂图像时，阈值设置如下式：When the cell is a complex image, the threshold is set as follows:

其中，t_crop2是多细胞区域S通道饱和度的众数，t1_crop2是多细胞区域S通道饱和度的otus阈值；利用阈值th_crop2对多细胞区域S通道二值化；经过半径为3的圆形结构体形态学变换后，遍历每个面积大于1200～1500的连通区域，找到每个连通区域的最大最小坐标，并以此作为矩形框图的坐标，对多细胞分割图、多细胞灰度图和多细胞二值图分别进行裁剪，从而得到对应的分割图 crop3_image、分割灰度图crop3_huidu和分割二值图crop3_erzhi。Among them, t _crop2 is the mode of S channel saturation in the multicellular area, and t1 _crop2 is the otus threshold value of S channel saturation in the multicellular area; use the threshold th _crop2 to binarize the S channel in the multicellular area; pass through a circle with a radius of 3 After the morphological transformation of the shape structure, traverse each connected region with an area greater than 1200-1500, find the maximum and minimum coordinates of each connected region, and use this as the coordinate of the rectangular frame diagram, for multi-cell segmentation map, multi-cell grayscale map and the multi-cell binary image are cropped separately to obtain the corresponding segmentation image crop3 _image , segmented grayscale image crop3 _huidu and segmented binary image crop3 _erzhi .

步骤(3-5-2)中，area_crop3、round_rate_crop3、ratio0_crop3计算具体如下：In step (3-5-2), area _crop3 , round_rate _crop3 , ratio0 _crop3 are calculated as follows:

area_crop3＝width_crop3*height_crop3 area _crop3 = width _crop3 * height _crop3

round_rate_crop3＝4*π*S_round_crop3/C_round_crop3 ² round_rate _crop3 = 4*π*S_round _crop3 /C_round _crop3 ²

ratio0_crop3＝S_round_crop3/area_crop3 ratio0 _crop3 = S_round _crop3 /area _crop3

其中，width_crop3，height_crop3分别为分割图crop3_image的宽度和高度， S_round_crop3为分割图crop3_image中细胞所占面积，C_round_crop3为分割图 crop3_image中细胞周长。Among them, width _crop3 and height _crop3 are the width and height of the crop3 _image respectively, S_round _crop3 is the area occupied by the cells in the crop3 _image , and C_round _crop3 is the perimeter of the cells in the crop3 _image .

步骤(3-5-3)所述提取(3-5-2)多细胞区域crop3_multi的H通道图像，并进行再分割，得到分割图crop4_image、分割灰度图crop4_huidu和分割二值图crop4_erzhi，具体为：Step (3-5-3) extracts (3-5-2) the H-channel image of the multicellular region crop3 _multi , and performs subdivision to obtain the segmentation image crop4 _image , segmentation grayscale image crop4 _huidu , and segmentation binary image crop4 _erzhi , specifically:

阈值设置如下式：The threshold is set as follows:

其中，阈值t_crop3是多细胞区域H通道饱和度的众数，利用阈值th_crop3对多细胞区域H通道二值化；经过半径为1的圆形结构体形态学变换后，再经过半径为3的圆形结构体腐蚀，半径为1的圆形结构体膨胀和半径为3的圆形结构体腐蚀后，遍历每个面积大于1200～1500的连通区域，找到每个连通区域的最大最小坐标，并以此作为矩形框图的坐标对多细胞分割图、多细胞灰度图和多细胞二值图分别进行裁剪，从而得到对应的分割图crop4_image、分割灰度图 crop4_huidu和分割二值图crop4_erzhi。Among them, the threshold t _crop3 is the mode of the saturation of the H channel in the multicellular area, and the threshold th _crop3 is used to binarize the H channel in the multicellular area; After the circular structure corrosion of the circular structure with a radius of 1 is expanded and the circular structure with a radius of 3 is corroded, each connected area with an area greater than 1200-1500 is traversed to find the maximum and minimum coordinates of each connected area. And use this as the coordinates of the rectangular frame to crop the multi-cell segmentation map, multi-cell grayscale image and multi-cell binary image respectively, so as to obtain the corresponding segmentation image crop4 _image , segmented grayscale image crop4 _huidu and segmented binary image crop4 _erzhi .

步骤(3-5-4)所述剔除(3-5-3)分割图中不完整细胞，具体为：Step (3-5-4) described in removing (3-5-3) incomplete cells in the segmentation map, specifically:

定义area_crop4为分割图crop4_image的面积；ratio0_crop4为分割图crop4_image宽高比，ratio0_crop4∈(0，1]；ratio1_crop4为分割图crop4_image饱和度在区间 [102，255]的占比；Define area _crop4 as the area of the crop4 _image ; ratio0 _crop4 is the aspect ratio of the crop4 _image , ratio0 _crop4 ∈ (0, 1]; ratio1 _crop4 is the ratio of the saturation of the crop4 _image in the interval [102, 255] ;

1)分割图crop4_image位于待分割多细胞区域crop3_multi边界时，其满足 area_crop4∈(1000，3000]，ratio0_crop4≥0.5且ratio1_crop4＞0.45，或者 area_crop4∈(3000，∞)且ratio1_crop4＞0.45则保留；2)分割图crop4_image位于非边界时，其满足ratio1_crop4＞0.45则保留，并将(3-5-3)中的矩形框图扩大 1.2～1.3倍，重新对crop3_multi裁剪得到单细胞区域crop4_new_image。1) When the segmentation image crop4 _image is located at the crop3 _multi boundary of the multi-cell area to be segmented, it satisfies area _crop4 ∈ (1000, 3000], ratio0 _crop4 ≥ 0.5 and ratio1 _crop4 > 0.45, or area _crop4 ∈ (3000, ∞) and ratio1 _crop4 ＞0.45, keep it; 2) When crop4 _image is located in the non-boundary, it satisfies ratio1 _crop4 ＞0.45, keep it, expand the rectangular frame in (3-5-3) by 1.2-1.3 times, and re-crop crop3 _multi to get Single-cell region crop4_new _image .

area_crop4、ratio0_crop4、ratio1_crop4、sign_crop4(i，j)的计算具体如下：The calculation of area _crop4 , ratio0 _crop4 , ratio1 _crop4 , sign _crop4 (i, j) is as follows:

area_crop4＝width_crop4*height_crop4 area _crop4 = width _crop4 * height _crop4

ratio0_crop4＝min(width_crop4，height_crop4)/max(width_crop4，height_crop4)ratio0 _crop4 = min(width _crop4 , height _crop4 )/max(width _crop4 , height _crop4 )

其中，width_crop4，height_crop4分别为分割图crop4_image的宽度和高度，S_crop4是分割图crop4_image提取其S通道的图像。Among them, width _crop4 and height _crop4 are the width and height of the crop4 _image , respectively, and S _crop4 is the image whose S channel is extracted from the crop4 _image .

步骤(4-2-1)所述对包含单一骨髓细胞的带分类标签的原始数据图库根据需要进行分类，具体为：根据骨髓细胞类别和对于急性白血病诊断重要性、FAB 标准因素分为13大类，分别是中幼红细胞、晚幼红细胞、其他红系细胞、原始细胞、成熟淋巴细胞、其他淋系细胞、单核系细胞、早幼粒细胞、中幼粒细胞、晚幼粒细胞、杆状核细胞、分叶核细胞和其他粒系细胞。In the step (4-2-1), the original data library with classification labels containing a single bone marrow cell is classified according to needs, specifically: according to the type of bone marrow cell and the importance for the diagnosis of acute leukemia, the FAB standard factors are divided into 13 categories Myelocytes, metamyelocytes, other erythroid cells, blasts, mature lymphocytes, other lymphocytes, monocytic cells, promyelocytes, myelocytes, metamyelocytes, rods cells, segmented nuclei, and other granulocytes.

与现有技术相比，本发明具有以下优点和有益效果：Compared with the prior art, the present invention has the following advantages and beneficial effects:

(1)本发明创造性地提出了一种骨髓细胞自动分类方法，整个过程完全实现自动骨髓细胞检测、分割和分类，无需人工参与，克服了现有方法需要手动操作，可重复性差、费时费力、不满足临床应用要求等缺陷。(1) The present invention creatively proposes a method for automatic classification of bone marrow cells. The whole process fully realizes automatic detection, segmentation and classification of bone marrow cells without manual participation. Defects such as not meeting clinical application requirements.

(2)本发明创造性地提出了基于稀疏表示的骨髓细胞检测方法。由于涂片制备、染色条件、图像获取设备等的差异，不同来源得到的细胞图像通常比较复杂；目标和背景的颜色、纹理等特征常常变化、特征缺失和特征混淆等情形普遍。基于稀疏表示的骨髓细胞检测方法可以自动检测不同来源细胞图像的背景，并基于该背景建立稀疏字典，从而很好地消除上述因素导致的差异，从而避免误判。(2) The present invention creatively proposes a bone marrow cell detection method based on sparse representation. Due to differences in smear preparation, staining conditions, and image acquisition equipment, cell images obtained from different sources are usually complex; features such as color and texture of the target and background often change, feature loss, and feature confusion are common. The bone marrow cell detection method based on sparse representation can automatically detect the background of cell images from different sources, and build a sparse dictionary based on the background, so as to eliminate the differences caused by the above factors and avoid misjudgment.

(3)本发明针对不同复杂程度的细胞图像创造性得定制了不同的分割手段。急性白血病细胞图像随个体差异、病情程度等而呈现不同的状态，一般增生极度活跃，病情严重的图像多出现细胞团聚、粘连现象，因此，需要不同对待。(3) The present invention creatively customizes different segmentation means for cell images with different degrees of complexity. Acute leukemia cell images show different states depending on individual differences and disease severity, etc. Generally, hyperplasia is extremely active, and cell aggregation and adhesion are often seen in images with severe disease, so they need to be treated differently.

(4)本发明创造性地全方位利用细胞图像的不同特征进行多角度分割，分别从图像全局和局部、色彩空间和饱和度空间，结合形态学变换对不同细胞图像进行分割与计数，很好地解决了骨髓中不同生长阶段的各种细胞，单核与多核细胞，核区分离细胞以及粘连、重叠细胞的分割。(4) The present invention creatively utilizes the different characteristics of cell images for multi-angle segmentation, respectively segmenting and counting different cell images from the image global and local, color space and saturation space, combined with morphological transformation, which is very good Solve the segmentation of various cells in different growth stages in the bone marrow, mononuclear and multinucleated cells, nuclear region separation cells, and adhesion and overlapping cells.

(5)本发明创造性地将骨髓细胞根据细胞类别和对于急性白血病诊断重要性等因素分为13大类：中幼红细胞、晚幼红细胞、其他红系细胞(主要为原红细胞和早幼红细胞)、原始细胞(包括原始淋巴细胞和原始粒细胞)、成熟淋巴细胞、其他淋系细胞(主要为浆细胞)、单核系细胞、早幼粒细胞、中幼粒细胞、晚幼粒细胞、杆状核细胞、分叶核细胞和其他粒系细胞。该分类基本囊括了骨髓细胞的所有类别；同时，贴合FAB标准对急性白血病的判定。(5) The present invention creatively divides the bone marrow cells into 13 categories according to the cell type and the importance for the diagnosis of acute leukemia: middle and early red blood cells, late red blood cells, and other red blood cells (mainly primary red blood cells and early red blood cells) , primitive cells (including primitive lymphocytes and myeloblasts), mature lymphocytes, other lymphoid cells (mainly plasma cells), monocytic cells, promyelocytes, myelocytes, metamyelocytes, rods cells, segmented nuclei, and other granulocytes. This classification basically includes all types of bone marrow cells; at the same time, it conforms to the FAB standard for the judgment of acute leukemia.

(5)本发明创造性地在深度网络中引入骨髓细胞领域学知识，将细胞细胞核作为一路输入到网络中，提高了分类的准确率。(5) The present invention creatively introduces the field knowledge of bone marrow cells into the deep network, and inputs cell nuclei into the network as a channel, thereby improving the classification accuracy.

附图说明Description of drawings

图1为本发明的实施例的骨髓细胞自动分类方法的工作流程图。Fig. 1 is a working flow chart of the method for automatic classification of bone marrow cells according to the embodiment of the present invention.

图2为本发明的实施例的基于饱和度的骨髓细胞预检测的流程图。Fig. 2 is a flow chart of the pre-detection of bone marrow cells based on saturation in an embodiment of the present invention.

图3为本发明的实施例的基于稀疏表示的骨髓细胞检测的流程图。Fig. 3 is a flowchart of bone marrow cell detection based on sparse representation according to an embodiment of the present invention.

图4为本发明的实施例的基于形态学的多角度骨髓细胞分割和计数的流程图。Fig. 4 is a flowchart of multi-angle bone marrow cell segmentation and counting based on morphology according to an embodiment of the present invention.

图5为本发明的实施例的骨髓细胞分类方法的工作流程图。Fig. 5 is a workflow diagram of a method for classifying bone marrow cells according to an embodiment of the present invention.

具体实施方式Detailed ways

下面结合实施例，对本发明作进一步地详细说明，但本发明的实施方式不限于此。The present invention will be described in further detail below in conjunction with the examples, but the embodiments of the present invention are not limited thereto.

实施例Example

本实施例的骨髓细胞自动分类方法，包括以下步骤：The method for automatic classification of bone marrow cells in this embodiment comprises the following steps:

(1)基于饱和度的骨髓细胞预检测：(1) Pre-detection of bone marrow cells based on saturation:

在RGB彩色空间中，不同光谱也可能产生相同的颜色，因而对于同样的一张骨髓相图片颜色不变，却可能因为操作的不一致导致三个颜色分量产生很大变化。而在HSV空间中，一个物体的色度和饱和度通常只由物体原材料的光线吸收和反射特性决定。因此，在色度和饱和度空间，条件同色不会影响计算机的判断。因而，色度和饱和度对骨髓细胞分割至关重要。In the RGB color space, different spectrums may also produce the same color. Therefore, for the same bone marrow phase image, the color remains unchanged, but the three color components may change greatly due to inconsistencies in operations. In HSV space, the hue and saturation of an object are usually determined only by the light absorption and reflection properties of the object's raw material. Therefore, in the space of hue and saturation, conditional homochromaticity will not affect the judgment of the computer. Thus, hue and saturation are critical for myeloid cell segmentation.

骨髓细胞染色时是细胞核与细胞浆着色，在光照下细胞会呈现出较饱和的颜色，而背景的饱和度总体来说明显低于细胞核，比细胞胞浆略低。色调也会因细胞核和细胞浆着色而与背景出现差异，但是不同成长阶段的细胞着色会出现偏红或偏蓝等不同基色；因此，饱和度对于骨髓细胞的预检测更有效。When the bone marrow cells are stained, the nuclei and cytoplasm are stained. Under the light, the cells will show a saturated color, and the saturation of the background is generally lower than that of the nucleus and slightly lower than that of the cytoplasm. The color tone will also be different from the background due to the staining of the nucleus and cytoplasm, but the staining of cells at different growth stages will have different base colors such as reddish or blue; therefore, saturation is more effective for the pre-detection of bone marrow cells.

具体步骤如图2所示，将原始图像由RGB彩色空间转为HSV空间，利用 otus阈值对S通道进行二值化，同时，根据骨髓细胞饱和度先验知识，当阈值小于区间[70，75]的任意一值时，将其设为区域[80，87]中的一值，否则保持不变，得到骨髓细胞预检测位置的二值化图像；The specific steps are shown in Figure 2. The original image is converted from RGB color space to HSV space, and the S channel is binarized using the otus threshold. At the same time, according to the prior knowledge of bone marrow cell saturation, when the threshold is less than the interval [70, 75 ], set it as a value in the area [80, 87], otherwise keep it unchanged, and obtain the binarized image of the pre-detection position of bone marrow cells;

(2)如图3，基于稀疏表示的骨髓细胞检测：(2) As shown in Figure 3, bone marrow cell detection based on sparse representation:

由于涂片制备、染色条件、图像获取设备等的差异，不同来源得到的细胞图像通常比较复杂；目标和背景的颜色、纹理等特征常常变化、特征缺失和特征混淆等情形普遍。本发明的基于稀疏表示的骨髓细胞检测方法，它可以根据骨髓细胞预检测位置自动找到背景区域，并根据背景区域高效而准确地检测到骨髓细胞，具体步骤如下：Due to differences in smear preparation, staining conditions, and image acquisition equipment, cell images obtained from different sources are usually complex; features such as color and texture of the target and background often change, feature loss, and feature confusion are common. The bone marrow cell detection method based on the sparse representation of the present invention can automatically find the background area according to the pre-detection position of the bone marrow cells, and efficiently and accurately detect the bone marrow cells according to the background area. The specific steps are as follows:

(2-1)超像素分割：用SLIC算法将步骤(1)得到的二值化图像进行超像素分割；用SLIC算法对原始图像进行超像素分割；(2-1) Superpixel segmentation: use SLIC algorithm to carry out superpixel segmentation to the binarized image obtained in step (1); use SLIC algorithm to carry out superpixel segmentation to original image;

分别用半径为10和5的圆形结构体对(1)得到的二值化图像进行形态学变换，先膨胀后腐蚀，重复三次；然后采用简单的线性迭代聚类算法(Simple Liner IterativeClustering，SLIC)算法将图像分割成一个个大小相似并且边缘贴近图像边缘的超像素；以超像素为基本操作单元，不仅能极大地减少运算量，而且能很好地保留图像边缘信息和局部的结构信息，甚至能增加算法对噪声的鲁棒性。本实施例将超声图像分割为600个超像素；The binarized image obtained in (1) is morphologically transformed with a circular structure with a radius of 10 and 5, respectively, which is first expanded and then corroded, and repeated three times; then a simple linear iterative clustering algorithm (Simple Liner Iterative Clustering, SLIC ) algorithm divides the image into superpixels of similar size and whose edges are close to the edge of the image; using superpixels as the basic operation unit can not only greatly reduce the amount of computation, but also preserve image edge information and local structural information well. It can even increase the robustness of the algorithm to noise. In this embodiment, the ultrasonic image is divided into 600 superpixels;

(2-2)超像素特征提取：具体为使用一个23维的向量v来表示每个超像素，其中，v＝{F1，F2，F3，F4}，F1是每个超像素的平均亮度、F2是每个超像素在洋红色至绿色色彩空间的均值、F3是每个超像素在黄色至蓝色色彩空间的均值和F4代表每个超像素在S通道上的饱和度分布。其中，饱和度分布特征F4是一个 20维的向量，通过将图像的最小饱和度到最大饱和度范围之间均匀划分20个饱和度区间，统计每个超像素在这20个饱和度间隔的直方图作为其饱和度分布特征；(2-2) Superpixel feature extraction: Specifically, a 23-dimensional vector v is used to represent each superpixel, where v={F1, F2, F3, F4}, F1 is the average brightness of each superpixel, F2 is the mean value of each superpixel in the magenta to green color space, F3 is the mean value of each superpixel in the yellow to blue color space and F4 represents the saturation distribution of each superpixel on the S channel. Among them, the saturation distribution feature F4 is a 20-dimensional vector, by evenly dividing the image into 20 saturation intervals from the minimum saturation to the maximum saturation range, and counting the histogram of each superpixel in these 20 saturation intervals graph as its saturation distribution feature;

(2-3)构建背景字典：(2-3) Build a background dictionary:

(2-3-1)备选背景区域选择：遍历(2-1)得到的二值化图像超像素，找到当前超像素平均亮度为0且邻接超像素平均亮度也为0的超像素点，作为备选背景超像素；(2-3-1) Alternative background area selection: Traverse the superpixels of the binarized image obtained in (2-1), find the superpixels whose average brightness of the current superpixel is 0 and the average brightness of adjacent superpixels is also 0, as an alternative background superpixel;

(2-3-2)背景区域选择：根据(2-3-1)得到的备选背景超像素点的坐标得到(2-1)中原始图像对应的超像素点，当备选背景超像素区域覆盖原始图像中对应超像素区域的一半以上，则设原始图像对应的超像素点为背景区域，否则视为前景区域；以最终背景超像素的特征作为列向量组合成背景字典矩阵D，即 D＝[v₁，v₂，...，v_m]，其中m为背景超像素个数；(2-3-2) Background area selection: According to the coordinates of the candidate background superpixel obtained in (2-3-1), the superpixel corresponding to the original image in (2-1) is obtained, when the candidate background superpixel If the area covers more than half of the corresponding superpixel area in the original image, the superpixel point corresponding to the original image is set as the background area, otherwise it is regarded as the foreground area; the characteristics of the final background superpixel are used as column vectors to form a background dictionary matrix D, namely D=[v ₁ , v ₂ ,...,v _m ], where m is the number of background superpixels;

(2-4)骨髓细胞检测：(2-4) Bone marrow cell detection:

(2-4-1)按照稀疏分解公式，计算原始图像中每个超像素在背景字典下的稀疏系数，如下式所示：(2-4-1) According to the sparse decomposition formula, calculate the sparse coefficient of each superpixel in the original image under the background dictionary, as shown in the following formula:

其中b_j是求得的稀疏系数，j∈[1，2，...，n]，n为原始图像中超像素个数；λ为正则系数；Where b _j is the obtained sparse coefficient, j ∈ [1, 2, ..., n], n is the number of superpixels in the original image; λ is the regular coefficient;

(2-4-2)利用求得的稀疏系数对原超像素v_j进行重建，得到稀疏重建后的残差ε_j为，如下式：(2-4-2) Use the obtained sparse coefficients to reconstruct the original superpixel v _j , and obtain the residual ε _j after sparse reconstruction as follows:

(2-4-3)将计算得到的残差作为原始图像超像素的显著度，得到骨髓细胞的检测图image_huidu；(2-4-3) using the calculated residual as the saliency of the original image superpixels to obtain the detection map image _huidu of bone marrow cells;

(3)如图4所示，基于形态学的多角度骨髓细胞分割和计数：(3) As shown in Figure 4, multi-angle bone marrow cell segmentation and counting based on morphology:

(3-1)分类简单细胞图像和复杂细胞图像：(3-1) Classify simple cell images and complex cell images:

急性白血病细胞图像随个体差异、病情程度等而呈现不同的状态，一般增生极度活跃，病情严重的图像多出现细胞团聚、粘连现象，分割难度较大，因此，需要挑出进行特殊分析。Acute leukemia cell images show different states depending on individual differences and disease severity, etc. Generally, hyperplasia is extremely active, and severe disease images often show cell aggregation and adhesion, which makes segmentation difficult. Therefore, it needs to be picked out for special analysis.

具体为：利用otus阈值对(2-4-3)得到的检测图image_huidu进行二值化，得到图像image_erzhi；当骨髓细胞的面积占总图像面积的40％～45％以上，即为复杂细胞图像；否则为简单细胞图像；Specifically: use the otus threshold to binarize the detection image image _huidu obtained in (2-4-3) to obtain the image image _erzhi ; when the area of bone marrow cells accounts for more than 40% to 45% of the total image area, it is complex cell image; otherwise simple cell image;

(3-2)细胞全局分割：(3-2) Cell global segmentation:

(3-2-1)遍历(3-1)中二值图像image_erzhi的连通区域，找到每个连通区域的最大最小坐标，以此为矩形框图的坐标，对原始图像、(2-4-3)得到的检测图 image_huidu和(3-1)得到的二值图像image_erzhi分别进行裁剪，从而得到骨髓细胞对应的分割图crop1_image、分割灰度图crop1_huidu和分割二值图crop1_erzhi；(3-2-1) Traverse the connected regions of the binary image image _erzhi in (3-1), find the maximum and minimum coordinates of each connected region, and use this as the coordinates of the rectangular frame, for the original image, (2-4- 3) The obtained detection image image _huidu and the binary image image _erzhi obtained in (3-1) are respectively cropped to obtain the segmentation image crop1 _image corresponding to bone marrow cells, the segmentation grayscale image crop1 _huidu and the segmentation binary image crop1 _erzhi ;

(3-2-2)剔除(3-2-1)分割图中的不完整细胞和由于涂片制备、染色条件和手工操作导致的干扰背景；(3-2-2) Eliminate the incomplete cells in the (3-2-1) segmentation map and the interference background caused by smear preparation, staining conditions and manual operations;

定义area_crop1为分割图crop1_image的面积；ratio0_crop1为分割图Define area _crop1 as the area of crop1 _image ; ratio0 _crop1 is the segmentation map

crop1_image宽高比，ratio0_crop1∈(0，1]，ratio1_crop1为分割图crop1_image饱和度在区间[102，255]的占比；crop1 _image aspect ratio, ratio0 _crop1 ∈ (0, 1], ratio1 _crop1 is the proportion of crop1 _image saturation in the interval [102, 255];

area_crop1、ratio0_crop1、ratio1_crop1的计算具体如下：The calculation of area _crop1 , ratio0 _crop1 , ratio1 _crop1 is as follows:

area_crop1＝width_crop1*height_crop1 area _crop1 = width _crop1 * height _crop1

ratio0_crop1＝min(width_crop1，height_crop1)/max(width_crop1，height_crop1)ratio0 _crop1 = min(width _crop1 , height _crop1 )/max(width _crop1 , height _crop1 )

其中，width_crop1，height_crop1分别为分割图crop1_image的宽度和高度，S_crop1是分割图crop1_image提取其S通道的图像，i∈[1，2，...，width_crop1]，j∈[1，2，...，height_crop1]；Among them, width _crop1 and height _crop1 are the width and height of crop1 _image respectively, S _crop1 is the image of S channel extracted from crop1 _image , i ∈ [1, 2, ..., width _crop1 ], j ∈ [ 1,2,...,height _crop1 ];

1)分割图crop1_image位于原始图像边界时，其满足area_crop1∈ (1000，3000]且ratio0_crop1，ratio1_crop1≥0.45，或者area_crop1∈(3000，∞) 且ratio1_crop1≥0.45则保留；2)分割图crop1_image位于原始图像非边界时，其满足ratio1_crop1≥0.45则保留；1) When the segmentation image crop1 _image is located at the boundary of the original image, it satisfies area _crop1 ∈ (1000, 3000] and ratio0 _crop1 , ratio1 _crop1 ≥ 0.45, or area _crop1 ∈ (3000, ∞) and ratio1 _crop1 ≥ 0.45 is reserved; 2) When the segmentation map crop1 _image is located on the non-boundary of the original image, if it satisfies ratio1 _crop1 ≥ 0.45, it will be retained;

(3-3)细胞局部再分割：(3-3) Partial subdivision of cells:

骨髓细胞的多变性导致它不可能通过单一手法就实现很好的分割结果，因此，需要针对全局分割的结果找到包含多细胞的聚集区域，根据聚集区域局部特征进行二值化，并根据不同复杂程度的图像进行不同的形态学变换，从而得到更好的分割结果。The variability of bone marrow cells makes it impossible to achieve a good segmentation result by a single method. Therefore, it is necessary to find the aggregated area containing multiple cells for the result of global segmentation, and perform binarization according to the local characteristics of the aggregated area. Different morphological transformations are performed on images of different degrees to obtain better segmentation results.

(3-3-1)对(3-2-2)保留下来的分割灰度图crop1_huidu进行阈值区间为[6，10]的二值化处理，经过边长区间为[2，4]的正方形结构体形态学变换后，定义 round_rate_crop1为细胞曲圆率；当分割图crop1_image满足area_crop1＞35000，或者area_crop1∈(17000，35000]且round_rate_crop1＜0.46时，则判定为待分割的多细胞区域crop1_multi，否则为单细胞区域crop1_single；(3-3-1) Perform binarization processing on the segmented grayscale image crop1 _huidu retained in (3-2-2) with a threshold value interval of [6, 10], and after the edge length interval is [2, 4] After the morphological transformation of the square structure, define round_rate _crop1 as the cell curvature; when the segmentation image crop1 _image satisfies area _crop1 > 35000, or area _crop1 ∈ (17000, 35000] and round_rate _crop1 < 0.46, it is determined to be segmented Multi-cell region crop1 _multi , otherwise single-cell region crop1 _single ;

细胞曲圆率round_rate_crop1的计算如下：The cell curvature round_rate _crop1 is calculated as follows:

round_rate_crop1＝4*π*S_round_crop1/C_round_crop1 ² round_rate _crop1 = 4*π*S_round _crop1 /C_round _crop1 ²

其中，S_round_crop1为分割图crop1_image中细胞所占面积，C_round_crop1为分割图crop1_image中细胞的周长Among them, S_round _crop1 is the area occupied by the cells in the crop1 _image , and C_round _crop1 is the perimeter of the cells in the crop1 _image

(3-3-2)利用otus阈值对(3-3-1)中待分割的多细胞区域crop1_multi对应的分割灰度图crop1_huidu进行二值化，当阈值大于区间[112，117]之间的值时，设阈值为[220，240]中的值；图像为简单图像时，设圆形结构体的半径为1；图像为复杂图像时，设圆形结构体的半径为3；经过形态学变换，遍历每个面积大于 1200～1500的连通区域，找到每个连通区域的最大最小坐标，并以此为矩形框图的坐标，对多细胞分割图、多细胞灰度图和多细胞二值图分别进行裁剪，从而得到对应的分割图crop2_image、分割灰度图crop2_huidu和分割二值图crop2_erzhi；(3-3-2) Use the otus threshold to binarize the segmented grayscale image crop1 _huidu corresponding to the multicellular region crop1 _multi to be segmented in (3-3-1), when the threshold is greater than the interval [112, 117] When the value between , the threshold is set to the value in [220, 240]; when the image is a simple image, the radius of the circular structure is set to 1; when the image is a complex image, the radius of the circular structure is set to 3; Morphological transformation, traversing each connected region with an area greater than 1200-1500, finding the maximum and minimum coordinates of each connected region, and using this as the coordinates of the rectangular frame diagram, multi-cell segmentation map, multi-cell grayscale map and multi-cell binary The value map is cropped separately to obtain the corresponding segmentation image crop2 _image , segmented grayscale image crop2 _huidu and segmented binary image crop2 _erzhi ;

(3-3-3)剔除(3-3-2)分割图中不完整细胞(3-3-3) Eliminate (3-3-2) incomplete cells in the segmentation map

局部分割时，会产生一些不完整细胞，这些细胞要么本身是不完整细胞，位于原始图像的边界；要么是已经分割出去的单细胞区域但同时被部分框进了多细胞区域，因此，需要剔除；During local segmentation, some incomplete cells will be generated. These cells are either incomplete cells themselves and located at the boundary of the original image; or they are single-cell regions that have been segmented but are partially framed into multi-cell regions at the same time. Therefore, they need to be removed. ;

当(3-3-2)分割图crop2_image位于对应的(3-3-1)多细胞区域crop1_multi的边界时，分割图crop2_image面积低于区间[14000，16000]的值且宽高比低于区间 [0.5，0.55]的值时剔除，否则保留该分割图crop2_image；When the (3-3-2) segmentation image crop2 _image is located at the boundary of the corresponding (3-3-1) multicellular area crop1 _multi , the area of the segmentation image crop2 _image is lower than the value of the interval [14000, 16000] and the aspect ratio Eliminate when the value is lower than the interval [0.5, 0.55], otherwise keep the segmentation map crop2 _image ;

(3-4)细胞S通道再分割：(3-4) Cell S channel subdivision:

(3-4-1)对(3-3-3)保留的分割图对应的灰度图crop2_huidu进行阈值区间为 [6，10]的二值化处理，经过边长区间为[2，4]的正方形结构体形态学变换后，定义 area_crop2是分割图crop2_image的面积；round_rate_crop2为细胞曲圆率； ratio0_crop2为细胞所占面积比；(3-4-1) Perform binarization processing with a threshold interval of [6, 10] on the grayscale image crop2 _huidu corresponding to the segmented image retained in (3-3-3), after the side length interval is [2, 4 ] After the morphological transformation of the square structure, define area _crop2 as the area of the crop2 _image ; round_rate _crop2 is the cell curvature; ratio0 _crop2 is the area ratio of the cell;

area_crop2、round_rate_crop2、ratio0_crop2的计算如下：area _crop2 , round_rate _crop2 , ratio0 _crop2 are calculated as follows:

area_crop2＝width_crop2*height_crop2 area _crop2 = width _crop2 * height _crop2

round_rate_crop2＝4*π*S_round_crop2/C_round_crop2 ² round_rate _crop2 = 4*π*S_round _crop2 /C_round _crop2 ²

ratio0_crop2＝S_round_crop2/area_crop2 ratio0 _crop2 = S_round _crop2 /area _crop2

其中，width_crop2，height_crop2分别为分割图crop2_image的宽度和高度， S_round_crop2为分割图crop2_image中细胞所占面积，C_round_crop2为分割图crop2_image中细胞周长；Among them, width _crop2 and height _crop2 are the width and height of the crop2 _image , respectively, S_round _crop2 is the area occupied by the cells in the crop2 _image , and C_round _crop2 is the perimeter of the cells in the crop2 _image ;

1)area_crop2＞27500；2)area_crop2∈(19000，27500]且 round_rate_crop2＜0.56，或者area_crop2∈(19000，27500]且 round_rate_crop2≥0.56但ratio0_crop2＜0.5，则判定为待分割的多细胞区域 crop2_multi，否则为单细胞区域crop2_single；并将(3-3-2)中单细胞区域的矩形框图扩大1.1～1.2倍重新对crop1_multi裁剪得到crop2_new_single；1) area _crop2 >27500; 2) area _crop2 ∈ (19000, 27500] and round_rate _crop2 < 0.56, or area _crop2 ∈ (19000, 27500] and round_rate _crop2 ≥ 0.56 but ratio0 _crop2 < 0.5, then it is determined that there are many crops to be divided The cell area crop2 _multi , otherwise it is the single-cell area crop2 _single ; and the rectangular frame diagram of the single-cell area in (3-3-2) is enlarged by 1.1 to 1.2 times to re-crop crop1 _multi to obtain crop2_new _single ;

(3-4-2)提取(3-4-1)的多细胞区域crop2_multi的S通道图像，并进行再分割，得到分割图crop3_image、分割灰度图crop3_nuidu和分割二值图crop3_erzhi；(3-4-2) Extract the S-channel image of the multi-cellular region crop2 _multi in (3-4-1), and then segment it again to obtain the segmented image crop3 _image , the segmented grayscale image crop3 _nuidu and the segmented binary image crop3 _erzhi ;

细胞为简单图像时，设阈值为区间[125，130]中的一值对S通道进行二值化；经过半径为1的圆形结构体形态学变换二次，遍历每个面积大于1200～1500的连通区域，找到每个连通区域的最大最小坐标，并以此作为矩形框图的坐标，对多细胞分割图crop2_multi、多细胞灰度图和多细胞二值图分别进行裁剪，从而得到对应的分割图crop3_image、分割灰度图crop3_nuidu和分割二值图crop3_erzhi；When the cell is a simple image, set the threshold value to a value in the interval [125, 130] to binarize the S channel; after two morphological transformations of a circular structure with a radius of 1, traverse each area greater than 1200 to 1500 Connected areas, find the maximum and minimum coordinates of each connected area, and use them as the coordinates of the rectangular frame to crop the multi-cell segmentation map crop2 _multi , multi-cell grayscale image and multi-cell binary image respectively, so as to obtain the corresponding Segmentation image crop3 _image , segmentation grayscale image crop3 _nuidu and segmentation binary image crop3 _erzhi ;

细胞为复杂图像时，阈值设置如下式：When the cell is a complex image, the threshold is set as follows:

其中，t_crop2是多细胞区域S通道饱和度的众数，t1_crop2是多细胞区域S通道饱和度的otus阈值；利用阈值th_crop2对多细胞区域S通道二值化；经过半径为3的圆形结构体形态学变换后，遍历每个面积大于1200～1500的连通区域，找到每个连通区域的最大最小坐标，并以此作为矩形框图的坐标，对多细胞分割图、多细胞灰度图和多细胞二值图分别进行裁剪，从而得到对应的分割图 crop3_image、分割灰度图crop3_huidu和分割二值图crop3_erzhi；Among them, t _crop2 is the mode of S channel saturation in the multicellular area, and t1 _crop2 is the otus threshold value of S channel saturation in the multicellular area; use the threshold th _crop2 to binarize the S channel in the multicellular area; pass through a circle with a radius of 3 After the morphological transformation of the shape structure, traverse each connected region with an area greater than 1200-1500, find the maximum and minimum coordinates of each connected region, and use this as the coordinate of the rectangular frame diagram, for multi-cell segmentation map, multi-cell grayscale map and the multi-cell binary image are cropped separately to obtain the corresponding segmentation image crop3 _image , segmented grayscale image crop3 _huidu and segmented binary image crop3 _erzhi ;

(3-5)细胞H通道再分割：(3-5) Cell H channel subdivision:

一般情况下，经过三次分割的细胞图像已经可以分割得很好了，但是当其为复杂图像时，细胞往往粘连、重叠、堆积现象严重，需要对该现象进行大力度的形态学变换；同时，堆积、重叠严重的细胞在饱和度上不易区分，但是往往在色度上呈现可分割状态，即细胞核、胞浆之间色度有所差异，而内部色度又保持一致。Under normal circumstances, the cell image that has been segmented three times can already be segmented very well, but when it is a complex image, the cells often adhere, overlap, and accumulate seriously, and a large-scale morphological transformation is required for this phenomenon; at the same time, Cells with severe accumulation and overlap are not easy to distinguish in terms of saturation, but they are often separable in terms of chroma, that is, the chroma of the nucleus and cytoplasm is different, while the internal chroma remains the same.

当图像为简单图像时，将(3-4-2)中的矩形框图扩大1.2～1.3倍，重新对crop2_multi裁剪得到单细胞区域crop3_new_image；When the image is a simple image, expand the rectangular frame in (3-4-2) by 1.2 to 1.3 times, and re-crop the crop2 _multi to obtain the single-cell region crop3_new _image ;

当图像为复杂图像时，进行细胞H通道再分割：When the image is a complex image, the cell H channel is re-segmented:

(3-5-1)剔除(3-4-2)分割图中不完整细胞：当(3-4-2)分割图crop3_image位于 (3-4-1)待分割多细胞区域crop2_multi的边界时，分割图crop3_image面积低于区间 [14000，16000]的值且宽高比低于区间[0.5，0.55]的值时剔除，否则保留该分割图 crop3_image；(3-5-1) Eliminate incomplete cells in the (3-4-2) segmentation map: when the (3-4-2) segmentation map crop3 _image is located in the (3-4-1) multi-cell region crop2 _multi to be segmented When bordering, if the crop3 _image area of the segmentation map is lower than the value of the interval [14000, 16000] and the aspect ratio is lower than the value of the interval [0.5, 0.55], it will be removed, otherwise the segmentation image crop3 _image will be kept;

(3-5-2)对(3-5-1)保留下来的分割图对应的灰度图crop3_huidu进行阈值区间为[6，10]的二值化处理，经过边长区间为[2，4]的正方形结构体形态学变换后，定义area_crop3是分割图crop3_image的面积；round_rate_crop3为细胞曲圆率； ratio0_crop3为细胞所占面积比；(3-5-2) Perform binarization processing with a threshold interval of [6, 10] on the grayscale image crop3 _huidu corresponding to the segmented image retained in (3-5-1), after the side length interval is [2, 4] After the morphological transformation of the square structure, define area _crop3 as the area of the crop3 _image ; round_rate _crop3 is the cell curvature; ratio0 _crop3 is the area ratio of the cell;

area_crop3、round_rate_crop3、ratio0_crop3计算具体如下：area _crop3 , round_rate _crop3 , ratio0 _crop3 are calculated as follows:

area_crop3＝width_crop₃*height_crop3 area _crop3 = width _crop p ₃ * height _crop3

round_rate_crop3＝4*π*S_round_crop3/C_round_crop3 ² round_rate _crop3 = 4*π*S_round _crop3 /C_round _crop3 ²

ratio0_crop3＝S_round_crop3/area_crop3 ratio0 _crop3 = S_round _crop3 /area _crop3

其中，width_crop3，height_crop3分别为分割图crop3_image的宽度和高度，S_round_crop3为分割图crop3_image中细胞所占面积，C_round_crop3为分割图 crop3_image中细胞周长；Among them, width _crop3 and height _crop3 are the width and height of the crop3 _image , respectively, S_round _crop3 is the area occupied by the cells in the crop3 _image , and C_round _crop3 is the perimeter of the cells in the crop3 _image ;

1)area_crop3＞27500；2)area_crop3∈(19000，27500]且 round_rate_crop3＜0.56，或者area_crop3∈(19000，27500]且round_rate_crop3≥ 0.56但ratio0_crop3＜0.5，则判定为待分割的多细胞区域crop3_multi，否则为单细胞区域crop3_single，并将(3-4-2)中的矩形框图扩大1.2～1.3倍，重新对crop2_multi裁剪得到crop3_new_single；1) area _crop3 >27500; 2) area _crop3 ∈ (19000, 27500] and round_rate _crop3 < 0.56, or area _crop3 ∈ (19000, 27500] and round_rate _crop3 ≥ 0.56 but ratio0 _crop3 < 0.5, then it is determined that there are many crops to be divided The cell area crop3 _multi , otherwise it is the single-cell area crop3 _single , and the rectangular frame in (3-4-2) is enlarged by 1.2 to 1.3 times, and crop2 _multi is re-cropped to obtain crop3_new _single ;

(3-5-3)提取(3-5-2)多细胞区域crop3_multi的H通道图像，并进行再分割，得到分割图crop4_image、分割灰度图crop4_huidu和分割二值图crop4_erzhi：(3-5-3) Extract (3-5-2) the H-channel image of the multi-cellular area crop3 _multi , and then segment it again to obtain the segmented image crop4 _image , the segmented grayscale image crop4 _huidu and the segmented binary image crop4 _erzhi :

阈值设置如下式：The threshold is set as follows:

其中，阈值t_crop3是多细胞区域H通道饱和度的众数，利用阈值th_crop3对多细胞区域H通道二值化；经过半径为1的圆形结构体形态学变换后，再经过半径为3的圆形结构体腐蚀，半径为1的圆形结构体膨胀和半径为3的圆形结构体腐蚀后，遍历每个面积大于1200～1500的连通区域，找到每个连通区域的最大最小坐标，并以此作为矩形框图的坐标对多细胞分割图、多细胞灰度图和多细胞二值图分别进行裁剪，从而得到对应的分割图crop4_image、分割灰度图 crop4_huidu和分割二值图crop4_erzhi；Among them, the threshold t _crop3 is the mode of the saturation of the H channel in the multicellular area, and the threshold th _crop3 is used to binarize the H channel in the multicellular area; After the circular structure corrosion of the circular structure with a radius of 1 is expanded and the circular structure with a radius of 3 is corroded, each connected area with an area greater than 1200-1500 is traversed to find the maximum and minimum coordinates of each connected area. And use this as the coordinates of the rectangular frame to crop the multi-cell segmentation map, multi-cell grayscale image and multi-cell binary image respectively, so as to obtain the corresponding segmentation image crop4 _image , segmented grayscale image crop4 _huidu and segmented binary image crop4 _erzhi ;

(3-5-4)剔除(3-5-3)分割图crop4_image中不完整细胞：(3-5-4) Eliminate (3-5-3) incomplete cells in the crop4 _image of the segmentation map:

定义area_crop4为分割图crop4_image的面积；ratio0_crop4为分割图crop4_image宽高比，ratio0_crop4∈(0，1]；ratio1_crop4为分割图crop4_image饱和度在区间 [102，255]的占比；Define area _crop4 as the area of the crop4 _image ; ratio0 _crop4 is the aspect ratio of the crop4 _image , ratio0 _crop4 ∈ (0, 1]; ratio1 _crop4 is the ratio of the saturation of the crop4 _image in the interval [102, 255] ;

area_crop4、ratio0_crop4、ratio1_crop4的计算具体如下：The calculation of area _crop4 , ratio0 _crop4 , ratio1 _crop4 is as follows:

area_crop4＝width_crop4*height_crop4 area _crop4 = width _crop4 * height _crop4

ratio0_crop4＝min(width_crop4，height_crop4)/max(width_crop4，height_crop4)ratio0 _crop4 = min(width _crop4 , height _crop4 )/max(width _crop4 , height _crop4 )

其中，width_crop4，height_crop4分别为分割图crop4_image的宽度和高度，S_crop4是分割图crop4_image提取其S通道的图像；Wherein, width _crop4 , height _crop4 are respectively the width and the height of crop4 _image of segmentation figure, S _crop4 is the image of its S channel extracted from crop4 _image of segmentation figure;

1)分割图crop4_image位于待分割多细胞区域crop3_multi边界时，其满足 area_crop4∈(1000，3000]，ratio0_crop4≥0.5且ratio1_crop4＞0.45，或者 area_crop4∈(3000，∞)且ratio1_crop4＞0.45则保留；2)分割图crop4_image位于非边界时，其满足ratio1_crop4＞0.45则保留，并将(3-5-3)中的矩形框图扩大 1.2～1.3倍，重新对crop3_multi裁剪得到单细胞区域crop4_new_image；1) When the segmentation image crop4 _image is located at the crop3 _multi boundary of the multi-cell area to be segmented, it satisfies area _crop4 ∈ (1000, 3000], ratio0 _crop4 ≥ 0.5 and ratio1 _crop4 > 0.45, or area _crop4 ∈ (3000, ∞) and ratio1 _crop4 ＞0.45, keep it; 2) When crop4 _image is located in the non-boundary, it satisfies ratio1 _crop4 ＞0.45, keep it, expand the rectangular frame in (3-5-3) by 1.2-1.3 times, and re-crop crop3 _multi to get single cell region crop4_new _image ;

(3-6)汇总原始图像上分割出来的染色单细胞：图像为简单图像时，汇总 (3-3-1)单细胞区域crop1_single、(3-4-1)单细胞区域crop2_new_single和(3-5)单细胞区域crop3-new_image，作为原始图像上待分类的染色单细胞；图像为复杂图像时，汇总(3-3-1)单细胞区域crop1_single、(3-4-1)单细胞区域crop2_new_single、 (3-5-2)单细胞区域crop3-new_single和(3-5-4)单细胞区域crop4_new_image，作为原始图像上待分类的染色单细胞。(3-6) Summarize the stained single cells segmented on the original image: when the image is a simple image, summarize (3-3-1) single cell region crop1 _single , (3-4-1) single cell region crop2_new _single and ( 3-5) single cell area crop3-new _image , as the dyed single cell to be classified on the original image; when the image is a complex image, summarize (3-3-1) single cell area crop1 _single , (3-4-1) Single-cell region crop2_new _single , (3-5-2) single-cell region crop3-new _single and (3-5-4) single-cell region crop4_new _image are used as stained single cells to be classified on the original image.

(4)基于深度学习的骨髓细胞分类(4) Bone marrow cell classification based on deep learning

常用的白细胞识别方法主要涉及特征提取和分类器的选择。然而手工提取的有些特征并不能很好的表达细胞的特性；对白细胞的分类也不是很高效。近年来，深度网络发展迅猛，并在各个领域都取得了不错的效果。因此，本方法采用深度网络自动学习骨髓细胞的特征，并进行分类。Commonly used leukocyte identification methods mainly involve feature extraction and classifier selection. However, some features extracted manually cannot express the characteristics of cells well; the classification of white blood cells is not very efficient. In recent years, the deep network has developed rapidly and achieved good results in various fields. Therefore, this method uses a deep network to automatically learn the characteristics of bone marrow cells and classify them.

(4-1)三路并行网络结构设计：包括RGB彩色空间网络CNN-RGB、HSV彩色空间网络CNN-HSV和领域学知识细胞核网络CNN-NUCLEUS设计(4-1) Three-way parallel network structure design: including RGB color space network CNN-RGB, HSV color space network CNN-HSV and domain knowledge cell nucleus network CNN-NUCLEUS design

由于医疗数据匮乏，很难满足深度网络中大量超参数充分学习的要求。因此，本方法采用较小的深度网络，同时引入骨髓细胞的领域学知识，提高网络的学习能力。Due to the scarcity of medical data, it is difficult to meet the requirement of adequate learning of a large number of hyperparameters in deep networks. Therefore, this method adopts a smaller deep network and at the same time introduces the domain knowledge of bone marrow cells to improve the learning ability of the network.

如图5所示，本发明的模型主要包括四个部分：数据预处理、RGB彩色空间模块CNN-RGB、HSV彩色空间模块CNN-HSV和领域学知识细胞核模块 CNN-NUCLEUS。采用Alex网络的前六层作为单路基准网络。As shown in Figure 5, the model of the present invention mainly includes four parts: data preprocessing, RGB color space module CNN-RGB, HSV color space module CNN-HSV and domain knowledge cell nucleus module CNN-NUCLEUS. The first six layers of the Alex network are used as the single-way benchmark network.

网络结构输入图像大小为256*256*3，这三维数值分别是宽度、高度、通道数。每一张输入图像数据被裁剪到224*224*3的大小。前5层是5个卷积层。第一层卷积层有96个卷积核，大小为11*11*3。第2层有256个卷积核，大小为5*5*48。第三层卷积层有384个核，大小为3*3*256。第四层卷积层有384 个核，大小为3*3*192。第5层卷积层有256个核，大小为3*3*192。前两层和第五层卷积层的后面都接一层池化层(Pooling)和一层正则化层(Normalization)。 CNN-RGB、CNN-HSV和CNN-NUCLEUS通过一个神经元数量为4096的全连接层实现并行，使得模型同时从多角度学习分类，并增加了医学领域学知识。网络的最后一层包含13个神经元，进行骨髓细胞13分类输出。The input image size of the network structure is 256*256*3, and the three-dimensional values are width, height, and number of channels. Each input image data is cropped to a size of 224*224*3. The first 5 layers are 5 convolutional layers. The first convolution layer has 96 convolution kernels with a size of 11*11*3. The second layer has 256 convolution kernels with a size of 5*5*48. The third convolutional layer has 384 cores with a size of 3*3*256. The fourth convolutional layer has 384 cores with a size of 3*3*192. The 5th layer convolutional layer has 256 cores with a size of 3*3*192. The first two layers and the fifth layer of convolutional layer are followed by a layer of pooling layer (Pooling) and a layer of regularization layer (Normalization). CNN-RGB, CNN-HSV, and CNN-NUCLEUS are parallelized through a fully connected layer with 4096 neurons, which enables the model to learn classification from multiple perspectives at the same time, and increases medical domain knowledge. The last layer of the network contains 13 neurons, which output 13 classifications of bone marrow cells.

(4-2)输入预处理与数据集平衡化处理(4-2) Input preprocessing and data set balance processing

近年来的识别方法主要是针对五类成熟白细胞的分类，训练和测试样本量都很小，实验中效果比较理想；但是，在临床应用中，骨髓细胞除了这五类成熟白细胞外，还有很多其他类别的细胞；即使是白细胞也含有不成熟阶段各种类型的细胞。仅仅对五类成熟白细胞进行分类对于急性白血病诊断是远远不够的。同时，由于数据采集困难，耗费大量人力物力，需要医疗人员协助，往往很难满足大数据的要求。并且，不平衡的训练数据集会对分类结果产生不良影响，弱化学习得到特征的表达能力。因此，需要采用图像处理手段进行合理的医疗图像扩增与平衡化处理，同时对骨髓细胞类别进行合理分类。In recent years, the identification methods are mainly aimed at the classification of five types of mature white blood cells. The training and test samples are very small, and the experimental results are ideal; however, in clinical applications, bone marrow cells have many other types besides these five types of mature white blood cells Other classes of cells; even white blood cells contain various types of cells in immature stages. It is far from enough to classify the five types of mature white blood cells for the diagnosis of acute leukemia. At the same time, due to the difficulty of data collection, which consumes a lot of manpower and material resources, and requires the assistance of medical personnel, it is often difficult to meet the requirements of big data. Moreover, an unbalanced training data set will have a negative impact on the classification results and weaken the expressive ability of the learned features. Therefore, it is necessary to use image processing methods to perform reasonable medical image amplification and balance processing, and at the same time to reasonably classify bone marrow cell types.

(4-2-1)输入预处理，具体为将从医生那里得到的包含单一骨髓细胞的带分类标签的原始数据图库根据骨髓细胞类别和对于急性白血病诊断重要性、FAB标准等因素分为13大类，它们分别是中幼红细胞、晚幼红细胞、其他红系细胞(主要为原红细胞和早幼红细胞)、原始细胞(包括原始淋巴细胞和原始粒细胞)、成熟淋巴细胞、其他淋系细胞(主要为浆细胞)、单核系细胞、早幼粒细胞、中幼粒细胞、晚幼粒细胞、杆状核细胞、分叶核细胞和其他粒系细胞，作为网络 CNN-RGB的输入。然后，对这些原始细胞进行彩色空间变换，转换为HSV，作为网络CNN-HSV的输入；同时利用细胞染色的饱和度差异提取细胞的细胞核，作为网络CNN-NUCLEUS的输入。(4-2-1) Input preprocessing, specifically, the original data library with classification labels containing a single bone marrow cell obtained from the doctor is divided into 13 according to the type of bone marrow cell and the importance for the diagnosis of acute leukemia, FAB standards and other factors Major categories, they are medocytes, metamyelocytes, other erythrocytes (mainly proerythrocytes and promyelocytes), blasts (including primitive lymphocytes and myeloblasts), mature lymphocytes, other lymphocytes (mainly plasma cells), monocytic cells, promyelocytes, myelocytes, metamyelocytes, rod cells, segmented nuclei and other granulocytes, as input to the network CNN-RGB. Then, these original cells are transformed into color space and converted to HSV, which is used as the input of the network CNN-HSV; at the same time, the nuclei of the cells are extracted by using the saturation difference of cell staining, which is used as the input of the network CNN-NUCLEUS.

(4-2-2)数据集平衡化处理，具体为通过对图像旋转和镜像方式，对所有类别图像数据进行增加样本处理，以满足深度网络对数据量的需求；并对图像数量少的某些类别进行更多的图像旋转操作增加样本，达到与多数类样本数量平衡。并且，所有样本的尺寸统一调整为256*256。(4-2-2) Data set balance processing, specifically, through image rotation and mirroring, increase sample processing for all types of image data to meet the demand for data volume of the deep network; Some categories perform more image rotation operations to increase the number of samples to achieve a balance with the number of samples in the majority category. Moreover, the size of all samples is uniformly adjusted to 256*256.

(4-3)三路并行网络模型训练，具体训练步骤如下：(4-3) Three-way parallel network model training, the specific training steps are as follows:

(4-3-1)模型初始化：采用了fine-tune策略，前五层卷积层采用AlexNet模型前五层的网络权值来初始化；全连接层的初始化设置为随机值初始化。(4-3-1) Model initialization: The fine-tune strategy is adopted, and the first five convolutional layers are initialized with the network weights of the first five layers of the AlexNet model; the initialization of the fully connected layer is set to random value initialization.

(4-3-2)设置训练参数：前5层的初始学习率设置为0.0001。全连接层参数的初始学习率为0.001。训练过程设为每8次遍历样本集后，学习率降低40％。(4-3-2) Set training parameters: The initial learning rate of the first 5 layers is set to 0.0001. The initial learning rate of fully connected layer parameters is 0.001. The training process is set to reduce the learning rate by 40% after traversing the sample set every 8 times.

(4-3-3)加载训练数据：采用经(4-2-2)处理后的总共约有4万张图像的细胞图像数据集对网络模型进行训练，对应网络的输入大小，所有样本尺寸统一调整为256*256。所有图片具有13类标签中的一种。(4-3-3) Load training data: use the cell image data set with a total of about 40,000 images processed by (4-2-2) to train the network model, corresponding to the input size of the network, and all sample sizes Unified adjustment is 256*256. All images have one of 13 categories of labels.

(4-3-4)采用随即梯度下降算法对图5的并行深度卷积神经网络模型进行迭代训练，每迭代1000次保存一次模型参数，经过不断迭代，取得网络最优解。综合考虑在验证集上准确率高、损失函数低的模型作为本发明的最优网络模型。(4-3-4) Use the random gradient descent algorithm to iteratively train the parallel deep convolutional neural network model in Figure 5, save the model parameters every 1000 iterations, and obtain the optimal solution of the network after continuous iterations. A model with high accuracy and low loss function on the verification set is considered comprehensively as the optimal network model of the present invention.

(4-4)对(3-6)待分类染色单细胞使用已训练好的分类模型进行评价，系统把待分类染色单细胞图像输入到分类模型中，模型对图像进行分类，输出分类结果，并给出分类准确率和每个类别下的细胞数量。(4-4) Evaluate (3-6) the stained single cell to be classified using the trained classification model, the system inputs the image of the stained single cell to be classified into the classification model, the model classifies the image, and outputs the classification result, And give the classification accuracy and the number of cells under each category.

利用本发明设计的骨髓细胞分类模型，给定病人骨髓细胞染色图像后，系统自动给出染色细胞总数和每种类别下细胞数量。Using the bone marrow cell classification model designed in the present invention, the system automatically gives the total number of stained cells and the number of cells in each category after a patient's bone marrow cell staining image is given.

上述实施例为本发明较佳的实施方式，但本发明的实施方式并不受所述实施例的限制，其他的任何未背离本发明的精神实质与原理下所作的改变、修饰、替代、组合、简化，均应为等效的置换方式，都包含在本发明的保护范围之内。The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the embodiment, and any other changes, modifications, substitutions and combinations made without departing from the spirit and principle of the present invention , simplification, all should be equivalent replacement methods, and are all included in the protection scope of the present invention.