CN110084237A - Detection model construction method, detection method and the device of Lung neoplasm - Google Patents
- ️Fri Aug 02 2019
技术领域technical field
本发明涉及人工智能与医学影像技术领域，具体涉及一种肺结节的检测模型构建方法、检测方法和装置。The invention relates to the technical field of artificial intelligence and medical imaging, in particular to a method for constructing a detection model of pulmonary nodules, a detection method and a device.
背景技术Background technique
肺癌是世界上最常见的恶性肿瘤，也是排名第一位的肿瘤死因。目前早期发现肺癌存在较高难度，患者无法及时察觉自身身体状况异常从而及时就诊，直到出现咳血等明显症状。早发现、早干预、早治疗对提高肺癌患者的存活率有着十分显著的作用。因此，CT被认为是及早检测肺癌的最有效的手段之一，对肺部CT图像进行检测，确定肺结节的位置是肺癌“早发现、早治疗”的必要手段。Lung cancer is the most common malignant tumor and the number one cause of cancer death in the world. At present, it is very difficult to detect lung cancer at an early stage. Patients cannot detect their abnormal physical conditions in time and seek medical treatment in time until obvious symptoms such as coughing up blood appear. Early detection, early intervention, and early treatment have a very significant effect on improving the survival rate of lung cancer patients. Therefore, CT is considered to be one of the most effective means for early detection of lung cancer. Detection of lung CT images to determine the location of lung nodules is a necessary means for "early detection and early treatment" of lung cancer.
计算机辅助诊断系统可以通过相关算法对肺部CT图像进行一系列的处理，最终对肺部CT图像的分类进行预测。不仅可以大大地减轻医生的工作负担，进而降低了因疲劳等主观因素造成的误诊、漏诊的可能性，更为医生提供判别肺结节的有效建议，为肺癌的早期防治与监控提供了有力保障，对于肺癌的诊断有着重要的意义。The computer-aided diagnosis system can perform a series of processing on the lung CT images through related algorithms, and finally predict the classification of the lung CT images. It can not only greatly reduce the workload of doctors, but also reduce the possibility of misdiagnosis and missed diagnosis caused by subjective factors such as fatigue, and provide effective suggestions for doctors to identify pulmonary nodules, providing a strong guarantee for the early prevention and monitoring of lung cancer , which is of great significance for the diagnosis of lung cancer.
但是，目前的计算机辅助检测和诊断的方法中使用的检测模型大多是传统机器学习的方法或者采用单输入单通道的卷积神经网络结构的深度学习的方法，传统机器学习的方法采用人工特征提取的手段，不适合处理数据量大的问题，然而患者和病变的形态是形形色色且千变万化的，随着新的患者数据的加入，旧的特征可能会出现不适用的情况，从而带来严重的误差，而单输入单通道的卷积神经网络结构仅能针对某一种尺度的肺结节进行特征提取，但肺结节的大小不一，影响检测准确率。However, most of the detection models used in the current computer-aided detection and diagnosis methods are traditional machine learning methods or deep learning methods using single-input single-channel convolutional neural network structures. Traditional machine learning methods use artificial feature extraction. However, the shape of patients and lesions is varied and ever-changing. With the addition of new patient data, old features may appear inapplicable, resulting in serious errors. , while the convolutional neural network structure with single input and single channel can only extract features for pulmonary nodules of a certain scale, but the size of pulmonary nodules is different, which affects the detection accuracy.
因此，现有技术中对肺结节进行检测的检测模型和检测方法均会导致检测结果不准确，降低诊断结果的准确率。Therefore, the detection models and detection methods for detecting pulmonary nodules in the prior art will lead to inaccurate detection results and reduce the accuracy of diagnosis results.
发明内容SUMMARY OF THE INVENTION
有鉴于此，本发明的目的在于提供一种肺结节的检测模型构建方法、检测方法和装置，以解决现有技术中肺结节检测模型误差大，检测结果不准确，降低了诊断结果的准确率的问题。In view of this, the purpose of the present invention is to provide a method for constructing a detection model of pulmonary nodules, a detection method and a device, so as to solve the problem that the detection model of pulmonary nodules in the prior art has large errors, inaccurate detection results, and reduces the reliability of the diagnostic results. The problem of accuracy.
为实现以上目的，本发明采用如下技术方案：To achieve the above object, the present invention adopts the following technical solutions:
一种肺结节的检测模型构建方法，包括：A method for constructing a detection model of pulmonary nodules, comprising:
根据预先获取的肺部图像和所述肺部图像对应的描述信息，确定感兴趣区域图像；Determine the image of the region of interest according to the pre-acquired lung image and the description information corresponding to the lung image;
对所述感兴趣区域图像进行处理，得到多尺度目标样本集；Processing the image of the region of interest to obtain a multi-scale target sample set;
利用所述多尺度目标样本集，对预先设计的目标集成卷积神经网络进行训练，得到肺结节检测模型。Using the multi-scale target sample set, the pre-designed target integrated convolutional neural network is trained to obtain a pulmonary nodule detection model.
进一步地，上述所述的方法中，所述根据预先获取的肺部图像和所述肺部图像对应的描述信息，确定感兴趣区域图像，包括：Further, in the above-mentioned method, the determining the image of the region of interest according to the pre-acquired lung image and the description information corresponding to the lung image includes:
对预先获取的所述肺部图像进行图像增强处理，得到目标图像；performing image enhancement processing on the pre-acquired lung image to obtain a target image;
对预先获取的所述肺部图像对应的所述描述信息进行分析，得到坐标信息；Analyzing the description information corresponding to the pre-acquired lung image to obtain coordinate information;
根据所述目标图像和所述坐标信息，确定所述感兴趣区域图像。The image of the region of interest is determined according to the target image and the coordinate information.
进一步地，上述所述的方法中，所述坐标信息包括肺结节坐标信息和非肺结节坐标信息；Further, in the method described above, the coordinate information includes lung nodule coordinate information and non-pulmonary nodule coordinate information;
所述根据所述目标图像和所述坐标信息，确定所述感兴趣区域图像，包括：The determining the image of the region of interest according to the target image and the coordinate information includes:
在所述目标图像上截取所述肺结节坐标信息对应的图像，得到正样本图像；在所述目标图像上截取所述非肺结节坐标信息对应的图像，得到负样本图像；Intercepting an image corresponding to the lung nodule coordinate information on the target image to obtain a positive sample image; intercepting an image corresponding to the non-pulmonary nodule coordinate information on the target image to obtain a negative sample image;
将所述正样本图像和所述负样本图像组合，得到所述感兴趣区域图像。Combining the positive sample image and the negative sample image to obtain the ROI image.
进一步地，上述所述的方法中，所述对所述感兴趣区域图像进行处理，得到多尺度目标样本集，包括：Further, in the method described above, the processing of the image of the region of interest to obtain a multi-scale target sample set includes:
基于预设的n个不同尺度，对所述感兴趣区域图像进行归一化处理，得到具有n个所述尺度的原始数据样本集，所述n为大于等于2的正整数；Perform normalization processing on the image of the region of interest based on preset n different scales to obtain an original data sample set with n scales, where n is a positive integer greater than or equal to 2;
对所述原始数据样本集进行数据扩增处理，得到扩增数据样本集；performing data amplification processing on the original data sample set to obtain the amplified data sample set;
将所述原始数据样本集和所述扩增数据样本集进行合并，得到所述多尺度目标样本集。The original data sample set and the augmented data sample set are combined to obtain the multi-scale target sample set.
进一步地，上述所述的方法，所述利用所述多尺度目标样本集，对预先设计的目标集成卷积神经网络进行训练，得到肺结节检测模型之前，还包括：Further, the method described above, using the multi-scale target sample set to train the pre-designed target integrated convolutional neural network, before obtaining the pulmonary nodule detection model, also includes:
获取与每个所述尺度相对应的卷积神经网络；obtaining a convolutional neural network corresponding to each of said scales;
将获取的所述n个所述卷积神经网络进行融合，得到所述目标集成卷积神经网络。The obtained n convolutional neural networks are fused to obtain the target integrated convolutional neural network.
进一步地，上述所述的方法中，所述利用所述多尺度目标样本集，对预先设计的目标集成卷积神经网络进行训练，得到肺结节检测模型，包括：Further, in the above-mentioned method, the multi-scale target sample set is used to train the pre-designed target integrated convolutional neural network to obtain a pulmonary nodule detection model, including:
将所述多尺度目标样本集进行分类，得到训练样本集、验证样本集和测试样本集；Classifying the multi-scale target sample set to obtain a training sample set, a verification sample set and a test sample set;
利用所述训练样本集和所述验证样本集对所述目标集成卷积神经网络进行训练和验证，得到集成卷积神经网络模型；Using the training sample set and the verification sample set to train and verify the target integrated convolutional neural network to obtain an integrated convolutional neural network model;
将所述测试样本集输入到所述集成卷积神经网络模型，得到测试结果；The test sample set is input to the integrated convolutional neural network model to obtain test results;
根据所述测试结果，确定测试准确率；According to the test result, determine the test accuracy rate;
检测所述测试准确率是否大于预设准确率；Detecting whether the test accuracy rate is greater than a preset accuracy rate;
若是，将所述集成卷积神经网络模型作为肺结节检测模型。If so, use the integrated convolutional neural network model as a pulmonary nodule detection model.
本发明还提供一种肺结节的检测方法，包括：The present invention also provides a method for detecting pulmonary nodules, comprising:
获取待检测图像；Obtain the image to be detected;
基于肺结节检测模型，输入所述待检测图像，得到检测结果；Based on the pulmonary nodule detection model, the image to be detected is input to obtain a detection result;
所述肺结节检测模型通过上述肺结节的检测模型构建方法构建。The pulmonary nodule detection model is constructed by the above method for constructing a detection model of pulmonary nodules.
本发明还提供一种肺结节的检测模型构建装置，包括：The present invention also provides a detection model building device for pulmonary nodules, comprising:
确定模块，用于根据预先获取的肺部图像和所述肺部图像对应的描述信息，确定感兴趣区域图像；A determining module, configured to determine the image of the region of interest according to the pre-acquired lung image and the description information corresponding to the lung image;
处理模块，用于对所述感兴趣区域图像进行处理，得到多尺度目标样本集；A processing module, configured to process the image of the region of interest to obtain a multi-scale target sample set;
训练模块，用于利用所述多尺度目标样本集，对预先设计的目标集成卷积神经网络进行训练，得到肺结节检测模型。The training module is used to use the multi-scale target sample set to train the pre-designed target integrated convolutional neural network to obtain a pulmonary nodule detection model.
进一步地，上述所述的装置中，所述确定模块包括：第一处理单元、分析单元和图像确定单元；Further, in the above-mentioned device, the determination module includes: a first processing unit, an analysis unit, and an image determination unit;
所述第一处理单元，用于对预先获取的所述肺部图像进行图像增强处理，得到目标图像；The first processing unit is configured to perform image enhancement processing on the pre-acquired lung image to obtain a target image;
所述分析单元，用于对预先获取的所述肺部图像对应的所述描述信息进行分析，得到坐标信息；The analysis unit is configured to analyze the description information corresponding to the pre-acquired lung image to obtain coordinate information;
所述图像确定单元，用于根据所述目标图像和所述坐标信息，确定所述感兴趣区域图像。The image determining unit is configured to determine the image of the region of interest according to the target image and the coordinate information.
本发明还提供一种肺结节的检测装置，包括：图像获取模块和检测模块；The present invention also provides a detection device for pulmonary nodules, including: an image acquisition module and a detection module;
所述图像获取模块，用于获取待检测图像；The image acquisition module is used to acquire the image to be detected;
所述检测模块，用于基于肺结节检测模型，输入所述待检测图像，得到检测结果；所述肺结节检测模型通过上述肺结节的检测模型构建方法构建。The detection module is configured to input the image to be detected based on a pulmonary nodule detection model to obtain a detection result; the pulmonary nodule detection model is constructed by the above method for constructing a pulmonary nodule detection model.
本发明的肺结节的检测模型构建方法、检测方法和装置，根据预先获取的肺部图像和肺部图像对应的描述信息，确定感兴趣区域图像；对感兴趣区域图像进行处理，得到多尺度目标样本集；利用多尺度目标样本集，对预先设计的目标集成卷积神经网络进行训练，得到肺结节检测模型。采用本发明的技术方案，通过利用多尺度目标样本集对目标集成卷积神经网络进行训练得到肺结节检测模型，卷积神经网络区分于传统机器学习人工提取特征，是通过卷积操作对图像特征进行深度学习从而实现自动提取特征的效果，深度学习卷积神经网络模型的内在需求就是大的数据量，适用于数据不断加入和更新的情况，从而减小误差，采用本发明构建的肺结节检测模型，能够增强检测结果的准确性，提高诊断结果的准确率，且通过多尺度样本集训练集成卷积神经网络得到的模型是基于多尺度输入的集成卷积神经网络模型，能规避单输入单通道提取特征不全面性导致的对识别结果的影响，避免错判误判现象，提高识别准确率和检测效率。The detection model construction method, detection method and device for pulmonary nodules of the present invention determine the image of the region of interest according to the pre-acquired lung image and the corresponding description information of the lung image; process the image of the region of interest to obtain a multi-scale Target sample set: Using the multi-scale target sample set, the pre-designed target integrated convolutional neural network is trained to obtain a lung nodule detection model. Using the technical solution of the present invention, the pulmonary nodule detection model is obtained by using the multi-scale target sample set to train the target integrated convolutional neural network. The convolutional neural network is different from the traditional machine learning manual extraction of features. Features are subjected to deep learning so as to realize the effect of automatic feature extraction. The inherent requirement of the deep learning convolutional neural network model is a large amount of data, which is suitable for the situation where data is continuously added and updated, thereby reducing errors. The pulmonary nodules constructed by the present invention The node detection model can enhance the accuracy of detection results and improve the accuracy of diagnosis results, and the model obtained by training the integrated convolutional neural network through multi-scale sample sets is an integrated convolutional neural network model based on multi-scale input, which can avoid single The impact on the recognition results caused by the incompleteness of the input single-channel extraction features can avoid misjudgment and misjudgment, and improve the recognition accuracy and detection efficiency.
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本发明。It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.
图1是本发明的肺结节的检测模型构建方法实施例一的流程图；Fig. 1 is the flowchart of embodiment one of the detection model construction method of pulmonary nodules of the present invention;
图2是本发明的肺结节的检测模型构建方法实施例二的流程图；Fig. 2 is the flow chart of embodiment 2 of the detection model construction method of pulmonary nodules of the present invention;
图3是本发明的肺结节的检测模型构建方法中训练样本集的精度折线图；Fig. 3 is the accuracy line chart of training sample set in the detection model construction method of pulmonary nodule of the present invention;
图4是本发明的肺结节的检测模型构建方法中验证样本集的精度折线图；Fig. 4 is the line graph of the accuracy of the verification sample set in the detection model construction method of pulmonary nodules of the present invention;
图5是本发明的肺结节的检测方法实施例的结构示意图；5 is a schematic structural view of an embodiment of a method for detecting pulmonary nodules of the present invention;
图6是本发明的肺结节的检测模型构建装置实施例一的结构示意图；6 is a schematic structural view of Embodiment 1 of the device for constructing a detection model of pulmonary nodules of the present invention;
图7是本发明的肺结节的检测模型构建装置实施例二的结构示意图；Fig. 7 is a schematic structural diagram of Embodiment 2 of the device for constructing a detection model of pulmonary nodules of the present invention;
图8是本发明的肺结节的检测装置实施例的结构示意图。Fig. 8 is a schematic structural diagram of an embodiment of a device for detecting pulmonary nodules of the present invention.
具体实施方式Detailed ways
为使本发明的目的、技术方案和优点更加清楚，下面将对本发明的技术方案进行详细的描述。显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所得到的所有其它实施方式，都属于本发明所保护的范围。In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other implementations obtained by persons of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.
图1是本发明的肺结节的检测模型构建方法实施例一的流程图。如图1所示，本实施例的肺结节的检测模型构建方法具体可以包括如下步骤：FIG. 1 is a flow chart of Embodiment 1 of the method for constructing a detection model for pulmonary nodules of the present invention. As shown in Figure 1, the method for constructing a detection model of pulmonary nodules in this embodiment may specifically include the following steps:
S101、根据预先获取的肺部图像和肺部图像对应的描述信息，确定感兴趣区域图像；S101. Determine the image of the region of interest according to the pre-acquired lung image and the description information corresponding to the lung image;
本实施例的肺结节的检测模型构建方法首先需要获取肺部图像和肺部图像对应的描述信息，其中肺部图像优选为肺部CT图像，肺部图像对应的描述信息优选为XML格式的标注信息，那么本实施例可以从LIDC-IDRI肺结节公开数据库中获取肺部CT图像和肺部CT图像对应的XML标注信息，本实施例中获取的图像数量为300位病人的肺部CT图像，XML标注信息包括300位病人的肺部CT图像分别对应的XML标注信息，本实施例并不限制获取的肺部图像和描述信息的数量。获取到肺部CT图像和肺部CT图像对应的XML标注信息后，根据该图像和信息确定感兴趣区域图像。The lung nodule detection model construction method of this embodiment first needs to obtain the lung image and the description information corresponding to the lung image, wherein the lung image is preferably a lung CT image, and the description information corresponding to the lung image is preferably in XML format Annotation information, then this embodiment can obtain lung CT images and XML annotation information corresponding to lung CT images from the LIDC-IDRI pulmonary nodule public database. The number of images acquired in this embodiment is the lung CT images of 300 patients. The image, XML annotation information includes the XML annotation information corresponding to the lung CT images of 300 patients respectively, and this embodiment does not limit the number of acquired lung images and description information. After the lung CT image and the XML annotation information corresponding to the lung CT image are obtained, the image of the region of interest is determined according to the image and information.
S102、对感兴趣区域图像进行处理，得到多尺度目标样本集；S102. Process the image of the region of interest to obtain a multi-scale target sample set;
通过上述步骤，确定感兴趣区域图像之后，对该感兴趣区域图像进行处理，将处理后的图像作为多尺度目标样本集。Through the above steps, after the image of the region of interest is determined, the image of the region of interest is processed, and the processed image is used as a multi-scale target sample set.
S103、利用多尺度目标样本集，对预先设计的目标集成卷积神经网络进行训练，得到肺结节检测模型。S103. Using the multi-scale target sample set, train the pre-designed target integrated convolutional neural network to obtain a pulmonary nodule detection model.
通过上述步骤，得到多尺度目标样本集后，利用该多尺度目标样本集对预先设计的目标集成卷积神经网络进行训练，将最后经多尺度目标样本集训练得到的集成卷积神经网络模型作为肺结节检测模型。Through the above steps, after the multi-scale target sample set is obtained, the multi-scale target sample set is used to train the pre-designed target integrated convolutional neural network, and the final integrated convolutional neural network model obtained through multi-scale target sample set training is used as Lung nodule detection model.
本实施例的肺结节的检测模型构建方法，根据预先获取的肺部图像和肺部图像对应的描述信息，确定感兴趣区域图像；对感兴趣区域图像进行处理，得到目标样本集；利用多尺度目标样本集，对预先设计的目标集成卷积神经网络进行训练，得到肺结节检测模型。这样通过利用多尺度目标样本集对目标集成卷积神经网络进行训练得到肺结节检测模型，卷积神经网络区分于传统机器学习人工提取特征，是通过卷积操作对图像特征进行深度学习从而实现自动提取特征的效果，深度学习卷积神经网络模型的内在需求就是大的数据量，适用于数据不断加入和更新的情况，从而减小误差，那么，通过本实施例构建的肺结节检测模型，便能够增强检测结果的准确性，提高诊断结果的准确率，且通过多尺度样本集训练集成卷积神经网络得到的模型是多尺度输入的集成卷积神经网络模型，能规避单输入单通道提取特征不全面性导致的对识别结果的影响，避免错判误判现象，提高识别准确率和检测效率。。In the method for constructing a detection model of pulmonary nodules in this embodiment, the image of the region of interest is determined according to the pre-acquired lung image and the description information corresponding to the lung image; the image of the region of interest is processed to obtain the target sample set; The scale target sample set is used to train the pre-designed target ensemble convolutional neural network to obtain a lung nodule detection model. In this way, the pulmonary nodule detection model is obtained by using the multi-scale target sample set to train the target integrated convolutional neural network. The convolutional neural network is different from the traditional machine learning manual extraction of features. It is realized by deep learning of image features through convolution operations. The effect of automatic feature extraction, the inherent requirement of the deep learning convolutional neural network model is a large amount of data, which is suitable for the situation where data is continuously added and updated, thereby reducing errors. Then, the pulmonary nodule detection model constructed by this embodiment , it can enhance the accuracy of detection results and improve the accuracy of diagnosis results, and the model obtained by training the integrated convolutional neural network through multi-scale sample sets is an integrated convolutional neural network model with multi-scale input, which can avoid single-input and single-channel The impact on the recognition results caused by the incompleteness of the extracted features can avoid misjudgment and misjudgment, and improve the recognition accuracy and detection efficiency. .
图2是本发明的肺结节的检测模型构建方法实施例二的流程图。如图2所示，本实施例的肺结节的检测模型构建方法具体可以包括如下步骤：FIG. 2 is a flow chart of Embodiment 2 of the method for constructing a detection model for pulmonary nodules of the present invention. As shown in Figure 2, the method for constructing a detection model of pulmonary nodules in this embodiment may specifically include the following steps:
S201、对预先获取的所述肺部图像进行图像增强处理，得到目标图像；S201. Perform image enhancement processing on the pre-acquired lung image to obtain a target image;
本实施例的肺结节的检测模型构建方法首先获取肺部图像，其中肺部图像优选为肺部CT图像，可以从LIDC-IDRI肺结节公开数据库中获取，获取到肺部CT图像后，对其进行图像增强处理，从而得到目标图像。The method for constructing a detection model of pulmonary nodules in this embodiment first obtains a lung image, wherein the lung image is preferably a lung CT image, which can be obtained from the LIDC-IDRI pulmonary nodule public database. After obtaining the lung CT image, Image enhancement processing is performed on it to obtain the target image.
图像增强处理的方法包括直方图均衡化和中值滤波。直方图均衡化是增强图像对比度的一种方法，它的基本思想是将一幅图像的灰度直方图变平，使变换后的图像中每个灰度值的分布概率都相同。首先通过累计分布函数获得原始图像的直方图，再将直方图修改成均匀分布的直方图，得到均衡化之后的图像；中值滤波的基本原理是将图像中的每一个像素点的灰度值设为其邻域内的所有灰度值的中值，进而消除孤立的噪声点，以达到图像增强的目的，本实施例可以选取3×3中值滤波器对原始图像进行滤波操作从而得到增强后的图像。Image enhancement processing methods include histogram equalization and median filtering. Histogram equalization is a method to enhance image contrast. Its basic idea is to flatten the gray histogram of an image so that the distribution probability of each gray value in the transformed image is the same. First, the histogram of the original image is obtained through the cumulative distribution function, and then the histogram is modified into a uniformly distributed histogram to obtain an equalized image; the basic principle of median filtering is to convert the gray value of each pixel in the image Set it as the median of all gray values in its neighborhood, and then eliminate isolated noise points to achieve the purpose of image enhancement. In this embodiment, a 3×3 median filter can be selected to filter the original image to obtain the enhanced Image.
S202、对预先获取的肺部图像对应的描述信息进行分析，得到坐标信息；S202. Analyze the descriptive information corresponding to the pre-acquired lung image to obtain coordinate information;
其中，坐标信息包括肺结节坐标信息和非肺结节坐标信息；Wherein, the coordinate information includes pulmonary nodule coordinate information and non-pulmonary nodule coordinate information;
首先在获取肺部图像的同时，也获取该肺部图像对应的描述信息，每个肺部图像皆对应一套描述信息。如果肺部图像为肺部CT图像，描述信息可以为该肺部CT图像对应的XML标注信息，XML标注信息可从LIDC-IDRI肺结节公开数据库中获取，XML标注信息是由经验丰富的影像学专家标注的。获取到描述信息后，对其进行分析，得到坐标信息，其中，坐标信息包括肺结节坐标信息和非肺结节坐标信息，此处的非肺结节为类似肺结节却不是肺结节的部分。Firstly, while obtaining the lung image, the description information corresponding to the lung image is also obtained, and each lung image corresponds to a set of description information. If the lung image is a lung CT image, the description information can be XML annotation information corresponding to the lung CT image. The XML annotation information can be obtained from the LIDC-IDRI pulmonary nodule public database. The XML annotation information is provided by experienced image marked by experts. After obtaining the description information, analyze it to obtain the coordinate information, wherein the coordinate information includes the coordinate information of lung nodules and the coordinate information of non-pulmonary nodules, where the non-pulmonary nodules are similar to lung nodules but not lung nodules part.
若描述信息为XML标注信息，则获取坐标信息的基本流程如下：If the description information is XML annotation information, the basic process of obtaining coordinate information is as follows:
(1)获取病患编号，位于<SeriesInstanceUid></SeriesInstanceUid>之间的字符串为患者的编号信息。(1) Get the patient number, the string between <SeriesInstanceUid></SeriesInstanceUid> is the patient number information.
(2)循环提取标注信息。XML文件包含<readingSession></readingSession>，分别对应着每个专家的标注信息。在每个<readingSession></readingSession>之间进行以下操作：(2) Recursively extract annotation information. The XML file contains <readingSession></readingSession>, corresponding to the marking information of each expert. Do the following between each <readingSession></readingSession>:
①搜索<unblindedReadNodule></unblindedReadNodule>。此标签内存储肺结节信息。如果该标签包含<characteristics></characteristics>标签，则代表此肺结节为直径介于3mm-30mm之间的肺结节，在每个<roi></roi>中保存着肺结节的所有坐标信息。其中，<imageZposition></imageZposition>对应CT图像对应帧数；每一对<edgeMap></edgeMap>存放的<xCoord>和<yCoord>信息是肺结节的轮廓坐标。如果不包含<characteristics></characteristics>标签，则说明该结节为小结节，只需提取<roi></roi>中的一个坐标<imageZposition>、<xCoord>和<yCoord>信息，该坐标信息代表了小结节的中心坐标。①Search <unblindedReadNodule></unblindedReadNodule>. Lung nodule information is stored in this tag. If the tag contains the <characteristics></characteristics> tag, it means that the lung nodule is a lung nodule with a diameter between 3mm and 30mm, and the lung nodule is saved in each <roi></roi> All coordinate information. Among them, <imageZposition></imageZposition> corresponds to the number of frames corresponding to the CT image; the <xCoord> and <yCoord> information stored in each pair of <edgeMap></edgeMap> is the contour coordinates of the pulmonary nodule. If the <characteristics></characteristics> tag is not included, it means that the nodule is a small nodule, and only one coordinate <imageZposition>, <xCoord> and <yCoord> information in <roi></roi> needs to be extracted, the The coordinate information represents the central coordinates of the small nodules.
②搜索<nonNodule></nonNodule>。该标签保存的是标注的非肺结节信息，该标签下只需提取<imageZposition>信息和<locus></locus>中的一个坐标<xCoord>和<yCoord>信息，该坐标信息代表了非肺结节组织的中心坐标。②Search <nonNodule></nonNodule>. This label saves the labeled non-pulmonary nodule information, and only needs to extract <imageZposition> information and a coordinate <xCoord> and <yCoord> information in <locus></locus> under this label, which represents the non-pulmonary nodule information. Center coordinates of lung nodule tissue.
通过上述对XML文件的解析步骤，可以得到肺结节坐标信息和非肺结节坐标信息。Through the above steps of parsing the XML file, coordinate information of lung nodules and coordinate information of non-pulmonary nodules can be obtained.
S203、根据目标图像和坐标信息，确定感兴趣区域；S203. Determine the region of interest according to the target image and coordinate information;
通过上述步骤，得到目标图像和坐标信息之后，在目标图像上截取坐标信息中肺结节坐标信息对应的图像，将该图像作为正样本图像；在目标图像上截取坐标信息中非肺结节坐标信息对应的图像，将该图像作为负样本图像。Through the above steps, after obtaining the target image and coordinate information, intercept the image corresponding to the coordinate information of the lung nodule in the coordinate information on the target image, and use this image as a positive sample image; intercept the non-pulmonary nodule coordinates in the coordinate information on the target image The image corresponding to the information is used as a negative sample image.
得到正样本图像和负样本图像后，将正样本图像和负样本图像组合，将组合后的所有图像作为感兴趣区域图像，即感兴趣区域图像包括正样本图像和负样本图像。After the positive sample image and the negative sample image are obtained, the positive sample image and the negative sample image are combined, and all the combined images are used as the ROI image, that is, the ROI image includes the positive sample image and the negative sample image.
S204、基于预设的n个不同尺度，对感兴趣区域图像进行归一化处理，得到具有n个尺度的原始数据样本集，n为大于等于2的正整数；S204. Based on the preset n different scales, normalize the image of the region of interest to obtain an original data sample set with n scales, where n is a positive integer greater than or equal to 2;
通过上述步骤，得到感兴趣区域图像后，对感兴趣区域图像进行归一化处理，其中，归一化处理可以采用双三次插值算法实现，双三次插值又叫双立方插值，是二维空间中最常用的插值方法。在这种方法中，函数f在点(x，y)的值可以通过矩形网格中最近的十六个采样点的加权平均得到，在这里需要使用两个多项式插值三次函数，每个方向使用一个。对数据进行归一化处理，将所有图片分别归一化至n个尺度得到原始数据样本集。其中，n为大于等于2的正整数。本实施例中，n优选为3，对数据进行归一化处理，将所有图片分别归一化至32×32，64×64以及128×128三个尺度得到原始数据样本集。Through the above steps, after the image of the region of interest is obtained, the image of the region of interest is subjected to normalization processing, wherein the normalization processing can be realized by using the bicubic interpolation algorithm, and the bicubic interpolation is also called bicubic interpolation, which is the The most commonly used interpolation method. In this method, the value of the function f at the point (x, y) can be obtained by the weighted average of the nearest sixteen sampling points in the rectangular grid, where two polynomial interpolation cubic functions are required, and each direction is used One. The data is normalized, and all pictures are normalized to n scales to obtain the original data sample set. Wherein, n is a positive integer greater than or equal to 2. In this embodiment, n is preferably 3, the data is normalized, and all pictures are normalized to three scales of 32×32, 64×64 and 128×128 respectively to obtain the original data sample set.
S205、对原始数据样本集进行数据扩增处理，得到扩增数据样本集；S205. Perform data amplification processing on the original data sample set to obtain the amplified data sample set;
通过上述步骤，得到原始数据样本集后，需要对该原始数据样本集进行数据扩增处理，得到扩增数据样本集。其中，对原始数据样本集进行数据扩增处理包括对原始数据样本集中的所有图像进行左右翻转和上下翻转，从而得到诸多变换后的图像，将变换后的所有图像作为扩增数据样本集。Through the above steps, after the original data sample set is obtained, data amplification processing needs to be performed on the original data sample set to obtain the expanded data sample set. Wherein, performing data amplification processing on the original data sample set includes performing left-right flip and up-down flip on all images in the original data sample set, thereby obtaining many transformed images, and using all transformed images as the augmented data sample set.
S206、将原始数据样本集和扩增数据样本集进行合并，得到多尺度目标样本集；S206. Merge the original data sample set and the augmented data sample set to obtain a multi-scale target sample set;
通过上述步骤，得到原始数据样本集和扩增数据样本集之后，将原始数据样本集和扩增数据样本集进行合并，从而得到多尺度样本集，将多尺度样本集作为多尺度目标样本集。即多尺度目标样本集中包括原始数据样本集和扩增数据样本集中的所有图像。Through the above steps, after the original data sample set and the augmented data sample set are obtained, the original data sample set and the augmented data sample set are combined to obtain a multi-scale sample set, and the multi-scale sample set is used as a multi-scale target sample set. That is, the multi-scale target sample set includes all images in the original data sample set and the augmented data sample set.
S207、获取与每个尺度相对应的卷积神经网络；S207. Obtain a convolutional neural network corresponding to each scale;
通过上述步骤得知，将感兴趣区域图像归一化至n个尺度，因此，需要获取n个卷积神经网络，每个卷积神经网络与每个尺度相对应。Through the above steps, it is known that the image of the region of interest is normalized to n scales, therefore, it is necessary to obtain n convolutional neural networks, and each convolutional neural network corresponds to each scale.
卷积神经网络一般由卷积层、池化层、全连接层等组成。卷积层负责获得图像的局部特征，并将局部特征从网络中向后传递；池化层在空间维度上进行下采样，负责减小数据量；全连接层将会计算分类评分，损失函数会根据分类评分与目标的差距进行网络权重的调整。Convolutional neural networks are generally composed of convolutional layers, pooling layers, and fully connected layers. The convolutional layer is responsible for obtaining the local features of the image and passing the local features back from the network; the pooling layer performs downsampling in the spatial dimension and is responsible for reducing the amount of data; the fully connected layer will calculate the classification score, and the loss function will The network weights are adjusted according to the gap between the classification score and the target.
S208、将获取的n个卷积神经网络进行融合，得到目标集成卷积神经网络；S208. Fusing the obtained n convolutional neural networks to obtain a target integrated convolutional neural network;
通过上述步骤，获取n个卷积神经网络后，将n个独立的卷积神经网络进行融合，从而得到集成卷积神经网络，将所述集成卷积神经网络作为目标集成卷积神经网络。本实施例中n优选为3，表1为3个独立的卷积神经网络CNN结构，如表1所示：Through the above steps, after obtaining n convolutional neural networks, the n independent convolutional neural networks are fused to obtain an integrated convolutional neural network, and the integrated convolutional neural network is used as a target integrated convolutional neural network. In the present embodiment, n is preferably 3, and Table 1 is 3 independent convolutional neural network CNN structures, as shown in Table 1:
表1Table 1
其中，卷积神经网络CNN1共有1个输入层，2个卷积层，2个池化层，2个全连接层，1个输出层，其中两个卷积层的卷积核数量分别为8、16，池化采用最大池化策略；卷积神经网络CNN2共有1个输入层，3个卷积层，3个池化层，2个全连接层，1个输出层，其中三个卷积层的卷积核数量分别为8、16、32，池化采用最大池化策略；卷积神经网络CNN3共有1个输入层，4个卷积层，4个池化层，2个全连接层，1个输出层，其中四个卷积层的卷积核数量分别为8、16、32、64；池化采用最大池化策略。Among them, the convolutional neural network CNN1 has 1 input layer, 2 convolutional layers, 2 pooling layers, 2 fully connected layers, and 1 output layer, and the number of convolutional kernels in the two convolutional layers is 8. , 16, the pooling adopts the maximum pooling strategy; the convolutional neural network CNN2 has 1 input layer, 3 convolutional layers, 3 pooling layers, 2 fully connected layers, 1 output layer, and three convolutional layers The number of convolution kernels in each layer is 8, 16, and 32, and the pooling adopts the maximum pooling strategy; the convolutional neural network CNN3 has 1 input layer, 4 convolutional layers, 4 pooling layers, and 2 fully connected layers. , 1 output layer, the number of convolution kernels of the four convolution layers are 8, 16, 32, 64 respectively; the pooling adopts the maximum pooling strategy.
三个独立网络卷积核数量依次递增，因为随着网络深度增加，更多的卷积核可以提取到更为深层次的图像特征。所有卷积操作后接入批归一化Batch Normalization和随机失活Dropout，进一步提升模型的泛化能力。网络中采用非线性激活函数ReLU作为激活函数，相比传统的Sigmoid函数能够防止数据在两端产生饱和的现象，从而避免权重无法进行更新的状况。其中，Batch Normalization是神经网络训练的技巧，它不仅可以加快了模型的收敛速度，而且更重要的是在一定程度缓解了深层网络中“梯度弥散”的问题，从而使得训练深层网络模型更加容易和稳定。Dropout就是指在每个训练批次中，可以明显地减少过拟合现象，使模型泛化性更强。The number of three independent network convolution kernels increases sequentially, because as the network depth increases, more convolution kernels can extract deeper image features. After all convolution operations, batch normalization and random inactivation Dropout are connected to further improve the generalization ability of the model. The nonlinear activation function ReLU is used as the activation function in the network. Compared with the traditional Sigmoid function, it can prevent the data from being saturated at both ends, thereby avoiding the situation that the weight cannot be updated. Among them, Batch Normalization is a technique for neural network training. It can not only speed up the convergence speed of the model, but more importantly, it alleviates the problem of "gradient dispersion" in the deep network to a certain extent, thus making it easier and easier to train the deep network model. Stablize. Dropout means that in each training batch, the phenomenon of overfitting can be significantly reduced, making the model more generalizable.
S209、将多尺度目标样本集进行分类，得到训练样本集、验证样本集和测试样本集；S209. Classify the multi-scale target sample set to obtain a training sample set, a verification sample set, and a test sample set;
通过步骤S206得到多尺度目标样本集之后，对多尺度目标样本集进行分类，分为训练样本集、验证样本集和测试样本集三种，且训练样本集、验证样本集和测试样本集均为多尺度的样本集。After the multi-scale target sample set is obtained through step S206, the multi-scale target sample set is classified into three types: training sample set, verification sample set and test sample set, and the training sample set, verification sample set and test sample set are all Multi-scale sample set.
S210、利用训练样本集和验证样本集对目标集成卷积神经网络进行训练和验证，得到集成卷积神经网络模型；S210. Using the training sample set and the verification sample set to train and verify the target integrated convolutional neural network to obtain an integrated convolutional neural network model;
通过上述步骤，得到目标集成卷积神经网络、训练样本集和验证样本集之后，利用训练样本集对目标集成卷积神经网络进行训练，训练后再利用验证样本集对训练后的目标集成卷积神经网络进行验证，从而得到集成卷积神经网络模型，由于训练样本集和验证样本集均为多尺度的样本集，所以经过训练样本集训练以及验证样本集验证后得到的集成卷积神经网络模型为基于多尺度输入的集成卷积神经网络模型。Through the above steps, after obtaining the target integrated convolutional neural network, training sample set and verification sample set, use the training sample set to train the target integrated convolutional neural network, and then use the verification sample set to convolve the trained target integration The neural network is verified to obtain the integrated convolutional neural network model. Since the training sample set and the verification sample set are both multi-scale sample sets, the integrated convolutional neural network model obtained after training with the training sample set and verification of the verification sample set It is an integrated convolutional neural network model based on multi-scale input.
将肺结节图片进行多尺度输入，利用集成卷积神经网络检测识别可以挖掘到不同尺度下肺结节的图像特征和规律信息，这样可以最大可能地规避单输入单通道提取特征的不全面性导致的对识别结果的影响，从而达到提升检测准确率的目的。Multi-scale input of pulmonary nodule images, using integrated convolutional neural network detection and recognition can mine the image features and regular information of pulmonary nodules at different scales, so as to avoid the incompleteness of single-input single-channel extraction features to the greatest possible extent The resulting impact on the recognition results, so as to achieve the purpose of improving the detection accuracy.
其中，训练样本集是进行模型拟合的数据样本，直接参与了模型调参的过程；验证样本集是模型训练过程中单独留出的样本集，它可以用于调整模型的超参数和对模型的能力进行初步评估。在神经网络中，用验证数据集去寻找最优的网络深度，或者决定反向传播算法的停止点或者在神经网络中选择隐藏层神经元的数量。Among them, the training sample set is the data sample for model fitting, which directly participates in the process of model tuning; the verification sample set is a sample set set aside during the model training process, which can be used to adjust the hyperparameters of the model and optimize the model. capacity for an initial assessment. In neural networks, the validation data set is used to find the optimal network depth, or to decide the stopping point of the backpropagation algorithm or to choose the number of hidden layer neurons in the neural network.
对目标集成卷积神经网络进行训练和验证是采用分批训练的方式，一个轮次完成后会返回该轮次训练过程的损失值，损失函数再将损失率从网络中反向传播进行网络权重参数的调整，使得能够获得更低的损失率。验证与训练损失收敛后停止训练，并保存模型为.h5文件，作为最终训练结果，即基于多尺度输入的集成卷积神经网络模型，此处的集成卷积神经网络模型为集成学习模型。The training and verification of the target integrated convolutional neural network adopts the method of batch training. After a round is completed, the loss value of the training process of the round will be returned, and the loss function will then backpropagate the loss rate from the network for network weighting. The adjustment of the parameters makes it possible to obtain a lower loss rate. Stop training after the verification and training loss converge, and save the model as a .h5 file as the final training result, that is, the integrated convolutional neural network model based on multi-scale input, and the integrated convolutional neural network model here is an integrated learning model.
集成学习从字面意义上来理解就是集合多个学习器的机器学习方法，有时也被成为多分类器系统。与单一的学习模型相比，集成学习模型的可获得比单一学习模型显著优越的泛化性能。集成卷积神经网络将三种尺度的样本数据分别送入CNN1、CNN2和CNN3中进行检测，最后对三个网络的检测结果进行集成得到最终结果。集成学习理念中CNN1、CNN2和CNN3均为弱分类器，弱分类器集成强分类器的方法有求平均、投票和加权求和等。本实施例中选用投票器的方式进行集成，即只有三种分类结果都为1时最终输出结果才为1，否则结果为0。Literally, ensemble learning is a machine learning method that integrates multiple learners, and is sometimes called a multi-classifier system. Compared with a single learning model, the ensemble learning model can obtain significantly superior generalization performance than a single learning model. The integrated convolutional neural network sends the sample data of three scales to CNN1, CNN2 and CNN3 for detection, and finally integrates the detection results of the three networks to obtain the final result. In the concept of integrated learning, CNN1, CNN2, and CNN3 are all weak classifiers, and methods for integrating weak classifiers with strong classifiers include averaging, voting, and weighted summation. In this embodiment, the voter is selected for integration, that is, the final output result is 1 only when the three classification results are all 1, otherwise the result is 0.
S211、将测试样本集输入到集成卷积神经网络模型，得到测试结果；S211. Input the test sample set into the integrated convolutional neural network model to obtain the test result;
通过上述步骤，得到集成卷积神经网络模型和测试样本集之后，将测试样本集输入到集成卷积神经网络模型中，得到测试结果，即是否为肺结节。由于多尺度目标样本集包含n个尺度的图像，集成卷积神经网络模型为基于多尺度输入的集成卷积神经网络模型，所以，测试样本集中也包含n个尺度的图像，集成卷积神经网络模型中也包含n个与多尺度目标样本集中图像尺度对应的卷积神经网络，在测试样本集输入到集成卷积神经网络模型的过程中，测试样本集中的图像是根据尺度对应输入到相应的卷积神经网络中的，卷积神经网络模型包含的每个卷积神经网络都会输出一个识别结果，集成卷积神经网络模型中还包含一个投票机制，即集成卷积神经网络模型中包含的n个卷积神经网络都会输出一个识别结果，则会得到n个识别结果，只有n个识别结果都表示为肺结节时，该测试结果才为是肺结节，只要n个识别结果中有一个表示不是肺结节，则测试结果便为不是肺结节。这样做的原因在于对真结节做三次确认，保证强分类模型的准确性。Through the above steps, after the integrated convolutional neural network model and the test sample set are obtained, the test sample set is input into the integrated convolutional neural network model to obtain the test result, that is, whether it is a pulmonary nodule. Since the multi-scale target sample set contains images of n scales, the integrated convolutional neural network model is an integrated convolutional neural network model based on multi-scale input, so the test sample set also contains images of n scales, and the integrated convolutional neural network The model also contains n convolutional neural networks corresponding to the image scales in the multi-scale target sample set. During the process of inputting the test sample set to the integrated convolutional neural network model, the images in the test sample set are input to the corresponding corresponding scales. In the convolutional neural network, each convolutional neural network included in the convolutional neural network model will output a recognition result, and the integrated convolutional neural network model also includes a voting mechanism, that is, the n included in the integrated convolutional neural network model Each convolutional neural network will output a recognition result, and n recognition results will be obtained. Only when the n recognition results are all expressed as pulmonary nodules, the test result is a pulmonary nodule, as long as one of the n recognition results is means not a lung nodule, the test result is not a lung nodule. The reason for this is to confirm the true nodules three times to ensure the accuracy of the strong classification model.
测试样本集用来评估模最终模型的泛化能力，但不能作为调参、选择特征等算法相关的选择的依据。测试样本集是网络训练完成最终的检验数据，从测试集上网络的准确度可以看出这个模型的好坏。The test sample set is used to evaluate the generalization ability of the final model, but it cannot be used as the basis for algorithm-related choices such as parameter tuning and feature selection. The test sample set is the final test data after network training. The accuracy of the network on the test set can tell whether the model is good or not.
S212、根据测试结果，确定测试准确率；S212. Determine the test accuracy rate according to the test result;
通过上述步骤，得到测试结果后，将测试结果与正确结果进行对比，从而确定测试准确率。其中，测试准确率的计算方式为，测试结果与正确结果相同的样本数量除以测试样本集中样本的总数量。Through the above steps, after the test result is obtained, the test result is compared with the correct result to determine the test accuracy. Wherein, the calculation method of the test accuracy rate is that the number of samples whose test result is the same as the correct result is divided by the total number of samples in the test sample set.
S213、检测测试准确率是否大于预设准确率；S213. Check whether the test accuracy rate is greater than the preset accuracy rate;
通过上述步骤，得到测试准确率后，检测测试准确率是否大于预设准确率，其中预设准确率为预先设置的数值，如果测试准确率能够超过预设准确率，则把表示上述步骤中得到的集成卷积神经网络模型为准确率合格的模型。Through the above steps, after the test accuracy rate is obtained, check whether the test accuracy rate is greater than the preset accuracy rate, wherein the preset accuracy rate is a preset value, if the test accuracy rate can exceed the preset accuracy rate, then it means that obtained in the above steps The integrated convolutional neural network model is a model with a qualified accuracy rate.
S214、若测试准确率大于预设准确率，将集成卷积神经网络模型作为肺结节检测模型。S214. If the test accuracy rate is greater than the preset accuracy rate, use the integrated convolutional neural network model as a pulmonary nodule detection model.
通过上述步骤，如果检测到测试准确率大于预设准确率，则将集成卷积神经网络模型作为肺结节检测模型。如果，检测到测试准确率不大于预设准确率，则需要重新获取肺部图像和描述信息，并且调整集成卷积神经网络模型结构，从而继续对集成卷积神经网络模型进行训练和验证，直到测试准确率大于预设准确率后，再停止训练。Through the above steps, if it is detected that the test accuracy rate is greater than the preset accuracy rate, the integrated convolutional neural network model is used as the pulmonary nodule detection model. If it is detected that the test accuracy rate is not greater than the preset accuracy rate, it is necessary to re-acquire the lung image and description information, and adjust the structure of the integrated convolutional neural network model, so as to continue to train and verify the integrated convolutional neural network model until After the test accuracy is greater than the preset accuracy, stop the training.
图3是本发明的肺结节的检测模型构建方法中训练样本集的精度折线图；Fig. 3 is the accuracy line chart of training sample set in the detection model construction method of pulmonary nodule of the present invention;
图4是本发明的肺结节的检测模型构建方法中验证样本集的精度折线图。本实施例中，对目标集成卷积神经网络进行训练和验证过程中，训练样本集和验证样本集训练和验证的精度(即准确率)如图3和图4所示。其中，横坐标是模型训练时的epoch，一个epoch指代所有的数据送入网络中完成一次前向计算及反向传播的过程。由于一个epoch常常太大，计算机无法负荷，我们会将它分成几个较小的batches。batch就是每次送入网络中训练的一部分数据，而batch Size就是每个batch中训练样本的数量。iterations是完成一次epoch所需的batch个数。例如，有2000个数据，分成4个batch，那么batch size就是500。运行所有的数据进行训练，完成1个epoch，需要进行4次iterations。卷积神经网络训练前，可以自行设置batch size的大小和epoch的次数；纵坐标是准确率。Fig. 4 is a broken line graph of the accuracy of the verification sample set in the method for constructing the detection model of pulmonary nodules of the present invention. In this embodiment, in the process of training and verifying the target integrated convolutional neural network, the training and verification accuracy (ie accuracy rate) of the training sample set and the verification sample set are shown in Fig. 3 and Fig. 4 . Among them, the abscissa is the epoch during model training, and one epoch refers to the process of sending all the data into the network to complete a forward calculation and back propagation. Since an epoch is often too large for the computer to load, we divide it into several smaller batches. The batch is a part of the data sent to the network for training each time, and the batch size is the number of training samples in each batch. iterations is the number of batches required to complete an epoch. For example, if there are 2000 data, divided into 4 batches, then the batch size is 500. Run all the data for training, and to complete 1 epoch, 4 iterations are required. Before training the convolutional neural network, you can set the size of the batch size and the number of epochs; the vertical axis is the accuracy rate.
本实施例中，并不限制步骤S207-S208的执行顺序，步骤S207-S208可以在步骤S209之前的任一步骤间执行。In this embodiment, the execution sequence of steps S207-S208 is not limited, and steps S207-S208 may be executed between any steps before step S209.
本实施例的肺结节的检测模型构建方法，对预先获取的肺部图像进行图像增强处理，得到目标图像；对预先获取的肺部图像对应的描述信息进行分析，得到坐标信息；在目标图像上截取肺结节坐标信息对应的图像，得到正样本图像；在目标图像截取上非肺结节坐标信息对应的图像，得到负样本图像；将正样本图像和负样本图像组合，得到感兴趣区域图像；基于预设的n个不同尺度，对感兴趣区域图像进行归一化处理，得到具有n个尺度的原始数据样本集；对原始数据样本集进行数据扩增处理，得到扩增数据样本集；将多尺度原始数据样本集和扩增数据样本集进行合并，得到多尺度目标样本集；获取与每个尺度相对应的卷积神经网络；将获取的n个卷积神经网络进行融合，得到目标集成卷积神经网络；将多尺度目标样本集进行分类，得到训练样本集、验证样本集和测试样本集；利用训练样本集和验证样本集对目标集成卷积神经网络进行训练和验证，得到集成卷积神经网络模型；将测试样本集输入到集成卷积神经网络模型，得到测试结果；根据测试结果，确定测试准确率；检测测试准确率是否大于预设准确率；若是，将集成卷积神经网络模型作为肺结节检测模型。这样通过利用多尺度目标样本集对目标集成卷积神经网络进行训练、验证和测试，得到肺结节检测模型，卷积神经网络区分于传统机器学习人工提取特征，是通过卷积操作对图像特征进行深度学习从而实现自动提取特征的效果，深度学习卷积神经网络模型的内在需求就是大的数据量，适用于数据不断加入和更新的情况，从而减小误差，那么，通过本实施例构建的肺结节检测模型，便能够增强检测结果的准确性，提高诊断结果的准确率，并且本实施例中采用的目标样本集为多尺度样本集，采用的卷积神经网络为集成卷积神经网络，训练得到的肺结节检测模型为基于多尺度输入的集成卷积神经网络模型，将肺结节图片进行多尺度输入，利用集成卷积神经网络检测识别可以挖掘到不同尺度下肺结节的图像特征和规律信息，这样可以最大可能地规避单输入单通道提取特征的不全面性导致的对识别结果的影响，网络在最后判定时集成了三条网络的判定结果，能给尽可能避免错判误判的现象，从而使识别准确率更高。而且三条网络可以实现并行运算，在节约计算资源的同时提升了检测效率。The method for constructing a detection model of a pulmonary nodule in this embodiment performs image enhancement processing on a pre-acquired lung image to obtain a target image; analyzes the description information corresponding to the pre-acquired lung image to obtain coordinate information; Intercept the image corresponding to the lung nodule coordinate information to obtain the positive sample image; intercept the image corresponding to the non-pulmonary nodule coordinate information on the target image to obtain the negative sample image; combine the positive sample image and the negative sample image to obtain the region of interest Image; Based on the preset n different scales, normalize the image of the region of interest to obtain the original data sample set with n scales; perform data amplification processing on the original data sample set to obtain the expanded data sample set ;Merge the multi-scale original data sample set and the augmented data sample set to obtain a multi-scale target sample set; obtain the convolutional neural network corresponding to each scale; fuse the obtained n convolutional neural networks to obtain Target integrated convolutional neural network; classify the multi-scale target sample set to obtain training sample set, verification sample set and test sample set; use the training sample set and verification sample set to train and verify the target integrated convolutional neural network, and obtain Integrate the convolutional neural network model; input the test sample set into the integrated convolutional neural network model to obtain the test result; determine the test accuracy according to the test result; check whether the test accuracy is greater than the preset accuracy; if so, integrate the convolution The neural network model is used as a pulmonary nodule detection model. In this way, by using the multi-scale target sample set to train, verify and test the target integrated convolutional neural network, a pulmonary nodule detection model is obtained. The convolutional neural network is different from the traditional machine learning manual extraction of features. Carry out deep learning to achieve the effect of automatic feature extraction. The inherent requirement of the deep learning convolutional neural network model is a large amount of data, which is suitable for the situation where data is continuously added and updated, thereby reducing errors. Then, the model constructed by this embodiment The pulmonary nodule detection model can enhance the accuracy of the detection results and improve the accuracy of the diagnosis results, and the target sample set used in this embodiment is a multi-scale sample set, and the convolutional neural network used is an integrated convolutional neural network , the trained pulmonary nodule detection model is an integrated convolutional neural network model based on multi-scale input, and multi-scale input of pulmonary nodule pictures is used to detect and identify pulmonary nodules at different scales. Image features and regular information, which can avoid the impact on the recognition results caused by the incompleteness of single-input and single-channel extraction features. The network integrates the judgment results of the three networks in the final judgment, which can avoid misjudgments as much as possible. The phenomenon of misjudgment makes the recognition accuracy higher. Moreover, the three networks can realize parallel computing, which improves the detection efficiency while saving computing resources.
图5是本发明的肺结节的检测方法实施例的结构示意图。如图5所示，本实施例的肺结节的检测方法具体可以包括如下步骤：Fig. 5 is a schematic structural diagram of an embodiment of a method for detecting pulmonary nodules of the present invention. As shown in Figure 5, the detection method of the pulmonary nodule of the present embodiment may specifically include the following steps:
S301、获取待检测图像；S301. Obtain an image to be detected;
本实施例中，患者若想要检测肺部是否存在肺结节，肺结节的检测方法首先需要获取用户的待检测图像。其中，通过对患者的肺部图像进行形态学分割，将肺部图像中所有类似肺结节的图像均分割出来，作为待检测图像。另外，待检测图像也可以通过医生针对肺部图像进行人工标注，将肺部图像中所有类似肺结节的图像标注并分割出来。本实施例中，肺部图像优选为肺部CT图像，通过CT机拍摄。In this embodiment, if the patient wants to detect whether there are pulmonary nodules in the lungs, the method for detecting pulmonary nodules first needs to obtain the user's image to be detected. Wherein, by performing morphological segmentation on the patient's lung image, all images similar to lung nodules in the lung image are segmented as images to be detected. In addition, the image to be detected can also be manually marked by a doctor on the lung image, and all images similar to lung nodules in the lung image are marked and segmented. In this embodiment, the lung image is preferably a lung CT image, taken by a CT machine.
S302、基于肺结节检测模型，输入待检测图像，得到检测结果。S302. Based on the pulmonary nodule detection model, input the image to be detected, and obtain a detection result.
通过上述步骤，得到待检测图像后，将待检测图像输入到通过上述实施例得到的肺结节检测模型中，从而得到输出的检测结果，其中得到的检测结果包括表示是肺结节和表示不是肺结节。Through the above steps, after the image to be detected is obtained, the image to be detected is input into the pulmonary nodule detection model obtained by the above embodiment, so as to obtain the output detection result, wherein the obtained detection result includes indicating that it is a pulmonary nodule and indicating that it is not lung nodules.
本实施例的肺结节的检测方法，获取待检测图像；基于肺结节检测模型，输入待检测图像，得到检测结果。肺结节检测模型是利用目标样本集对目标卷积神经网络进行训练、验证和测试得到的，其中，卷积神经网络区分于传统机器学习人工提取特征，是通过卷积操作对图像特征进行深度学习从而实现自动提取特征的效果，深度学习卷积神经网络模型的内在需求就是大的数据量，适用于数据不断加入和更新的情况，从而减小误差，那么，本实施例的肺结节的检测方法通过上述实施例构建的肺结节检测模型对待检测图像进行检测，能够增强检测结果的准确性，提高诊断结果的准确率，并且本实施例中采用的肺结节检测模型为基于多尺度输入的集成卷积神经网络模型，将肺结节图片进行多尺度输入，利用集成卷积神经网络检测识别可以挖掘到不同尺度下肺结节的图像特征和规律信息，这样可以最大可能地规避单输入单通道提取特征的不全面性导致的对识别结果的影响，网络在最后判定时集成了三条网络的判定结果，能给尽可能避免错判误判的现象，从而使识别准确率更高。而且三条网络可以实现并行运算，在节约计算资源的同时提升了检测效率。The method for detecting a pulmonary nodule in this embodiment acquires an image to be detected; based on a pulmonary nodule detection model, the image to be detected is input to obtain a detection result. The pulmonary nodule detection model is obtained by using the target sample set to train, verify and test the target convolutional neural network. Among them, the convolutional neural network is different from the traditional machine learning manual extraction of features. learning so as to achieve the effect of automatic feature extraction, the inherent requirement of the deep learning convolutional neural network model is a large amount of data, which is suitable for the situation where data is continuously added and updated, thereby reducing errors. Then, the pulmonary nodules of this embodiment The detection method uses the pulmonary nodule detection model constructed in the above embodiment to detect the image to be detected, which can enhance the accuracy of the detection result and improve the accuracy of the diagnosis result, and the pulmonary nodule detection model adopted in this embodiment is based on the multi-scale The input integrated convolutional neural network model, multi-scale input of pulmonary nodule pictures, and the detection and recognition of integrated convolutional neural network can mine the image characteristics and regular information of pulmonary nodules at different scales, so as to avoid single The impact on the recognition results caused by the incompleteness of the input single-channel extraction features, the network integrates the judgment results of the three networks in the final judgment, which can avoid the phenomenon of misjudgment and misjudgment as much as possible, so that the recognition accuracy is higher. Moreover, the three networks can realize parallel computing, which improves the detection efficiency while saving computing resources.
为了更全面，对应于本发明实施例提供的肺结节的检测模型构建方法，本申请还提供了肺结节的检测模型构建装置。To be more comprehensive, corresponding to the method for constructing a detection model of pulmonary nodules provided in the embodiment of the present invention, the present application also provides a device for constructing a detection model of pulmonary nodules.
图6是本发明的肺结节的检测模型构建装置实施例一的结构示意图。如图6所示，本实施例的肺结节的检测模型构建装置包括：确定模块11、处理模块12和训练模块13。FIG. 6 is a structural schematic diagram of Embodiment 1 of the apparatus for constructing a detection model of pulmonary nodules according to the present invention. As shown in FIG. 6 , the apparatus for constructing a detection model of pulmonary nodules in this embodiment includes: a determination module 11 , a processing module 12 and a training module 13 .
确定模块11，用于根据预先获取的肺部图像和肺部图像对应的描述信息，确定感兴趣区域图像；A determining module 11, configured to determine the image of the region of interest according to the pre-acquired lung image and the description information corresponding to the lung image;
处理模块12，用于对感兴趣区域图像进行处理，得到多尺度目标样本集；A processing module 12, configured to process the image of the region of interest to obtain a multi-scale target sample set;
训练模块13，用于利用多尺度目标样本集，对预先设计的目标集成卷积神经网络进行训练，得到肺结节检测模型。The training module 13 is configured to use the multi-scale target sample set to train the pre-designed target integrated convolutional neural network to obtain a pulmonary nodule detection model.
本实施例的肺结节的检测模型构建模型，确定模块11根据预先获取的肺部图像和肺部图像对应的描述信息，确定感兴趣区域图像；处理模块12对感兴趣区域图像进行处理，得到多尺度目标样本集；训练模块13利用多尺度目标样本集，对预先设计的目标集成卷积神经网络进行训练，得到肺结节检测模型。这样通过利用多尺度目标样本集对目标集成卷积神经网络进行训练得到肺结节检测模型，卷积神经网络区分于传统机器学习人工提取特征，是通过卷积操作对图像特征进行深度学习从而实现自动提取特征的效果，深度学习卷积神经网络模型的内在需求就是大的数据量，适用于数据不断加入和更新的情况，从而减小误差，那么，通过本实施例构建的肺结节检测模型，便能够增强检测结果的准确性，提高诊断结果的准确率，且通过多尺度样本集训练集成卷积神经网络得到的模型是多尺度输入的集成卷积神经网络模型，能规避单输入单通道提取特征不全面性导致的对识别结果的影响，避免错判误判现象，提高识别准确率和检测效率。The detection model of the pulmonary nodule in the present embodiment constructs a model, and the determination module 11 determines the image of the region of interest according to the pre-acquired lung image and the description information corresponding to the lung image; the processing module 12 processes the image of the region of interest to obtain Multi-scale target sample set; the training module 13 uses the multi-scale target sample set to train the pre-designed target integrated convolutional neural network to obtain a pulmonary nodule detection model. In this way, the pulmonary nodule detection model is obtained by using the multi-scale target sample set to train the target integrated convolutional neural network. The convolutional neural network is different from the traditional machine learning manual extraction of features. It is realized by deep learning of image features through convolution operations. The effect of automatic feature extraction, the inherent requirement of the deep learning convolutional neural network model is a large amount of data, which is suitable for the situation where data is continuously added and updated, thereby reducing errors. Then, the pulmonary nodule detection model constructed by this embodiment , it can enhance the accuracy of detection results and improve the accuracy of diagnosis results, and the model obtained by training the integrated convolutional neural network through multi-scale sample sets is an integrated convolutional neural network model with multi-scale input, which can avoid single-input and single-channel The impact on the recognition results caused by the incompleteness of the extracted features can avoid misjudgment and misjudgment, and improve the recognition accuracy and detection efficiency.
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。Regarding the apparatus in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments related to the method, and will not be described in detail here.
图7是本发明的肺结节的检测模型构建装置实施例二的结构示意图。如图7所示，本实施例的肺结节的检测模型构建装置中的确定模块11具体包括：第一处理单元111、分析单元112和图像确定单元113，图像确定单元113包括：截取子单元1131和组合子单元1132。Fig. 7 is a schematic structural diagram of Embodiment 2 of the apparatus for constructing a detection model of pulmonary nodules of the present invention. As shown in FIG. 7 , the determination module 11 in the device for constructing a pulmonary nodule detection model in this embodiment specifically includes: a first processing unit 111, an analysis unit 112, and an image determination unit 113, and the image determination unit 113 includes: an interception subunit 1131 and combination subunit 1132.
第一处理单元111，用于对预先获取的肺部图像进行图像增强处理，得到目标图像；The first processing unit 111 is configured to perform image enhancement processing on the pre-acquired lung image to obtain a target image;
分析单元112，用于对预先获取的肺部图像对应的描述信息进行分析，得到坐标信息；The analysis unit 112 is configured to analyze the descriptive information corresponding to the pre-acquired lung image to obtain coordinate information;
图像确定单元113，用于根据目标图像和坐标信息，确定所述感兴趣区域图像。The image determining unit 113 is configured to determine the image of the region of interest according to the target image and coordinate information.
其中，坐标信息包括肺结节坐标信息和非肺结节坐标信息；图像确定单元113首先，在目标图像上截取肺结节坐标信息对应的图像，得到正样本图像；在目标图像上截取非肺结节坐标信息对应的图像，得到负样本图像；其次，将正样本图像和负样本图像组合，得到感兴趣区域图像。Wherein, the coordinate information includes lung nodule coordinate information and non-pulmonary nodule coordinate information; the image determination unit 113 first intercepts the image corresponding to the lung nodule coordinate information on the target image to obtain a positive sample image; intercepts the non-lung nodule coordinate information on the target image The image corresponding to the nodule coordinate information is obtained to obtain the negative sample image; secondly, the positive sample image and the negative sample image are combined to obtain the image of the region of interest.
进一步地，本实施例的处理模块12具体包括：第二处理单元121、第三处理单元122和合并单元123。Further, the processing module 12 of this embodiment specifically includes: a second processing unit 121 , a third processing unit 122 and a combining unit 123 .
第二处理单元121，用于基于预设的n个不同尺度，对感兴趣区域图像进行归一化处理，得到具有n个尺度的原始数据样本集，n为大于等于2的正整数；The second processing unit 121 is configured to perform normalization processing on the image of the region of interest based on preset n different scales to obtain an original data sample set with n scales, where n is a positive integer greater than or equal to 2;
第三处理单元122，用于对原始数据样本集进行数据扩增处理，得到扩增数据样本集；The third processing unit 122 is configured to perform data amplification processing on the original data sample set to obtain the expanded data sample set;
合并单元123，用于将原始数据样本集和扩增数据样本集进行合并，得到多尺度目标样本集。The merging unit 123 is configured to combine the original data sample set and the augmented data sample set to obtain a multi-scale target sample set.
进一步地，本实施例的肺结节的检测模型构建装置还包括神经网络获取模块14和融合模块15。Further, the device for constructing a detection model of pulmonary nodules in this embodiment also includes a neural network acquisition module 14 and a fusion module 15 .
神经网络获取模块14，用于获取与每个尺度相对应的卷积神经网络；A neural network acquisition module 14, configured to acquire a convolutional neural network corresponding to each scale;
融合模块15，用于将获取的n个卷积神经网络进行融合，得到目标集成卷积神经网络。The fusion module 15 is configured to fuse the acquired n convolutional neural networks to obtain a target integrated convolutional neural network.
进一步地，本实施例的训练模块13具体包括：样本分类单元131、模型处理单元132、模型测试单元133、第一确定单元134、检测单元135和第二确定单元136。Further, the training module 13 of this embodiment specifically includes: a sample classification unit 131 , a model processing unit 132 , a model testing unit 133 , a first determination unit 134 , a detection unit 135 and a second determination unit 136 .
样本分类单元131，用于将多尺度目标样本集进行分类，得到训练样本集、验证样本集和测试样本集；A sample classification unit 131, configured to classify the multi-scale target sample set to obtain a training sample set, a verification sample set and a test sample set;
模型处理单元132，用于利用训练样本集和验证样本集对目标集成卷积神经网络进行训练和验证，得到集成卷积神经网络模型；The model processing unit 132 is used to train and verify the target integrated convolutional neural network by using the training sample set and the verification sample set to obtain an integrated convolutional neural network model;
模型测试单元133，用于将测试样本集输入到集成卷积神经网络模型，得到测试结果；The model testing unit 133 is used to input the test sample set into the integrated convolutional neural network model to obtain test results;
第一确定单元134，用于根据测试结果，确定测试准确率；The first determination unit 134 is used to determine the test accuracy rate according to the test result;
检测单元135，用于检测测试准确率是否大于预设准确率；A detection unit 135, configured to detect whether the test accuracy rate is greater than the preset accuracy rate;
第二确定单元136，用于若测试准确率大于预设准确率，将集成卷积神经网络模型作为肺结节检测模型。The second determining unit 136 is configured to use the integrated convolutional neural network model as the pulmonary nodule detection model if the test accuracy rate is greater than the preset accuracy rate.
本实施例的肺结节的检测模型构建装置，通过第一处理单元111对预先获取的肺部图像进行图像增强处理，得到目标图像；通过分析单元112对预先获取的肺部图像对应的描述信息进行分析，得到坐标信息；通过图像确定单元113根据目标图像和坐标信息，得到感兴趣区域图像；通过第二处理单元121基于预设的n个不同尺度，对感兴趣区域图像进行归一化处理，得到具有n个尺度的原始数据样本集；通过第三处理单元122对原始数据样本集进行数据扩增处理，得到扩增数据样本集；通过合并单元123将原始数据样本集和扩增数据样本集进行合并，得到多尺度目标样本集；通过神经网络获取模块14获取与每个尺度相对应的卷积神经网络；通过融合模块15将获取的n个卷积神经网络进行融合，得到目标集成卷积神经网络；通过样本分类单元131将多尺度目标样本集进行分类，得到训练样本集、验证样本集和测试样本集；通过模型处理单元132利用训练样本集和验证样本集对目标集成卷积神经网络进行训练和验证，得到集成卷积神经网络模型；通过模型测试单元133将测试样本集输入到集成卷积神经网络模型，得到测试结果；通过第一确定单元134根据测试结果，确定测试准确率；通过检测单元135检测测试准确率是否大于预设准确率；若是，通过第三确定单元136将集成卷积神经网络模型作为肺结节检测模型。这样通过利用多尺度目标样本集对目标集成卷积神经网络进行训练、验证和测试，得到肺结节检测模型，卷积神经网络区分于传统机器学习人工提取特征，是通过卷积操作对图像特征进行深度学习从而实现自动提取特征的效果，深度学习卷积神经网络模型的内在需求就是大的数据量，适用于数据不断加入和更新的情况，从而减小误差，那么，通过本实施例构建的肺结节检测模型，便能够增强检测结果的准确性，提高诊断结果的准确率，并且本实施例中采用的目标样本集为多尺度样本集，采用的卷积神经网络为集成卷积神经网络，训练得到的肺结节检测模型为基于多尺度输入的集成卷积神经网络模型，将肺结节图片进行多尺度输入，利用集成卷积神经网络检测识别可以挖掘到不同尺度下肺结节的图像特征和规律信息，这样可以最大可能地规避单输入单通道提取特征的不全面性导致的对识别结果的影响，网络在最后判定时集成了三条网络的判定结果，能给尽可能避免错判误判的现象，从而使识别准确率更高。而且三条网络可以实现并行运算，在节约计算资源的同时提升了检测效率。In the device for constructing a detection model of pulmonary nodules in this embodiment, the first processing unit 111 performs image enhancement processing on the pre-acquired lung image to obtain the target image; the analysis unit 112 performs descriptive information corresponding to the pre-acquired lung image Perform analysis to obtain coordinate information; use the image determination unit 113 to obtain an image of the region of interest based on the target image and coordinate information; use the second processing unit 121 to perform normalization processing on the region of interest image based on preset n different scales , to obtain the original data sample set with n scales; the third processing unit 122 performs data amplification processing on the original data sample set to obtain the expanded data sample set; the original data sample set and the expanded data sample are combined by the merging unit 123 The multi-scale target sample set is obtained; the convolutional neural network corresponding to each scale is obtained through the neural network acquisition module 14; the obtained n convolutional neural networks are fused through the fusion module 15 to obtain the target integrated volume The multi-scale target sample set is classified by the sample classification unit 131 to obtain a training sample set, a verification sample set and a test sample set; the model processing unit 132 utilizes the training sample set and the verification sample set to integrate the convolution neural network of the target The network is trained and verified to obtain the integrated convolutional neural network model; the test sample set is input to the integrated convolutional neural network model by the model testing unit 133 to obtain the test result; according to the test result by the first determining unit 134, the test accuracy is determined ; use the detection unit 135 to detect whether the test accuracy rate is greater than the preset accuracy rate; if so, use the integrated convolutional neural network model as the pulmonary nodule detection model through the third determination unit 136 . In this way, by using the multi-scale target sample set to train, verify and test the target integrated convolutional neural network, a pulmonary nodule detection model is obtained. The convolutional neural network is different from the traditional machine learning manual extraction of features. Carry out deep learning to achieve the effect of automatic feature extraction. The inherent requirement of the deep learning convolutional neural network model is a large amount of data, which is suitable for the situation where data is continuously added and updated, thereby reducing errors. Then, the model constructed by this embodiment The pulmonary nodule detection model can enhance the accuracy of the detection results and improve the accuracy of the diagnosis results, and the target sample set used in this embodiment is a multi-scale sample set, and the convolutional neural network used is an integrated convolutional neural network , the trained pulmonary nodule detection model is an integrated convolutional neural network model based on multi-scale input, and the pulmonary nodule image is input at multiple scales, and the detection and recognition of the integrated convolutional neural network can be used to mine pulmonary nodules at different scales Image features and regular information, which can avoid the impact on the recognition results caused by the incompleteness of single-input and single-channel extraction features. The network integrates the judgment results of the three networks in the final judgment, which can avoid misjudgments as much as possible. The phenomenon of misjudgment makes the recognition accuracy higher. Moreover, the three networks can realize parallel computing, which improves the detection efficiency while saving computing resources.
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。Regarding the apparatus in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments related to the method, and will not be described in detail here.
图8是本发明的肺结节的检测装置实施例的结构示意图。如图8所示，本实施例的肺结节的检测装置包括图像获取模块21和检测模块22。Fig. 8 is a schematic structural diagram of an embodiment of a device for detecting pulmonary nodules of the present invention. As shown in FIG. 8 , the device for detecting pulmonary nodules in this embodiment includes an image acquisition module 21 and a detection module 22 .
图像获取模块21，用于获取待检测图像；An image acquisition module 21, configured to acquire an image to be detected;
检测模块22，用于基于肺结节检测模型，输入待检测图像，得到检测结果；肺结节检测模型通过上述实施例中肺结节的检测模型构建方法构建。The detection module 22 is configured to input the image to be detected based on the pulmonary nodule detection model, and obtain the detection result; the pulmonary nodule detection model is constructed by the method for constructing the pulmonary nodule detection model in the above-mentioned embodiment.
本实施例的肺结节的检测装置，通过图像获取模块21获取待检测图像；通过检测模块22基于肺结节检测模型，输入待检测图像，得到检测结果。肺结节检测模型是利用多尺度目标样本集对目标集成卷积神经网络进行训练、验证和测试得到的，其中，卷积神经网络区分于传统机器学习人工提取特征，是通过卷积操作对图像特征进行深度学习从而实现自动提取特征的效果，深度学习卷积神经网络模型的内在需求就是大的数据量，适用于数据不断加入和更新的情况，从而减小误差，那么，本实施例的肺结节的检测方法通过上述实施例构建的肺结节检测模型对待检测图像进行检测，能够增强检测结果的准确性，提高诊断结果的准确率，并且本实施例中采用的肺结节检测模型为基于多尺度输入的集成卷积神经网络模型，将肺结节图片进行多尺度输入，利用集成卷积神经网络检测识别可以挖掘到不同尺度下肺结节的图像特征和规律信息，这样可以最大可能地规避单输入单通道提取特征的不全面性导致的对识别结果的影响，网络在最后判定时集成了三条网络的判定结果，能给尽可能避免错判误判的现象，从而使识别准确率更高。而且三条网络可以实现并行运算，在节约计算资源的同时提升了检测效率。The device for detecting pulmonary nodules in this embodiment acquires the image to be detected through the image acquisition module 21; the detection module 22 inputs the image to be detected based on the pulmonary nodule detection model, and obtains the detection result. The pulmonary nodule detection model is obtained by using the multi-scale target sample set to train, verify and test the target integrated convolutional neural network. Among them, the convolutional neural network is different from the traditional machine learning manual feature extraction. The feature is deeply learned to achieve the effect of automatic feature extraction. The inherent requirement of the deep learning convolutional neural network model is a large amount of data, which is suitable for the situation where data is continuously added and updated, thereby reducing errors. Then, the lungs of this embodiment Nodule detection method The pulmonary nodule detection model constructed in the above embodiment is used to detect the image to be detected, which can enhance the accuracy of the detection result and improve the accuracy of the diagnosis result, and the pulmonary nodule detection model used in this embodiment is The integrated convolutional neural network model based on multi-scale input, multi-scale input of pulmonary nodule pictures, and the use of integrated convolutional neural network detection and recognition can mine the image characteristics and regular information of pulmonary nodules at different scales, so that the maximum possible To avoid the impact on the recognition results caused by the incompleteness of single-input and single-channel extraction features, the network integrates the judgment results of three networks in the final judgment, which can avoid the phenomenon of misjudgment and misjudgment as much as possible, so that the recognition accuracy rate higher. Moreover, the three networks can realize parallel computing, which improves the detection efficiency while saving computing resources.
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。Regarding the apparatus in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments related to the method, and will not be described in detail here.
可以理解的是，上述各实施例中相同或相似部分可以相互参考，在一些实施例中未详细说明的内容可以参见其他实施例中相同或相似的内容。It can be understood that, the same or similar parts in the above embodiments can be referred to each other, and the content that is not described in detail in some embodiments can be referred to the same or similar content in other embodiments.
需要说明的是，在本发明的描述中，术语“第一”、“第二”等仅用于描述目的，而不能理解为指示或暗示相对重要性。此外，在本发明的描述中，除非另有说明，“多个”的含义是指至少两个。It should be noted that, in the description of the present invention, the terms "first", "second" and so on are only used for description purposes, and should not be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" means at least two.
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本发明的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本发明的实施例所属技术领域的技术人员所理解。Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, which shall It is understood by those skilled in the art to which the embodiments of the present invention pertain.
应当理解，本发明的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现，和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques known in the art: Discrete logic circuits, ASICs with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be executed when the program is executed. When, one or a combination of the steps of the method embodiment is included.
此外，在本发明各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
上述提到的存储介质可以是只读存储器，磁盘或光盘等。The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like.
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
尽管上面已经示出和描述了本发明的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本发明的限制，本领域的普通技术人员在本发明的范围内可以对上述实施例进行变化、修改、替换和变型。Although the embodiments of the present invention have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Embodiments are subject to variations, modifications, substitutions and variations.