CN115018957A - Hyperspectral image virtual staining method for unstained section - Google Patents

The invention relates to a virtual staining method for a hyperspectral image of an unstained section, and aims to solve the technical problems that the existing virtual staining method for the unstained section is insufficient in texture information expression and inaccurate in virtual staining. The method comprises the following steps: 1. acquiring and storing a plurality of groups of hyperspectral original data; 2. preprocessing data to obtain hyperspectral data; 3. forming an experimental data set by the hyperspectral data of the unstained section, and selecting k clustering centers according to the experimental data set to ensure that the color similarity of the primary pseudo-color image under the k value and the hyperspectral image of the stained section is highest; 4. obtaining an optimal preliminary pseudo-color image; 5. obtaining a standard color ratio; 6. and synthesizing the final pseudo-color image of the unstained section of the tissue sample to be detected with the single spectral band image of the microscopic tissue to obtain a hyperspectral virtual staining image of the unstained section of the tissue sample to be detected. The method provided by the invention is beneficial to extracting more accurate hyperspectral data structure texture information.

一种未染色切片的高光谱图像虚拟染色方法A virtual staining method for hyperspectral images of unstained sections

技术领域technical field

本发明涉及组织切片的标记方法，具体涉及一种未染色切片的高光谱图像虚拟染色方法。The invention relates to a method for labeling tissue slices, in particular to a method for virtual staining of hyperspectral images of unstained slices.

背景技术Background technique

常规染色方法是苏木精-伊红(H-E)染色及免疫组织化学(IHC)染色，如图1所示为现有H-E染色显微结构示意图，H-E染色是最基础的染色方式，主要利用细胞内不同区域对酸碱性染料的不同着色(细胞核显紫蓝色，细胞质显粉红色)来体现组织结构。如图2所示为现有IHC染色显微结构示意图，IHC通常被描述为“分子病理学”方法，因为它减少了对形态特征的依赖，而是使用化学成分的变化(某些抗体标记分子的存在或过度表达)来作为区分正常组织与癌变组织的主要手段。IHC具有高灵敏度和特异性，但仅对所使用抗体的标记能够产生检测信号，而不能检测未知的癌变标记。相比而言，未染色切片的显微图像通常没有丰富颜色，且缺少细节纹理，因此在目前的组织病理学研究中，染色过程仍然是必不可少的，显微组织结构需要通过化学染色来突出重要的诊断信息。The conventional staining methods are hematoxylin-eosin (H-E) staining and immunohistochemical (IHC) staining. Figure 1 shows the schematic diagram of the existing H-E staining microstructure. H-E staining is the most basic staining method, mainly using cells. Different coloration of acid-base dyes in different regions (nucleus is purplish blue, cytoplasm is pink) to reflect tissue structure. Figure 2 shows a schematic representation of the microstructure of existing IHC staining. IHC is often described as a "molecular pathology" method because it reduces the reliance on morphological features and instead uses changes in chemical composition (certain antibodies to label molecules) presence or overexpression) as the main means to distinguish normal tissue from cancerous tissue. IHC has high sensitivity and specificity, but can only generate a detection signal for the label of the antibody used, and cannot detect unknown cancerous markers. In contrast, microscopic images of unstained sections are usually not rich in color and lack detailed texture, so in current histopathological studies, the staining process is still essential, and the microstructure needs to be chemically stained. Highlight important diagnostic information.

在切片染色过程中，由于实验仪器差异，不同化学配剂的差别以及人为操作的影响，同一组织切片可能会产生不同的染色效果，在进行自动化的图像分析时，这类切片外观的变化确实给高效模型的建立带来了极大挑战。此外，染色过程通常比较耗时，如果癌症类别较难确定的话，还需要进行多次、多种类的染色，对于快速判断癌症类别是极为不利的，可能会影响到患者的最佳治疗时间。尤其是对于术中的冰冻切片而言，手术时间非常宝贵，冰冻切片的染色和诊断效率就显得更为关键。In the process of section staining, the same tissue section may have different staining effects due to differences in experimental instruments, differences in chemical formulations, and the influence of human operations. During automated image analysis, changes in the appearance of such sections do give The establishment of efficient models brings great challenges. In addition, the staining process is usually time-consuming. If the type of cancer is difficult to determine, multiple stainings of multiple types are required, which is extremely unfavorable for quickly judging the type of cancer and may affect the optimal treatment time for patients. Especially for intraoperative frozen sections, the operation time is very precious, and the staining and diagnostic efficiency of frozen sections are even more critical.

目前，部分研究成果已证明了多光谱技术应用于未染色样本研究的可行性，例如，Irshad等人在Multi-channels statistical and morphological features basedmitosis detection in breast cancer histopathology[J].ConfProc IEEE Eng MedBiolSoc,2013,2013(2013):6091-6094.中公开了利用多光谱成像自动检测乳腺癌病理图像中的有丝分裂，证明了光谱信息与H-E染色特征之间的相关性；Kopriva等人在Unsupervised decomposition of low-intensity low-dimensional multi-spectralfluorescent images for tumour demarcation[J].Medical Image Analysis，2009，13(3):507-518.公开了采用多光谱技术对未染色的癌变显微图像进行了分割，通过对多通道图像的非线性映射，将其转换为光谱表达，这种非线性映射也是虚拟染色的一种形式，但是，多光谱技术存在对纹理信息描述不足，虚拟染色不精确的问题。At present, some research results have proved the feasibility of applying multispectral technology to the study of unstained samples, for example, Irshad et al. in Multi-channels statistical and morphological features basedmitosis detection in breast cancer histopathology[J].ConfProc IEEE Eng MedBiolSoc,2013 , 2013 (2013): 6091-6094. The use of multispectral imaging to automatically detect mitosis in breast cancer pathology images was disclosed, demonstrating the correlation between spectral information and H-E staining features; Kopriva et al. in Unsupervised decomposition of low- intensity low-dimensional multi-spectralfluorescent images for tumor demarcation[J]. Medical Image Analysis, 2009, 13(3): 507-518. It is disclosed that the unstained cancerous microscopic images were segmented by multi-spectral technology. The nonlinear mapping of multi-channel images is converted into spectral representation. This nonlinear mapping is also a form of virtual dyeing. However, multi-spectral technology has the problems of insufficient description of texture information and imprecise virtual dyeing.

发明内容SUMMARY OF THE INVENTION

本发明目的在于解决目前未染色切片虚拟染色方法纹理信息表述不足，虚拟染色不精确的技术问题，提出一种未染色切片的高光谱图像虚拟染色方法，旨在借助显微高光谱技术，采用光谱聚类的方法对不同组织结构进行光谱标记，结合单谱段的显著形态学信息，实现对未染色切片的高光谱虚拟染色，代替或者模拟染色过程的方法，同时不影响诊断效率和精度。The purpose of the invention is to solve the technical problems of insufficient representation of texture information and imprecise virtual dyeing in the current virtual dyeing method for unstained sections, and proposes a virtual dyeing method for hyperspectral images of unstained sections, aiming at using microscopic hyperspectral technology and using spectral The clustering method performs spectral labeling of different tissue structures, and combines the significant morphological information of a single spectral segment to realize hyperspectral virtual staining of unstained sections, replacing or simulating the method of the staining process, without affecting the diagnostic efficiency and accuracy.

本发明的技术方案为：The technical scheme of the present invention is:

一种未染色切片的高光谱图像虚拟染色方法，其特殊之处在于，包括以下步骤：A hyperspectral image virtual staining method for unstained sections, which is special in that it includes the following steps:

S1、采用显微高光谱成像技术采集组织样本的多组高光谱原始数据，并将高光谱原始数据进行存储；各组高光谱原始数据均包括同一组织样本的未染色切片高光谱原始数据和染色切片高光谱原始数据；S1. Use microscopic hyperspectral imaging technology to collect multiple sets of hyperspectral raw data of tissue samples, and store the hyperspectral raw data; each group of hyperspectral raw data includes unstained sections of the same tissue sample. Slice hyperspectral raw data;

S2、采用平滑去噪方式对高光谱原始数据进行图像维的数据预处理，得到高光谱数据，高光谱数据包括未染色切片高光谱数据和染色切片高光谱数据，根据染色切片高光谱数据得到染色切片高光谱图像；S2. Perform image-dimensional data preprocessing on the hyperspectral raw data by smoothing and denoising to obtain hyperspectral data. The hyperspectral data includes hyperspectral data of unstained sections and hyperspectral data of dyed sections, and dyed sections are obtained according to the hyperspectral data of dyed sections. Slice hyperspectral images;

S3、未染色切片高光谱数据构成实验数据集，根据实验数据集选择k个聚类中心，使得k值下的初步伪彩色图像与染色切片高光谱图像的颜色相似度最高；S3. The hyperspectral data of the unstained section constitutes an experimental data set, and k cluster centers are selected according to the experimental data set, so that the color similarity between the preliminary pseudo-color image and the hyperspectral image of the stained section under the k value is the highest;

S4、根据步骤S3选择的k值，采用光谱聚类法对未染色切片高光谱数据的各个像素点的光谱特征进行划分，根据划分结果对各个像素点进行标记，得到未染色切片高光谱数据光谱标记后的最优初步伪彩色图像；S4, according to the k value selected in step S3, adopt the spectral clustering method to divide the spectral characteristics of each pixel point of the hyperspectral data of the unstained slice, and mark each pixel point according to the division result to obtain the spectrum of the hyperspectral data of the unstained slice The optimal preliminary pseudocolor image after labeling;

S5、根据颜色相似性，将最优初步伪彩色图像与染色切片高光谱图像进行比较，获取标准颜色配比，根据标准颜色配比调节未染色切片中各类组织的颜色配比，得到最终伪彩色图像；S5. According to the color similarity, compare the optimal preliminary pseudo-color image with the hyperspectral image of the stained section to obtain a standard color ratio, and adjust the color ratio of various tissues in the unstained section according to the standard color ratio to obtain the final pseudo-color ratio. color image;

S6、采集待测组织样本的未染色切片高光谱原始数据，采用平滑去噪方式预处理后，得到待测组织样本未染色切片高光谱数据；S6, collecting the hyperspectral raw data of the unstained section of the tissue sample to be tested, and after preprocessing with a smoothing and denoising method, the hyperspectral data of the unstained section of the tissue sample to be tested is obtained;

根据步骤S3选择的k值，采用光谱聚类法处理待测组织样本未染色切片高光谱数据，并根据步骤S5中标准颜色配比，调节待测组织样本未染色切片中各类组织的颜色配比，得到待测组织样本未染色切片的最终伪彩色图像；According to the k value selected in step S3, the spectral clustering method is used to process the hyperspectral data of the unstained section of the tissue sample to be tested, and according to the standard color ratio in step S5, the color matching of various tissues in the unstained section of the tissue sample to be tested is adjusted. ratio to obtain the final pseudo-color image of the unstained section of the tissue sample to be tested;

采用两次层叠的主成分变换方法提取待测组织样本未染色切片高光谱数据的结构纹理信息，得到显微组织单谱段图像；将显微组织单谱段图像与待测组织样本未染色切片的最终伪彩色图像合成，得到待测组织样本未染色切片的高光谱虚拟染色图像。The structure and texture information of the hyperspectral data of the unstained section of the tissue sample to be tested is extracted by the principal component transformation method of two layers, and the single-spectrum image of the microstructure is obtained. The final pseudo-color image is synthesized to obtain a hyperspectral virtual stained image of the unstained section of the tissue sample to be tested.

进一步地，步骤S3中，选择k个聚类中心，使得k值下的初步伪彩色图像与染色切片高光谱图像的颜色相似度最高，具体过程为：Further, in step S3, k cluster centers are selected, so that the color similarity between the preliminary pseudo-color image under the k value and the hyperspectral image of the stained section is the highest, and the specific process is as follows:

S3.1、从实验数据集中随机选取一个聚类中心作为初始聚类中心c₁，计算每个坐标样本点x_i与当前聚类中心之间的最短距离D(x)，然后计算x_i被选为下一个聚类中心的概率P_i，计算公式为S3.1. Randomly select a cluster center from the experimental data set as the initial cluster center c ₁ , calculate the shortest distance D(x) between each coordinate sample point x _i and the current cluster center, and then calculate that x _i is is selected as the probability P _i of the next cluster center, the calculation formula is

其中，i为第i个样本点，c_i为第i个样本点是聚类中心；Among them, i is the ith sample point, and c _i is the ith sample point, which is the cluster center;

选择概率最大的样本点作为下一个聚类中心；Select the sample point with the highest probability as the next cluster center;

S3.2、重复步骤S3.1，直到选择出k个聚类中心，k为整数且2≤k≤8，得到初步伪彩色图像；S3.2. Repeat step S3.1 until k cluster centers are selected, where k is an integer and 2≤k≤8, to obtain a preliminary pseudo-color image;

S3.3、取不同的k值，重复步骤S3.1-步骤S3.2，得到各个k值下的初步伪彩色图像，将各个初步伪彩色图像与染色切片高光谱图像比较颜色相似度，选出相似度最高的初步伪彩色图像对应的k值。S3.3. Take different k values, repeat steps S3.1 to S3.2 to obtain preliminary pseudo-color images under each k value, compare the color similarity between each preliminary pseudo-color image and the hyperspectral image of the stained section, and select The k value corresponding to the preliminary pseudo-color image with the highest similarity is obtained.

进一步地，步骤S1中，采集组织样本具体为，调节显微高光谱成像设备的调焦机构，使组织样本处于观察位置进行高光谱原始数据采集，之后保持放大倍数与光源强度不变，移动切片或更换其他切片，采集高光谱原始数据。Further, in step S1, collecting the tissue sample is specifically: adjusting the focusing mechanism of the microscopic hyperspectral imaging device so that the tissue sample is in the observation position to collect hyperspectral raw data, and then keeping the magnification and the intensity of the light source unchanged, moving the slice. Or replace other slices and collect hyperspectral raw data.

进一步地，步骤S1中，同一组织样本的未染色切片高光谱原始数据和染色切片高光谱原始数据的获取方式为：同一组织样本切下相邻的两个切片，一个按照正常流程进行染色和封片作为染色切片，采集得到的数据为染色切片高光谱原始数据；另一个直接封片作为未染色切片，采集得到的数据为未染色切片高光谱原始数据。Further, in step S1, the acquisition method of the hyperspectral raw data of the unstained section and the hyperspectral raw data of the stained section of the same tissue sample is as follows: two adjacent sections are cut from the same tissue sample, and one is stained and sealed according to the normal process. The slices were used as stained slices, and the collected data were the hyperspectral raw data of the stained slices; the other directly mounted slides were used as the unstained slices, and the collected data were the original hyperspectral data of the unstained slices.

进一步地，步骤S6中，所述两次层叠的主成分变换方法包括第一次变换和第二次变换；Further, in step S6, the two-layered principal component transformation method includes a first transformation and a second transformation;

所述第一次变换基于未染色切片高光谱数据中不同谱段的噪声协方差矩阵分离噪声，用于使变换后的噪声数据只有最小方差，且没有谱段之间的相关性；The first transformation separates noise based on the noise covariance matrix of different spectral segments in the hyperspectral data of unstained slices, so that the transformed noise data has only minimum variance and no correlation between spectral segments;

所述第二次变换是对噪声数据的标准主成分变换以去除噪声数据，得到未染色切片高光谱数据的特征图像，用于提取未染色切片高光谱数据的特征图像中的高光谱数据的结构纹理信息。The second transformation is a standard principal component transformation of the noise data to remove the noise data to obtain a characteristic image of the hyperspectral data of the unstained slice, which is used to extract the structure of the hyperspectral data in the characteristic image of the hyperspectral data of the unstained slice texture information.

本发明的有益效果：Beneficial effects of the present invention:

1、本发明提出高光谱虚拟染色方法，依靠显微高光谱技术实现了与传统H-E染色类似的效果，可以辅助进行病理判断，同时为无标记病理诊断的研究带来了新思路。1. The present invention proposes a hyperspectral virtual staining method, which relies on microscopic hyperspectral technology to achieve a similar effect to traditional H-E staining, which can assist in pathological judgment, and at the same time brings new ideas for the study of label-free pathological diagnosis.

2、本发明通过对选择最佳的聚类中心个数，使用最佳的聚类中心个数采用光谱聚类法对高光谱原始数据处理，得到最优初步伪彩色图像，提高了高光谱虚拟染色的精确性。2. In the present invention, the optimal number of cluster centers is selected, and the optimal number of cluster centers is used to process the hyperspectral raw data by using the spectral clustering method to obtain the optimal preliminary pseudo-color image, which improves the hyperspectral virtual image. Dyeing Accuracy.

3、本发明在两次层叠的主成分变换中，通过信号与噪声分离，使重要信息集中在有限的特征集中，光谱特征向类特征向量汇集，某些微弱信号会在去噪过程中得到增强，有利于提取到更准确的高光谱数据结构纹理信息。3. In the two layered principal component transformations, the present invention separates the signal from the noise, so that the important information is concentrated in a limited feature set, the spectral features are gathered into the class feature vector, and some weak signals will be enhanced during the denoising process. , which is beneficial to extract more accurate hyperspectral data structure texture information.

4、本发明提供的方法可以加快组织样本处理时间，有效降低成本，同时防止染色过程对样本的不良影响，增加病理性判断的客观性。4. The method provided by the present invention can speed up the processing time of the tissue sample, effectively reduce the cost, and at the same time prevent the adverse effect of the staining process on the sample, and increase the objectivity of the pathological judgment.

5、本发明提供的方法不仅适用于未染色切片，还可应用于手术中的冰冻切片，可以提高冰冻切片的染色和诊断效率。5. The method provided by the present invention is not only applicable to unstained sections, but also to frozen sections in surgery, which can improve the staining and diagnostic efficiency of frozen sections.

附图说明Description of drawings

图1为现有H-E染色显微结构示意图；Fig. 1 is a schematic diagram of the existing H-E staining microstructure;

图2为现有IHC染色显微结构示意图；Fig. 2 is the schematic diagram of existing IHC staining microstructure;

图3为本发明实施例中未染色胃组织切片的10×高光谱伪彩色图像的黑白图；3 is a black and white diagram of a 10× hyperspectral pseudocolor image of an unstained gastric tissue section in the embodiment of the present invention;

图4为本发明实施例中未染色胃组织切片的4×高光谱伪彩色图像的黑白图；4 is a black and white diagram of a 4× hyperspectral pseudo-color image of an unstained gastric tissue section in an embodiment of the present invention;

图5为本发明实施例中未染色胃组织切片的20×高光谱伪彩色图像的黑白图；5 is a black-and-white image of a 20× hyperspectral pseudo-color image of an unstained gastric tissue section in the embodiment of the present invention;

图6为本发明实施例中第一胃癌样本染色切片的高光谱伪彩色图像的黑白图(A)与未染色切片的高光谱虚拟染色图像的黑白图(B)对比示意图；6 is a schematic diagram showing the comparison of a black-and-white image (A) of a hyperspectral pseudo-color image of a stained section of the first gastric cancer sample and a black-and-white image (B) of an unstained section of a hyperspectral pseudo-colored image in the embodiment of the present invention;

图7为本发明实施例中第二胃癌样本染色切片的高光谱伪彩色图像的黑白图(C)与未染色切片的高光谱虚拟染色图像的黑白图(D)对比示意图；7 is a schematic diagram showing the comparison of a black-and-white image (C) of a hyperspectral pseudo-color image of a stained section of a second gastric cancer sample and a black-and-white image (D) of an unstained section of a hyperspectral pseudo-colored image in the embodiment of the present invention;

图8为本发明实施例中第三胃癌样本染色切片的高光谱伪彩色图像的黑白图(A)与未染色切片的高光谱虚拟染色图像的黑白图(B)对比示意图；8 is a schematic diagram showing the comparison between a black-and-white image (A) of a hyperspectral pseudo-color image of a stained section of a third gastric cancer sample and a black-and-white image (B) of an unstained section of the hyperspectral pseudo-colored image in the embodiment of the present invention;

图9为本发明实施例中第四胃癌样本染色切片的高光谱伪彩色图像的黑白图(C)与未染色切片的高光谱虚拟染色图像的黑白图(D)对比示意图；9 is a schematic diagram showing the comparison of a black-and-white image (C) of a hyperspectral pseudo-color image of a stained section of a fourth gastric cancer sample and a black-and-white image (D) of an unstained section of a hyperspectral pseudo-colored image in the embodiment of the present invention;

图10为本发明实施例中大网膜转移性腺癌第一样本的最终伪彩色图像的黑白图(A)、显微组织单谱段图像(B)与高光谱虚拟染色图像的黑白图(C)对比示意图；10 is a black and white image of the final pseudo-color image of the first sample of metastatic adenocarcinoma of the greater omentum (A), a single-segment image of the microstructure (B), and a black and white image of a hyperspectral virtual staining image in the embodiment of the present invention ( C) Comparative schematic diagram;

图11为本发明实施例中大网膜转移性腺癌第二样本的最终伪彩色图像的黑白图(D)、显微组织单谱段图像(E)与高光谱虚拟染色图像的黑白图(F)对比示意图；11 is a black and white image (D) of a final pseudo-color image of a second sample of metastatic adenocarcinoma of the greater omentum, a black and white image of a single-segment image of the microstructure (E), and a hyperspectral virtual staining image (F) in the embodiment of the present invention ) comparison diagram;

图12为本发明实施例中高光谱虚拟染色图像的黑白图(B)与H-E染色图像的黑白图(A)对比示意图。FIG. 12 is a schematic diagram comparing the black and white image (B) of the hyperspectral virtual stained image and the black and white image (A) of the H-E stained image in the embodiment of the present invention.

具体实施方式Detailed ways

本实施例提供一种未染色切片的高光谱图像虚拟染色方法，该方法包括以下步骤：This embodiment provides a hyperspectral image virtual staining method for unstained sections, and the method includes the following steps:

S1、采用显微高光谱成像技术采集胃组织样本的多组高光谱原始数据，并将高光谱原始数据进行存储；各组高光谱原始数据均包括同一胃组织样本的未染色切片高光谱原始数据和染色切片高光谱原始数据。S1. Use microscopic hyperspectral imaging technology to collect multiple sets of hyperspectral raw data of gastric tissue samples, and store the hyperspectral raw data; each group of hyperspectral raw data includes the hyperspectral raw data of unstained sections of the same gastric tissue sample and stained section hyperspectral raw data.

具体的，采集胃组织样本为调节显微高光谱成像设备的调焦机构，使胃组织样本处于观察位置进行高光谱原始数据采集，之后保持放大倍数与光源强度不变，移动切片或更换其他切片，采集高光谱原始数据；采集的数据保存为596×716×256的数据立方体；同一胃组织样本的未染色切片高光谱原始数据和染色切片高光谱原始数据的获取方式为：同一胃组织样本切下相邻的两个切片，一个按照正常流程进行染色和封片作为染色切片，采集得到的数据为染色切片高光谱原始数据；另一个直接封片作为未染色切片，采集得到的数据为未染色切片高光谱原始数据。Specifically, collecting the gastric tissue sample is to adjust the focusing mechanism of the microscopic hyperspectral imaging device, so that the gastric tissue sample is in the observation position to collect hyperspectral raw data, and then keep the magnification and the intensity of the light source unchanged, move the slice or replace other slices , collect hyperspectral raw data; the collected data is saved as a data cube of 596 × 716 × 256; the hyperspectral raw data of unstained sections and the hyperspectral raw data of stained sections of the same gastric tissue sample are obtained by: Two adjacent sections below, one is stained and mounted as a stained section according to the normal process, and the collected data is the hyperspectral raw data of the stained section; the other is directly mounted as an unstained section, and the collected data is unstained Slice hyperspectral raw data.

本实施例的试验样本包括胃癌组织术中冰冻切片(来自解放军301医院)和胃癌组织未染色切片(来自广州番禺区第一医院)，采集参见图3至图5，未染色胃组织切片的4×、10×及20×高光谱伪彩色图像，从图中可以明显看出，相比于染色切片，未染色切片缺乏丰富的色彩信息和结构纹理信息，对于未染色切片，无法获取足够的细胞和组织的形态学特征，因此难以直接进行诊断。The test samples in this example include intraoperative frozen sections of gastric cancer tissue (from PLA 301 Hospital) and unstained sections of gastric cancer tissue (from the First Hospital of Panyu District, Guangzhou). ×, 10× and 20× hyperspectral pseudo-color images, it can be clearly seen from the figure that compared with the stained sections, the unstained sections lack rich color information and structural texture information, and for the unstained sections, enough cells cannot be obtained. and morphological features of the tissue, making it difficult to make a direct diagnosis.

S2、采用平滑去噪方式对高光谱原始数据进行图像维的数据预处理，得到高光谱数据，高光谱数据包括未染色切片高光谱数据和染色切片高光谱数据，根据染色切片高光谱数据得到染色切片高光谱图像。S2. Perform image-dimensional data preprocessing on the hyperspectral raw data by smoothing and denoising to obtain hyperspectral data. The hyperspectral data includes hyperspectral data of unstained sections and hyperspectral data of dyed sections, and dyed sections are obtained according to the hyperspectral data of dyed sections. Slice hyperspectral images.

S3、未染色切片高光谱数据构成实验数据集，选择k个聚类中心，使得k值下的初步伪彩色图像与染色切片高光谱图像的颜色相似度最高；具体的为，S3. The hyperspectral data of the unstained section constitutes the experimental data set, and k cluster centers are selected, so that the initial pseudo-color image under the k value has the highest color similarity with the hyperspectral image of the stained section; specifically,

S3.1、从实验数据集中随机选取一个聚类中心作为初始聚类中心c₁；计算实验数据集中的每个坐标样本点x_i与当前已有的聚类中心之间的最短距离D(x)，然后计算x_i被选为下一个聚类中心的概率P_i，计算公式为S3.1. Randomly select a cluster center from the experimental data set as the initial cluster center c ₁ ; calculate the shortest distance D(x) between each coordinate sample point x _i in the experimental data set and the currently existing cluster center ), and then calculate the probability P _i that x _i is selected as the next cluster center, the calculation formula is

其中，i为第i个样本点，c_i为第i个样本点是聚类中心；Among them, i is the ith sample point, and c _i is the ith sample point, which is the cluster center;

选择概率最大的样本点作为下一个聚类中心。Select the sample point with the highest probability as the next cluster center.

S3.2、重复步骤S3.1，直到选择出k个聚类中心，，k为整数且2≤k≤8，得到初步伪彩色图像。S3.2. Repeat step S3.1 until k cluster centers are selected, where k is an integer and 2≤k≤8, and a preliminary pseudo-color image is obtained.

S3.3、设置不同的k值，重复步骤S3.1-步骤S3.2，得到各个k值下的初步伪彩色图像，将各个初步伪彩色图像与染色切片高光谱图像进行比较颜色相似度，选出相似度最高时对应的k值。S3.3. Set different k values, repeat steps S3.1-S3.2 to obtain preliminary pseudo-color images under each k value, and compare the color similarity between each preliminary pseudo-color image and the hyperspectral image of the stained section, The k value corresponding to the highest similarity is selected.

采用的光谱聚类以欧式距离为基础进行样本之间的相似度度量，并根据相似度对各样本进行聚类，在此过程中，聚类中心个数对整体结果有较大影响，一旦选择不好，可能就无法得到有效的相似性结果；因此将聚类中心k依次设置为2～8，对比实验效果确定最佳k值。选择聚类中心的思路为，已选取了n个初始聚类中心(0<n<k)，则在选取第n+1个聚类中心时，距离当前这n个聚类中心越远的点，就越有几率被选为第n+1个聚类中心。本实施例中，经过颜色相似度比较，将各个初步伪彩色图像与染色切片高光谱图像进行比较，将k值设为6更符合实际情况。The adopted spectral clustering measures the similarity between samples based on Euclidean distance, and clusters each sample according to the similarity. In this process, the number of cluster centers has a great influence on the overall results. If not, effective similarity results may not be obtained; therefore, the cluster center k is set to 2 to 8 in turn, and the optimal k value is determined by comparing the experimental results. The idea of selecting the cluster center is that n initial cluster centers (0<n<k) have been selected, then when selecting the n+1th cluster center, the farther the point from the current n cluster centers is. , the more likely it is to be selected as the n+1th cluster center. In this embodiment, after comparing the color similarity, each preliminary pseudo-color image is compared with the hyperspectral image of the stained section, and setting the value of k to 6 is more in line with the actual situation.

S4、根据步骤S3选择的k值，采用光谱聚类对未染色切片高光谱数据的各个像素点的光谱特征进行划分，根据划分结果对各个像素点进行标记，得到未染色切片高光谱数据光谱标记后的最优初步伪彩色图像。S4. According to the k value selected in step S3, spectral clustering is used to divide the spectral characteristics of each pixel point of the hyperspectral data of the unstained slice, and each pixel point is marked according to the division result to obtain the spectral mark of the hyperspectral data of the unstained slice. Post-optimal preliminary pseudocolor image.

S5、根据颜色相似性，将最优初步伪彩色图像与染色切片高光谱图像进行比较，获取标准颜色配比，根据标准颜色配比调节切片中各类组织的颜色配比，得到最终伪彩色图像。S5. According to the color similarity, compare the optimal preliminary pseudo-color image with the hyperspectral image of the stained section to obtain a standard color ratio, adjust the color ratio of various tissues in the section according to the standard color ratio, and obtain a final pseudo-color image .

本实施例展示了低分化胃癌组织的处理结果，参见图6，A中主要展示了第一组胃癌样本中染色切片细胞的特征，B中采用光谱聚类法对第一组胃癌样本中未染色切片进行标记，得到高光谱虚拟染色图像；参见图7，C中主要展示了第二组胃癌样本中染色切片细胞区域特征，D中采用光谱聚类法对第二组胃癌样本中未染色切片进行标记，得到高光谱虚拟染色图像，清晰的展示了胃癌样本中纤维组织的区域特征；参见图8，A中主要展示了第三组胃癌样本中染色切片细胞的特征，B中采用光谱聚类法对第三组胃癌样本中未染色切片进行标记，得到高光谱虚拟染色图像；参见图9，C中主要展示了第四组胃癌样本中染色切片细胞区域特征，D中采用光谱聚类法对第四组胃癌样本中未染色切片进行标记，得到高光谱虚拟染色图像，清晰的展示了胃癌样本中纤维组织的区域特征。This example shows the processing results of poorly differentiated gastric cancer tissues, see Figure 6, A mainly shows the characteristics of the stained section cells in the first group of gastric cancer samples, and B uses the spectral clustering method to analyze the unstained cells in the first group of gastric cancer samples Sections are marked to obtain hyperspectral virtual staining images; see Figure 7, C mainly shows the cell area characteristics of the stained sections in the second group of gastric cancer samples, and D adopts spectral clustering method to analyze the unstained sections in the second group of gastric cancer samples. mark to obtain a hyperspectral virtual staining image, which clearly shows the regional characteristics of fibrous tissue in gastric cancer samples; see Figure 8, A mainly shows the characteristics of stained section cells in the third group of gastric cancer samples, and spectral clustering method is used in B The unstained sections in the third group of gastric cancer samples were marked to obtain hyperspectral virtual stained images; see Figure 9, C mainly shows the cell area characteristics of the stained sections in the fourth group of gastric cancer samples, D uses spectral clustering method to analyze The unstained sections in the four groups of gastric cancer samples were marked to obtain hyperspectral virtual staining images, which clearly showed the regional characteristics of fibrous tissue in gastric cancer samples.

S6、采集待测组织样本的未染色切片高光谱原始数据，采用平滑去噪方式预处理后，得到待测组织样本未染色切片高光谱数据；根据步骤S3选择的k值，采用光谱聚类法处理待测组织样本未染色切片高光谱数据，并根据步骤S5中标准颜色配比，对待测组织样本未染色切片的进行颜色配比，得到待测组织样本未染色切片的最终伪彩色图像；采用两次层叠的主成分变换方法提取待测组织样本未染色切片高光谱数据的结构纹理信息，得到显微组织单谱段图像；将显微组织单谱段图像与待测组织样本未染色切片的最终伪彩色图像合成，得到待测组织样本未染色切片的高光谱虚拟染色图像。S6. Collect the hyperspectral raw data of the unstained section of the tissue sample to be tested, and after preprocessing by smoothing and denoising, obtain the hyperspectral data of the unstained section of the tissue sample to be tested; according to the k value selected in step S3, use the spectral clustering method Process the hyperspectral data of the unstained section of the tissue sample to be tested, and perform color matching of the unstained section of the tissue sample to be tested according to the standard color ratio in step S5, to obtain a final pseudo-color image of the unstained section of the tissue sample to be tested; The structure and texture information of the hyperspectral data of the unstained section of the tissue sample to be tested is extracted by the principal component transformation method of two layers, and the single-spectrum image of the microstructure is obtained; The final pseudo-color image is synthesized to obtain a hyperspectral virtual stained image of the unstained section of the tissue sample to be tested.

可以理解的是，最终伪彩色图像体现了未染色切片的图像信息，但是在对数据进行图像维的数据预处理时，造成了局部结构纹理信息的丢失，为弥补细节纹理信息的不足，需要采用两次层叠的主成分变换方法在最终伪彩色图像上添加结构纹理信息。It is understandable that the final pseudo-color image reflects the image information of the unstained section, but when the data is preprocessed in the image dimension, the local structure and texture information is lost. In order to make up for the lack of detailed texture information, it is necessary to use A two-layered PCT method adds structural texture information to the final pseudocolor image.

参见图10，A为光谱聚类的处理结果，针对未染色切片模拟了染色信息，得到的最终伪彩色图像；B为经过两次层叠的主成分变换提取的显微组织单谱段图像，包含较为丰富的组织纹理信息；C为将最终伪彩色图像与显微组织单谱段图像融合后，形成的高光谱虚拟染色图像，因为加入了细节信息，使得细胞结构更为清晰，可以看出其中红细胞的特征比较明显。Referring to Fig. 10, A is the processing result of spectral clustering, the final pseudo-color image obtained by simulating the staining information for the unstained section; B is the single-spectrum image of the microstructure extracted by two stacked principal component transformations, including Relatively rich tissue texture information; C is the hyperspectral virtual staining image formed by fusing the final pseudo-color image with the single-spectrum image of the microstructure. Because of the addition of detailed information, the cell structure is clearer, and it can be seen that the The characteristics of red blood cells are more obvious.

参见图11，D为光谱聚类的处理结果，针对未染色切片模拟了染色信息，得到的最终伪彩色图像；E为经过两次层叠的主成分变换提取的显微组织单谱段图像，包含较为丰富的组织纹理信息；F为将最终伪彩色图像与显微组织单谱段图像融合后，形成的高光谱虚拟染色图像，使得F中纤维组织的纹理特征更为突出，有效提升了虚拟染色的效果。Referring to Figure 11, D is the processing result of spectral clustering, the final pseudo-color image obtained by simulating the staining information for the unstained section; E is the single-segment image of the microstructure extracted by two stacked principal component transformations, including Relatively rich tissue texture information; F is the hyperspectral virtual staining image formed by fusing the final pseudo-color image with the single spectrum image of the microstructure, which makes the texture features of the fibrous tissue in F more prominent and effectively improves the virtual staining. Effect.

具体的，两次层叠的主成分变换方法包括第一次变换和第二次变换；第一次变换基于未染色切片高光谱数据中不同谱段的噪声协方差矩阵分离噪声，用于使变换后的噪声数据只有最小方差，且没有谱段之间的相关性；第二次变换是对噪声数据的标准主成分变换以去除噪声数据，得到未染色切片高光谱数据的特征图像，用于提取未染色切片高光谱数据的特征图像中的高光谱数据的结构纹理信息。Specifically, the two-layered principal component transformation method includes a first transformation and a second transformation; the first transformation separates the noise based on the noise covariance matrix of different spectral segments in the hyperspectral data of unstained slices, and is used to make the transformed The noise data has only the minimum variance and no correlation between spectral segments; the second transformation is the standard principal component transformation of the noise data to remove the noise data, and obtain the characteristic image of the hyperspectral data of the unstained section, which is used to extract the unstained section hyperspectral data. Structural texture information of hyperspectral data in characteristic images of stained section hyperspectral data.

两次层叠的主成分变换后会使特征域中不同谱段具有不同的物理意义，对于未染色切片图像而言，本实施例中，变换后的谱段1常代表整个谱段的亮度背景，在图像上比其他谱段更亮；光谱信息主要集中在第2～6个谱段，图像比较清晰，同时噪声也逐渐出现；通常在第10谱段以后会出现随机噪声，覆盖目标物的光谱信息及空间信息。总体而言，在两次层叠的主成分变换中，通过信号与噪声分离，使重要信息集中在有限的特征集中，光谱特征向类特征向量汇集，某些微弱信号会在去噪过程中得到增强，用这样的方法提取显微组织单谱段图像，与待测组织样本未染色切片的最终伪彩色图像进行合成从而实现整体高光谱虚拟染色。After two layered principal component transformations, different spectral segments in the feature domain have different physical meanings. For unstained slice images, in this embodiment, the transformed spectral segment 1 often represents the brightness background of the entire spectral segment. It is brighter than other spectral bands on the image; the spectral information is mainly concentrated in the 2nd to 6th spectral bands, the image is relatively clear, and the noise gradually appears; usually after the 10th spectral band, random noise will appear, covering the spectral information of the target object and Spatial information. In general, in the two stacked principal component transformations, the important information is concentrated in a limited feature set by separating the signal from the noise, the spectral features are aggregated into the class feature vector, and some weak signals will be enhanced in the denoising process. , using this method to extract a single spectrum image of the microstructure, and synthesize it with the final pseudo-color image of the unstained section of the tissue sample to be tested to achieve overall hyperspectral virtual staining.

为进一步说明问题，将本实施例的高光谱虚拟染色图像与采用H-E方法染色的图像进行对比，参见图12可以看出，B中显示的最终伪彩色图像与显微组织单谱段图像的组合能够显示更精细的纤维结构细节，从而有效提高图像的分辨率。In order to further illustrate the problem, the hyperspectral virtual staining image of this embodiment is compared with the image stained by the H-E method. Referring to Figure 12, it can be seen that the combination of the final pseudo-color image shown in B and the single spectrum image of the microstructure It can display finer fiber structure details, thereby effectively improving the resolution of the image.

1.一种未染色切片的高光谱图像虚拟染色方法，其特征在于，包括以下步骤：1. a hyperspectral image virtual dyeing method of undyed section, is characterized in that, comprises the following steps: S1、采用显微高光谱成像技术采集组织样本的多组高光谱原始数据，并将高光谱原始数据进行存储；各组高光谱原始数据均包括同一组织样本的未染色切片高光谱原始数据和染色切片高光谱原始数据；S1. Use microscopic hyperspectral imaging technology to collect multiple sets of hyperspectral raw data of tissue samples, and store the hyperspectral raw data; each group of hyperspectral raw data includes unstained sections of the same tissue sample. Slice hyperspectral raw data; S2、采用平滑去噪方式对高光谱原始数据进行图像维的数据预处理，得到高光谱数据，高光谱数据包括未染色切片高光谱数据和染色切片高光谱数据，根据染色切片高光谱数据得到染色切片高光谱图像；S2. Perform image-dimensional data preprocessing on the hyperspectral raw data by smoothing and denoising to obtain hyperspectral data. The hyperspectral data includes hyperspectral data of unstained sections and hyperspectral data of dyed sections, and dyed sections are obtained according to the hyperspectral data of dyed sections. Slice hyperspectral images; S3、未染色切片高光谱数据构成实验数据集，根据实验数据集选择k个聚类中心，使得k值下的初步伪彩色图像与染色切片高光谱图像的颜色相似度最高；S3. The hyperspectral data of the unstained section constitutes an experimental data set, and k cluster centers are selected according to the experimental data set, so that the color similarity between the preliminary pseudo-color image and the hyperspectral image of the stained section under the k value is the highest; S4、根据步骤S3选择的k值，采用光谱聚类法对未染色切片高光谱数据的各个像素点的光谱特征进行划分，根据划分结果对各个像素点进行标记，得到未染色切片高光谱数据光谱标记后的最优初步伪彩色图像；S4, according to the k value selected in step S3, adopt the spectral clustering method to divide the spectral characteristics of each pixel point of the hyperspectral data of the unstained slice, and mark each pixel point according to the division result to obtain the spectrum of the hyperspectral data of the unstained slice The optimal preliminary pseudocolor image after labeling; S5、根据颜色相似性，将最优初步伪彩色图像与染色切片高光谱图像进行比较，获取标准颜色配比，根据标准颜色配比调节未染色切片中各类组织的颜色配比，得到最终伪彩色图像；S5. According to the color similarity, compare the optimal preliminary pseudo-color image with the hyperspectral image of the stained section to obtain a standard color ratio, and adjust the color ratio of various tissues in the unstained section according to the standard color ratio to obtain the final pseudo-color ratio. color image; S6、采集待测组织样本的未染色切片高光谱原始数据，采用平滑去噪方式预处理后，得到待测组织样本未染色切片高光谱数据；S6, collecting the hyperspectral raw data of the unstained section of the tissue sample to be tested, and after preprocessing with a smoothing and denoising method, the hyperspectral data of the unstained section of the tissue sample to be tested is obtained; 根据步骤S3选择的k值，采用光谱聚类法处理待测组织样本未染色切片高光谱数据，并根据步骤S5中标准颜色配比，调节待测组织样本未染色切片中各类组织的颜色配比，得到待测组织样本未染色切片的最终伪彩色图像；According to the k value selected in step S3, the spectral clustering method is used to process the hyperspectral data of the unstained section of the tissue sample to be tested, and according to the standard color ratio in step S5, the color matching of various tissues in the unstained section of the tissue sample to be tested is adjusted. ratio to obtain the final pseudo-color image of the unstained section of the tissue sample to be tested; 采用两次层叠的主成分变换方法提取待测组织样本未染色切片高光谱数据的结构纹理信息，得到显微组织单谱段图像；将显微组织单谱段图像与待测组织样本未染色切片的最终伪彩色图像合成，得到待测组织样本未染色切片的高光谱虚拟染色图像。The structure and texture information of the hyperspectral data of the unstained section of the tissue sample to be tested is extracted by the principal component transformation method of two layers, and the single-spectrum image of the microstructure is obtained. The final pseudo-color image is synthesized to obtain a hyperspectral virtual stained image of the unstained section of the tissue sample to be tested. 2.根据权利要求1所述的未染色切片的高光谱图像虚拟染色方法，其特征在于：2. the hyperspectral image virtual dyeing method of undyed section according to claim 1, is characterized in that: 步骤S3中，选择k个聚类中心，使得k值下的初步伪彩色图像与染色切片高光谱图像的颜色相似度最高，具体过程为：In step S3, k cluster centers are selected, so that the color similarity between the preliminary pseudo-color image under the k value and the hyperspectral image of the stained section is the highest, and the specific process is as follows: S3.1、从实验数据集中随机选取一个聚类中心作为初始聚类中心c₁；计算每个坐标样本点x_i与当前聚类中心之间的最短距离D(x)，然后计算x_i被选为下一个聚类中心的概率P_i，计算公式为S3.1. Randomly select a cluster center from the experimental data set as the initial cluster center c ₁ ; calculate the shortest distance D(x) between each coordinate sample point x _i and the current cluster center, and then calculate that x _i is is selected as the probability P _i of the next cluster center, the calculation formula is

其中，i为第i个样本点；Among them, i is the ith sample point; 选择概率最大的样本点作为下一个聚类中心；Select the sample point with the highest probability as the next cluster center; S3.2、重复步骤S3.1，直到选择出k个聚类中心，k为整数且2≤k≤8，得到初步伪彩色图像；S3.2. Repeat step S3.1 until k cluster centers are selected, where k is an integer and 2≤k≤8, to obtain a preliminary pseudo-color image; S3.3、取不同的k值，重复步骤S3.1-步骤S3.2，得到各个k值下的初步伪彩色图像，将各个初步伪彩色图像与染色切片高光谱图像比较颜色相似度，选出相似度最高的初步伪彩色图像对应的k值。S3.3. Take different k values, repeat steps S3.1 to S3.2 to obtain preliminary pseudo-color images under each k value, compare the color similarity between each preliminary pseudo-color image and the hyperspectral image of the stained section, and select The k value corresponding to the preliminary pseudo-color image with the highest similarity is obtained. 3.根据权利要求2所述的未染色切片的高光谱图像虚拟染色方法，其特征在于：3. the hyperspectral image virtual dyeing method of undyed section according to claim 2, is characterized in that: 步骤S1中，采集组织样本具体为，调节显微高光谱成像设备的调焦机构，使组织样本处于观察位置进行高光谱原始数据采集，之后保持放大倍数与光源强度不变，移动切片或更换其他切片，采集高光谱原始数据。In step S1, collecting the tissue sample is specifically: adjusting the focusing mechanism of the microscopic hyperspectral imaging device, so that the tissue sample is in the observation position to collect hyperspectral raw data, and then keeping the magnification and the intensity of the light source unchanged, moving the slice or changing other Slice and collect hyperspectral raw data. 4.根据权利要求3所述的未染色切片的高光谱图像虚拟染色方法，其特征在于：4. the hyperspectral image virtual dyeing method of undyed section according to claim 3, is characterized in that: 步骤S1中，同一组织样本的未染色切片高光谱原始数据和染色切片高光谱原始数据的获取方式为：同一组织样本切下相邻的两个切片，一个按照正常流程进行染色和封片作为染色切片，采集得到的数据为染色切片高光谱原始数据；另一个直接封片作为未染色切片，采集得到的数据为未染色切片高光谱原始数据。In step S1, the acquisition method of the hyperspectral raw data of the unstained section and the hyperspectral raw data of the stained section of the same tissue sample is as follows: two adjacent sections are cut from the same tissue sample, and one is stained and sealed according to the normal process. Section, the collected data is the hyperspectral raw data of the stained section; another directly mounted section is used as the unstained section, and the collected data is the hyperspectral raw data of the unstained section. 5.根据权利要求1-4任一所述的未染色切片的高光谱图像虚拟染色方法，其特征在于：5. according to the arbitrary described hyperspectral image virtual dyeing method of undyed section of claim 1-4, it is characterized in that: 步骤S6中，所述两次层叠的主成分变换方法包括第一次变换和第二次变换；In step S6, the two-layered principal component transformation method includes a first transformation and a second transformation; 所述第一次变换基于未染色切片高光谱数据中不同谱段的噪声协方差矩阵分离噪声，用于使变换后的噪声数据只有最小方差，且没有谱段之间的相关性；The first transformation separates noise based on the noise covariance matrix of different spectral segments in the unstained slice hyperspectral data, so that the transformed noise data has only a minimum variance and no correlation between spectral segments; 所述第二次变换是对噪声数据的标准主成分变换以去除噪声数据，得到未染色切片高光谱数据的特征图像，用于提取未染色切片高光谱数据的特征图像中的高光谱数据的结构纹理信息。The second transformation is the standard principal component transformation of the noise data to remove the noise data, and obtain the characteristic image of the hyperspectral data of the unstained section, which is used to extract the structure of the hyperspectral data in the characteristic image of the hyperspectral data of the unstained section. texture information.