TWI663960B - Foot deformity detection model, foot deformity detection system and foot deformity detection method - Google Patents

本發明提供一種足畸形檢測系統，包含一影像擷取單元以及一非暫態機器可讀媒體。影像擷取單元用以取得一受試者的一目標足內側位X光影像資料。非暫態機器可讀媒體包含一儲存單元及一處理單元。儲存單元用以儲存一足畸形檢測程式。當足畸形檢測程式由處理單元執行時用以判斷受試者之足畸形類型以及足畸形發生程度。藉此，本發明之足畸形檢測系統可有效提升足畸形檢測的準確度與敏感度，並可提供即時且有效之足畸形檢測評估結果。 The invention provides a foot deformity detection system, which includes an image capturing unit and a non-transitory machine-readable medium. The image acquisition unit is used to obtain X-ray image data of a target foot medial position of a subject. The non-transitory machine-readable medium includes a storage unit and a processing unit. The storage unit is used for storing a foot deformity detection program. When the foot deformity detection program is executed by the processing unit, it is used to determine the type of foot deformity and the degree of occurrence of the foot deformity. Thus, the foot deformity detection system of the present invention can effectively improve the accuracy and sensitivity of foot deformity detection, and can provide immediate and effective results of foot deformity detection and evaluation.

足畸形檢測模型、足畸形檢測系統及足畸形檢測方法 Foot deformity detection model, foot deformity detection system, and foot deformity detection method

本發明是有關於一種醫療資訊分析模型、系統以及方法，特別是一種足畸形檢測模型、足畸形檢測系統及足畸形檢測方法。 The invention relates to a medical information analysis model, system and method, in particular to a foot deformity detection model, a foot deformity detection system and a foot deformity detection method.

直至今日，檢測足畸形發生與否最為快速且最為直接的方法是透過影像分析的方式對足部的解剖幾何構造進行評估，其中又以X光影像分析為檢測足畸形最為廣泛應用的影像分析檢測方法。然而，以X光影像分析方式檢測足畸形發生與否仍需透過分析者對骨骼影像進行分析，是以在後續判讀X光影像時容易因為不同分析者的不同比對習慣而產生不同的診斷結果，致使習知透過X光影像分析方式進行足畸形判定的準確度較為不佳。 Until today, the fastest and most direct way to detect the occurrence of foot deformity is to evaluate the anatomical geometry of the foot through image analysis. Among them, X-ray image analysis is the most widely used image analysis detection for detecting foot deformity. method. However, detecting the occurrence of foot deformity by X-ray image analysis still needs to analyze the bone image by the analyst. It is easy to produce different diagnosis results due to the different comparison habits of different analysts in the subsequent interpretation of the X-ray image. As a result, the accuracy of the conventional method for determining foot deformity through X-ray image analysis is relatively poor.

由於醫學影像技術的發展，核磁共振成像(Magnetic Resonance Imaging，MRI)開始應用於與足畸形相關的檢測上，由於核磁共振成像可有效獲得完整之足部三維結構影像，使其具有高效的檢測精準度。然而，以核磁 共振成像進行足畸形檢測不僅需耗費較高的成本，檢測的時間亦較長，是以利用核磁共振成像進行足畸形的診斷於目前實務應用方面較不普及。 Due to the development of medical imaging technology, Magnetic Resonance Imaging (MRI) has begun to be used for the detection of foot deformities. Because MRI can effectively obtain a complete three-dimensional image of the foot, it has an efficient and accurate detection degree. However, with NMR The detection of foot deformity by resonance imaging not only requires a high cost, but also takes a long time to detect. The diagnosis of foot deformity using MRI is less popular in current practical applications.

因此，如何發展出一種具有高度準確率及快速檢測之足畸形檢測系統，實為一具有商業價值之技術課題。 Therefore, how to develop a foot deformity detection system with high accuracy and rapid detection is a technical issue with commercial value.

本發明的目的是在於提供一種足畸形檢測模型、足畸形檢測系統及足畸形檢測方法，其透過自動化之判定模組而可在大量的X光影像資料的基礎下提供即時(real time)且準確之足畸形檢測評估結果，以作為後續之病情監控或療效評估的依據。 The object of the present invention is to provide a foot deformity detection model, a foot deformity detection system and a foot deformity detection method, which can provide real time and accuracy based on a large amount of X-ray image data through an automated determination module. The results of foot deformity detection and evaluation are used as the basis for subsequent disease monitoring or efficacy evaluation.

本發明之一態樣是在於提供一種足畸形檢測模型，其包含以下建立步驟。取得一參照資料庫，其中前述之參照資料庫包含複數個參照足內側位X光影像資料。進行一影像前處理步驟，其係利用一影像資料編輯模組調整各參照足內側位X光影像資料的一影像大小及一影像黑白對比度，以取得複數個標準化足內側位X光影像資料。進行一特徵選取步驟，其係利用一特徵選取模組分析前述之標準化足內側位X光影像資料後以得至少一影像特徵值。進行一訓練步驟，其係將前述之至少一影像特徵值透過一卷積神經網路學習分類器進行訓練而達到收斂，以得前述之足畸形檢測模型，其中前述之足畸形檢測模型係用以判斷一受試者之一足畸形類型以及一足畸形發生程度。 One aspect of the present invention is to provide a foot deformity detection model, which includes the following establishment steps. A reference database is obtained, wherein the aforementioned reference database includes a plurality of reference medial foot X-ray image data. An image pre-processing step is performed, which uses an image data editing module to adjust an image size and an image black-and-white contrast of each reference medial foot X-ray image data to obtain a plurality of standardized medial foot X-ray image data. A feature selection step is performed, which uses a feature selection module to analyze the aforementioned standardized medial foot X-ray image data to obtain at least one image feature value. A training step is performed, which is to converge the at least one image feature value through a convolutional neural network learning classifier to obtain the aforementioned foot deformity detection model, wherein the aforementioned foot deformity detection model is used to Determine the type of foot deformity and the degree of foot deformity in a subject.

依據前述之足畸形檢測模型，其中前述之卷積神經網路學習分類器可為Inception-ResNet-v2卷積神經網路。 According to the aforementioned foot deformity detection model, the aforementioned convolutional neural network learning classifier may be an Inception-ResNet-v2 convolutional neural network.

依據前述之足畸形檢測模型，其中前述之影像前處理步驟可更對各參照足內側位X光影像資料進行一影像色度擴展處理。 According to the aforementioned foot deformity detection model, the aforementioned image pre-processing step can further perform an image chroma expansion process on each reference medial foot X-ray image data.

依據前述之足畸形檢測模型，其中前述之參照足內側位X光影像資料的影像格式可為數位醫療影像儲存標準協定(Digital Imaging and Communications in Medicine，DICOM)之影像格式。 According to the aforementioned foot malformation detection model, the aforementioned image format of the reference medial foot X-ray image data may be an image format of the Digital Imaging and Communications in Medicine (DICOM).

依據前述之足畸形檢測模型，其中前述之足畸形類型可為柔軟性扁平足或結構性扁平足。 According to the aforementioned foot deformity detection model, the aforementioned foot deformity type may be a soft flat foot or a structural flat foot.

依據前述之足畸形檢測模型，其中前述之足畸形發生程度可為無足畸形、輕度足畸形、中度足畸形或重度足畸形。 According to the aforementioned foot deformity detection model, the degree of occurrence of the aforementioned foot deformity may be an ankle deformity, a mild foot deformity, a moderate foot deformity, or a severe foot deformity.

依據前述之足畸形檢測模型，其中各參照足內側位X光影像資料可包含一左足內側位X光影像資料及一右足內側位X光影像資料。 According to the aforementioned foot deformity detection model, each reference medial foot X-ray image data may include a left medial foot X-ray image data and a right medial foot X-ray image data.

本發明之另一態樣是在於提供一種足畸形檢測系統，用以判斷受試者之一足畸形類型以及一足畸形發生程度，且足畸形檢測系統包含一影像擷取單元以及一非暫態機器可讀媒體。影像擷取單元用以取得受試者的一目標足內側位X光影像資料。非暫態機器可讀媒體訊號連接前述之影像擷取單元，且非暫態機器可讀媒體包含一儲存單元及一處理 單元，其中儲存單元用以儲存前述之目標足內側位X光影像資料及一足畸形檢測程式，而處理單元用以執行前述之足畸形檢測程式。其中，前述之足畸形檢測程式包含一足畸形檢測模型建立子程式及一足畸形檢測子程式。足畸形檢測模型建立子程式包含一參照資料庫取得模組、一參照影像資料編輯模組、一特徵選取模組及一訓練模組。參照資料庫取得模組係用以取得一參照資料庫，其中前述之參照資料庫包含複數個參照足內側位X光影像資料。參照影像資料編輯模組係調整各參照足內側位X光影像資料的一影像大小及一影像黑白對比度，以取得複數個標準化足內側位X光影像資料。特徵選取模組係用以分析前述之標準化足內側位X光影像資料後以得至少一參照影像特徵值。訓練模組係用以將前述之至少一參照影像特徵值透過一卷積神經網路學習分類器進行訓練而達到收斂，以得一足畸形檢測模型。足畸形檢測子程式包含一目標影像資料編輯模組、一目標特徵選取模組及一比對模組。目標影像資料編輯模組係調整前述之目標足內側位X光影像資料的一影像大小及一影像黑白對比度，以取得一標準化目標足內側位X光影像資料。目標特徵選取模組係用以分析前述之標準化目標足內側位X光影像資料後以得至少一目標影像特徵值。比對模組係用以將前述之至少一目標影像特徵值以前述之足畸形檢測模型進行分析，以得一目標影像特徵值權重數據，並分析前述之目標影像特徵值權重數據，以得受試者之足畸形類型以及足畸形發生程度。 Another aspect of the present invention is to provide a foot deformity detection system for determining a type of foot deformity and a degree of occurrence of a foot deformity, and the foot deformity detection system includes an image acquisition unit and a non-transitory machine. Read the media. The image capture unit is used to obtain X-ray image data of a target foot medial position of the subject. The non-transitory machine-readable medium signal is connected to the aforementioned image capture unit, and the non-transitory machine-readable medium includes a storage unit and a processing unit. A unit in which a storage unit is used to store the aforementioned target medial foot X-ray image data and a foot deformity detection program, and a processing unit is used to execute the aforementioned foot deformity detection program. The aforementioned foot deformity detection program includes a foot deformity detection model creation subroutine and a foot deformity detection subroutine. The foot deformity detection model creation subroutine includes a reference database acquisition module, a reference image data editing module, a feature selection module, and a training module. The reference database acquisition module is used to obtain a reference database, wherein the aforementioned reference database includes a plurality of reference medial position X-ray image data. The reference image data editing module adjusts an image size and an image black-and-white contrast of each reference medial foot X-ray image data to obtain a plurality of standardized medial foot X-ray image data. The feature selection module is used to obtain at least one reference image feature value after analyzing the aforementioned standardized medial foot X-ray image data. The training module is used to train the at least one reference image feature value through a convolutional neural network learning classifier to achieve convergence to obtain a foot deformity detection model. The foot deformity detection subroutine includes a target image data editing module, a target feature selection module, and a comparison module. The target image data editing module adjusts an image size and an image black-and-white contrast of the aforementioned target medial position X-ray image data to obtain a standardized target medial position X-ray image data. The target feature selection module is used to analyze at least one target image feature value after analyzing the above-mentioned standardized target foot medial position X-ray image data. The comparison module is used to analyze the at least one target image feature value using the aforementioned foot deformity detection model to obtain a target image feature value weight data, and analyze the aforementioned target image feature value weight data to obtain The type of foot deformity and the degree of foot deformity.

依據前述之足畸形檢測系統，其中前述之卷積神經網路學習分類器可為Inception-ResNet-v2卷積神經網路。 According to the aforementioned foot deformity detection system, the aforementioned convolutional neural network learning classifier may be an Inception-ResNet-v2 convolutional neural network.

依據前述之足畸形檢測系統，其中前述之參照影像資料編輯模組可更對各參照足內側位X光影像資料進行一影像色度擴展處理，前述之目標影像資料編輯模組可更對目標足內側位X光影像資料進行一影像色度擴展處理。 According to the aforementioned foot deformity detection system, the aforementioned reference image data editing module can further perform an image chroma expansion process on each reference foot medial position X-ray image data, and the aforementioned target image data editing module can further target the foot The inside X-ray image data is subjected to an image chroma expansion process.

依據前述之足畸形檢測系統，其中前述之目標足內側位X光影像資料的影像格式可為數位醫療影像儲存標準協定之影像格式，前述之參照足內側位X光影像資料的影像格式可為數位醫療影像儲存標準協定之影像格式。 According to the aforementioned foot deformity detection system, the image format of the target medial foot X-ray image data may be an image format of a digital medical image storage standard agreement, and the image format of the aforementioned medial foot X-ray image data may be digital Medical image storage standard agreement image format.

依據前述之足畸形檢測系統，其中前述之足畸形類型可為柔軟性扁平足或結構性扁平足。 According to the aforementioned foot deformity detection system, the aforementioned type of foot deformity may be a soft flat foot or a structural flat foot.

依據前述之足畸形檢測系統，其中前述之足畸形發生程度可為無足畸形、輕度足畸形、中度足畸形或重度足畸形。 According to the aforementioned foot deformity detection system, the degree of occurrence of the aforementioned foot deformity may be an ankle deformity, a mild foot deformity, a moderate foot deformity or a severe foot deformity.

依據前述之足畸形檢測系統，其中各參照足內側位X光影像資料可包含一左足內側位X光影像資料及一右足內側位X光影像資料，前述之目標足內側位X光影像資料可包含一左足內側位X光影像資料及一右足內側位X光影像資料。 According to the aforementioned foot deformity detection system, each reference medial foot X-ray image data may include a left foot medial X-ray image data and a right foot medial X-ray image data, and the aforementioned target medial foot X-ray image data may include A left foot medial X-ray image data and a right foot medial X-ray image data.

本發明之又一態樣是在於提供一種足畸形檢測方法，其包含下述步驟。提供一如前段所述之足畸形檢測模型。提供一受試者之一目標足內側位X光影像資料。對前述 之目標足內側位X光影像資料進行前處理，其係利用前述之影像資料編輯模組調整目標足內側位X光影像資料的一影像大小及一影像黑白對比度，以取得一標準化目標足內側位X光影像資料。利用前述之特徵選取模組分析標準化目標足內側位X光影像資料後以得至少一影像特徵值。利用前述之足畸形檢測模型分析前述之至少一影像特徵值，以判斷受試者之一足畸形類型以及一足畸形發生程度。 Another aspect of the present invention is to provide a method for detecting a foot deformity, which includes the following steps. A foot deformity detection model as described in the previous paragraph is provided. Provide an X-ray image of the medial foot position of a subject. To the foregoing The target medial foot X-ray image data is pre-processed, which uses the aforementioned image data editing module to adjust an image size and an image black and white contrast of the target medial foot X-ray image data to obtain a standardized target medial foot position. X-ray image data. The at least one image feature value is obtained after analyzing the normalized target's medial position X-ray image data by using the aforementioned feature selection module. The aforementioned foot deformity detection model is used to analyze the at least one image feature value to determine the type of foot deformity and the degree of occurrence of a foot deformity.

依據前述之足畸形檢測方法，其中前述之影像資料編輯模組可更對目標足內側位X光影像資料進行一影像色度擴展處理。 According to the aforementioned method of detecting a foot deformity, the aforementioned image data editing module can further perform an image chroma expansion process on the target medial position X-ray image data.

依據前述之足畸形檢測方法，其中前述之目標足內側位X光影像資料的影像格式可為數位醫療影像儲存標準協定之影像格式。 According to the foregoing method for detecting a foot deformity, the image format of the aforementioned target medial foot X-ray image data may be an image format of a digital medical image storage standard agreement.

依據前述之足畸形檢測方法，其中前述之目標足內側位X光影像資料可包含一左足內側位X光影像資料及一右足內側位X光影像資料。 According to the foregoing method of detecting a foot deformity, the aforementioned target medial position X-ray image data may include a left medial position X-ray image data and a right medial position X-ray image data.

藉此，本發明之足畸形檢測模型、足畸形檢測系統及足畸形檢測方法透過將足內側位X光影像資料與目標足內側位X光影像資料進行影像標準化前處理，並利用特徵選取模組分析並得至少一影像特徵值後，再以卷積神經網路學習分類器對影像特徵值進行訓練，以快速且準確地判斷受試者之足畸形發生與否、足畸形類型以及足畸形發生程度，並具有高度的檢測再現性。再者，透過包含卷積神經網路學習分類器之足畸形檢測模型能有效提升足畸形檢測的 準確度與敏感度，使本發明之足畸形檢測模型、足畸形檢測系統及足畸形檢測方法具有優良的臨床應用潛力。 In this way, the foot deformity detection model, the foot deformity detection system and the foot deformity detection method of the present invention perform image standardization preprocessing on the medial foot X-ray image data and the target medial foot X-ray image data, and use a feature selection module After analyzing and obtaining at least one image feature value, the convolutional neural network learning classifier is used to train the image feature value to quickly and accurately determine whether the subject's foot deformity occurs, the type of foot deformity, and the occurrence of foot deformity. Degree and has high detection reproducibility. Furthermore, a foot deformity detection model including a convolutional neural network learning classifier can effectively improve the detection of foot deformities. The accuracy and sensitivity make the foot deformity detection model, the foot deformity detection system and the foot deformity detection method of the present invention have excellent clinical application potential.

100‧‧‧足畸形檢測模型 100‧‧‧foot deformity detection model

110‧‧‧取得一參照資料庫 110‧‧‧Get a reference database

120‧‧‧進行一影像前處理步驟 120‧‧‧ Perform an image pre-processing step

130‧‧‧進行一特徵選取步驟 130‧‧‧Perform a feature selection step

140‧‧‧進行一訓練步驟 140‧‧‧ perform a training step

200‧‧‧足畸形檢測系統 200‧‧‧foot deformity detection system

300‧‧‧影像擷取單元 300‧‧‧Image capture unit

400‧‧‧非暫態機器可讀媒體 400‧‧‧ Non-transitory machine-readable media

411‧‧‧目標足內側位X光影像資料 411‧‧‧ target X-ray image data of medial foot position

410‧‧‧儲存單元 410‧‧‧Storage unit

420‧‧‧處理單元 420‧‧‧processing unit

430‧‧‧足畸形檢測程式 430‧‧‧foot deformity detection program

440‧‧‧足畸形檢測模型建立子程式 440‧‧‧ Foot deformity detection model establishment subroutine

441‧‧‧參照資料庫取得模組 441‧‧‧Refer to the database for modules

442‧‧‧參照影像資料編輯模組 442‧‧‧Reference image data editing module

443‧‧‧特徵選取模組 443‧‧‧Feature Selection Module

444‧‧‧訓練模組 444‧‧‧ Training Module

450‧‧‧足畸形檢測子程式 450‧‧‧foot deformity detection subroutine

451‧‧‧目標影像資料編輯模組 451‧‧‧Target image data editing module

452‧‧‧目標特徵選取模組 452‧‧‧Target Feature Selection Module

453‧‧‧比對模組 453‧‧‧comparison module

500‧‧‧足畸形檢測方法 500‧‧‧foot deformity detection method

510‧‧‧提供足畸形檢測模型 510‧‧‧ provides foot deformity detection model

520‧‧‧提供受試者之一目標足內側位X光影像資料 520‧‧‧ Provides X-ray image data of the medial foot of one of the subjects

530‧‧‧對目標足內側位X光影像資料進行前處理 530‧‧‧ Pre-processing the target medial foot X-ray image data

540‧‧‧利用特徵選取模組分析標準化目標足內側位X光影像資料後以得至少一影像特徵值 540‧‧‧Using the feature selection module to analyze the X-ray image data of the medial foot of the standardized target to obtain at least one image feature value

550‧‧‧利用足畸形檢測模型分析至少一影像特徵值 550‧‧‧Using a foot deformity detection model to analyze at least one image feature value

600‧‧‧卷積神經網路學習分類器 600‧‧‧ Convolutional neural network learning classifier

601‧‧‧目標影像特徵值權重數據 601‧‧‧ Target image feature value weight data

為讓本發明之上述和其他目的、特徵、優點與實施例能更明顯易懂，所附圖式之說明如下：第1圖係繪示本發明一實施方式之足畸形檢測模型的建立步驟流程圖；第2A圖係繪示本發明另一實施方式之足畸形檢測系統的架構示意圖；第2B圖係繪示第2A圖之足畸形檢測系統的足畸形檢測程式的架構示意圖；第3圖係繪示本發明另一實施方式之足畸形檢測方法的步驟流程圖；以及第4圖係繪示本發明之足畸形檢測模型的卷積神經網路學習分類器的架構示意圖。 In order to make the above and other objects, features, advantages, and embodiments of the present invention more comprehensible, the description of the drawings is as follows: FIG. 1 is a flow chart showing the steps of establishing a foot deformity detection model according to an embodiment of the present invention FIG. 2A is a schematic diagram showing the architecture of a foot deformity detection system according to another embodiment of the present invention; FIG. 2B is a schematic diagram showing the architecture of a foot deformity detection program of the foot deformity detection system of FIG. 2A; FIG. 4 is a flowchart showing steps of a foot deformity detection method according to another embodiment of the present invention; and FIG. 4 is a schematic diagram of a convolutional neural network learning classifier of the foot deformity detection model of the present invention.

下述將更詳細討論本發明各實施方式。然而，此實施方式可為各種發明概念的應用，可被具體實行在各種不同的特定範圍內。特定的實施方式是僅以說明為目的，且不受限於揭露的範圍。 Embodiments of the present invention will be discussed in more detail below. However, this embodiment may be an application of various inventive concepts, and may be embodied in various specific ranges. Specific embodiments are for the purpose of illustration only, and are not limited to the scope of the disclosure.

請參照第1圖，其係繪示本發明一實施方式之足畸形檢測模型100的建立步驟流程圖。足畸形檢測模型100的建立步驟包含步驟110、步驟120、步驟130以及步驟140。 Please refer to FIG. 1, which is a flowchart illustrating the steps of establishing a foot deformity detection model 100 according to an embodiment of the present invention. The steps of establishing the foot deformity detection model 100 include steps 110, 120, 130 and 140.

步驟110為取得一參照資料庫，其中參照資料庫包含複數個參照足內側位X光影像資料。較佳地，前述之參照足內側位X光影像資料的影像格式可為數位醫療影像儲存標準協定(Digital Imaging and Communications in Medicine，DICOM)之影像格式，以將各參照足內側位X光影像資料的生理年齡資訊、性別資訊等基本資料儲存於參照足內側位X光影像資料的檔頭(header)中，以利於後續的分析。較佳地，參照足內側位X光影像資料可包含一左足內側位X光影像資料及一右足內側位X光影像資料，以分別建立左足與右足之足畸形參照資料庫，進而利於對左右二足分別進行更精準的足畸形檢測。 Step 110 is to obtain a reference database, where the reference database contains a plurality of reference medial foot X-ray image data. Preferably, the image format of the aforementioned medial foot X-ray image data may be an image format of a Digital Medical Image Storage Standard Agreement (Digital Imaging and Communications in Medicine, DICOM), so as to convert each reference medial foot X-ray image data Basic information such as the physiological age information and gender information are stored in a header that refers to the medial foot X-ray image data to facilitate subsequent analysis. Preferably, the reference medial foot X-ray image data may include a medial left foot X-ray image data and a medial right foot X-ray image data, so as to establish a reference database of deformities of the left foot and the right foot respectively, thereby facilitating the comparison of the left and right feet. For more accurate foot deformity detection.

步驟120為進行一影像前處理步驟，其係利用一影像資料編輯模組調整各參照足內側位X光影像資料的一影像大小及一影像黑白對比度，以取得複數個標準化足內側位X光影像資料。詳細而言，影像資料編輯模組可分別將不同的參照足內側位X光影像資料的影像大小調整為1024像素(pixel)×1024像素後，並調整其黑白對比度為0至255之灰階分布的影像強度，以減少不同參照足內側位X光影像資料之間的黑白色度差異以及增加影像的清晰度，以利於後續的分析。另外，在步驟120中，影像資料編輯模組可進一步對各參照足內側位X光影像資料進行影像色度擴展處 理。詳細而言，影像資料編輯模組可計算各參照足內側位X光影像資料的影像灰階程度，並依據前述之計算結果而依序對各參照足內側位X光影像資料之影像像素行、列自動填補色彩，以將呈現灰階色調之各參照足內側位X光影像資料轉換為彩色色調，以增加各參照足內側位X光影像資料的資料強度，進而提升後續分析的準確度，但本發明並不以前述說明與圖式揭露的內容為限。 Step 120 is an image pre-processing step, which uses an image data editing module to adjust an image size and an image black and white contrast of each reference medial foot X-ray image data to obtain a plurality of standardized medial foot X-ray images. data. In detail, the image data editing module can adjust the image size of different reference medial X-ray image data to 1024 pixels (pixel) × 1024 pixels, and adjust the grayscale distribution of black and white contrast from 0 to 255. To reduce the difference in black and white between different reference medial X-ray image data and increase the sharpness of the image to facilitate subsequent analysis. In addition, in step 120, the image data editing module may further perform image chroma extension on each reference medial position X-ray image data. Management. In detail, the image data editing module can calculate the image gray level of each reference medial foot X-ray image data, and sequentially perform image pixel row, The rows are automatically filled with color to convert the gray media to the reference medial foot X-ray image data into color tones to increase the data intensity of each reference medial foot X-ray image data, thereby improving the accuracy of subsequent analysis, but The present invention is not limited to the contents of the foregoing description and drawings.

步驟130為進行一特徵選取步驟，其係利用一特徵選取模組分析前述之標準化足內側位X光影像資料後以得至少一影像特徵值。詳細而言，本發明之足畸形檢測模型100可利用特徵選取模組自動地對標準化足內側位X光影像資料的影像資訊進行分析，並自動提取對應的影像特徵值，藉以增進本發明之足畸形檢測模型100的足畸形診斷效率。 Step 130 is a feature selection step, which uses a feature selection module to analyze the aforementioned normalized medial foot X-ray image data to obtain at least one image feature value. In detail, the foot deformity detection model 100 of the present invention can automatically analyze the image information of the standardized medial foot X-ray image data by using a feature selection module, and automatically extract corresponding image feature values, thereby improving the foot of the present invention. The deformity detection model 100 has a foot deformity diagnosis efficiency.

步驟140為進行一訓練步驟，其係將前述之至少一影像特徵值透過一卷積神經網路學習分類器進行訓練而達到收斂，以得足畸形檢測模型100，其中足畸形檢測模型100係用以判斷一受試者之一足畸形類型以及一足畸形發生程度。較佳地，前述之卷積神經網路學習分類器可為Inception-ResNet-v2卷積神經網路。Inception-ResNet-v2卷積神經網路為一種基於ImageNet可視化數據資料庫的大規模視覺辨識(Large Scale Visual Recognition)卷積神經網路，其透過殘差連接(Residual connections)的方式而可有效擴展卷積神經網路的訓練深 度，進而使Inception-ResNet-v2卷積神經網路於圖像分類與辨識方面具有相當高的準確率。 Step 140 is a training step that trains at least one of the aforementioned image feature values through a convolutional neural network learning classifier to achieve convergence to obtain a foot deformity detection model 100. The foot deformity detection model 100 is used for To determine the type of foot deformity and the degree of foot deformity in a subject. Preferably, the aforementioned convolutional neural network learning classifier may be an Inception-ResNet-v2 convolutional neural network. Inception-ResNet-v2 convolutional neural network is a large-scale visual recognition (Large Scale Visual Recognition) convolutional neural network based on ImageNet visual data database, which can be effectively extended by means of residual connections Deep training of convolutional neural networks Degree, which in turn makes the Inception-ResNet-v2 convolutional neural network have a fairly high accuracy in image classification and recognition.

較佳地，前述之足畸形類型可為柔軟性扁平足或結構性扁平足，而前述之足畸形發生程度則可為無足畸形、輕度足畸形、中度足畸形或重度足畸形。詳細而言，正常人的腳底呈現一自然的弓形，其即為腳底之足弓，當人體在步行或跑步時，足弓會根據各種地形提供適度的彈力和扭力，以達到吸震和平衡的目的。然而，扁平足患者的足弓甚不明顯或幾無足弓，當其站立或平踩於地面時，足底內側的足弓將會下陷而使足底呈現扁平狀態，甚至平貼於地面，進而影響正常的行走，其中柔軟性扁平足多肇因於蹠底較厚之皮下脂肪或較為鬆散的足部關節，而結構性扁平足則是因為關節異常或周圍肌肉痙攣所造成之足部堅硬現象。在站立姿的足側位X光影像資料中，罹患結構性扁平足的患者其跟骨(calcaneus)最下緣連線與第五蹠骨(5th metatarsal)之夾角足弓角度(arch angle)大於168度時認定為扁平足，而本發明之足畸形檢測模型100則可自動根據標準化足內側位X光影像資料而判斷受試者是否罹患扁平足症及其足畸形發生程度，避免因為不同分析者的不同比對習慣及量測方式不同而產生不同的診斷結果，進而使本發明之足畸形檢測模型100的檢測準確度大幅提升。 Preferably, the aforementioned foot deformity type may be a soft flat foot or a structural flat foot, and the occurrence degree of the aforementioned foot deformity may be an ankle deformity, a mild foot deformity, a moderate foot deformity, or a severe foot deformity. In detail, the foot of a normal person presents a natural arch, which is the arch of the foot. When the human body is walking or running, the arch will provide moderate elasticity and torque according to various terrains to achieve the purpose of shock absorption and balance. . However, the arch of a patient with a flat foot is very inconspicuous or has few arches. When they stand or step on the ground flat, the arch on the inner side of the plantar will sink and make the sole flat, or even flat against the ground. Affects normal walking. The soft flat feet are mostly caused by thick subcutaneous fat or loose foot joints, and the structural flat feet are due to joint stiffness or foot stiffness caused by surrounding muscle spasms. In the lateral X-ray image data of the standing foot, the arch angle of the angle between the lowermost edge of the calcaneus and the 5th metatarsal in patients with structural flat feet is greater than 168 degrees. It is identified as a flat foot, and the foot deformity detection model 100 of the present invention can automatically determine whether a subject suffers from flat foot disease and the degree of foot deformity based on standardized X-ray image data of the medial foot, so as to avoid the different ratios of different analysts. Different diagnosis results are generated due to different habits and measurement methods, thereby further improving the detection accuracy of the foot deformity detection model 100 of the present invention.

請參照第2A圖與第2B圖，第2A圖係繪示本發明另一實施方式之足畸形檢測系統200的架構示意圖，第2B圖係繪示第2A圖之足畸形檢測系統200的足畸形檢測程式 430的架構示意圖。足畸形檢測系統200用以判斷一受試者之一足畸形類型以及一足畸形發生程度，且足畸形檢測系統200包含一影像擷取單元300以及一非暫態機器可讀媒體400。 Please refer to FIG. 2A and FIG. 2B. FIG. 2A is a schematic structural diagram of a foot deformity detection system 200 according to another embodiment of the present invention. FIG. 2B is a foot deformity detection system 200 of FIG. 2A Detection program 430 architecture diagram. The foot deformity detection system 200 is used to determine a type of foot deformity and the degree of occurrence of a foot deformity in a subject. The foot deformity detection system 200 includes an image capturing unit 300 and a non-transitory machine-readable medium 400.

影像擷取單元300用以取得受試者的一目標足內側位X光影像資料411。詳細而言，影像擷取單元300可為一X光檢測儀器，其利用低劑量之X光射線照射受試者之足部，以取得解析度適當之目標足內側位X光影像資料411。較佳地，目標足內側位X光影像資料411的影像格式可為數位醫療影像儲存標準協定之影像格式，以將目標足內側位X光影像資料411的生理年齡資訊、性別資訊等基本資料儲存於目標足內側位X光影像資料411的檔頭中，以利於後續的分析。較佳地，目標足內側位X光影像資料411可包含一左足內側位X光影像資料及一右足內側位X光影像資料，以利於對左右二足分別進行更精準的足畸形檢測。 The image capturing unit 300 is used to obtain a target medial foot X-ray image data 411 of the subject. In detail, the image capturing unit 300 may be an X-ray detection apparatus, which irradiates the subject's feet with low-dose X-rays to obtain the target medial foot X-ray image data 411 with an appropriate resolution. Preferably, the image format of the target medial foot X-ray image data 411 may be an image format of a digital medical image storage standard agreement to store basic data such as physiological age information and gender information of the target medial foot X-ray image data 411. It is located in the file of the target medial position X-ray image data 411 to facilitate subsequent analysis. Preferably, the target medial foot X-ray image data 411 may include a left foot medial X-ray image data and a right foot medial X-ray image data, so as to facilitate more accurate detection of foot deformities on the left and right legs, respectively.

非暫態機器可讀媒體400訊號連接影像擷取單元300，且非暫態機器可讀媒體400包含一儲存單元410及一處理單元420，其中儲存單元410用以儲存目標足內側位X光影像資料411及一足畸形檢測程式430，處理單元420用以執行足畸形檢測程式430。足畸形檢測程式430包含一足畸形檢測模型建立子程式440及一足畸形檢測子程式450。 The signal of the non-transitory machine-readable medium 400 is connected to the image capturing unit 300, and the non-transitory machine-readable medium 400 includes a storage unit 410 and a processing unit 420, where the storage unit 410 is used to store the target medial foot X-ray image The data 411 and a foot deformity detection program 430 are used to process the foot deformity detection program 430. The foot deformity detection program 430 includes a foot deformity detection model creation subroutine 440 and a foot deformity detection subroutine 450.

足畸形檢測模型建立子程式440包含一參照資料庫取得模組441、一參照影像資料編輯模組442、一特徵選取模組443及一訓練模組444。 The foot deformity detection model creation subroutine 440 includes a reference database acquisition module 441, a reference image data editing module 442, a feature selection module 443, and a training module 444.

參照資料庫取得模組441係用以取得一參照資料庫，其中前述之參照資料庫包含複數個參照足內側位X光影像資料。較佳地，前述之參照足內側位X光影像資料的影像格式可為數位醫療影像儲存標準協定之影像格式，以將各參照足內側位X光影像資料的生理年齡資訊、性別資訊等基本資料儲存於參照足內側位X光影像資料的檔頭中，以利於後續的分析。較佳地，參照足內側位X光影像資料可包含一左足內側位X光影像資料及一右足內側位X光影像資料，以分別建立左足與右足之足畸形參照資料庫，以利於對左右二足分別進行更精準的足畸形檢測。 The reference database acquisition module 441 is used to obtain a reference database, wherein the aforementioned reference database includes a plurality of reference foot medial position X-ray image data. Preferably, the aforementioned image format of the medial foot X-ray image data may be an image format of a digital medical image storage standard agreement, so as to reference basic data such as physiological age information and gender information of each medial foot X-ray image data. It is stored in the file head with reference to the medial position X-ray image data to facilitate subsequent analysis. Preferably, the reference medial foot X-ray image data may include a medial left foot X-ray image data and a medial right foot X-ray image data, so as to establish reference database of deformity of the left foot and the right foot respectively, so as to facilitate the comparison of left and right feet. For more accurate foot deformity detection.

參照影像資料編輯模組442係調整各參照足內側位X光影像資料的一影像大小及一影像黑白對比度，以取得複數個標準化足內側位X光影像資料。詳細而言，參照影像資料編輯模組442可分別將不同的參照足內側位X光影像資料的影像大小調整為1024像素×1024像素後，並調整其黑白對比度為0至255之灰階分布的影像強度，以減少不同參照足內側位X光影像資料之間的黑白色度差異以及增加影像的清晰度，以利於後續的分析。另外，參照影像資料編輯模組442可進一步對各參照足內側位X光影像資料進行影像色度擴展處理。詳細而言，影像資料編輯模組可計算各參照足內側位X光影像資料的影像灰階程度，並依據前述之計 算結果而依序對各參照足內側位X光影像資料之影像像素行、列自動填補色彩，以將呈現灰階色調之各參照足內側位X光影像資料轉換為彩色色調，以增加各參照足內側位X光影像資料的資料強度，進而提升後續分析的準確度。 The reference image data editing module 442 adjusts an image size and an image black and white contrast of each reference medial foot X-ray image data to obtain a plurality of standardized medial foot X-ray image data. In detail, the reference image data editing module 442 can adjust the image size of different reference medial X-ray image data to 1024 pixels × 1024 pixels, and adjust the grayscale distribution of the black and white contrast of 0 to 255. Image intensity to reduce the difference in black and white between different reference medial X-ray image data and increase the sharpness of the image to facilitate subsequent analysis. In addition, the reference image data editing module 442 may further perform image chroma expansion processing on each reference foot medial position X-ray image data. In detail, the image data editing module can calculate the image gray level of each reference medial foot X-ray image data, and based on the aforementioned calculations The calculation results are used to automatically fill the image pixel rows and columns of each reference medial position X-ray image data in order to convert each reference medial position X-ray image data showing grayscale tones to color tones, so as to increase each reference. The data intensity of the medial foot X-ray image data improves the accuracy of subsequent analysis.

特徵選取模組443係用以分析標準化足內側位X光影像資料後以得至少一參照影像特徵值。詳細而言，特徵選取模組443可自動地對標準化足內側位X光影像資料的影像資訊進行分析，並自動提取對應的影像特徵值。 The feature selection module 443 is used to analyze at least one reference image feature value after analyzing the medial foot X-ray image data. In detail, the feature selection module 443 can automatically analyze the image information of the standardized medial foot X-ray image data, and automatically extract the corresponding image feature values.

訓練模組444係用以將至少一參照影像特徵值透過一卷積神經網路學習分類器進行訓練而達到收斂，以得一足畸形檢測模型。較佳地，前述之卷積神經網路學習分類器可為Inception-ResNet-v2卷積神經網路，以有效擴展卷積神經網路的訓練深度，進而提升訓練模組444的圖像分類與辨識能力。 The training module 444 is used to train at least one reference image feature value through a convolutional neural network learning classifier to achieve convergence to obtain a foot deformity detection model. Preferably, the aforementioned convolutional neural network learning classifier may be an Inception-ResNet-v2 convolutional neural network to effectively extend the training depth of the convolutional neural network, thereby improving the image classification and training of the training module 444. Discernment.

足畸形檢測子程式450包含一目標影像資料編輯模組451、一目標特徵選取模組452及一比對模組453。 The foot deformity detection subroutine 450 includes a target image data editing module 451, a target feature selection module 452, and a comparison module 453.

目標影像資料編輯模組451係調整目標足內側位X光影像資料411的一影像大小及一影像黑白對比度，以取得一標準化目標足內側位X光影像資料。詳細而言，目標影像資料編輯模組451係將目標足內側位X光影像資料411的影像大小調整為1024像素×1024像素後，並調整其黑白對比度為0至255之灰階分布的影像強度，進而獲得前述之標準化目標足內側位X光影像資料。較佳地，目標影像資料編輯模組451可進一步對目標足內側位X光影像資料411進 行影像色度擴展處理，其係計算目標足內側位X光影像資料411的影像灰階程度，並根據前述之計算結果而依序對目標足內側位X光影像資料411之影像像素行、列自動填補色彩，以將呈現灰階色調之目標足內側位X光影像資料411轉換為彩色色調，以增加目標足內側位X光影像資料的資料強度，並與前述之各參照足內側位X光影像資料的影像格式相符，進而提升後續分析的準確度。 The target image data editing module 451 adjusts an image size and an image black-and-white contrast of the target medial foot X-ray image data 411 to obtain a standardized target medial foot X-ray image data. In detail, the target image data editing module 451 adjusts the image size of the target medial position X-ray image data 411 to 1024 pixels × 1024 pixels, and adjusts the image intensity of the grayscale distribution with a black and white contrast ratio of 0 to 255. , And then obtain the aforementioned normalized target medial foot X-ray image data. Preferably, the target image data editing module 451 can further input the target medial position X-ray image data 411 Row image chroma expansion processing, which calculates the gray level of the image of the target medial position X-ray image data 411, and sequentially ranks the image pixel rows and columns of the target medial position X-ray image data 411 according to the aforementioned calculation results. Automatically fill colors to convert the target medial foot X-ray image data 411 to gray tones to increase the data intensity of the target medial foot X-ray image data and compare with the aforementioned reference medial foot X-rays The image format of the image data matches, which improves the accuracy of subsequent analysis.

目標特徵選取模組452係用以分析標準化目標足內側位X光影像資料後以得至少一目標影像特徵值。詳細而言，目標特徵選取模組452可自動地對標準化目標足內側位X光影像資料的影像資訊進行分析，並自動提取對應的影像特徵值。 The target feature selection module 452 is used to analyze at least one target image feature value after analyzing the X-ray image data of the medial foot of the normalized target. In detail, the target feature selection module 452 can automatically analyze the image information of the standardized target medial position X-ray image data, and automatically extract the corresponding image feature values.

比對模組453係用以將至少一目標影像特徵值以足畸形檢測模型進行分析，以得一目標影像特徵值權重數據，並分析目標影像特徵值權重數據，以得受試者之足畸形類型以及足畸形發生程度。 The comparison module 453 is used to analyze at least one target image feature value using a foot deformity detection model to obtain a target image feature value weight data, and analyze the target image feature value weight data to obtain a subject's foot deformity. Type and degree of foot deformity.

較佳地，前述之足畸形類型可為柔軟性扁平足或結構性扁平足，而前述之足畸形發生程度則可為無足畸形、輕度足畸形、中度足畸形或重度足畸形，但本發明並不以此為限。 Preferably, the aforementioned foot deformity type may be a soft flat foot or a structural flat foot, and the occurrence degree of the aforementioned foot deformity may be an ankle deformity, a mild foot deformity, a moderate foot deformity, or a severe foot deformity, but the present invention It is not limited to this.

請參照第3圖，其係繪示本發明另一實施方式之足畸形檢測方法500的步驟流程圖。足畸形檢測方法500包含步驟510、步驟520、步驟530、步驟540以及步驟550。 Please refer to FIG. 3, which is a flowchart illustrating steps of a foot deformity detection method 500 according to another embodiment of the present invention. The foot deformity detection method 500 includes steps 510, 520, 530, 540, and 550.

步驟510為提供足畸形檢測模型，而足畸形檢測模型係經由前述步驟110至步驟140所建立。 Step 510 is to provide a foot deformity detection model, and the foot deformity detection model is established through the foregoing steps 110 to 140.

步驟520為提供受試者之一目標足內側位X光影像資料。較佳地，目標足內側位X光影像資料的影像格式可為數位醫療影像儲存標準協定之影像格式，以將目標足內側位X光影像資料的生理年齡資訊、性別資訊等基本資料儲存於目標足內側位X光影像資料的檔頭中，以利於後續的分析。較佳地，目標足內側位X光影像資料可包含一左足內側位X光影像資料及一右足內側位X光影像資料，以利於對左右二足分別進行更精準的足畸形檢測。 Step 520 is to provide X-ray image data of the medial foot of one of the subjects. Preferably, the image format of the target medial foot X-ray image data may be an image format of a digital medical image storage standard agreement to store basic data such as physiological age information and gender information of the target medial foot X-ray image data in the target. In the file of the medial foot X-ray image data, to facilitate subsequent analysis. Preferably, the target medial foot X-ray image data may include a left foot medial X-ray image data and a right foot medial X-ray image data, so as to facilitate more accurate detection of foot deformities on the left and right feet, respectively.

步驟530為對目標足內側位X光影像資料進行前處理，其係利用影像資料編輯模組整目標足內側位X光影像資料的一影像大小及一影像黑白對比度，以取得一標準化目標足內側位X光影像資料。詳細而言，目標影像資料編輯模組係將目標足內側位X光影像資料的影像大小調整為1024像素×1024像素後，並調整其黑白對比度為0至255之灰階分布的影像強度，進而獲得前述之標準化目標足內側位X光影像資料。較佳地，目標影像資料編輯模組可進一步對目標足內側位X光影像資料進行影像色度擴展處理，其係計算目標足內側位X光影像資料的影像灰階程度，並根據前述之計算結果而依序對目標足內側位X光影像資料之影像像素行、列自動填補色彩，以將呈現灰階色調之目標足內側位X光影像資料轉換為彩色色調，以增加目標足內側位X光影像資料的資料強度，進而提升後續分析的準確度。 Step 530 is for pre-processing the target medial foot X-ray image data. It uses an image data editing module to adjust an image size and an image black and white contrast of the target medial foot X-ray image data to obtain a standardized target medial foot. X-ray image data. In detail, the target image data editing module adjusts the image size of the target medial position X-ray image data to 1024 pixels × 1024 pixels, and adjusts the image intensity of the grayscale distribution with a black and white contrast ratio of 0 to 255, and further Obtain the aforementioned standardized target X-ray image data of the foot. Preferably, the target image data editing module can further perform image chroma expansion processing on the target medial position X-ray image data, which is to calculate the image gray level of the target medial position X-ray image data, and according to the aforementioned calculation As a result, the image pixel rows and columns of the target medial foot position X-ray image data are automatically filled in order to convert the target medial foot position X-ray image data showing grayscale tones to color tones to increase the target medial position X The intensity of the light image data improves the accuracy of subsequent analysis.

步驟540為利用特徵選取模組分析標準化目標足內側位X光影像資料後以得至少一影像特徵值。詳細而言，足畸形檢測方法500可利用特徵選取模組自動地對標準化目標足內側位X光影像資料的影像資訊進行分析，並自動提取對應的影像特徵值，藉以增進本發明之足畸形檢測方法500的足畸形診斷效率。 Step 540 is to obtain at least one image feature value after analyzing the normalized target's medial position X-ray image data using the feature selection module. In detail, the foot deformity detection method 500 can automatically analyze the image information of the normalized target's medial position X-ray image data by using a feature selection module, and automatically extract corresponding image feature values, thereby improving the foot deformity detection of the present invention. Method 500 for diagnosis of foot deformity.

步驟550為利用足畸形檢測模型分析至少一影像特徵值，以判斷受試者之一足畸形類型以及一足畸形發生程度。 Step 550 is to analyze at least one image feature value using a foot deformity detection model to determine a type of foot deformity and the degree of occurrence of a foot deformity.

根據上述實施方式，以下提出具體試驗例並配合圖式予以詳細說明。 According to the above embodiments, specific test examples are provided below and described in detail with reference to the drawings.

<試驗例><Test Example> 一、參照資料庫I. Reference database

本發明所使用的參照資料庫為國軍臺中總醫院(Taichung Armed Forces General Hospital)所蒐集的回溯性去連結化之受檢者的臨床足內側位X光影像資料，為經中國醫藥大學暨附設醫院研究倫理委員會中區區域性審查委員會核准之臨床試驗計劃，其編號為：CRREC-107-079。前述之參照資料庫包含6000位受試者的參照足內側位X光影像資料，且前述之參照足內側位X光影像資料的影像格式皆為數位醫療影像儲存標準協定之影像格式，以將各受試者的生理年齡資訊、性別資訊、病歷號 碼、受試編號等相關資料儲存於影像資料的檔頭中，以利於後續的分析。 The reference database used in the present invention is the retrospectively de-linked clinical foot X-ray image data of the subject collected by the Taichung Armed Forces General Hospital of the National Army. The clinical trial plan approved by the Central Research Regional Review Committee of the Hospital Research Ethics Committee is numbered as CRREC-107-079. The aforementioned reference database contains reference medial foot X-ray image data of 6000 subjects, and the image formats of the aforementioned medial foot X-ray image data are all image formats of the digital medical image storage standard agreement, so that Subject's biological age information, gender information, medical record number Relevant data such as codes and test numbers are stored in the header of the image data to facilitate subsequent analysis.

另外，參照資料庫中的每位受試者的參照足內側位X光影像資料其拍攝時之足部擺放位置及擺放方向皆一致，且各足內側位X光影像資料包含一左足內側位X光影像資料及一右足內側位X光影像資料，以排除使用隨機水平反轉(Random Horizontal Inversion)、隨機平移(Random Translation)、旋轉、剪切等資料擴增處理等方式對左右二足進行足畸形的分析，進而提高本發明之足畸形檢測模型、足畸形檢測系統及足畸形檢測方法的分析準確率。 In addition, the reference medial foot X-ray image data of each subject in the reference database has the same foot placement position and orientation when shooting, and each medial foot X-ray image data includes a left medial side X-ray image data of the right foot and X-ray image data of the medial side of the right foot, in order to exclude the use of random horizontal inversion (Random Horizontal Inversion), random translation (Random Translation), rotation, shearing and other data amplification processing methods The analysis of the foot deformity is performed to further improve the analysis accuracy of the foot deformity detection model, the foot deformity detection system and the foot deformity detection method of the present invention.

二、本發明之足畸形檢測模型Second, the foot deformity detection model of the present invention

本發明之足畸形檢測模型在取得參照資料庫後，各參照足內側位X光影像資料將利用一影像資料編輯模組進行調整，以將各參照足內側位X光影像資料的影像大小調整為1024像素×1024像素，並調整其黑白對比度為0至255之灰階分布的影像強度，以減少不同參照足內側位X光影像資料之間的黑白色度差異以及增加影像的清晰度，並進一步對各參照足內側位X光影像資料之影像像素行、列自動填補色彩，以將呈現灰階色調之各參照足內側位X光影像資料轉換為彩色色調，並取得複數個標準化足內側位X光影像資料。 After obtaining the reference database of the foot deformity detection model of the present invention, each reference medial foot X-ray image data will be adjusted by using an image data editing module to adjust the image size of each reference medial foot X-ray image data to 1024 pixels × 1024 pixels, and adjust the image intensity of the grayscale distribution with a black and white contrast ratio of 0 to 255 to reduce the difference in black and whiteness between different reference medial X-ray image data and increase the sharpness of the image, and further The image pixel rows and columns of each reference medial foot X-ray image data are automatically filled with color to convert each reference medial foot X-ray image data showing grayscale tones to color tones, and obtain a plurality of standardized medial foot X Light image data.

詳細而言，由於目前的深度神經網路模型在運作上需要大量的訓練資料(Training Data，即本發明之足 畸形檢測模型的參照足內側位X光影像資料)來達成穩定收斂及高度的分類準確率，倘若訓練資料的數目不夠充足將會使深度神經網路產生過擬合現象(Overfitting)而導致判斷結果的誤差值過高，致使深度神經網路模型的可信度較低。為了解決前述問題，本發明之足畸形檢測模型透過影像前處理步驟對各參照足內側位X光影像資料進行資料擴增(Data Augmentation)，以透過改變各參照足內側位X光影像資料的幾何大小、影像強度分布以及加入雜訊的方式來降低過擬合現象的發生機率，使其在增加影像資料數據的同時亦能保留各參照足內側位X光影像資料的原始訊息。 In detail, since the current deep neural network model requires a large amount of training data (Training Data, The malformation detection model refers to the medial foot X-ray image data) to achieve stable convergence and high classification accuracy. If the number of training data is insufficient, the deep neural network will cause overfitting and result in judgment. The error value is too high, which makes the deep neural network model less reliable. In order to solve the foregoing problem, the foot deformity detection model of the present invention performs data augmentation (Data Augmentation) on each reference medial foot X-ray image data through an image pre-processing step to change the geometry of each reference medial foot X-ray image data. The size, image intensity distribution, and the way noise is added to reduce the occurrence of overfitting, so that it can increase the image data data while retaining the original information of each reference medial foot X-ray image data.

接著，各標準化足內側位X光影像資料將以特徵選取模組進行分析，以得至少一影像特徵值。詳細而言，特徵選取模組可進一步區別各標準化足內側位X光影像資料中的骨骼區域以及背景區域，並分析骨骼區域之X光影像資料而獲得各標準化足內側位X光影像資料中之影像特徵值。 Next, each standardized foot medial X-ray image data will be analyzed by a feature selection module to obtain at least one image feature value. In detail, the feature selection module can further distinguish the skeletal region and background region in each standardized medial foot X-ray image data, and analyze the X-ray image data of the skeletal region to obtain each of the standardized medial foot X-ray image data. Image feature value.

接著，前述之影像特徵值將透過一卷積神經網路學習分類器進行訓練而達到收斂，以得本發明之足畸形檢測模型。在本試驗例中，足畸形檢測模型將應用於判斷足畸形類型中的柔軟性扁平足或結構性扁平足，並進一步根據分析結果而輸出受試者之扁平足的足畸形發生程度判斷結果。 Then, the aforementioned image feature values will be trained through a convolutional neural network learning classifier to achieve convergence, so as to obtain the foot deformity detection model of the present invention. In this test example, the foot deformity detection model will be used to determine the soft flat or structural flat foot in the type of foot deformity, and further output the judgment result of the degree of foot deformity of the flat foot according to the analysis result.

請參照第4圖，其係繪示本發明之足畸形檢測模型的卷積神經網路學習分類器600的架構示意圖。在第4圖的試驗例中，卷積神經網路學習分類器600為 Inception-ResNet-v2卷積神經網路，其包含複數個卷積層(Convolution)、複數個最大池化層(MaxPool)、複數個平均池化層(AvgPool)以及複數個級聯層(Concat)，以對影像特徵值進行訓練與分析。 Please refer to FIG. 4, which is a schematic diagram showing the architecture of a convolutional neural network learning classifier 600 of the foot deformity detection model of the present invention. In the experimental example in Fig. 4, the convolutional neural network learning classifier 600 is Inception-ResNet-v2 convolutional neural network, which includes a plurality of convolutional layers (Convolution), a plurality of maximum pooling layers (MaxPool), a plurality of average pooling layers (AvgPool), and a plurality of cascade layers (Concat), In order to train and analyze image feature values.

詳細而言，Inception-ResNet-v2卷積神經網路是基於ImageNet可視化數據資料庫的大規模視覺辨識卷積神經網路，且ImageNet可視化數據資料庫裡面的影像資料皆為二維之彩色圖像，因此習知的GoogLeNet卷積神經網路模型在其第一卷積層中具有RGB三通道之濾波器。然而，各參照足內側位X光影像資料的原始影像檔案皆為三維之灰階影像，是以本發明之足畸形檢測模型進一步將包含RGB三通道之濾波器的GoogLeNet卷積神經網路模型透過算術平均法而轉換為單一通道，並將隨機梯度下降法(Stochastic Gradient Descent，SGD)應用於本發明之足畸形檢測模型的預訓練模型神經網路中，以優化其訓練過程，其訓練次數可為100期(Epochs)及採用96Mini-Batch Size之梯度下降法，並透過改變初始學習率(Learning Rates)以進行調變，其中學習率是對神經網路進行訓練時控制權重(weight)和偏差(bias)變化的重要參數，是以本發明之足畸形檢測模型透過調整學習率的數值可進一步確保損失函數(Loss Function)可達穩定收斂。 In detail, the Inception-ResNet-v2 convolutional neural network is a large-scale visual recognition convolutional neural network based on the ImageNet visual data database, and the image data in the ImageNet visual data database are two-dimensional color images Therefore, the conventional GoogLeNet convolutional neural network model has RGB three-channel filters in its first convolution layer. However, each original image file referring to the medial foot X-ray image data is a three-dimensional grayscale image. The GoogLeNet convolutional neural network model including RGB three-channel filters is further transmitted by the foot malformation detection model of the present invention. The arithmetic average method is converted into a single channel, and the Stochastic Gradient Descent (SGD) method is applied to the pre-trained model neural network of the foot malformation detection model of the present invention to optimize the training process, and the number of trainings can be It is 100 periods (Epochs) and a gradient descent method using 96Mini-Batch Size, and is adjusted by changing the initial learning rate (Learning Rate), where the learning rate is the control weight and deviation when training the neural network The important parameter of (bias) change is that the foot deformity detection model of the present invention can further ensure that the loss function (Loss Function) can reach stable convergence by adjusting the value of the learning rate.

在本發明之足畸形檢測模型對影像特徵值進行訓練的過程中，各標準化足內側位X光影像資料的影像特徵值進行二層卷積層及一層最大池化層(MaxPool)處理，以將 所提取之影像特徵值進行最大輸出，並再次重複前述之二層卷積層與一層最大池化層輸出後，利用複數個卷積層進行並行塔(parallel towers)訓練，以完成影像特徵值的初級訓練(Inception)。 In the process of training the image feature values of the foot malformation detection model of the present invention, the image feature values of each normalized medial foot X-ray image data are processed by two layers of convolution layers and one layer of maximum pooling (MaxPool). After extracting the image feature values for maximum output, and repeating the output of the two convolutional layers and one maximum pooling layer described above, use multiple convolution layers to perform parallel towers training to complete the primary training of image feature values (Inception).

在完成前述之初級訓練後，各標準化足內側位X光影像資料的影像特徵值將進行10次(10×)、20次(20×)與10次(10×)的不同深度、不同階層與不同態樣之殘差(Residual)模塊訓練，以對各標準化足內側位X光影像資料的影像特徵值進行訓練並達到收斂。詳細而言，由於Inception-ResNet卷積神經網路在經過複數個階層的權重運算後，因為每一殘差模塊均對各標準化足內側位X光影像資料的影像特徵值進行不同的運算與判斷，致使誤差累積，因此Inception-ResNet卷積神經網路的訓練將會把特定階層的節點運算值拉回到該階層的輸入端再次進行運算，以防止卷積神經網路學習分類器600對前述之影像特徵值進行多層的權重運算訓練後發生梯度消失的退化現象，以及避免誤差累積導致資訊遺失，並可有效提升卷積神經網路學習分類器600的訓練效率。 After completing the aforementioned preliminary training, the image feature values of each standardized medial foot X-ray image data will be performed at different depths, different levels and levels of 10 times (10 ×), 20 times (20 ×), and 10 times (10 ×). Residual module training of different aspects to train the image feature values of the standardized medial foot X-ray image data and achieve convergence. In detail, after the Inception-ResNet convolutional neural network has undergone weight calculations of multiple layers, each residual module performs different calculations and judgments on the image feature values of the standardized medial foot X-ray image data. , Causing errors to accumulate, so the training of the Inception-ResNet convolutional neural network will pull the node operation value of a specific layer back to the input of that layer and perform the operation again to prevent the convolutional neural network learning classifier 600 from performing After the image feature values are subjected to multi-layer weight calculation training, the degradation phenomenon of gradient disappears, and information accumulation is avoided to cause loss of information, and the training efficiency of the convolutional neural network learning classifier 600 can be effectively improved.

在完成深層且重複之殘差模塊訓練後，將依序以一層卷積層、一平均池化層、一取代全局平均池化層(Global Average Pooling 2D，GloAvePool2D)以及一線性整流單元訓練層(Rectified Linear Unit，ReLU)對收斂之影像特徵值進行最終訓練與處理，藉以判斷受試者之扁平足發生類型以及扁平足發生程度。其中，平均池化層可先 對完成殘差模塊訓練之影像特徵值進行計算，以求各影像特徵值的平均值，取代全局平均池化層則可對卷積神經網路學習分類器600的整體網路架構進行正則化(Regularization)處理，防止卷積神經網路學習分類器600在追求低誤差之訓練模式下發生過擬合現象，而導致判斷結果的誤差值過高，最後，線性整流單元訓練層則進一步對完成訓練後之影像特徵值進行激活，並輸出一目標影像特徵值權重數據601，以進行後續的比對與分析。前述之線性整流單元訓練層可避免足畸形檢測模型輸出的目標影像特徵值權重數據601趨近於零或趨近於無限大，以利於後續比對步驟的進行，進而提升本發明之足畸形檢測模型的判斷準確率。 After completing the deep and repeated training of the residual module, a convolutional layer, an average pooling layer, a global average pooling 2D (GloAvePool2D), and a linear rectifier unit training layer (Rectified) Linear Unit (ReLU) performs final training and processing on the convergent image feature values to determine the type of flatfoot occurrence and the degree of flatfoot occurrence in the subject. Among them, the average pooling layer can be first Calculate the image feature values after completing the residual module training to find the average value of each image feature value. Instead of the global average pooling layer, the overall network architecture of the convolutional neural network learning classifier 600 can be regularized ( (Regularization) processing, to prevent the convolutional neural network learning classifier 600 from overfitting in the training mode that pursues low error, which causes the error value of the judgment result to be too high. Finally, the training layer of the linear rectifier unit further completes the training. The subsequent image feature values are activated, and a target image feature value weight data 601 is output for subsequent comparison and analysis. The aforementioned training layer of the linear rectification unit can prevent the target image feature value weight data 601 output from the foot malformation detection model from approaching zero or approaching infinity to facilitate subsequent comparison steps and further improve the foot malformation detection of the present invention. Model judgment accuracy.

接著，前述受試者之扁平足發生類型以及扁平足發生程度判斷結果將進一步整合於參照資料庫中，以對本發明之足畸形檢測模型進行優化，進而使本發明之足畸形檢測模型的訓練效果及判斷準確度進一步提升。 Next, the results of determining the occurrence type of flat feet and the degree of occurrence of flat feet in the aforementioned subjects will be further integrated into the reference database to optimize the foot deformity detection model of the present invention, and further the training effect and judgment of the foot deformity detection model of the present invention. Accuracy is further improved.

藉由上述內容可知，足畸形檢測模型、足畸形檢測系統及足畸形檢測方法透過將足內側位X光影像資料與目標足內側位X光影像資料進行影像標準化前處理，並利用特徵選取模組分析並得至少一影像特徵值後，再以卷積神經網路學習分類器對影像特徵值進行訓練，以快速且準確地判斷受試者之足畸形發生與否、足畸形類型以及足畸形發生程度，並具有高度的檢測再現性。再者，透過包含卷積神經網路學習分類器之足畸形檢測模型能有效提升足畸形檢測 的準確度與敏感度，使本發明之足畸形檢測模型、足畸形檢測系統及足畸形檢測方法可於大量的X光影像資料的情形下提供即時且準確之足畸形檢測評估結果，以作為後續之病情監控或療效評估的依據，使其具有優良的臨床應用潛力。 Based on the above, it can be known that the foot deformity detection model, foot deformity detection system and foot deformity detection method perform image pre-processing on the medial foot X-ray image data and the target medial foot X-ray image data, and use a feature selection module After analyzing and obtaining at least one image feature value, the convolutional neural network learning classifier is used to train the image feature value to quickly and accurately determine whether the subject's foot deformity occurs, the type of foot deformity, and the occurrence of foot deformity. Degree and has high detection reproducibility. Furthermore, a foot deformity detection model including a convolutional neural network learning classifier can effectively improve foot deformity detection The accuracy and sensitivity of the foot deformity detection model, foot deformity detection system, and foot deformity detection method of the present invention can provide instant and accurate foot deformity detection and evaluation results under a large amount of X-ray image data for subsequent follow-up. The basis of disease monitoring or efficacy evaluation makes it have excellent clinical application potential.

雖然本發明已以實施方式揭露如上，然其並非用以限定本發明，任何熟習此技藝者，在不脫離本發明之精神和範圍內，當可作各種之更動與潤飾，因此本發明之保護範圍當視後附之申請專利範圍所界定者為準。 Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Any person skilled in the art can make various modifications and retouches without departing from the spirit and scope of the present invention. Therefore, the protection of the present invention The scope shall be determined by the scope of the attached patent application.