醫療影像處理在診斷上之應用
嘉義大學資工系 教授
時間: 2010年5月3日
柯建全
Outline







Introduction
Object of medical image processing
Imaging devices
applications
Related techniques for Medical
imaging
Research Results
Future works
Introduction



What is Medical imaging?
Why do we need digital image
processing?
What kind of problems are often
caused in medical images?




Blurring caused by respiratory or motion
Low contrast caused by imaging device or
resolution
Complicated textures
Research trends have been transferred
from 2-D to 3-D reconstruction
Introduction (continue)

Integrate all possible methods in
the filed of DIP, pattern recognition,
and computer graphics
Qualitative
 Quantitative


Three categories of imaging in
different modalities



Structural image
Functional image
Molecular image
Object

Help physicians diagnose
(Computer-assisted diagnosis; CAD)




Reduce inter- and intra-variability
Produce qualitative and quantitative
assessment by computer
technologies
Determine appropriate treatments
according to the analyses
Surgical simulation or skills to
reduce possible errors
Medical Imaging Modalities







X-ray
Ultrasound: non-invasive
Computed tomography
Magnetic resonance imaging
SPECT (Single photon emission
tomography)
PET( Positron emission tomography)
Microscopy: OM,LSCM, EM, FMAFM,
STM
X-ray
Ultrasound



2-D sonography
3-D sonography
Doppler color sonography



A series of 2-D projection
Reconstruction
4-D sonography
Computed tomography

無法顯現人體組織和器官功能
MRI



可以觀察活體三度空間的斷層影像
磁振影像取影像時可以適當控制而得到不同參數
的影像，如溫度、流場(flow)、水含量、分子擴
散( diffusion)、 灌流(perfusion)、化學位移
(chemical shift)、功能性(functional MRI) 及
不同核種如氫、碳、磷
人體內若有金屬物質，可能形成假訊號，造成磁
場分佈不 均勻，或是干擾體內儀器運作，所以
裝有心律調整器、人工金屬關節、接受腦血管動
脈者
MRI-structural and functional image
Related techniques

Image processing




Pre-Processing
Segmentation
Registration
Feature Extraction




Shape feature
Texture
Motion tracking
Pattern recognition






Supervised learning
Un-supervised learning
Neural network
Fuzzy
Support vector machine (SVM)
Genetic algorithm
Related techniques

3-D graphic



Virtual diagnose or visualization
Fusion between different modalities
Bio-medical visualization
SPECT-functional image
PET(Positron Emission Tomography )


PET以分子細胞學為基礎，將帶有特殊標記的葡
萄糖合成藥劑注入受檢者體內，利用PET掃瞄儀
的高解析度與靈敏度作全身的掃描，藉由癌細胞
分裂迅速，新陳代謝特別旺盛，攝取葡萄糖達到
正常細胞二至十倍，造成掃描圖像上出現明顯的
「光點」
能於癌細胞的早期(約0.5公分)準確地判定癌細
胞，提供醫師作為診斷及治療的依據，診斷率高
達87-91％，30歲以上的成年人及有癌症家族史
的民眾，建議每隔1～2年做一次PET檢查。
PET (Positron emission tomography)
Cell identification via microscope

Tools

Traditional optical microscope


Fluorescent microscope


Identification for nuclear and gene
expression
Laser confocal microscope


Stained specimen
Identification from 2-D to 3-D
Multi-photon microscope

Identification from 2-D to 3-D
Applications in a hospital


Assist surgeon plan surgical operation or
diagnose
Picture archiving system (PACS)




將醫療系統中所有的影像，以數位化的方式儲存，並經
由網路傳遞至同系統中，供使用者於遠側電腦螢幕閱讀
影像並判讀。
Telemedicine
Surgical simulation: Medical Visualization，
Surgical augmented Reality, Medicalpurpose robot, Surgery Simulation，Image
Guided Surgery，Computer Aided Surgery
Estimate the location, size and shape of
tumor
PACS System
Virtual Surgery
Related techniques

Classification of normal or abnormal
tissues such as carcinoma


Pre-processing: Contrast enhancement,
noise removal, and edge detection
Lesion segmentation: extract contours
of interest
thresholding
 2-D segmentation
 3-D segmentation based on voxel data
 Color image processing

Our study




Virtual colonoscopy
Bone tumor segmentation with
MRI and virtual display
Breast carcinoma based on
histology and cytology
Visualization of cell activities
using confocal laser scanning
microscope
Virtual colonscopy-Browsing or navigation
within a colon








Helical CT –patients injected contrast
medium
Re-sampling—Voxel-based
Interpolation
Automatic segmentation (seed)
 threshloding
Determination of the skeleton of the colon
Connected-Component Labeling
Surface rendering and volume rendering
Extraction of suspicious sub-volumes for
diagnosis
Automatic segmentation
Determination of the skeleton of the
colon
Display and measurement
Bone tumor segmentation with MRI and
virtual display—Contrast medium

Otsu thresholding





Region growing
Tri-linear interpolation
Morphological post-processing
Surface rendering
Measurement
Histogram of T1 weighted and T2 weighted
(a) 0度
(b) 45度
Classification of Breast Carcinoma
開始
輸入組織影像
(1524*1012)
色彩分離
(RGB)
影像分割
(Gray level、Otsu、Laplacian)
特徵參數分析
(導管比例、管腔個數、組織紋理...)
貝式網路判斷
正常
異常
系統判斷為正常
12
6
系統判斷為異常
1
11
準確性
76.67%
敏感度
有效性
64.71%
92.31%
螢究研構重維三像影鏡微顯焦軛共射雷用應
動活之胞細光-例為胞細癌頸宮子以
細胞結構簡介
雷射共軛焦顯微鏡之成像原理
雷射共軛焦顯微鏡解析度:
46
雷射共軛焦顯微鏡雜訊生成之原因

大部分的生物樣本，潛在一些特性會降低
CLSM 影像解析度：


第一個是具有散射的特性
第二個特性是折射率的不匹配(refractiveindex mismatch)所產生的。

47
由於折射率的不匹配會引入球面像差，而使得縱
向與橫向解析度變差。
散射的特性
散射光強度:
48
散射的特性
49
研究影像
50
實驗方法與架構
選取重要影像
重要影像初始輪
廓偵測
Snake自動偵測
重要影像輪廓
二質化細胞質
區域
排除對比較差的
細胞核輪廓
利用適應性
Snake自動偵測
整體輪廓
重建三維細胞
51
分割蛋白質劇烈
活動區域
選取重要影像

52
由於雷射共軛焦取像環境的限制，在細胞邊界處通常訊號
較弱且較為模糊，使得初始輪廓分割相當困難，因此本步
驟選取出對比最好的影像作為重要影像(分割時的初始切
片)，偵測出其初始輪廓。
重要影像初始輪廓偵測
53
Snake自動偵測重要影像細胞輪廓去除人工雜物

54
由於雷射共軛焦取像的特性，其邊緣部分通常模糊不清，
因此初始輪廓的結果偶爾會產生過小得輪廓，本研究排除
輪廓長度小於100pixel得輪廓(可能是雜訊的輪廓)，僅以
較大的輪廓作為初始輪廓。
利用適應性Snake自動偵測整體影像輪廓
55
利用適應性Snake自動偵測整體影像輪廓
56
輪廓重疊偵測-濾除對比較差細胞核輪廓
偵測

細胞核訊號分佈模式:


57
重要影像前半部
重要影像後半部
= 細胞質區域Mean
= 細胞核區域Mean
輪廓重疊偵測-濾除對比較差細胞核
輪廓偵測
58
59
60
細胞內蛋白質反應劇烈區域的分割
61
細胞內蛋白質反應劇烈區域的分割

62
初始點決定與K-Means群法偵測最亮區域
不同角度顯示
63
其他範例(Case3)
64
不同角度顯示
65
三維重建資料比較
66
體積比例量測
Case 1
Case 2
Case3
域區質胞細(Voxels)
224396
521563
629562
蛋質白活域區烈劇動
3819
3387
7785
1.7019%
0.649%
1.236%
(Voxels)
例比
67
效能評估
Process
Case1:Time(Sec)
Case2:Time(Sec)
Case3:Time(Sec)
68
74
83
7
6
7
建重維三體整
9
10
9
間時體整
84
90
99
割分域區質胞細
區烈劇動活質白蛋
割分域
68
拉普拉斯三維平滑
69
拉普拉斯三維平滑
70
Requirements for medical image
processing system in clinical diagnosis




Automatic and less human interaction
Qualitative and quantitative measurements
Stable and reliable (experiments with much
more cases)
Performance evaluation



True positive, true negative, false positive, false
negative
Accuracy, sensitivity, and specificity
Receiving operating characteristic curve (An index
for evaluating the effectiveness of classification
 Optimal classification threshold
 Area under ROC approach 1 – better classification
ROC curve
Analyses of prognosis on breast cancer
for a stained tissue



Microscopy with different resolution
(400 or 100) for a stained tissue
Fluorescent microscopy in detecting
the number of chromosome
Immunohistochemistry(IHC)
Her-2 IHC image
Fish image(normal)
Fish image (abnormal)
Preliminaries or problems ?





Blurring often caused by patient
motion or respiration
Clinical opinion or idea obtained from
an experienced surgeon
Non-absolute answers at some
specific conditions
Trade-off between complexity and
performance
Large variations for different image
modality
Preliminaries or problems ?



Automation is necessary so as to
help physicians
Prove identification accuracy—
comparison between manual and
image processing approaches
Classification based on neural
network, pattern recognition, or
fuzzy,.. etc is crucial in practical
applications

Thanks for your attention!