Automated abdominal segmentation of CT scans for body composition analysis using U-Net

Alexander Weston, Panagiotis Korfiatis PhD, Timothy Kline PhD, Kenneth Philbrick PhD, Petro Kostandy PhD, Motokazu Sugimoto

MD, Naoki Takahashi MD, Bradley Erickson MD, PhD

Radiology Informatics Lab, Mayo Clinic

CMIMI ConferenceSeptember 9th 2018

Body CompositionAmount and distribution of fat and muscle

Linked to outcomes in a number of conditions

diabetesheart diseasecancerkidney diseasesurgical outcomesquality of life

Topic of research –obesity paradoxmetabolic syndromevisceral vs subcutaneous obesity

Woman with Mirror, Francisco Botero

Measuring Body Composition

Several clinical techniquesBody mass indexSkin-fold testGrip strengthAir-displacement plethysmographyBioelectric Impedance AnalysisHydrostatic WeighingDual-energy X-ray absorptiometry

CT – most available?

Limitations: Segmentation

Dataset

Parameter N = 1429

Male, number (%) 878 (61)

Age range, min - max 29 – 97

Average age, mean (SD) 66 (11.3)

Scan date range 1997–2015

Parameter N (%) or mean ± SD

mA 362 ± 148

kVp

100 171 (6)

120 2384 (88)

130 53 (2)

140 89 (3)

Other 10 (1)

Slice Thickness

<2mm 37 (1)

2 - 3mm 1105 (41)

3 - 4mm 277 (10)

5mm 1152 (43)

>5mm 136 (5)

Patient parameters Acquisition parameters

ModelModel

Layers 12

Activation Function

Hyperbolic tangent (tanh)

Output activation

Softmax

Hyper-parametersLearning rate 0.0005

Optimizer Adam

Loss Categorical Crossentropy

Batch Size 16

TrainingEpochs: (early stopping) 196

Software Keras with TF backend

Hardware Nvidia Tesla V100 GPU (16GB RAM)

Results on the test setCompartment Prediction vs

gold-standardInter-rater Prediction vs

gold-standard

Dice score,mean (SD)

Dice score, mean (SD)

Area difference, mean (SD)

Subcutaneous adipose tissue

0.98 (0.01) 0.95 (0.02) 1.1% (2.0%)

Muscle 0.96 (0.02) 0.93 (0.02) 2.1% (2.5%)

Visceral adipose tissue 0.94 (0.12) N/A 1.4% (1.8%)

Visceral organ tissue 0.97 (0.01) N/A 3.7% (12.2%)

Bone 0.98 (0.02) 0.95 (0.02) 4.3% (3.6%)

Visual results on the test set

Prediction Gold-standard

853 cm2 845 cm2

177 cm2 171 cm2

418 cm2 415 cm2

Prediction Gold-standard

103 cm2 104 cm2

186 cm2 185 cm2

89 cm2 89 cm2

Scan Gold-standardPredictionObese

Healthy-weight

Subcutaneous adipose tissue

Muscle

Visceral adipose tissue

Visceral organ tissue

Results on a secondary datasetScan

Gold-standardPrediction

Compartment Prediction vs gold-standard, liver

Dice score, mean (SD) Area diff, mean (SD)

SubQ adipose tissue 0.94 (0.05) 6.6% (8.1%)

Muscle 0.92 (0.04) 4.3% (4.8%)

Viscera (combined) 0.99 (0.01) 0.9% (1.0%)

3D segmentation

Annotation by iterative deep-learning (AID)Original Update 1 Update 2

Conclusions and implicationsGenerate accurate metrics related to patient health in seconds, not hours

Increasingly accurate algorithm through iterative retraining

In research –Faster analysisMore accurate data2D vs 3D

In clinical practiceRuns on existing, unused clinical dataHigh quality, novel metrics of patient health

Thank you,Radiology Informatics Lab

Bradley Erickson, MD, PhD

Naoki Takahashi, MD

Arunnit Boonrod, MD

Panagiotis Korfiatis, PhD

Bill RyanPetro Kostandy, MD

Zeynettin Akkus, PhD

Timothy Kline, PhD

Issa Ali

Kenneth Philbrick, PhD

Motokazu Sugimoto, MD

Scott SquiresAtefeh Zeinoddini, MD

Gian Marco Conte, MD

Supplemental Slides

Metrics

BADice = ( 2 * 2 ) / ( 3 + 4 ) = 0.57

TPF = ( 2 ) / ( 3 ) = 0.67

FPF = ( ( 3 + 4 – 2) – 3 ) / ( 3 ) = 0.67|A| = 3|B| = 4|A∩B| = 2

A∩B2 ∙ | A ∩ B || A + B |

Dice =

Metrics

Dice score

Jaccard coefficient

True Positive Fraction (TPF)

False Positive Fraction (FPF)

𝐴𝐴 ∪ 𝐵𝐵𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐵𝐵

2 ∙ | A ∩ B || A + B |

| A ∩ B || A U B |

| A ∩ B || A |

| (B U A) – A || A |

Biomarkers of body composition

Image from Mayo Clinic, https://www.mayoclinic.org/diseases-conditions/metabolic-syndrome/multimedia/apple-and-pear-body-shapes/img-20006114

visceral adiposity

intramuscular adipose deposits

lean muscle

subcutaneous adiposity

Dual-Energy X-Ray Absorptiometry (DXA)

Image from Mayo Clinic, https://mayoclinichealthsystem.org/locations/springfield/services-and-treatments/radiology-and-imaging/dexa-scan

Image from Queensland X-Ray, https://www.qldxray.com.au/our-services/body-composition-scanning/

Low energy whole-body X-Ray Developed for osteoporosis,

can also measure adiposity Additional scan

Results against gold-standard

Compare to: Lee, H., Troschel, F. M., Tajmir, S., Fuchs, G., Mario, J., Fintelmann, F. J., & Do, S. (2017). Pixel-level deep segmentation: artificial intelligence quantifies muscle on computed tomography for body morphometric analysis. Journal of digital imaging, 30(4), 487-498.

Results against expert physician

Results on HCC Dataset

Bland-Altman plots

Performance on L4 vertebra of model trained on L3