An Insight Software Consortium Tutorial:

Medical Image Analysis with ITK

and Related Open-Source Software

http://www.InsightSoftwareConsortium.org

This course introduces attendees to select open-source efforts in the field of medical image
analysis. Opportunities for users and developers are presented.
The course particularly focuses on the open-source Insight Toolkit (ITK) for medical image
segmentation and registration. The course describes the procedure for downloading and installing
the toolkit and covers the use of its data representation and filtering classes. Attendees are shown
how ITK can be used in their research, rapid prototyping, and application development.

LEARNING OUTCOMES
After completing this course, attendees will be able to:
• contribute to and benefit from open-source software for medical image analysis
• download and install the ITK toolkit
• start their own software project based on ITK
• design and construct an image processing pipeline
• combine ITK filters for medical image segmentation
• combine ITK components for medical image registration

INTENDED AUDIENCE
This course is intended for anyone involved in medical image analysis. In particular it targets
graduate students, researchers and professionals in the areas of computer science and medicine.
Attendees should have an intermediate level on object oriented programming with C++ and must
be familiar with the basics of medical image processing and analysis.

COURSE LEVEL
Intermediate

COURSE LENGTH
Full-day (6.5 Hours)

COURSE OUTLINE
1) The Insight Software Consortium: contributing and using open-source
2) The architecture and installation of the Insight Toolkit
3) Segmentation methods of the Insight Toolkit
4) Registration methods of the Insight Toolkit
5) Image IO using the Insight Toolkit
6) The Image-Guided Surgery Toolkit: architecture overview
7) Using the Insight Toolkit with TK/TCL
8) Applications of the Insight Toolkit:
VTK/ITK, The Virtual Soldier, Industrial Research and NA-MIC’s Slicer

CONTRIBUTORS
Many people contributed to the development of these presentations. The ISC is grateful to those
peoples’ support of open source software and education. Major contributors include the following:

Stephen Aylward is Chief Medical Scientist at Kitware, Inc. He joined Kitware in December 2005,
after serving as a tenured Associate Professor of Radiology, Computer Science, and Surgery at the
University of North Carolina at Chapel Hill. At Kitware, Stephen is working with leaders in the
open-source movement on research and development for clinical image analysis software. Dr.
Aylward is also President of the Insight Software Consortium and an associate editor of IEEE
Transactions on Medical Imaging. He has computer science degrees from Purdue (BS), Georgia
Institute of Technology (MS), and UNC (PhD).

Luis Ibanez is a Research Engineer at Kitware, Inc. He received a BSc in Physics in 1989 and a
MSc in Optics in 1994 from the Universidad Industrial de Santander (Bucaramanga, Colombia). He
received a PhD in 2000 from the Universite de Rennes I (Rennes, France). From 1999 to 2001 he
was Research Assistant Professor in the Division of Neurosurgery at the University of North
Carolina at Chapel Hill. His current research interest the application of genomics paradigms to
computation.

Josh Cates is a member of the research staff of the Scientific Computing and Imaging Institute at
the University of Utah's School of Computer Science. He received his BS degree in Biology in 1995
and MS in Computer Science in 1999 from the University of Tennessee. His interests include
software engineering and problems in computer vision, including image processing (differential
geometry and p.d.e.-based methods), image segmentation, and computed tomography.

Lydia Ng is a member of the Informatics group at the Allen Institute for Brain Science. She
received a BE in Electrical Engineering and a BSc in Computer Science in 1994 from The University
of New South Wales (Sydney, Australia). She received a Ph.D. in 2000 from Macquarie University
(Sydney, Australia). Her research interests include: image registration, PDE based image
processing methods, motion estimation and development of software for medical image processing
and analysis.

Julien Jomier is a research member at Kitware, Inc. He joined Kitware in December, 2005 after
being a research faculty member in the Computer-Aided Diagnosis and Display Laboratory at the
University of North Carolina at Chapel Hill. He has extensive C++ programming and software
design experience as well as medical image analysis and data handling expertise. He is currently
continuing his development of ITK’s spatialObject library and his freely available spatial object
viewer library. Julien is also contributing to the development of the Insight Journal, the Image
Guided Surgery Toolkit, and numerous other open-source efforts.

William Lorensen is a Graphics Engineer in the Electronic Systems Laboratory at GE's Corporate
Research and Development Center in Schenectady, NY. He has over 35 years of experience in
computer graphics and software engineering. He is currently working on algorithms for 3D medical
graphics and scientific visualization.
 The Insight Software
Consortium

http://www.InsightSoftwareConsortium.org

Educating users and developers

of open-source, medical image
analysis software

Insight Software Consortium

2005 Overview

Outline
• History of the ISC

• Charter of the ISC

• Contributing to and benefiting from

open-source…

Insight Software Consortium

2005 Overview

1
 History
• Initiated by the original developers of the
National Library of Medicine’s Insight Toolkit
– Drs. Terry Yoo, Bill Lorensen, Will Schroeder, …
– 2003 Developers’ Meeting

• Motivation
– Near term:
• Hold the copyright of ITK
– Long term:
• Charter of the ISC…

Insight Software Consortium

2005 Overview

Charter
ISC incorporated in New York State in 2005.
A non-profit, educational consortium.
The purpose of this Consortium is to

1. support the maintenance of,

2. guide the development of, and

3. promote the use of

open-source, medical image analysis

software, data, and publications

for teaching, research, and commercial endeavors

Insight Software Consortium

2005 Overview

2
 What the ISC means to you…
• The ISC provides proven, documented, open-source
software environments and data for developers and
users involved in medical image analysis research.

• The haunting past of open-source:

“You get what you pay for…”
– Open-source is poorly written code
– Open-source is inefficient
– Open-source is poorly documented

– Users: My task is “special.”

…I can do “it” better myself…

– Developers: Users are “annoying.”

Insight Software Consortium
…weird platforms, errors, emails…
2005 Overview

Process of ISC Open-Source

Ultimate goal: ISC certification

Steps:
1. Use Development Environment
2. Contribute to Insight Software Journal
3. Participate in Community Review
4. Seek Protection
5. Support Distribution and Education
6. Benefit from Community support

Insight Software Consortium

2005 Overview

3
 1. Development Environment
• ISC Certified
– ITK
– IGSTK

• Companion projects
– CMake
– Dart / Dashboards
– Cable / CSwig
– DCMTK
– FLTK
– Slicer
– VTK

Insight Software Consortium

2005 Overview

Safety of development environment

Policies of the ISC:
– Programming style
– Backward compatibility
– Intellectual property
– Documentation
– Open-source

– Tested
– Maintained
– Cross-platform

Insight Software Consortium

2005 Overview

4
 I can do “it” better myself…
Segmentation • Mayo Clinic
• Statistical, Fuzzy Logic, Markov • Harvard / Brigham and Women’s
Random Fields, Mixture Modeling, Hospital
Parzen Windows, Nearest Neighbor, • Cognita, Inc.
K-Means, … • Imperial and King’s College London
• Level Set, Finite Element, Region • University of Iowa
Growing, Hybrid, Watershed, • Georgetown University
Connected Components, • Carnegie Mellon University
Parameterized Models, … • GE Research / Harvard
Registration • Kitware, Inc.
• Insightful / UPenn
• Rigid, Similarity, Affine, Vector Field, • UNC / UPitt
Hierarchical, Quaternion, Versor,
• UPenn / Columbia
Parameterized Deformation, Euler,
• University of Utah
3D/2D, …
• + Over 70 machine configurations
• Mutual Information, Normalized
recompile ITK every night
Correlation, Demons, Mean
Squared, Landmark, … • 20,000+ downloads
Insight Software Consortium
2005 Overview • 30+ countries

2. Insight Software Journal

http://www.insight-journal.com
• Initiated by Dr. Luis Ibanez (Kitware) and developed
by Julien Jomier, Zack Galbreath, …

• Open, online publication

– Upload, download, and review
– By-attribution distribution license – not copyright transfer

• Components of an open-science submission:

1. Paper that describes and demonstrates the method
2. Source code that implements the method
3. The data needed to generate the demonstration
Insight Software Consortium
2005 Overview

5
 3. Community Review
Insight Journal
– Review by peers
– Review by automated compilation process

Criterion
– Code
– Documentation
– Utility
– Adherence to policy

Iterative and only a component

– Revisions, respond to reviewers
Insight Software Consortium – Cite in technical journals
2005 Overview

4. Protection
Once consensus builds for incorporation into an ISC
project…

• Transfer copyright to the ISC

– A level of protection for developers
– Ensure the open-source status of the software for users
• Right to modify and distribute
• Teaching, research, and commercial

• Challenges
– Acquiring copyright from developers
– Copyright infringement (known and unknown)
– Patented code (known and unknown)
Insight Software Consortium
2005 Overview

6
 5. Distribution and Education
– Integration into ISC development environment

– Tutorials at international conferences:

SPIE Medical Imaging 2003-2005, MICCAI 2003-2004, IEEE
Visualization 2003-2005, SIGGRAPH 2005

– Freely available courseware:

Developed collaboratively by CMU, UPitt, Rutgers, and Kitware Inc.

– Examples, Documentation, Users Lists

– Books:

Insight Software Consortium

2005 Overview

6. Community Support
www.InsightSoftwareConsortium.org
• Web portal
• Certification - not sourceforge
• Dedicated to medical image analysis
– Jobs, News, Links

• Benefit from ISC certification and development

environment…education…

• Needs
– Contributions: Comments, Commented Code, …
– Long term dedication
• Good of the community vs. dissertation / proposal / …
Insight Software Consortium
2005 Overview

7
 Community Limitations
• FDA
– FDA does not approve software libraries
– FDA only approves applications

• GNU General Public License (GPL) software

has limited distribution
– Complicates corporate acceptance of and
contribution to the open-source initiative
– Consider instead the BSD or MIT license

Insight Software Consortium

2005 Overview

Open-Science
-- Dr. Terry S. Yoo, NLM

Open Source + Open Data = Open Science

– Share data
– Share code
– Understanding methods
• Assumptions, parameters, & outputs
– Comparison of methods
– Extension of methods

– MICCAI 2006, Copenhagen: Open-Source Workshop

Insight Software Consortium

2005 Overview

8
 Board of the ISC
• Stephen Aylward
• Joshua Cates
• Luis Ibanez
• Bill Lorensen
• Dimitris Metaxas
• Jim Miller
• Lydia Ng
• Will Schroeder
• Ross Whitaker

Insight Software Consortium

2005 Overview

9
 ITK-Overview

Insight Software Consortium

What is ITK
z Image Processing
z Segmentation

z Registration

z No Graphical User Interface (GUI)

z No Visualization

1
ITK Sponsors

The National The National

Institute for Dental and Science
Craniofacial Research Foundation

The National Institute of Neurological

Disorders and Stroke

ITK Developers

2
ITK Developers
GE CRD Insightful U Penn
Bill Lorensen Lydia Ng Dimitris Metaxas

Harvard BWH* U Penn* Columbia U.*

Ron Kikinis Jim Gee Celina Imielinska

Kitware UNC-CH U Tennessee

Will Schroeder Stephen Aylward Ross Whitaker

U Pittsburgh* U Utah*
George Stetten Ross Whitaker

* indicates a subcontractor.

ITK by the Numbers

z March 2000
– First code check-in
z 1300
– # of nightly builds
z 1062
– tests run nightly
z 41
– # of platforms ( software + hardware )
z 700
– # of classes
z 1600
– # of files with code

3
ITK by the Numbers
z 400K
– # of lines of code
z 100K
– # of lines of test code
z 35K
– # of lines of examples
z 150K
– # of lines of Applications
z 240
– weekly t-cons
z 50
– unique developers

ITK by the Numbers

z 1032
– # of users subscribed to the mailing-list
z 400
– # of emails posted monthly to the users-list
z 819
– # of pages in the Software Guide PDF document
z 1800
– # of monthly hits to the URL of the Software Guide PDF
z 1900
– # of monthly hits to the URL of the Tutorial PDF
z 2400
– # of monthly hits to the source code files (.zip + .tar.gz)

4
How to Integrate ITK
in you application
C++ Glue Code
ITK GUI

Image {MFC,Qt,
Processing wxWin
FLTK}

Visualization
{OpenGL, VTK}

What do I need ?
C++ Compiler

gcc 2.95 – 4.0

Visual C++ 6.0
Visual C++ 7.0 CMake
Visual C++ 7.1
Visual C++ 8.0 www.cmake.org
Intel 7.1
Intel 8.0
IRIX CC
Borland 5.5
Mac - gcc

5
 Step 1. Download ITK

Stability Live on the

Edge
Release tar
files CVS

http://www.itk.org

Insight.tgz CVS anonymous

Copying ITK from the CD

/Source/
InsightToolkit-2.4.1.zip
InsightToolkit-2.4.1.tar.gz

6
 Step 2. Download CMake

Latest
ITK
CMake
Requires
Release

http://www.cmake.org

Install Get the

2.2.3
Binaries Binaries

Installing CMake from the CD

/CMake/
cmake-2.2.3.tar.gz
cmake-2.2.3-x86-win.zip
cmake-2.2.3-x86-linux.tar.gz
CMakeSetup223.exe

7
 Step 3. Configure ITK
Source Tree Out Binary Tree
Source Build d
!
de
en
ITK ITKb m
m
co
Re

Common Common

Algorithms Algorithms

BasicFilter BasicFilter

Numerics Numerics
In
IO Source IO
Build

Configuring ITK – MS-Windows

z Run CMake

z Select the SOURCE directory

z Select the BINARY directory

z Select your Compiler

8
Configuring ITK – MS-Windows

Configuring ITK – MS-Windows

z Disable BUILD_EXAMPLES

z Disable BUILD_SHARED_LIBS

z Disable BUILD_TESTING

z Click “Configure” to configure

z Click “OK” to generate project files

9
 Configuring ITK – Unix
z Create the BINARY directory (mkdir)

z Change directory to the BINARY directory (cd)

z Set the environment variables CC and CXX

setenv CC /usr/bin/gcc; setenv CXX /usr/bin/g++ OR

export CC=/usr/bin/gcc; export CXX=/usr/bin/g++

z Type ccmake with argument the SOURCE directory

Configuring ITK – Unix

10
Configuring ITK – Unix
z Disable BUILD_EXAMPLES

z Disable BUILD_SHARED_LIBS

z Disable BUILD_TESTING

z Type “c” to configure

z Type “g” to generate the Makefiles
z Type “make” to start building

Building ITK

11
Building ITK

z Open ITK.dsw in the Binary Directory

z Select ALL_BUILD project
z Build it
…It will take about 15 minutes …

Building ITK

12
Building ITK

z Open ITK.sln in the Binary Directory

z Select ALL_BUILD project
z Build it
…It will take about 15 minutes …

Building ITK

z Most of ITK classes are C++ Templates

z Basic libraries are small
they only contain non-templated classes

z Basic libraries are built in about 15 min

13
Step 5. Verify the Built

Libraries will be found in

ITK_BINARY / bin / { Debug, Release }

Step 5. Verify the Build

The following libraries should be there

z ITKCommon z ITKIO
z ITKBasicFilters z ITKStatistics

z ITKAlgorithms z ITKMetaIO

z ITKNumerics z itkpng

z ITKFEM z itkzlib

14
Step 6. Use ITK from an
external Project

Copy
“HelloWorld.cxx”
“CMakeLists.txt”
from the Run • Select Source Dir
Examples/Installation CMake • Select Binary Dir
Directory
into another
directory

Using ITK – Hello World

15
Using ITK – Hello World

Step 6. Use ITK from an

external Project
z accept the default in
CMAKE_BACKBARD_COMPATIBILITY

z leave empty EXECUTABLE_OUTPUT_PATH

z leave empty LIBRARY_OUTPUT_PATH

z Set ITK_DIR to the binary directory

where ITK was built

16
Step 7. Build Sample Project

z Open HelloWorld.dsw (or .sln)

generated by CMake
z Select ALL_BUILD project
z Build it
…It will take about 3 seconds …

Step 7. Build Sample Project

17
Step 8. Run the example

z Locate the file HelloWorld.exe

z Run it…

z It should produce the message:

ITK Hello World !

Starting your own project

z Create a clean new directory
z Write a CMakeLists.txt file

z Write a simple .cxx file

z Configure with CMake

z Build

z Run

18
Step 9. Writing CMakeLists.txt
PROJECT( myProject )

FIND_PACKAGE ( ITK )
IF ( ITK_FOUND )
INCLUDE( ${ITK_USE_FILE} )
ENDIF( ITK_FOUND )

ADD_EXECUTABLE( myProject myProject.cxx )

TARGET_LINK_LIBRARIES ( myProject ITKCommon ITKIO)

Step 10. Writing myProject.cxx

#include "itkImage.h"
#include "itkImageFileReader.h"
#include "itkGradientMagnitudeImageFilter.h"

int main( int argc, char **argv ) {

typedef itk::Image<unsigned short,2> ImageType;
typedef itk::ImageFileReader<ImageType> ReaderType;
typedef itk::GradientMagnitudeImageFilter<
ImageType,ImageType> FilterType;

ReaderType::Pointer reader = ReaderType::New();

FilterType::Pointer filter = FilterType::New();

reader->SetFileName( argv[1] );
filter->SetInput( reader->GetOutput() );
filter->Update();
return 0;
}

19
Step 11. Run CMake

Step 12. How to find

what you need

http://www.itk.org/ItkSoftwareGuide.pdf

http://www.itk.org/Doxygen/html/index.html

z Follow the link Alphabetical List

z Follow the link Groups

z Post to the insight-users mailing list

20
21
The ITK Software Guide is
freely available as a PDF
document at

www.itk.org/
ItkSoftwareGuide.pdf

Its paper version can be ordered

from Amazon.com and from
Kitware’s e-store.

22
Enjoy ITK !

23
 ITK
Architecture

Kitware Inc.

ITK Basics
z C++ Generic Programming
z Data Pipeline

z Multi-threading

z Streaming

z Exceptions

z Events / Observers
z Tcl, Python and Java wrapping

1
Generic Programming
Example: STL Standard Template Library

Abstraction of Types and Behaviors

std::vector< T >

std::vector< int >

std::vector< double >
std::vector< char * >
std::vector< Point >
std::vector< Image >

itk::Image

itk::Image< PixelType , Dimension >

itk::Image< char , 2 >

itk::Image< char , 3 >
itk::Image< char , 4 >
itk::Image< float , 2 >
itk::Image< RGB , 3 >
itk::Image< unsigned short , 2 >
itk::Image< itk::Vector<float,2> , 2 >

2
 namespaces
Avoid naming collisions

itk::
itk::Statistics::
itk::fem::
itk::fem::itpack
itk::bio

Your favorite keyword

typedef

typedef itk::Image< char , 2 > ImageType

typedef itk::ImageFilter< ImageType , ImageType > FilterType

otherwise...

itk::ImageFilter< Image< char , 2 > ,

Image< char , 2 > > FilterType

3
 Smart Pointers

Smart
Pointer

Smart Smart
Object
Pointer Pointer
counter=3
counter=2
counter=1
counter=0

Self - Delete

SmartPointers
typedef itk::Image< char , 2 > ImageType
typedef itk::ImageFilter< ImageType , ImageType > FilterType

FilterType::Pointer filter = FilterType::New();

ImageType::Pointer image = filter->GetOutput();

Pointer notation
filter->Update();

NO NEED FOR
filter->Delete();

4
 Const Correctness

Knowing constancy is Insight.

Not knowing constancy leads to disaster.

Tao Te Ching, XVI. Lao Tsu

Const Smart Pointers

typedef itk::Image< char , 2 > ImageType
typedef itk::ImageFilter< ImageType , ImageType > FilterType

FilterType::Pointer filter = FilterType::New();

ImageType::ConstPointer image = filter->GetOutput();

Can only invoke “const” methods

image->GetSpacing ();

Compiler error for “non-const” methods

image->SetSpacing ( spacing );

5
Creating an Image
typedef itk::Image< char , 3 > ImageType

ImageType::Pointer image = ImageType::New();

ImageType::SizeType size;
size[ 0 ] = 512; // x direction
size[ 1 ] = 512; // y direction
size[ 2 ] = 50; // z direction

ImageType::IndexType start;
start[ 0 ] = 0; // x direction
start[ 1 ] = 0; // y direction
start[ 2 ] = 0; // z direction

Creating an Image
ImageType::RegionType region;
region.SetSize( size );
region.SetIndex( start );

image->SetRegions( region );
image->Allocate();
image->FillBuffer( 0 );

ImageType::SpacingType spacing;
spacing[ 0 ] = 0.83; // x direction
spacing[ 1 ] = 0.83; // y direction
spacing[ 2 ] = 2.15; // z direction

image->SetSpacing( spacing );

6
Exercise 3

Streaming
Processing Large Images

Input Output
Filter
Image Image

7
Image Regions
LargestPossibleRegion

BufferedRegion

RequestedRegion

Data Pipeline

Image Filter Image

Filter Image

Image Filter Image

8
 Simple Image IO

Image
ImageFileReader Image
File

Filter

Image
ImageFileWriter Image
File

Simple Image IO

Image
ImageFileReader Image
File

PNGImageIO MetaImageIO AnalyzeImageIO

GIPLImageIO VTKImageIO DICOMImageIO

CustomImageIO Loadable Factories

9
Simple Image IO
#include “itkImage.h”
#include “itkImageFileReader.h”
#include “itkImageFileWriter.h”

typedef itk::Image< char , 2 > ImageType;

typedef itk::ImageFileReader< ImageType > ReaderType;
typedef itk::ImageFileWriter< ImageType > WriterType;

ReaderType::Pointer reader = ReaderType::New();

WriterType::Pointer writer = WriterType::New();

reader->SetFileName( “inputImage.dcm” ); // DICOM

writer->SetFileName( “outputImage.hdr” ); // Analyze

writer->SetInput( reader->GetOutput() );
writer->Update();

Exceptions
Error Management

Application Layer

ITK Layer

10
Exceptions

try
{
filter->Update();
}
catch( itk::ExceptionObject & exp )
{
std::cerr << exp << std::endl;
}

Exercise 4
Image
ImageFileReader Filter
File

Image
Caster ImageFileWriter
File

11
 Events and Observers
Itk::Command
Itk::Object
Event
itk::Command
Event

Event
itk::Command
Event

Event
itk::Command

itk::Command

Events and Observers

Common Events

AnyEvent()
StartEvent()
EndEvent()
ProgressEvent()
IterationEvent()

12
 Events and Observers

Itk::Object Itk::Command

Itk::ProcessObject MyCommand
Execute()

Itk::FilterXY AnyEvent
AddObserver( )

MyEvent

Events and Observers

Itk::Object Itk::Command

Itk::SimpleMemberCommand
Itk::ProcessObject

AnyEvent

MyEvent MyClass
Itk::FilterXY MethodX()
AddObserver( )

13
 Exercise 5
Image File

ImageFileReader

Filter ProgressEvent()
StartEvent()
EndEvent()

ImageFileWriter
CommandObserver
Image File Execute()

GUI Communication

GUI Layer
Widget Widget

Observer
Command
Callback

ITK Layer
FilterX FilterX

14
Enjoy ITK !

15
 Introduction to ITK Segmentation

The Insight Consortium

presented by
Josh Cates
Scientific Computing and Imaging Institute
University of Utah

Session Objectives
• Review some important •Brief introduction to image
considerations when using processing “frameworks” in
ITK filters ITK
• Overview of low-level image •See examples
processing filters –simple command line
–denoising / scale space –full applications with user
–feature extraction interaction
• Overview some common
segmentation filters
–connected component
–pixel classification
–watersheds
–level-set methods

1
 Important Facts about ITK Filters
1. Most filters are N dimensional
2. Many filters run multi-threaded
3. Filters are implemented within frameworks
4. Most filters are documented
5. Filters are regression tested
6. Data type matters
7. Pixel spacing matters
8. Filters are not full applications
9. Filters require parameter tuning

Big Picture: The Role of ITK Filters

Raw Data
Dataflow architecture: ITK filters
fit together to produce
segmentation applications.
Filtering
Moral: ITK segmentation filters
Preprocessing

linear
nonlinear are not complete applications –
Feature components in a pipeline.
Extraction
differential
geom.
edge detection

Segmentation Visualization
region growing binary volume
watersheds meshes
level-sets labeled image
implicit surfaces

2
 Big Picture: The Role of ITK Filters
Raw Data

User
Interface
Filtering

Feature
Extraction

Segmentation Visualization

Where to go to really learn to use the filters

http://www.itk.org
Doxygen Manual Pages
Software Guide
Algorithm descriptions
Tutorials for using algorithms
Theory Book
Insight into Images, Terry Yoo, ed.
Users Mailing List
CVS Repositories:
Insight/Examples
InsightApplications

3
 What is an ITK Image?
Templated over data type
May be vector-valued LargestPossibleRegion
Templated over dimensionality BufferedRegion
Spacing, origin & orientation
information
Pixels may be addressed directly RequestedRegion
or through an assortment of
iterators
Interpolators may also be used
In general, implemented as a flat,
C++ array
Buffers may be imported &
exported

Thresholding
BinaryThresholdImageFilter

4
 Thresholding
ThresholdImageFilter

Intensity Transformations
CastImageFilter
RescaleIntensityImageFilter
ShiftScaleImageFilter
NormalizeImageFilter
SigmoidImageFilter

5
 Intensity Transformations
SigmoidImageFilter

Image Morphology
BinaryErodeImageFilter
BinaryDilateImageFilter

6
 Image Morphology
GrayscaleErodeImageFilter
GrayscaleDilateImageFilter

Edge Detection & Feature Extraction

CannyEdgeDetectionImageFilter
LaplacianImageFilter
GradientMagnitudeImageFilter

7
 Edge Detection & Feature Extraction
GradientMagnitudeRecursiveGaussianImageFilter

Edge Detection & Feature Extraction

DerivativeImageFilter

8
 Image Denoising: Linear
MeanImageFilter

Image Denoising: Linear

MedianImageFilter

9
 Image Denoising: Linear
BinomialBlurImageFilter
RecursiveGaussianImageFilter
DiscreteGaussianImageFilter

Image Denoising: Nonlinear

BilateralImageFilter
CurvatureFlowImageFilter
GradientAnisotropicDiffusionImageFilter
CurvatureAnisotropicDiffusionImageFilter

10
 Image Denoising: Nonlinear
VectorGradientAnisotropicDiffusionImageFilter
VectorCurvatureAnisotropicDiffusionImageFilter

Linear Diffusion
Destroys and moves edges

11
 Nonlinear Diffusion
Preserves Edges

Geometric Transformations
TransformImageFilter

12
 Remember
Filters generalize to N dimensions
Filters “work” on arbitrary data types, but only some
data types make sense
Filters are implemented within extensible frameworks

Image Neighborhood Framework

Neighborhood is a set of pixels local adjacent to one
another
Used to implement algorithms where result at pixel i
is based on some computation within the
neighborhood of i
Iterators abstract the API for working with a
neighborhood of pixels in arbitrary dimensions
Operators are applied to image neighborhood to
perform various calculations
Boundary conditions are handled automatically

13
Neighborhood Iterator Framework

Neighborhood Iterators

14
 ITK Segmentation Algorithms
Classification / Thresholding
Region Growing
Watersheds
Level-set Methods
“Hybrid” Methods

Statistical Pattern Classification

Idea
Find disjoint regions in a feature space
Classify image pixels according to feature vectors
Classifier
Multiple membership functions (each represents one
possible class) return scores from feature vectors
Decision rule based on scores
Implemented using the ITK statistics subsystem
(framework)

15
 Statistical Pattern Classification Framework
1. Measurement vectors are input to membership functions
2. Membership functions feed scores to decision rule
3. Decision rule compares scores and returns a class label

Training Data Container(+)

Membership Function(+) Parameter Estimator(+)

Data Container Data Container

(Test) Classifier (Classification Result)
•Sample •Membership Sample
•Image •Classified Image

Decision Rule

Classifier Framework Example

Segmentation of gene expression images of the mouse brain
at the Allen Institute for Brain Science, Seattle WA*
Designed with ITK components
Itk::ImageModelEstimatorBase & related classes
itk::Statistics::MembershipFunctionBase & related classes
itk::DecisionRuleBase & related classes
Other filters: morphological, connected component labeling, threshold
filters etc.
Requirements
Modular design / code reuse
Careful memory usage (each 2D image is on the order of 150 MPixels)
Robust and adaptable to change in image quality

*Courtesy of Dr. Lydia Ng, Allen Institute for Brain Science, www.brainatlas.org.

16
 Classifier Framework Example

(a) (b) (c)

(a) Original ISH stained image; (b) heat map representing the membership
values of each pixel representing expression, and (c) the final threshold
mask generated from the heat map

Region Growing
Idea
Start with set of seed pixels – region
Iteratively include neighboring pixels that satisfy
membership criteria
Membership criteria – similarity based metrics
Intensity interval, Regional statistics
Algorithms
Simple to complex variations
Easy to write using ND neighborhood tools
Several strategies
Connected Threshold, Otsu Threshold, Neighborhood
Connected, (Vector) Confidence Connected, Isolated
Connected

17
 Confidence Connected Filter
Threshold based region growing
Mean and standard deviation of region determine
upper and lower thresholds
Recomputes thresholds at intervals

Compute µ Flood fill with threshold

and σ interval
of region [µ−kσ, µ+kσ]

Repeat N times

Region Growing Segmentation

ConfidenceConnectedImageFilter

original white matter ventricle gray matter

(60,116) (81,112) (107,69)

smoothing iterations 5
smoothing time step 0.125
C.C. multiplier 2.5
C.C. iterations 5

18
 Region Growing Segmentation
VectorConfidenceConnectedImageFilter

Watershed Segmentation
Image treated as a topological relief map – intensity
represents height
Gradient descent defines segmented regions
Set of all pixels whose paths of steepest descent terminate
in same local minimum
Bounded by image features
“No parameters”

19
 ITK Watershed Transform

Watershed Depth
Image (filtered) Feature Extraction Watershed Transform
“Edge Map”

The Oversegmentation Problem

Watershed transform produces too many regions

One per local minimum
Especially in noisy or highly detailed data
To alleviate oversegmentation
Hierarchical approach – merge adjacent regions according
to increasing watershed depth
A. P. Mangan, R. T. Whitaker, Partitioning 3D surface
meshes using watershed segmentation, IEEE Transactions
on Visualization and Computer Graphics 5 (4) (1999) 308–
321.

20
 Watersheds Hierarchy
Enforce minimum watershed depths
at successively higher levels.

Watershed Depth
Undersegmented
Watershed Depth Threshold

Watershed Transform

= basin

Boolean Operations
Oversegmented

On Sub-trees
(e.g. user interaction)

Initial Watershed
Transform

Example: Watersheds GUI

InsightApplications/SegmentationEditor
Data with
Watershed overlay Segmentation
transform in progress

Sliders manipulate
watershed depth
and position in the 3D isosurface
hierarchy. rendering
Watershed Depth Threshold

21
Example: Watersheds GUI

Example: Watersheds GUI

22
 LevelSet Surface Modeling Theory
kth Level Set: set of all points of value k
Embed N dimensional surface as ZERO level set of
N+1 dim. volume
Model N dim. surface movement as evolving
wavefront – forward differences solution to PDE

Segmentation Using Level Sets

Define speed term(s) to go to zero at edges – data
fitting term
Surface motion/speed based on intensity-based
features

Solve the level-set equation where

23
 PDE Solver Framework
Purpose
Nonlinear image processing – e.g. anisotropic diffusion
Moving wave fronts – level set models
Deformable registration
Generic framework
Separate solvers from equations – interchangeable code
objects

PDE Solver Hierarchy

Finite Finite
Difference Difference
Solver Function

Level
Sparse Set
Dense Diffusion
Threaded
Narrow Sparse
Segment
Band .
Aniso. 4th
Diffusion Diff Order
Curv.
Limited
Deformable
Registration
Other Solvers
Other Functions

24
 Constructing a PDE Filter

Solver Object Subclass

Input Output
Image Image
(Filtered)
Function
Object

Parameters

LevelSet Segmentation Framework

Finite Difference
Finite Difference Solver
Solver

Sparse-Field Level-Set Solver

“Feature”
Image Level-Set Segmentation Filter
Curvature
Function

User-Defined LS Seg. Filter

Level Set
Function

Output
Model
Canny Laplacian
Initial Shape Edges Active- Function
Model Detection Function Contours Threshold
Function Function Function

25
 LevelSet Segmentation Algorithms in ITK
Fast marching
Geodesic active contours
CURVES (vessel segmentation)
Intensity interval (scalar and vector)
Canny edge distance
Laplacian edges
… and more.

Example: Threshold based LS Segmentation

Speed function (positive inside object)
Similar to confidence connected filter
Points Outside Points Inside Points Outside
Model Speed

Image Intensity

Low High
Threshold Threshold

•Lefohn, Kniss, Hansen, Whitaker, “GPU-Based, Interactive, Level-Set Models for Volume
Visualization and Analysis”, IEEE Vis 2003
•Lefohn, Whitaker, Cates, “Interactive Level-Set Models for Brain Tumor Segmentation”,
MICCAI 2003

26
 Example: LevelSet Segmentation GUI

Multiscale LevelSet 3D Segmentation

Scale Seed surface Data

1/4

1/2

1/1

27
 Advanced Features in the PDE Framework
Parallel Solvers – Narrowband,
Sparse field

50
45
40
35
Speedup
processor

30
(vs. 1

25
20 SGI Origin 3000
15 64 600 Mhz Processors
10
5
0
0 20 40 60

Number of processors

4th Order Flow Segmentation Framework

Finite Difference Solver
Finite Difference Solver
Sparse-Field Level-Set Solver Manifold Solver
(Refitting) (Normals Processing)

Input Level-Set Segmentation Filter

Image Curvature
Function

User-Defined LS Seg. Filter

Speed
Function

Output
Canny LaPlacian Model
Shape Edges Active- Function
Detection Function Contours
Initial Function
Threshold
Function Function
Model

28
 Segmentation Using 4th Order Flows
Special 4th order solver plugs into LS segmentation
framework – no change in function objects.

Speed term only Speed + Anisotropic (not real-time)

4th order terms

SNAP Tool
Aimed at clinical users – easy to learn and use
Implements various ITK level set algorithms
Implements both edge-based and
region competition
geodesic snake
methodology

InsightApplications/Snap

29
 SNAP User Interface

Scalpel tool for editing

segmentations
SNAP Segmentation Wizard
with VCR Controls

SNAP Segmentation of the

Caudate Nuclei Manual Segmentation
Dialog for setting evolution parameters

InsightApplications/Snap

“Hybrid” Segmentation Methods

Apply several algorithms in sequence
Utilize strengths of each
Speed / accuracy tradeoffs — e.g. connected component
vs. level-sets
Maximize use of information in data — e.g. region based
plus boundary based

30
 Hybrid Method: Region Growing + Level Sets
Generate initial model using confidence connected
filter
Fit to data using level-set methods – minimize
distance to Canny edges

Canny LS LS
Confidence Initial Segmentation Image
Connected model Filter (float)

Feature
Image

Anisotropic
Image
Diffusion

Confidence Connected + LevelSet Result

Initial confidence-
connected result
LS Speed Term: distance
from Canny edges

Post-processing with
Canny LS segmenter
Data: Warfield, Nabavi, Butz, Tuncali, Silverman, “Intraoperative segmentation and nonrigid
registration for image guided therapy, in: MICCAI'2000, SpringerVerlag, 2000, pp.176-185.

31
 enjoy ITK!
http://www.itk.org

32
 Overview: ITK
Registration Methods
Lydia Ng
Allen Institute for Brain Science

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 1

Overview
Introduction
ITK Registration Framework
„ Component Overview
„ Registration Strategies
„ GUI Communication
Deformable Registration in ITK
Resources
Application: Allen Brain Atlas
SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 2

1
 Introduction
Image registration
„ Process of finding the spatial transform that
maps all points from one image to their
homologous points in another image
Medical applications
„ Monitoring change in an individual
„ Fuse information from multiple sources to aid
clinical interpretation
„ Compare one subject to another
SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 3

Intra-subject Registration
Repeated image scans of the same
subject can be used to capture the effect
of disease development, treatment
progress, contrast bolus propagation etc.
Registration can be used to compensate
for differences in patient placement
Subtraction/overlay of registered images
can be used for visualization and
quantification

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 4

2
 3D Breast MR Contrast Uptake
Deformable Registration
Image data courtesy of University of Washington.

Unregistered:

Registered:

Image Data: 192 x 192 x 13 pixels, 0.94 x 0.94 x 8.00 mm

Registration: 2 levels, MI, LBFGS, B-spline deformation

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 5

Inter-subject Registration
Creation of mean images and atlases
„ Useful for objective and quantitative
assessment of abnormalities
„ Computed deformation field used to encode
pattern of anatomic variability within a
population
Atlas based segmentation
„ Automatic segmentation by mapping to a
labeled atlas

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 6

3
 3D Inter-subject MR-PD
Deformable Registration
Image data courtesy of N. C. Andreasen, University of Iowa, Psychiatry Dept.

Moving Volume
Fixed Volume

Image Data: 256 x 256 x 192 pixels, 1.02 x 1.02 x 1.02 mm

Registration: 3 levels, Demons algorithm, histogram matching
SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 7

3D Atlas-based Segmentation
Image data courtesy of N. C. Andreasen, University of Iowa, Psychiatry Dept.
Subject (Fixed) Volume
Atlas (Moving) Volume

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 8

4
 Multi-modality Registration
Combine or fuse information from multiple
sources to aid clinical interpretation
Some modalities
„ MR: soft tissue discrimination; lesion
identification
„ CT: bone localization; surgical guidance
„ PET/SPECT: functional information; tumor
localization

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 9

PET/CT Fusion Image data courtesy of University of Washington.

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 10

5
Image Registration Classification
Registration criteria
„ Quantitative measure of a “good match”
match”
„ Focus on intensity based measures
Spatial transform type
„ Allowable mapping from one image to another
„ Rigid versus non-
non-rigid
Optimization algorithm used
„ Optimize transform parameters with respect to match
criteria
Image interpolation method
„ Value of image at non-
non-grid position
SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 11

ITK Registration Framework

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 12

6
 ITK Registration Framework
Generic framework for building intensity based
registration algorithms
Each functionality encapsulated as components
Components are inter-changeable allowing a
combinatorial variety of registration methods
Components are generic
„ Can be used outside the registration framework

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 13

Registration Framework Components

Registration Framework
Fixed Transform
Image Parameters
Image Similarity
Metric

Moving Resampled
Image Image
Image Cost Function Resample Image
Interpolator Optimizer Filter

Transform

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 14

7
itk::ImageRegistrationMethod
Simple helper driver class that connects
up the components and starts the
optimizer
Examples in Software Guide

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 15

Examples/Registration
Declare types Connect up observers
Instantiate objects via New() StartRegistration()
StartRegistration()
Connect components and „ Should do this inside try/catch
images to the driver using Set block
methods Get the last transform
Set initial transform parameters
parameters Create registered image using
„ Don’
Don’t forget! ResampledImageFilter
Setup each component
„ optimization parameters: step
length, convergence …
„ MI parameters: number of
samples,

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 16

8
 Component Overview
Transforms
„ Forward versus inverse mapping
„ Rigid and non-rigid
Interpolators
Image to image similarity metrics
„ Intra-modality
„ Inter-modality (information-theoretic methods)
(Optimizers)
SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 17

itk::Transfrom
Encapsulates the mapping of points and vectors
from an “input” space to an “output” space
ITK provides a variety of transforms from simple
translation, rotation and scaling to general affine,
kernel and B-spline transfoms
Forward versus inverse mapping
Parameters representation
“Transform Jacobians”

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 18

9
 Forward and Inverse Mappings
Relationship between points of two images
can be expressed in two ways:
„ Forward: pixel of input (moving) image
mapped onto the output (fixed) image
„ Inverse: output (fixed image) pixels are
mapped back onto the input (moving) image

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 19

Forward Mapping

Input Output
Image Image

Input image pixel is mapped onto the output image

Output pixels with more than one hit: overlap
„ Value must be accumulated from overlapping pixels
Output pixels with no hits: hole

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 20

10
 Inverse Mapping

Input Output
Image Image

Output pixels are mapped back onto the input image

Output pixel value must be interpolated from a
neighborhood in the input image
Scheme avoids any holes and overlaps in the output
image because all pixels are scanned sequentially

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 21

Registration and Inverse Mapping

The registration framework uses inverse
mapping
The transform component maps points
from the fixed image space to the moving
image space
x′ = T(x p )

Point in moving image space Point in fixed image space

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 22

11
 Transform Parameters
Each transform class is defined by a set of
parameters
„ Translation, angle of rotation etc
Represented as a flat array of doubles to
facilitate communication with generic ITK
Optimizers
Transform parameters define the search space
for the optimizer
Goal of registration
„ Find the set of transform parameters that result in the
best value of an image similarity metric

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 23

Transform Jacobian
Some metrics require the knowledge of the
“transform Jacobian” in order to compute metric
derivatives
The “transform Jacobian” is a matrix whose
elements are the partial derivatives of the output
point with respect to the transform parameters
∂xi′
J ij =
∂p j

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 24

12
 itk::TranslationTransform
Maps all points by adding a constant vector:
x′ = x + t

Parameters:
„ i-th parameter represent the translation in the i-th
dimension p=t
Jacobian in 2D:

⎡1 0⎤
J=⎢ ⎥
⎣0 1 ⎦

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 25

itk::Euler2DTranform
Represents a rotation and translation in
2D
⎡ x′ ⎤ ⎡cos θ − sin θ ⎤ ⎡ x ⎤ ⎡t x ⎤
⎢ y′⎥ = ⎢ sin θ +⎢ ⎥
cos θ ⎥⎦ ⎢⎣ y ⎥⎦ ⎣t y ⎦
⎣ ⎦ ⎣

Parameters: p = {θ , t x , t y }

⎡ − (sin θ )x − (cos θ ) y 1 0⎤
Jacobian: J=⎢ ⎥
⎣+ (cos θ )x − (sin θ ) y 0 1⎦
Don’t forget TRANSFORM
CENTERING and PARAMETER SCALING!
SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 26

13
 Center of
Transformation/Rotation
Transformation/rotation is about the origin {0,0}
Can represent rotation at arbitrary rotation center but
at the expense of optimization performance,
especially for large auxiliary translation
x′ − c m = M (x − c f ) + t
x′ = Mx + (t + c m − Mc f )
144244 3
t′

Alternatives:
„ center of image
„ center of mass of an object

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 27

Parameter Scaling
Different type of parameters (translation vs.
angle vs. scaling) have different dynamic ranges
„ E.g. a unit change in rotation has a much larger
impact than a unit change in translation
Differences in scale appears as long narrow
valleys in the search space making optimization
difficult
Rescaling the parameters can help fix this
problem

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 28

14
 itk::Euler3DTranform

Represents 3D rotation and translation

„ Rotation about each coordinate axis
„ Ordering matters!

x′ = R Z R Y R X x + t
or
x′ = R Z R X R Y x + t

Parameters:
p = {θ x ,θ y , θ z , t x , t y , t z }

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 29

Alternative 3D Rigid Transforms

itk::QuaternionRigidTransform
„ 3D rotation represented by a quaternion
„ Represented by 4 numbers
„ Similar to axis/angle representation
„ Unit quaternion is equivalent to pure rigid
„ Does not suffer from “Gimbal lock”
lock”
itk::VersorRigidTransform
„ Strictly the rotational part of a quaternion (always rigid)
„ Represented by 3 numbers
Both quaternion and versor components do not form
vector spaces
„ Specialized optimizers required!

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 30

15
 I will not register images in pixel space
I will not register images in pixel space
I will not register images in pixel space
I will not register images in pixel space
I will not register images in pixel space
I will not register images in pix

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 31

Image Spacing and Origin

Medical image volume are typically anisotropic
„ In-
In-plane pixel size smaller than inter-
inter-slice spacing
A transform is rigid only with respect to physical
coordinates and not pixel coordinates
„ PhysCoord = PixelCoord x ImageSpacing + ImageOrigin
The registration is always with respect to physical
coordinates
Make sure spacing and origin information is set correctly
in the images!

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 32

16
 itk::AffineTransform
General affine transform can represent
rotation, scaling, shearing and translation
Parameters: matrix coefficient + translation
(6 in 2D, 12 in 3D)
Parallel lines are preserved
Don’t forget TRANSFORM
CENTERING and PARAMETER SCALING!

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 33

itk::BSplineDeformableTransform
Represent a low-dimensional smooth
deformable warp
Deformation field represented by B-splines
coefficient on a regular grid
Parameters: B-spline coefficient for each
dimension at each grid position

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 34

17
 B-Spline Grid Placement

Grid defined by origin, spacing, size

SplineOrder = 3

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 35

itk::InterpolateImageFunction
When a point is mapped from one image
space to another image space, it will
generally be mapped to a non-grid position
Interpolation is needed to compute the
intensity value at the mapped position

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 36

18
 Choice of Interpolation Method
Interpolation affects smoothness of metric
space
Interpolations computed 1000’s of times in
a single optimization cycle
Trade-off efficiency with ease of
optimization

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 37

Interpolation Schemes
itk::NearestNeighborInterpolateFunction
„ Assumes image is piecewise constant
itk::LinearInterpolateFunction
„ Assumes image is piecewise linear
itk::BSplineInterpolateFunction
„ Underlying image represented using B-
B-spline basis
functions
„ On connection, image of B-
B-spline coefficients is
computed

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 38

19
 itk::ImageToImageMetric
Measures how well a transformed moving
image “matches” the fixed image
The most critical component
Scalar function of the transform
parameters
„ For given, fixed image, moving image,
transformation type and interpolation type

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 39

Metrics, Transforms and

Interpolators
The metric typically samples
points within a defined region
Registration Framework
of the fixed image Fixed
For each point, the Image
Image Similarity
Metric
corresponding moving image
point is obtained using the Moving
transform component Image Image
Interpolator
The interpolator component is
then used to compute the
moving image intensity at the Transform

mapped position

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 40

20
 Metrics and Optimizers
The metric is used by the
optimizer to evaluate the
Registration Framework
quantitative criterion at various Fixed
positions in the transform Image
Image Similarity
Metric
parameter search space
For gradient-
gradient-based optimizer, Moving
the metric must also provide Image Image Cost Function
Interpolator Optimizer
the metric derivatives w.r.t
each transform parameter
„ Use chain rule with moving Transform
image derivatives and
transform Jacobian

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 41

Choice of Metric
Highly dependent on the registration problem to
be solved
„ Single-
Single-modality
„ Multi-
Multi-modality
„ Large capture range
„ Requires close initialization
Examples:
„ Mean squares, normalized correlation and mutual
information

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 42

21
 Mean Squares Metric
Interpolate the moving image

S (p F , M , T ) =
N

∑ (F (x ) − M (x′ ))
1 2
i i
N i

where Over a user specified

x′i = T(x i , p )
Transform from fixed fixed image region
image point to moving
image point

Simple to compute
Large capture range
Restricted to mono-modality applications
Linear differences in intensity results in poor
similarity measure

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 43

Mean Squares Metric

NearestNeighbor BSpline
Optimal Value at
Zero

Linear

Metric range:
image dependent

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 44
Translations

22
Normalized Correlation Metric
N

∑ F (x )M (x′ )
S (p F , M , T ) = −1×
i i
i
N N

∑ F (x )∑ M (x′ )
i
2
i
i
2
i

Pixel-wise cross-correlation between fixed

and moving image intensity
Factor -1 added to work with minimum
seeking generic optimizers
Insensitive to multiplicative difference

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 45

Normalized Correlation Metric

Linear Optimal Value at BSpline
-1

Metric range:
1 to -1

Translations
SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 46

23
 Mutual Information Metric
Information theoretic entity that
qualitatively measures how much
information is gained about one RV
(intensity in one image) by the knowledge
of another RV (intensity in another image)

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 47

Mutual Information Metric

Introduced in the context of multi-modality
registration by two different groups: Viola and
Wells (1997) and Collignon et al. (1995)
MI well suited since actual form of dependency
between the two RVs does not have to be
specified
MI defined in terms of entropy
„ Measure of information contained in a piece of data

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 48

24
 Entropy
Two RVs: A and B
Marginal entropies:
H ( A) = −∫ p A (a ) log p A (a ) da
H (B ) = −∫ pB (b) log pB (b) db

Joint entropy:

H ( A, B ) = − ∫ p AB (a, b ) log p AB (a, b ) da db

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 49

Mutual Information
If A and B independent:
H ( A, B ) = H ( A) + H ( B)

If A and B not independent:

H ( A, B ) < H ( A) + H ( B )

The difference is MI:

I ( A, B) = H ( A) + H ( B) − H ( A, B)

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 50

25
Estimating the Probabilistic Models
Entropy is defined in context of a probabilistic
model of the data source
„ Typically there is no direct access to these models
Marginal and joint densities has to be estimated
from the image data
„ Parzen windowing
„ Kernel density estimate
„ Histogram binning

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 51

Parzen Windowing

Density function is constructed by superimposing kernel functions

functions
centered on the intensity samples obtained from the image
Kernel type
„ Gaussian, boxcar, B-
B-Spline
Kernel width (crucial)
„ Depend on dynamic range of data
„ Normalize data
Number of samples

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 52

26
 Estimating Entropy
Using the density estimate, approximate
entropy integral by a sum
„ Evaluate at discrete positions/bins uniformly
spread within dynamic range
„ Computing a sampled mean using another set
of intensity samples randomly drawn from
image

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 53

Flavors of Mutual Information

itk::MutualInformationImageToImageMetric
„ Viola and Wells
„ New samples (50-100) each iteration
itk::MattesMutualInformationImageToImageMetric
„ Mattes et al, Thevenaz and Unser
„ One set of samples (5-10% of image)
itk::MutualInformationHistogramImageToImageMetric
„ Use all pixels

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 54

27
 Mutual Information Metric
ViolaWells Mattes
Optimal Value at
maximum

Metric range:
Image
dependent

Translations
SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 55

Joint Histograms: Mono-modality

Images Aligned Translated by 0 to 20 pixels

White = zero value Misalignment causes

Black = highest value dispersion

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 56

28
Joint Histograms: Multi-modality
Images Aligned Translated by 0 to 20 pixels

White = zero value Misalignment causes

Black = highest value dispersion

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 57

Joint Histograms and Registration

Seek transform that produces a small number of
high value bins and as many zero bins as
possible == minimizing joint entropy
Joint entropy metric favors transforms which
causes the images to be far apart as possible
(i.e. minimum overlap)
MI overcomes this problem by also trying to
maximize the information contributed by each
image in the overlap region

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 58

29
 Registration Strategies
Naive application of registration may not
work
Coarse to fine strategy
„ Improves computational speed and
robustness
Low to high dimensional transform
„ Translation to rigid to deformable
„ Sparse to fine grid

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 59

itk::MultiResolutionImageRegistrationMethod
Helper driver class to perform multi-
multi-resolution
registration

Initialize

Initialize

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 60

30
3D CT to MR -T1 Rigid Registration
MR-T1

Fixed Image: MR-T1, 256 x 256 x 52 pixels, 0.78 x 0.78 x 3.00 mm

Moving Image: CT, 512 x 512 x 44, 0.41 x 0.41 x 3.00 mm
Registration: 4 levels, MI, gradient descent, quaternion rigid

Images provided as part of the project: “Retrospective Image Registration Evaluation”,

NIH, Project No. 8R01EB002124-03, Principal Investigator, J. Michael Fitzpatrick, Vanderbilt University, Nashville, TN.

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 61

3D PET to MR -T2 Rigid Registration

MR-T2

Fixed Image: MR-T2, 256 x 256 x 26 pixels, 1.25 x 1.25 x 4.00 mm

Moving Image: PET, 128 x 128 x 15, 1.94 x 1.94 x 8.00 mm
Registration: 3 levels, MI, gradient descent, quaternion rigid

Images provided as part of the project: “Retrospective Image Registration Evaluation”,

NIH, Project No. 8R01EB002124-03, Principal Investigator, J. Michael Fitzpatrick, Vanderbilt University, Nashville, TN.

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 62

31
 Registration To GUI
Communication
Drive progress bar
Observe registration progress
„ Interrogate metric values
„ Interrogate current parameters
„ Display current results
Resample and display ROI
Change component parameters
Terminate registration

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 63

ITK Observers/Commands
itk::Object can invoke events
„ Start, End, Progress, Iteration
itk::Object maintains a linked list of event
observers
Observers register themselves to an
object and declare the type of events they
are interested in

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 64

32
 Observing Registration
ITK Optimizers typically execute an iterative process
Most Optimizers invoke an IterationEvent at the end of
each iteration
Observing IterationEvents provide periodic
communication facilitating progress feedback and
opportunity to control the registration
itk::MultiResolutionImageRegistrationMethod also
invokes an IterationEvent between levels to give the GUI
an opportunity to change components and/or parameters

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 65

ITK Registration Examples

ITK Software Guide
„ 9+ examples
InsightApplications
„ ImageRegistration
„ MRIRegistration
„ MultiResMIRegistration
„ MutualInformationEuler2DRegistration
Landmark Initialized 3D Registration Tool
„ CADD Lab, UNC

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 66

33
 Deformable Registration in ITK
Low dimensional transformations can be handled in the basic
registration framework
For high dimensional transformations:
„ Finite Element Methods (FEM)
„ Finite Different Methods
Metric optimization
„ Mean squares, normalized correlation, MI
FEM framework additionally supports regularization and landmark
constraints:
„ Diffeomorphic constraints
„ Linear elastic and large deformation (fluid and transient-
transient-quadratic)
models
Simple examples in Software Guide

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 67

Resources
ITK Software Guide
“Insight into Images”
Images”,
„ Yoo (ed), 2004
“A Survey of Medical Image Registration”
Registration”
„ Maintz and Viergever,
Viergever, Medical Image Analysis, 1998
“Handbook of Medical Imaging: Medical Image Processing and Analysis
Analysis””
„ Fitzpatrick et al (eds
(eds),
), SPIE, 2000
“Handbook of Medical Imaging: Processing and Analysis”
Analysis”
„ Bankman (ed.), Academic Press, 2000
“Medical Image Registration”
Registration”
„ Hajnal et al (eds
(eds),
), CRC Press, 2001
“Mutual-
Mutual-Information-
Information-Based Registration of Medical Images: A Survey”
Survey”
„ Pluim et al, IEEE-
IEEE-TMI, 22(8), 2003
IEEE Transaction of Medical Imaging Special Issue
„ November 2003

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 68

34
Data for Registration Testing
Retrospective Registration Project
„ http://www.vuse.vanderbilt.edu/~image/registration/
„ Multi-
Multi-modality images of the brain
Internet Brain Segmentation Repository
„ http://www.cma.mgh.harvard.edu/ibsr/
„ MR-
MR-TI images of brain with segmentations
BrainWeb
„ http://www.bic.mni.mcgill.ca/brainweb/
„ Simulated brain database
International Consortium For Brain Mapping (ICBM)
„ http://www.loni.ucla.edu/ICBM/ICBM_Databases.html
http://www.loni.ucla.edu/ICBM/ICBM_Databases.html

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 69

Application: Allen Brain Atlas

Detailed, cellular-resolution, genome-wide map
of gene expression in the mouse brain
„ http://www.brain-
http://www.brain-map.org
The Allen Reference Atlas is being developed to
allow comparison of the gene expression data in
a common anatomic framework
„ Created from 528 coronal sections from fresh frozen
tissue
„ 3D reconstruction is needed to take into account
differences between specimens and slicing
orientation

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 70

35
 SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 71

3D Atlas Reconstruction
Inherently 2D data
„ Spatial relationship between sections is lost
„ Artifacts (distortions, tears, smears,
inhomogeneity etc) appear independently on
per section basis
Blind application of section-to-section
registration could result in non-realistic 3D
reconstructions requiring post-processing
or constraints
SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 72

36
 Initialization

528 coronal images were converted to grayscale, downsampled,

downsampled, cropped
and padded to same size
Brain tissue mask were created to direct from where registration samples
information
Badly damaged sections were removed

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 73

Rigid Registration

Section to section rigid registration results in a 3D volume which

which is
fairly self-
self-consistent
Volume appears bent as registration tend to straighten high gradient
gradient
features (e.g
(e.g hippocampus, cerebellum)

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 74

37
Unbending Rigid Registration

Unbend the volume by applying the inverse of the

smoothed transform series
Result is then used as initialization for the next step:
affine registration

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 75

Affine Registration

Section to section affine registration results in very

smooth structures but causes severe warping
„ Due to straightening out of the hippocampus and cerebellum

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 76

38
De-warping Affine Registration

De-
De-warp the volume by applying the inverse of the
smoothed transform series
Result is then used as initialization for the next step:
deformable registration

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 77

Deformable Registration

Section to section deformable registration

„ Constraint movement to less than 125 microns
„ Cleaned up jags in hippocampus and cerebellum
„ Appearance of very fine line structures

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 78

39
Hippocampus (Azimuth)

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 79

Hippocampus (Elevation)

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 80

40
Cerebellum (Elevation)

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 81

Ventricles (Elevation)

SPIE 2006: Medical Image Analysis with ITK and Related Open-
Open-Source Software, February 11, 2006 82

41
 ITK Input/Output

Kitware Inc.

Overview

z IO Factory Mechanism

z Image File IO

z Transform File IO

z SpatialObject File IO

z Logger

1
IO Factory Mechanism

Why do we need a Factory?

How many file formats can you list?

…

(and many more exist…)

2
 Supported file formats
2D Only
- JPEG (.jpg/.jpeg)
- Bitmap (.bmp)
- PNG (.png)

2D/3D
- Analyze 3.5 - Siemens
- GIPL (.gipl) - Stimulate
- RAW (.raw) - TIFF
- DICOM - VTKImage
- GE 4x,5x - NRRD
- IPLCommon - LSM
- MetaImage (.mha/.mhd) - NIFTI

How does the Factory work?

reading

Image Filename Image IO itk::Image

Factory

Can you read this file?

…
JPEGImageIO DICOMImageIO

TIFFImageIO AnalyzeImageIO

3
How does the Factory work?
writing

itk::Image Image IO Image Filename

Factory

Can you write this file?

…
JPEGImageIO DICOMImageIO

TIFFImageIO AnalyzeImageIO

Image File IO

4
Reading my first image
#include <itkImageFileReader.h>

typedef unsigned char PixelType;

typedef itk::Image<PixelType, 2> ImageType;

itk::ImageFileReader<ImageType>::Pointer reader
= itk::ImageFileReader<ImageType>::New();

reader->SetFileName(“lena.jpg”);

try
{
reader->Update();
}
catch (itk::ExceptionObject & e)
{
std::cerr << e.GetDescription() << std::endl;
return EXIT_FAILURE;
}

ImageType::Pointer image = reader->GetOutput();

Writing my first image

#include <itkImageFileWriter.h>

typedef unsigned char PixelType;

typedef itk::Image<PixelType, 2> ImageType;

itk::ImageFileWriter<ImageType>::Pointer writer
= itk::ImageFileWriter<ImageType>::New();

writer->SetFileName(“lena.jpg”);
writer->SetInput(image);

try
{
writer- >Update();
}
catch (itk::ExceptionObject & e)
{
std::cerr << e.GetDescription() << std::endl;
return EXIT_FAILURE;
}

5
My file format converter
itk::ImageFileReader<ImageType>::Pointer reader
= itk::ImageFileReader<ImageType>::New();

itk::ImageFileWriter<ImageType>::Pointer writer
= itk::ImageFileWriter<ImageType>::New();

reader->SetFileName(“myImage.jpg”)
writer->SetFileName(“myImage.tiff”);
writer->SetInput(reader->GetOutput());

try
{
writer- >Update();
}
catch (itk::ExceptionObject & e)
{
std::cerr << e.GetDescription() << std::endl;
return EXIT_FAILURE;
}

How to avoid using the Factory?

z I know the type of images I want to
read/write

z Factory can be slow

#include <itkTIFFImageIO.h>

itk::TIFFImageIO::Pointer tiffImageIO = itk::TIFFImageIO::New();

reader->SetFilename(“myimage.tiff”);
reader->SetImageIO(tiffImageIO);
reader->Update();

6
Reading RAW images
#include <itkRawImageIO.h>

itk::RawImageIO<unsigned short,2>::Pointer io;

io = itk::RawImageIO<unsigned short,2>::New();

io->SetFileName(“myimage.raw”);
unsigned int dim[2] = {800,60};
double spacing[2] = {0.8, 0.8};
double origin[2] = {0.0,0.0};
for(unsigned int i=0; i<2; i++)
{
io->SetDimensions(i,dim[i]);
io->SetSpacing(i,spacing[i]);
io->SetOrigin(i,origin[i]);
}
io->SetHeaderSize(0);
io->SetByteOrderToLittleEndian();
io->SetPixelType(itk::ImageIOBase::SCALAR);
io->SetNumberOfComponents(1);

Reading RAW images (2)

itk::ImageFileReader<ImageType>::Pointer reader
= itk::ImageFileReader<ImageType>::New();

reader->SetFileName(“myImage.raw”);
reader->SetImageIO(io);

try
{
reader->Update();
}
catch (itk::ExceptionObject & e)
{
std::cerr << e.GetDescription() << std::endl;
return EXIT_FAILURE;
}

7
Creating a MetaImage Header
z Create a simple text file with the .mhd
extension

z Set the appropriate MetaData

z Example:
NDims = 3
DimSize = 100 200 300
ElementSpacing = 1.2 1.2 1.0
ElementType = MET_UCHAR
ElementByteOrderMSB = False
ElementDataFile = HeadMRVolume.raw
OR
ElementDataFile = HeadMRVolume%04d.raw 0 10 1

Dealing with DICOM images

z Often consists of several files

z Uses the GDCM library

z GDCMSeriesFileNames to construct the
serie:

1. Image Orientation & Image Position

2. 'Image Number‘
3. Lexicographical order

8
Reading DICOM images
// Select the correct files from the directory
typedef itk::ImageSeriesReader< ImageType > ReaderType;
ReaderType::Pointer reader = ReaderType::New();

typedef itk::GDCMSeriesFileNames NamesGeneratorType;

NamesGeneratorType::Pointer nameGenerator = NamesGeneratorType::New();
nameGenerator->SetUseSeriesDetails( true );

nameGenerator->SetDirectory( “./MyDirectory/” );
typedef std::vector< std::string > FileNamesContainer;
FileNamesContainer fileNames;
fileNames = nameGenerator->GetFileNames( seriesIdentifier );

reader->SetFileNames( fileNames );

// Set the DicomIO

typedef itk::GDCMImageIO ImageIOType;
ImageIOType::Pointer dicomIO = ImageIOType::New();
reader->SetImageIO( dicomIO );
reader->Update();

More on Series Filenames

z NumericSeriesFilenames
- ordered sequence of filenames
- unique, non-negative, integral value
- Set(Start/End)Index and SetIncrementIndex()

z RegularExpressionSeriesFilenames
- ordered sequence that matches RegEx
- ordered by submatch or numerically

9
Metadata

MetaData

z Extra information attached to an image

z itk::Image defines:
- Spacing (size of the voxels in mm)
- Origin (physical location of (0,0,0))
- Size (size of the largest region of the image)
- Orientation (direction cosines)

10
MetaData Dictionary
z itkMetaDataDictionary

z Filled in by the reader (when available)

z Not passed through the filters

z Accessing the dictionary:

ImageType::Pointer image = reader->GetOutput();

typedef itk::MetaDataDictionary DictionaryType;
DictionaryType & dictionary = image->GetMetaDataDictionary();

Transform File IO

11
Transform IO
z Special ITK File format

z Uses IO Factory

z Write Parameters of the transforms as

well as Fixed Parameters

z Concatenation of transforms in a single

file

Write Transform Example

#include "itkTransformFileWriter.h”
#include "itkAffineTransform.h”

typedef itk::AffineTransform<double,3> AffineTransformType;

AffineTransformType::Pointer affine = AffineTransformType::New();
itk::TransformFileWriter::Pointer writer;
writer = itk::TransformFileWriter::New();

writer->AddTransform(affine);
writer->SetFileName( “AffineTransform.txt" );
try
{
writer->Update();
}
catch( itk::ExceptionObject & excp )
{
std::cerr << excp << std::endl;
return EXIT_FAILURE;
}

12
Read Transform Example
#include "itkTransformFileReader.h”

itk::TransformFileReader::Pointer reader;
reader = itk::TransformFileReader::New();

reader->SetFileName( “AffineTransform.txt" );
reader->Update();

typedef itk::TransformFileReader::TransformListType * TransformListType;

TransformListType transforms = reader->GetTransformList();
itk::TransformFileReader::TransformListType::const_iterator it;
it = transforms->begin();
if(!strcmp((*it)->GetNameOfClass(),"AffineTransform"))
{
AffineTransformType::Pointer affineTransform =
static_cast<AffineTransformType*>((*it).GetPointer());
}

Object File IO

13
Spatial Objects
z Represents Physical Objects (not only
images)

z Scene Graph Concept

z Common coordinate frame

z Support a common IO framework through

MetaIO

Spatial Object Example

Abdomen

Liver

Kidneys
Blood Vessels

Tumor

14
Spatial Objects IO
z Uses the IO Factory

z Uses MetaIO file format

z Conversion is done internally

z Concatenation of objects in a single file

z Easy to extend

Writing Spatial Objects

#include "itkSpatialObjectWriter.h”
#include "itkEllipseSpatialObject.h”

typedef itk::EllipseSpatialObject<3> SphereType;

SphereType::Pointer sphere = SphereType::New();
sphere->SetRadius(2);

typedef itk::SpatialObjectWriter<3> WriterType;

WriterType::Pointer writer = WriterType::New();
writer->SetInput(sphere);
writer->SetFileName("ellipse.meta");
writer->Update();

15
 Reading Spatial Objects
#include "itkSpatialObjectReader.h”

typedef itk::SpatialObjectReader<3> ReaderType;

ReaderType::Pointer reader = ReaderType::New();
reader->SetFileName("ellipse.meta");
reader->Update();

// Return an itk::SceneSpatialObject with all the objects in the file

ReaderType::ScenePointer scene = reader->GetScene();

// Return an itk::GroupSpatialObject with all the objects in the file

ReaderType::GroupPointer group = reader->GetGroup();

ReaderType::SceneType::ObjectListType * sceneChildren = scene->GetObjects(999);

ReaderType::SceneType::ObjectListType::const_iterator objectIterator;
objectIterator = sceneChildren->begin();
if(!strcmp((* objectIterator)->GetTypeName(),“EllipseSpatialObject"))
{
SphereType::Pointer sphere = dynamic_cast<SphereType*>((*objectIterator).GetPointer());
}

Writing an itkMesh
#include "itkSpatialObjectWriter.h”
#include "itkMeshSpatialObject.h”

typedef itk::DefaultDynamicMeshTraits< float , 3, 3 > MeshTrait;

typedef itk::Mesh<float,3,MeshTrait> MeshType;

MeshType::Pointer mesh = MeshType::New();

// Create the mesh Spatial Object

MeshSpatialObjectType::Pointer meshSO = MeshSpatialObjectType::New();
meshSO->SetMesh(mesh);

// Writing the file

typedef itk::SpatialObjectWriter<3,float,MeshTrait> WriterType;
WriterType::Pointer writer = WriterType::New();
writer->SetInput(meshSO);
writer->SetFileName("metamesh.txt");
writer->Update();

16
Logging Capabilities

Logger
z Record output information and send the
output to a stream (or multiple streams)

z Priority Level (Fatal, Critical, Warning, Info,

Debug…)

z AddLogOutput() : attach an output stream to

the logger

z Write() : send information to the logger

17
Logger in use
#include <itkLogger.h>
#include <itkStdStreamLogOutput.h>

itk::Logger::Pointer logger = itk::Logger::New();

itk::StdStreamLogOutput::Pointer output =
itk::StdStreamLogOutput::New();
output->SetStream(std::cout);

logger->SetName(“MyLogger”);
logger->SetPriorityLevel(itk::LoggerBase::INFO);
logger->SetLevelForFlushing(itk::LoggerBase::CRITICAL);

logger->AddLogOutput(output);

logger->Write(itk::LoggerBase::INFO, "This is the INFO message.\n");

Logger Manager

z Centralize several loggers

z AddLogger()

z CreateLogger() / CreateThreadLogger

z Write()

z Flush()

18
References
z Culp, Timothy R. "Industrial Strength Pluggable Object Factories". C++
Report Online.
http://www.creport.com/html/from_pages/view_recent_articles_c.cfm?
ArticleID=1520

z P. Chandra, L. Ibanez, "ImageIO: Design of an Extensible Image

Input/Output Library", ACM Crossroad online magazine, April 2001.
Available online at http://www.acm.org/crossroads/xrds7-
4/imageIO.html

z E. Gamma, R. Helm, R. Johnson, J. Vlissides, "Design Patterns",

Addison Wesley, 1995.

Enjoy ITK !

19
 IGSTK
The Image-Guided
Surgery Toolkit

Kitware, Inc

Overview

z History
z Infrastructure
z Components
z Quality Control
z Software Process
z Applications

1
The Toolkit in a Nutshell
z Open Source Toolkit
z BSD-like License
z Written in C++
z Cross Platform
z Based on ITK and VTK
z GUI based on FLTK
z Designed for a Critical Application

History
z STTR Funded by NIBIB/NIH (Georgetown-
Kitware)
z Phase I from May to Sept 2003. Used in
example applications.
z Phase II started on October 2004
z CADDLab UNC Joined the project and
contributed Spatial Objects & RF ablation
Application.
z Atamai joined on 2005 and contributed
Tracker code.
z Releases done about every 6 months

2
Layer Architecture

IGS - APPLICATION

GUI: FLTK / Qt / MFC

IGSTK

ITK VTK

VNL GDCM Threads, XML,… OpenGL

Main Categories of Components

Geometry &
Infrastructure
Representation

Images &
Trackers Viewers
Readers

3
Infrastructure

z State Machine
z Time Stamp
z Pulse Generator
z Events
z Transforms
z Logger

Trackers
z Tracker (base class)
– PolarisTracker
– AuroraTracker
– Flock Of Birds
z Support
– SerialCommunication
z SerialCommunicationForWindows

z SerialCommunicationForPosix

– NDICommandInterpreter

4
 Object Representation
z SpatialObject z SpatialObjectRepresentation
– Ellipsoid – Ellipsoid
– Cylinder – Cylinder
– Images – Images
– Mesh – Mesh
– Tube – Tube
– TubeGroup – Box
– Group – Cone
– Box – VascularNetwork
– Cone – AirwaysNetwork
– Surgical Tools

Viewers

z View
– View2D
– View3D
z VTK Based
z GUI / VTK hybrid class

5
Images and Readers
z Image
– CTImage
– MRImage
– FluoroscopicImage

z Readers (DICOM)
– CTImageReader
– MRImageReader
– Fluoroscopy (video input)

Dynamics
Architecture

6
 General Interactions
Tracker
GUI
TrackerTool 1
View1
TrackerTool 2
Object
View2 Representation TrackerTool 3

View3 Object
Representation Spatial
Object
View4 Object
Representation

Tracking Interactions

T Tracker

Tick
T TrackerTool 3
Object T
Representation
Spatial
Object
Object
T
Representation
Transform
T = Event

7
Event Class

ITK Event

IGSTK Event

TransformEvent

TimeStampEvent

StringEvent

Event Class

8
Transform Class

Transform

Vector
Versor
TimeStamp
Error

Transform Class

9
Transform Event Class

Pulse Generator

Action

Action

How not to miss pulses ?

10
 Tick
Timing Control
Tick Tracker
GUI
TrackerTool 1
View1 TrackerTool 2
Object
Representation TrackerTool 3
View2
Object
View3 Representation Spatial
Object
View4 Object
Representation

Quality Control

11
Testing Platforms
z Windows
– Visual Studio 6.0 / 7.1 / 8.0
z Linux
– gcc 3.2 / 3.3 / 3.4 / 4.0
– icc 7.1 / 8.0
z Cygwin – Windows
– gcc 3.4
– gcc 3.3

Testing

z Goal
– 100 % Code Coverage
– 100 % State Machine Transitions Testing

z Current Code Coverage

– IGSTK 93.6 % ( untested 321 / 4714 lines )
– Sandbox 92.4 % ( untested 4 / 49 lines )

12
Dynamic Testing Platforms

z Valgrind on Linux
– gcc 3.3
– gcc 3.4
z Valgrind on Cygwin
z Purify (too expensive)

Software Process

13
Software Process

z Sandbox
z Extreme programming
z Release early, release often
z Code reviews
z Requirements
z Bug tracking

Typical Agenda for a release

z Create feature list
z Write requirements
z Develop code
z Tag Sandbox, create code review list
z Code Review
z Move code and tag
z Fix Documentation
z Fix remaining bugs
z Increase code coverage
z Create and release package

14
Applications

Applications

z Ultrasound-guided percutaneous
liver lesion RFA
z Needle Biopsy
z Guidewire Tracking

15
Ultrasound-Guided Liver RFA

GuideWire Tracking

16
Needle Biopsy

Download IGSTK and Try it!

z http://www.igstk.org
z Last stable release “Iteration 7”
z Demo: Sunday Feb 12 from 8:30 to 10:30 pm

17
Enjoy IGSTK!

18
 Applications:
Using Insight(itk) and
The Visualization Toolkit(vtk)
Bill Lorensen
Jim Miller
GE Research
Niskayuna, NY

Agenda

z Approaches to interfacing vtk and itk

z Case Studies / Demos
– Registration
z MR/MR

z CT/MR

– Segmentation
z Watershed

1
 Motivation

z itk has no visualization capabilities

z itk has some interesting segmentation and
registration algorithms
z Both itk and vtk have a pipeline architecture
z vtk has a nice wrapping facility to interpreters
like tcl and python

Avoid code duplication and use “best of breed”

from each toolkit.

vtk/itk Interfaces

z Connect vtk pipelines to itk pipelines

z Image filter class hierarchies
z Implement vtk filters with itk algorithms
z Callbacks for filter progress

2
 Connect pipelines - vtk to itk

vtkImageExport *movingExporter = vtkImageExport::New();

movingExporter->SetInput(movingReader->GetOutput());

typedef itk::VTKImageImport<InputType> ImageImportType;

ImageImportType::Pointer movingImporter =
ImageImportType::New();

ConnectPipelines(movingExporter, movingImporter);

InsightApplications/
InsightApplications/vtkITK/Common/
vtkITK/Common/vtkITKUtility
vtkITKUtility.h
.h

Connect Pipelines

z Implements itk pipeline protocol,

invoking vtk callbacks

PropagateRequestedRegion()
UpdateOutputInformation()
GenerateOutputInformation()
GenerateData()

3
 vtkITK Class Hierarchies

Class Hierarchy
z vtkImageToImageFilter
– vtkITKImageToImageFilter
z vtkITKImageToImageFilterFF

z vtkITKImageToImageFilterUSUS

z…
–vtkITKDiscreteGaussianImageFilter

vtkITKImageToImageFilter

z Changes exception macros to

errors/warnings
z Delegates
– Modified
– DebugOn/Off
– SetInput/Output
– Update

4
 vtkITKImageToImageFilterFF

z Defines Input/Output typedefs

typedef float InputImagePixelType;
typedef float OutputImagePixelType;
typedef itk::Image<InputImagePixelType, 3>
InputImageType;
typedef itk::Image<OutputImagePixelType, 3>
OutputImageType;
z Constructor
– Creates importers/exporters
– Connects vtk and itk pipelines

vtkITKDiscreteGaussianImageFilter

z Instantiates filter
z Delegates filter’s member data
set’s/get’s

5
 vtk Filters Implemented
with itk algorithms
z vtkITKMutualInformationTransform
z Implement InternalUpdate()
– vtkImageExport
– itk::VTKImageImport
– ConnectPipelines()
– Convert vtk transform to quaternion
– itk::RegistrationMethod
– Convert itk quaternion to vtk transform

Illustrative Examples

Registration

6
 Motivation for Registration

z Intra-modality
– Alzheimer’s
z Longitudinal analysis

– CT Lung
z Side-by-side slice reading

z Inter-modality
– PET/MR/CT/US/XRAY Tomo
z Fusion and analysis

Mutual Information
Registration
z Computes “mutual information” between
two datasets, a reference and target
– MI(X,Y) = H(X) + H(Y) – H(X,Y)
z Small parameter set
z Developed by Sandy Wells (BWH) and
Paul Viola (MIT) in 1995
z Defacto standard for automatic,
intensity based registration

7
 Longitudinal MRI Study

z Register multiple volumetric MRI

datasets of a patient taken over an
extended time
z Create a batch processing facility to
process dozens of datasets
z Resample the datasets

Approach

z Validate the algorithm

z Pick a set of parameters that can be
used across all the studies
z For each pair of datasets
– Perform registration
– Output a transform
z Viewthe resampled source dataset in
context with the target dataset

8
 The MI Algorithm Parameter
Space
z Learning Rate
z Standard Deviation of Parzen Window
estimate
z Number of Samples for density estimate
z Number of Iterations
z Image variance
z Translation scale factors

GE’s Design for Six Sigma (DFSS) Design of

Experiments Tool

9
10
 Division of Labor
Read data
vtk
Export data

Import Data
Normalize data MIRegistration.cxx
itk Shrink
Register
Report transform

Read data

vtk Reslice MultiCompare.tcl

Display

The Pipeline

z vtkImageReader
z vtkImageExport
z itk::vtkImporter
z itk::NormalizeImageFilter
z itk::ShrinkImageFilter
z itk::ImageRegistrationMethod
z vtkTransform

11
 Oregon Data

z 25 Registrations
z 13 Subjects, some with AD
z Qualitative comparison
z One set of parameters for all studies

Data Courtesy of Jeffrey Kaye, Oregon Health Sciences

Center

Difference

Checkerboard

Source Target
Original Original
image image

Longitudinal MRI
No Registration

12
 Difference

Checkerboard

Target
Original
Source
image
Original
image

Longitudinal MRI
Registration

Multi Field MRI Data

z Register 1.5T and 3T to 4T data

z Resampled 1.5T and 3T to correspond
to the 4T sampling
z Volume rendering of the 3 datasets from
the same view

z Demo mri1.bat, MultiCompareAD.tcl

InsightApplications/MRIRegistration

13
 Difference

Checkerboard

Target
Original
Source Image
Original
Image

1.5T vs 4T MRI
No Registration

Difference

Checkerboard

Target
Source Original
Original Image
Image

1.5T vs 4T MRI
Registration

14
 3D Visualization of the same subject
Scanned with different MR field Strengths

3T 1.5T

4T

All Registered
To 4T

CT Lung Longitudinal
Study
z Register two CT exams of the
same patient taken at two different
times
z Side-by-side synchronized view for
visual comparison

15
 Difference

Checkerboard

Target
Original
Image

Source
Original
Image

Lung CT No Registration

Difference

Checkerboard

Target
Original
Image

Source
Original
Image

Lung CT Registration

16
microPet/Volume CT

17
 microPet/Volume CT

Demo CT/MR fusion

18
 Implementing a vtk
Transform with itk
z vtkITKMutualInformationTransform

DemoMutual.tcl

Visible Woman Registration

Case Study
Bill Lorensen, GE Research
Peter Ratiu, Brigham and Womens Hospital

19
 Agenda

z Problem Statement
z Methods
z Results
z Discussion

Problem Statement

z Register the Visible Human Female

Computed Tomography and MRI Data to the
cryo-section data
z Validate the use of the itk Mutual Information
Registration algorithm on data of differing
modalities
z Assess the performance of the registration
using landmarks selected by an anatomical
expert

20
 Methods

z NormalizeImageFilter (itk)
z Mutual Information Registration (itk)
– Implemented as a vtk Transform
z MarchingCubes (vtk)
z Checkerboard Display (vtk)

Methods

z Multiple registration runs with increasing

scales
z Both RGB and CT data scaled to zero
mean and unit variance
z 1/16, 1/8, ¼, ½, 1 scale
z .01, .01, .004, .004, .001 learning rate
z 2000 iterations at each resolution

21
 CT Results

z Initialvertical translation of 20 mm
z Checkerboard display of two modalities
after each resolution run

22
23
24
25
26
 CT Results

z Generated isosurfaces of skin and bone

from CT data
z Transformed according to computed
transformation
z At each triangle vertex, sample the RGB
data to color the vertex

27
28
29
30
 CT Results

z CT Landmarks transformed by Mutual

Information matrix into RGB coordinate
system
z Compute distance between CT and
RGB points at each iteration

31
 Distances Between Corresponding Key Points
14

12

[234,280]
[264,506]
10 [282,131]
[291,107]
[270,104]
Distance (mm)

8 [239,608]
[118,334]
[431,347]
6 [41,376]
[365,362]
[265,377]
4 [260,485]
[402,265]
[144,252]

0
1 2 3 4 5 6 7 8 9 10
Iteration

MR Results

z Initial
transform computed using experts
anatomical landmarks as
correspondences
z Same parameter schedule as CT/RGB

32
33
34
35
36
37
 MRI Results

z MRI Landmarks transformed by Mutual

Information matrix into RGB coordinate
system
z Compute distance between MRI and
RGB points at each iteration

Distances Between Corresponding Key Points

18

16

14

[283,129]
12 [242,584]
[185,118]
[378,128]
Distance (mm)

10 [277,303]
[185,131]
[378,135]
8 [241,144]
[318,146]
[70,352]
6 [143,254]
[402,261]

0
1 2 3 4 5 6 7 8
Iteration

38
 CT Discussion

z CT to RGB rigid
body
registration is a
good first step
for subsequent
deformable
registration

MRI Discussion

z MRI to RGB
registration was not
effective
– Too much
anatomical distortion
occurred between
the MR and RGB
acquisitions

39
 Examples

Segmentation

Watershed Segmentation

z I/O and Visualization with vtk

z Image processing and segmentation
with itk
z User interface with tk
z Prototype in tcl

40
 Demo Watershed

Applications:
Using Insight(itk) and
The Visualization Toolkit(vtk)
Bill Lorensen
Jim Miller
GE Research
Niskayuna, NY

41
 ITK Meets the Virtual Soldier

GE Global Research
Bill Lorensen, Jim Miller
Dirk Padfield, James Ross
Wes Turner
[email protected]

Outline

z Atlas Quality Control

z Model Generation
z Implicit Anatomical Modeling
z Registration
z Model Refinement Usinig Level Sets

February 2005 SPIE Tutorial

1
 Activities

z Atlas from NLM

z Reconciled labels
z Registered slices
z Computed bounding boxes
z Calculated adjacencies
z Generated models using vtkDiscreteMarchingCubes
z Colored models
z Registered atlas to CT
z Atlas as seed for level sets
z Deformation of man to woman

February 2005 SPIE Tutorial

Atlas Quality Control

z Thorax Atlas consists of 411 70mm camera

cryo-section slices and corresponding
segmentation label maps

Slice
Resolution:
2048 x 1350

February 2005 SPIE Tutorial

2
 Atlas Quality Control

z Labeled 407 structures

z Registered 70mm film images
z Reconciled labels throughout entire
volume
z Computed bounding boxes for all labels
z Removed some “dusty” structures
– Automatic reassignment

February 2005 SPIE Tutorial

Overlay.tcl

February 2005 SPIE Tutorial

3
 Atlas Quality Control

Left Sixth Rib

Segmentation Dust removed via “voting”

February 2005 SPIE Tutorial

Model Generation

z Generated
four types of
models from
segmentation Discrete Smoothed
labels

Colored Decimated (90%)

February 2005 SPIE Tutorial

4
 Model Generation

February 2005 SPIE Tutorial

Model Generation

February 2005 SPIE Tutorial

5
 VHM Thorax Models

February 2005 SPIE Tutorial

Implicit Anatomical
Modeling
z Label = f(X,Y,Z)

X,Y,Z I,J,K Label

February 2005 SPIE Tutorial

6
 Implicit Anatomical
Modeling
0 mm: right_ventricle
2 mm: pericardium
3 mm: left_ventricle
52 mm: mediastinum
53 mm: left_upper_lobe_of_lung
66 mm: superior_lingular_branch_left_pulmonary
68 mm: left_upper_lobe_of_lung
107 mm: unknown
112 mm: left_fourth_rib
117 mm: serratus_anterior_muscle
123 mm: unknown
186 mm: pectoralis_minor_muscle
191 mm: deltoid_muscle
259 mm: unknown
261 mm: subcutaneous_tissue_of_trunk
271 mm: skin_of_trunk

February 2005 SPIE Tutorial

Implicit Anatomical
Modeling

February 2005 SPIE Tutorial

7
 RGB to CT Registration

February 2005 SPIE Tutorial

RGB to CT Registration

February 2005 SPIE Tutorial

8
 Deformable Registration

February 2005 SPIE Tutorial

Model Refinement
Using LevelSets

9
February 2005 SPIE Tutorial

February 2005 SPIE Tutorial

10
 LevelSet Model Refinement

z Powerful technique to track the evolution of surfaces

using image based constraints and forces
z Three techniques investigated
– Threshold Level Set
– Laplacian Level Set
– Canny Level Set
z All techniques are part of Insight Toolkit (itk)

But: Algorithm parameter space is high!

February 2005 SPIE Tutorial

Load Sharing Facility

z 256 3.2 Ghz Dual Processor Xeon

z 2 gig of memory each
z Jobs scheduled with Load Sharing
Facility (LSF)
– Platform Computing Corporation
z LSF Toolkit
– GE Research front-end and monitoring

February 2005 SPIE Tutorial

11
 LSF Toolkit

February 2005 SPIE Tutorial

Laplacian LevelSet DOE

Curvature

Advection

Threshold

1000 runs in 50 minutes!

February 2005 SPIE Tutorial

12
Curvature

Advection

Threshold

February 2005 SPIE Tutorial

LS Threshold Selection using

IsolatedConnected
Initial Using Threshold LS Using IsolatedConnected LS

February 2005 SPIE Tutorial

13
 ThresholdLevelSet

February 2005 SPIE Tutorial

3D IsolatedConnected LS Model
vs. Initial Model

Pulmonary Trunk

Left ventricle and

ascending aorta

February 2005 SPIE Tutorial

14
ITK Meets the Virtual Soldier

GE Global Research
Bill Lorensen, Jim Miller
Dirk Padfield, James Ross
Wes Turner
[email protected]

15
GE Research ITK in an Industrial Research Lab

ITK in an
Industrial Research Lab
GE Research

Jim Miller
Bill Lorensen
Dan Blezek
Majeid Alyassin
John Schenck
Zhu Li
Matt Turek
Paulo Mendonça
Dirk Padfield
James Ross

GE Research ITK in an Industrial Research Lab

ITK at GE Research
• Alzheimer’s Research
– Exploratory Research
– Molecular Imaging
– Diagnostic Pharmaceuticals
• Chronic Obstructive Pulmonary Disease
– Research Tool Development/Product Development
– Image Guided Clinical Trials
• Both projects are quantitative imaging projects
– Reduce images to numbers
• Volumes, Areas, Percent change
• Both projects are studying a disease from an imaging perspective
• Both projects use ITK along with other toolkits and applications

1
GE Research ITK in an Industrial Research Lab

Alzheimer’s Research
• Hypotheses:
– Alzheimer’s patients have elevated brain iron concentrations
• B Drayer, et al., Am. J. Neuroradiol. 7, 373-380, 1986.
• JF Schenck, J. Neurol. Sci 134 (Suppl.), 10-18, 1995.
– Alzheimer’s patients have volumetric differences in key brain
structures
• “A discriminant function analysis demonstrated that a linear
combination of the volumes of the hippocampus and the temporal horn
of the lateral ventricles differentiated 100% of the patients and controls
from one another.”
• Killiany, et al., “Temporal Lobe Regions on Magnetic Resonance
Imaging Identify Patients with Early Alzheimer's Disease”. Archives of
Neurology. 50:949-954, 1993

GE Research ITK in an Industrial Research Lab

Iron Concentration vs Mean T2 Value

48.0

46.0 %Iron for Normals

%Iron for AD
44.0 HIV

Brain Iron as AD
Mean T2

42.0

Marker
40.0

38.0

36.0
0 5 10 15 20 25 30 35 40 45
% Iron

Alzheimer’s Disease Normal HIV

2
GE Research ITK in an Industrial Research Lab

Brain Iron Regional Quantification

Global Regional

• Overall brain iron concentration can be used to assess

Alzheimer’s and monitor disease progression
– Global assessment
• Regional quantification and analysis will lead to:
– Increased understanding of disease locality
– Monitoring patient specific disease progression
• Requires longitudinal analysis
• Requires registration of patient data
• Change detection, change monitoring

GE Research ITK in an Industrial Research Lab

Longitudinal
MRI with no
registration

Checkerboard Difference

Source Target
Original Original
image image

3
GE Research ITK in an Industrial Research Lab

Longitudinal
MRI with mutual
information
registration

Checkerboard Difference

Source Target
Original Original
image image

GE Research ITK in an Industrial Research Lab

Design of Experiments

4
GE Research ITK in an Industrial Research Lab

Brain Iron Anatomical Quantification

Global Regional Structural

• 40 normals, 40 Alzheimer’s patients

• 8 structures per case
– Hippocampus, Caudate, Red Nucleus, Globus Pollidus, Putamen,
Substantia Nigra, Nucleus Basalis, Subthalamic Nucleus
• Structures with subtle boundaries
• Hand segmented
– Some on T1, some on T2
– 1 case per day

• Analyze brain iron concentrations on structure by structure basis

• Hand segmentations provide a rich set of segmentation priors

GE Research ITK in an Industrial Research Lab

5
GE Research ITK in an Industrial Research Lab

Automatic Brain Structure Segmentation

• Just starting this effort
• Demon’s-based atlas segmentation
• Statistical shape models

GE Research ITK in an Industrial Research Lab

Chronic Obstructive Pulmonary Disease

COPD is the 4th
leading cause of
death in the US.

• Emphysema
– Parenchyma disease
– Loss of lung tissue
• Reduced capability to exchange gases
– Loss of elastic recoil
• Reduce capability to exhale
• Chronic Bronchitis
– Airway disease
– Thickening of airway wall
• Restriction of airflow
• Quantitative analysis from CT images

6
GE Research ITK in an Industrial Research Lab

Emphysema
0.3

0.25

percentage
0.2

0.15

0.1

0.05

15
40

10
80
120
region region average

5
area

160
attenuation

0
• Partition lung field into homogeneous regions Class. rate (%) - Müller
• Calculate features for each region Chi-Square 63.16 92.11
– Intensity mean, intensity variance, region area, Jeffrey 57.89 94.74
number of adjacent regions Match 50.00 92.11
• Compare multi-dimensional histograms to EMD 52.63 94.74
canonical cases 5 class and 3 class performance
• Score severity on a 1-5 scale
• Legacy computer vision system

GE Research ITK in an Industrial Research Lab

Chronic Bronchitis

Segmentation Detection Grouping Measurement Fitting

Strong model, robust fits Higher order shape models

TPS

Inlier model

7
GE Research ITK in an Industrial Research Lab

Airway wall study

• Compare automated measurements derived from CT to manual
pathology measurements from resected lung
• Pathology measurements are based on high resolution digital
camera images
– 6 times the resolution of CT Gage R&R
• Systematic bias on radii of ¾ pixel
– 2X read • Similar to other techniques
• Due to scale and curvature interactions

GE Research ITK in an Industrial Research Lab

120-3
Lumen Error: 38.3%
Wall Error: 54.1%
177-3
120-2 Lumen Error: 57.7%
Lumen Error: 4.2% Wall Error: 410.9%
Wall Error: 38.3%

103-1
Lumen Error: 2.9%
12-2 Wall Error: 6.2%
Lumen Error: 24.6%
Wall Error: 25.7%

8
GE Research ITK in an Industrial Research Lab

Pig Airway 102-1

(9 mm Lumen Diameter)

Pathology Measurements Automatic CT Measurements

Lumen Area: mm2
75.02 Lumen Area: 75.27 mm2 (0.32 % Error)
Wall Area: 101.15 mm2 Wall Area: 107.38 mm2 (6.2 % Error)

GE Research ITK in an Industrial Research Lab

Pig Airway 177-3

(~1 mm Lumen Diameter)

Pathology Measurements Automatic CT Measurements

Lumen Area: 1.36 mm2 Lumen Area: 2.14 mm2 (57.7 % Error)
Wall Area: 2.54 mm2 Wall Area: 12.96 mm2 (410.9 % Error)

9
GE Research ITK in an Industrial Research Lab

Other ITK Applications

• PET Segmentation
– Otsu, Expectation-Maximization, Markov Random Field
• Small Animal Research
– CT + PET Mutual Information Registration
• Surgical Navigation
– CT + MR Mutual Information Registration
• Computer Aided Detection
– Watersheds, Level Sets, Connected Confidence, Bayesian Decisions
• Image Quality
– Edge preserving smoothing, Histogram matching
• Facial Reconstruction
– Geodesic morphology
• Virtual Soldier
– Atlas-based segmentation
• Product Teams
– Looking at reference implementations for registration, level sets, etc.

GE Research ITK in an Industrial Research Lab

Histogram

Background

Tumor

10
GE Research ITK in an Industrial Research Lab

Life at an Industrial Research Lab

Academia Industrial Industry

Research

Publish Product
Publish Evaluate Product
Publish Accelerate Product
Harden

How Extend What

Why Invent When

11
 NA-MIC
National Alliance for Medical Image Computing
http://na-mic.org

Slicer Overview

Steve Pieper, PhD

Outline
• Slicer
– Overview and History
– Architecture /
Implementation
– Image I/O and
Management
– Segmentation Tools
(EM, Editor)
– DTMRI Tools
– fMRI Tools
– Applications (IGT,
Clinical Research)

National Alliance for Medical Image Computing

http://na-mic.org

1
 What is 3D Slicer?
• 3D Slicer is…
– An end-user application for 3D
medical image computing research
and Image Guided Therapy
– A platform for research where new
techniques can be plugged into a
useful framework
– A freely-downloadable program with
source and binaries for Windows,
Linux, Solaris and (sort-of) Mac OSX
– NOT an FDA approved medical
device and CANNOT be used
clinically without proper research
controls (IRB etc.)
– NOT finished – some parts will work
better than others

National Alliance for Medical Image Computing

http://na-mic.org

Surgical Planning Example

Dr. Jose Miguel Selman,
Clinica Las Condes,
Santiago, CHILE
• Temporal Lobe
Cavernoma
• MR Cortex and White
Matter Extracted by
FreeSurfer (MGH
software, interface
created for BIRN)
• Registration to CT and
Visualization in Slicer

National Alliance for Medical Image Computing

http://na-mic.org

2
 Virtual Endoscopy Example
Delphine Nain, MIT AI
Lab, now at Georgia
Tech
• Automatic or Manual
Path Planning
• Animated Camera
and Controls

http://www.ai.mit.edu/projects/medical-vision/virtual-endoscopy/
National Alliance for Medical Image Computing
http://na-mic.org

Slicer Background
• SPL Image Guided Surgery and Visualization (Kikinis, Westin,
Hata, Halle, others)
• Slicer Application Pulled Together by Dave Gering 1997-1999
with VTK and Tcl
• Further Development and Architecture by Lauren O’Donnell
1999-2001
• Ongoing Development of Slicer’s Base Primarily by Steve
Pieper and Nicole Aucoin
• Many Modules and Contributions by Various Authors
– BWH, MIT, MGH, Georgia Tech, UCSD, JHU…

National Alliance for Medical Image Computing

http://na-mic.org

3
 Why Develop with Slicer?
• Start with a Powerful Platform
• Remove Obstacles to Problem Solving
– Access to Every Layer of Source Code:
Numeric, Graphic, Network, etc.
• Commit to an Environment that will Always
be Available
– Not Tied Up in IP of Old Institution
– Not Tied to Proprietary Platform with License
Fees
• Keep Your Own Work Available to You
National Alliance for Medical Image Computing
http://na-mic.org

Slicer Today
• 300K Lines of Code
– Cross-Platform Tcl/Tk
GUI
– VTK/ITK Based C++
Computing
• www.slicer.org
– 166 on slicer-users
– 117 on slicer-devel
– 4000 Registered
Downloads

National Alliance for Medical Image Computing

http://na-mic.org

4
 Architecture
• Plug-in Modules consist of Tcl and C++ code with cross-
platform file layout for building and loading
• Standard naming convention and hooks to add GUI and
processing components
• Personally, I find the speed of C++ and the interactive
development of Tcl to be a near-perfect environment

3D Slicer Application
Slicer Base Module 1 … Module N
VTK Tcl
OpenGL Window System
Computer Hardware

National Alliance for Medical Image Computing

http://na-mic.org

Features
• Load Medical Image Data: MR, CT in DICOM, GE, Analyze…
• XML-based File Format: MRML (Medical Reality Markup
Language)
• Interactive Editor: Draw, Threshold, Math Morphology…
• Automated Segmenters: EM Segmentation, Fast Marching,
Level Sets…
• Visualization: Model Building, Stereo Rendering, Animation…
• Registration: Manual, ITK, CNI
• Measurement: Fiducial-Based, Volumetric, Polyhedral
Intersection, Vessel Cross-Section, Osteotomy Planning
• IGT: Tracked Probes, Real-Time Images, Robot Control
• Additional Application-Specific Features in Modules…

National Alliance for Medical Image Computing

http://na-mic.org

5
 Administration
• Project Housed at Surgical Planning Lab, Brigham and
Women’s Hospital / Harvard Medical School, Many MIT
CSAIL (formerly AI Lab) Students and Faculty Involved
• CVS, Mailing Lists, etc at the SPL
• Funded Projects Supporting Slicer Base Development
– Neuroimage Analysis Center (NAC): NIH Center at SPL
– Biomedical Informatics Research Network (BIRN): NIH
Supported Neuroimaging Collaboration
– Computer Integrated Surgical Systems and Technology
(CISST): NSF Supported Robotics Collaboration Headed by
JHU
– Virtual Soldier: DARPA Organ Simulation Collaboration
– National Alliance for Medical Image Processing
• Module Development Supported by Application-Oriented
Grants

National Alliance for Medical Image Computing

http://na-mic.org

Image/Scene Management
• XML-Based MRML File
Stores Scene Description
– Volumes (Images, Label
Maps)
– Models
– Hierarchical Affine
Transforms
– Scene Data (Cameras,
Colors, Fiducials, etc).
• Manipulated in World
Coordinates based on
Patient RAS
– Automatically Extracted
from DICOM or GE Files

National Alliance for Medical Image Computing

http://na-mic.org

6
 Image Formats
• DICOM, GE, Headerless, Analyze
– Best Support for MR
– CT and RGB Support limited
– Real-Time from OpenMR and US
• Time Series
– Analyze Sequence
– DICOM Sequence
– BXH Files
• BIAC XML Header (Brian Image Analysis Center, Duke)
• Like MRML for fMRI; Integration Work Ongoing

National Alliance for Medical Image Computing

http://na-mic.org

User Segmentation Tools

• Label Map Editor
– Draw on Orthogonal
Planes
– Connected Component
“Island” Tools
– Math Morphology
– Image Masking and Logical
Operations
– Level Set, Fast Marching

National Alliance for Medical Image Computing

http://na-mic.org

7
 EM Segmenter
Segmentation tool designed for
fully automatic, high-quality
parcellation of the brain*:
• Segmentation of cortical
and ventricle substructures
• Multi channel input

• Hierarchical segmentation
based on anatomy
• Multi threaded
• User friendly interface
* For further information see Pohl et al. “Incorporating Non-Rigid Registration into
Expectation Maximization Algorithm to Segment MR Images”, MICCAI 2002, pp. 564-572
National Alliance for Medical Image Computing
http://na-mic.org

Models
• Triangle Meshes
from Label Maps
– Marching Cubes,
Decimation,
Smoothing
• Model Hierarchies
• Clipping By Slice
Planes

Image: Developmental Neuroinformatics, Simon Warfield

National Alliance for Medical Image Computing

http://na-mic.org

8
 Registration
• Interactive Manual
Transform Editing
• Landmark Based
Alignment
• Rigid Intensity Registration
– Mutual Information Metric
– ITK Implementation
• Non-Rigid Registration
– Demon’s Method
– Available by Request to CNI

National Alliance for Medical Image Computing

http://na-mic.org

Diffusion Tensor Tractography

Three crossing fiber tracts

• Multiple MR Gradient
Acquisitions
– Sensitive to Brownian Diffusion of
Water
– Cell Membranes Restrict Diffusion
• Post Processing to Extract
Probable White Matter Tracts
– Actual Tracts are Far Below the
Resolution of the Scan

Images Provided by Westin, Park, O’Donnell et al

National Alliance for Medical Image Computing

http://na-mic.org

9
 DTMRI Tools

• Convert Gradient
Images to Tensors
• Generate Scalars
– ADC, FA, etc
• Visualize Glyphs
• Tractography
– User Guided
– From ROI

National Alliance for Medical Image Computing

http://na-mic.org

Segmentation and Tractography

• Parcellation
– Freesurfer
(MGH)
• Tractography
– DoDTI (H.J.
Park)
• Visualization
– Slicer
• Full Integration
with Slicer
Planned
National Alliance for Medical Image Computing
http://na-mic.org

10
 DTMRI Examples
• White Matter Atlas
– Under Development in Slicer
by James Fallon, UCI

National Alliance for Medical Image Computing

http://na-mic.org

Slicer fMRI Goals

• Advanced GUI and Interactive

Visualization Environment
– iBrowser

• Platform for Activation Detector

Research
– fMRIEngine

• Framework for fMRI Integration

with Other Modalities

National Alliance for Medical Image Computing

http://na-mic.org

11
 Application: Pre-Operative Map
• Structural
– MRI Tumor Segmentation
• DTI
– Diffusion Tensor Imaging
• fMRI
– Functional MRI
• MEG
– Magneto Encephlogram
• Anatomy Atlas
– “Textbook” Information
National Alliance for Medical Image Computing
http://na-mic.org

Pre-Op Map Example

National Alliance for Medical Image Computing

http://na-mic.org

12
Medical Image Analysis with ITK and Related Open-Source
Software
This course introduces attendees to select open-source efforts in the field of medical image
analysis. Opportunities for users and developers are presented.
The course particularly focuses on the open-source Insight Toolkit (ITK) for medical image
segmentation and registration. The course describes the procedure for downloading and installing
the toolkit and covers the use of its data representation and filtering classes. Attendees are shown
how ITK can be used in their research, rapid prototyping, and application development.

LEARNING OUTCOMES
After completing this course, attendees will be able to:
• contribute to and benefit from open-source software for medical image analysis
• download and install the ITK toolkit
• start their own software project based on ITK
• design and construct an image processing pipeline
• combine ITK filters for medical image segmentation
• combine ITK components for medical image registration

INTENDED AUDIENCE
This course is intended for anyone involved in medical image analysis. In particular it targets
graduate students, researchers and professionals in the areas of computer science and medicine.
Attendees should have an intermediate level on object oriented programming with C++ and must
be familiar with the basics of medical image processing and analysis.

COURSE LEVEL
Intermediate

COURSE LENGTH
Full-day (6.5 hours)

COURSE SCHEDULE
Luis Ibanez, Kitware Inc.
1) The Insight Software Consortium: contributing and using open-source
2) The architecture and installation of the Insight Toolkit

Josh Cates, Univ. of Utah

1) Segmentation methods of the Insight Toolkit

Lydia Ng, Allen Brain Institute

1) Registration methods of the Insight Toolkit

Julien Jomier, CADDLab, Univ of North Carolina

1) Image IO using the Insight Toolkit
2) The Image-Guided Surgery Toolkit: architecture overview

Bill Lorensen, GE Research

1) Using the Insight Toolkit with TK/TCL
2) Applications of the Insight Toolkit

INSTRUCTORS
Luis Ibanez is a Research Engineer at Kitware, Inc. He received a BSc in Physics in 1989 and a
MSc in Optics in 1994 from the Universidad Industrial de Santander (Bucaramanga, Colombia). He
received a PhD in 2000 from the Universite de Rennes I (Rennes, France). From 1999 to 2001 he
was Research Assistant Professor in the Division of Neurosurgery at the University of North
Carolina at Chapel Hill. His current research interest the application of genomics paradigms to
computation.

Josh Cates is a member of the research staff of the Scientific Computing and Imaging Institute at
the University of Utah's School of Computer Science. He received his BS degree in Biology in 1995
and MS in Computer Science in 1999 from the University of Tennessee. His interests include
software engineering and problems in computer vision, including image processing (differential
geometry and p.d.e.-based methods), image segmentation, and computed tomography.

Lydia Ng is a member of the Informatics group at the Allen Institute for Brain Science. She
received a BE in Electrical Engineering and a BSc in Computer Science in 1994 from The University
of New South Wales (Sydney, Australia). She received a Ph.D. in 2000 from Macquarie University
(Sydney, Australia). Her research interests include: image registration, PDE based image
processing methods, motion estimation and development of software for medical image processing
and analysis.

Julien Jomier is a research faculty member in the Computer-Aided Diagnosis and Display
Laboratory at the University of North Carolina at Chapel Hill. He has extensive C++ programming
and software design experience as well as medical image analysis and data handling expertise. He
is currently continuing his development of ITK’s spatialObject library and his freely available spatial
object viewer library. Julien is also contributing to the development of the Insight Journal, the
Image Guided Surgery Toolkit, and numerous other open-source efforts.

William Lorensen is a Graphics Engineer in the Electronic Systems Laboratory at GE's Corporate
Research and Development Center in Schenectady, NY. He has over 35 years of experience in
computer graphics and software engineering. He is currently working on algorithms for 3D medical
graphics and scientific visualization.

Stephen Aylward is the director of the Computer-Aided Diagnosis and Display Laboratory and an
Associate Professor of Radiology, Computer Science, and Surgery at the University of North
Carolina at Chapel Hill. The CADDLab's research spans medical image analysis research for
disease detection, diagnosis, and treatment guidance. Dr. Aylward's research focuses on
segmenting and registering medical images using model-to-image registration strategies.