Scribd
Upload
20 views24 pages

Classifying Images: D.A. Forsyth

This document discusses image classification using visual words. It proposes representing images as histograms of visual words, where visual words are local image patches clustered using k-means clustering. The k-means algorithm is described as choosing cluster centers to minimize distances between points and centers. Images are then represented as histograms counting visual words in local regions, which provides a summary insensitive to spatial changes. Evaluation methods like accuracy, precision and recall are also discussed.

Uploaded by

AnhNguyễn
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
20 views24 pages

Classifying Images: D.A. Forsyth

This document discusses image classification using visual words. It proposes representing images as histograms of visual words, where visual words are local image patches clustered using k-means clustering. The k-means algorithm is described as choosing cluster centers to minimize distances between points and centers. Images are then represented as histograms counting visual words in local regions, which provides a summary insensitive to spatial changes. Evaluation methods like accuracy, precision and recall are also discussed.

Uploaded by

AnhNguyễn
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 24

Classifying images

D.A. Forsyth

Image classification - features

Issue:

category will not produce a single, simple pattern


but it might have components that are distinctive, but move around

look for distinctive local patches (visual words)


build a histogram

Idea:

Important trick: K-Means

Choose a fixed number of clusters


Choose cluster centers and point-cluster allocations to
minimize error

cant do this by search

iclusters

jelements
of i'th cluster

x j i

there are too many possible allocations.

fix cluster centers; allocate points to closest cluster


fix allocation; compute best cluster centers
x could be any set of features for which we can compute a distance
(careful about scaling)

Algorithm

K-means

Building visual words - I

Learn a dictionary

cluster patch representations with k-means


k will be big (1000s-100,000s)

Building visual words - II

Encode an image

find all interest points


for each patch around each interest point
map patch to closest cluster center
build histogram of interest points

Visual words

Visual words

Visual words

Features from visual words

Histogram

good summary of what is in image; quick and efficient


insensitive to spatial reorganization

build histograms of local blocks at various scales


less insensitive to spatial reorganization

Spatial pyramid

Spatial pyramids

Some standard tasks

Caltech 101, 256

images of isolated objects in 101, 256 categories

same, 1000s of categories

not isolated, fewer categories

eg indoor, etc.

Imagenet
Pascal image classification
Scenes
Materials

Evaluation methods

Total error rate

percentage of classification attempts that get the wrong answer

percentage of classification attempts that get the right answer

table showing how classes are mixed up

Accuracy
Class confusion matrix
Look at errors

Accuracy on Caltech

Material classification

Spatial pyramids + scenes

Evaluation

Precision

percentage of items in retrieved set that are relevant

percentage of relevant items that are retrieved

use classifier to label a collection of images


now plot precision against recall for different classifier thresholds

average precision
average of precision as a function of recall

Recall
Precision vs recall
AP

Precision vs recall

Can be hard

You might also like