Object Recognition

Digital Image Processing

Lecture 13,14,15 – Object

Regconition

Lecturer: Ha Dai Duong

Faculty of Information Technology

I. Introduction

The scope covered by out treatment of digital image

processing to include recognition of individual image

regions, which we called objects or patterns.

The approaches to pattern recognition are divided

into two principal areas:

Decision-theoretic: This catogory deals with patterns

described using quantitative descriptors, such as length,

area, texture …

Structural: Deals with patterns best described by qualitative

descriptors, such as the relational descriptors.

Digital Image Processing

2

1

Object Recognition

II. Patterns and pattern classes

A pattern is an arrangement of descriptors. The

name feature is used often in pattern recognition to

denote a descriptor.

A pattern class is a family of patterns that share

some common properties.

w1, w2, .., wK denotes pattern classes, Where K is the

number of classes

Pattern recognition by machine involves techniques for

assining patterns to their respective classes automatically

(and with as little human intervention as possible)

Digital Image Processing

3

II. Patterns and pattern classes

Three common pattern arrangements used in

practice are:

Vectors: for quantitative descriptions

Strings and trees: for qualitative descriptions

Pattern vectors are represented by bold

lowercase letters, such as z, y and z, and

take a form

or

Digital Image Processing

4

2

Object Recognition

II. Patterns and pattern classes

Example:

In

a classic paper to

recognize

three

types of iris flowers

(Setosa, virginica,

and versicolor) by

measuring

the

widths and lengths

of their petals

Digital Image Processing

5

II. Patterns and pattern classes

Another Example: We can form pattern vectors by

letting x1=r(θ1),…xn=r(θn). The vectors became

points in n-dimensions space.

Digital Image Processing

6

3

Object Recognition

II. Patterns and pattern classes

In some applications pattern characteristics are best

described by structural relationships.

For example: fingerprint recognition is based on the

interrelationships of print features. Together with

their relatives sizes and locations, these features are

primitive components that describe fingerprint ridge

properties, such as abrupt ending, branching, and

disconnected segments.

Recognition problems of this type, in which not only

quantitative mearsures about each feature but also

the spatial relationships between the features

determine class menbership, generally are best

solved by structural approachs.

Digital Image Processing

7

II. Patterns and pattern classes

Example

Digital Image Processing

8

4

Object Recognition

II. Patterns and pattern classes

Example

Digital Image Processing

9

II. Patterns and pattern classes

Example

Digital Image Processing

10

5

Object Recognition

III. Recognition Based on DecisionTheoretic Methods

Decision-theoretic appoaches to recognition

are based on the use of decision functions.

Let x=(x1, x2,.., xn)T represent an n-dimensional

pattern vector

ω1, ω2,.., ωW denote W pattern classes

The basic problem in decision-theoretic pattern

recognition is to find decision functions d1(x),

d2(x), .., dw(x) with property that, if pattern x

belongs to class ωi then:

Digital Image Processing

11

III. Recognition Based on DecisionTheoretic Methods

The decision boundary separating class ωi

from ωj is given by values of x for which

di(x)=dj(x), or equivalently, by value of x for

which:

Common practice is to identify the decision

boundary between two classes by the single

function dij(x)=di(x)-dj(x)=0

Thus dij(x)>0 for pattern of class ωi and

dij(x)<0 for pattern of class ωj

Digital Image Processing

12

6

Object Recognition

III.1 Matching by minimun distance

classifier

Suppose that we define the prototype of each

pattern class to be the mean vector of the pattern of

that class

where Nj is the number of pattern vector from class ωj

Using the Euclidean distance to determine closeness reduces

the problem to computing the distance measures:

Digital Image Processing

13

III.1 Matching by minimun distance

classifier

Assign x to class ωj if Di(x) is smallest distance

It is not difficult to show that selecting the

smallest distance is equivalent to evaluating the

functions

And assign x to class ωj if Di(x) is largest numerical

value. This formalation agrees with the concept of a

decision function as define in Eq (12.2-1)

Digital Image Processing

14

7

Object Recognition

III.1 Matching by minimun distance

classifier

Digital Image Processing

15

III. Recognition Based on DecisionTheoretic Methods

Matching by minimun distance classifier

Digital Image Processing

16

8

Object Recognition

III.1 Matching by minimun distance

classifier

Digital Image Processing

17

III.1 Matching by minimun distance

classifier

Digital Image Processing

18

9

Object Recognition

III.2 Matching by Correlation

Problem is finding matches of

subimage w(s,t) of size JxK

within a image f(x,y) of size

MxN, assume that J<=M,

K<=N

In its simplest form, the

correlation between f(x,y) and

w(x,y) is

For x=0,1, .., M-1, y=0,1,.., N-1 and the summation is taken

over the image region where w and f overlap

Digital Image Processing

19

III.2 Matching by Correlation

Move w around the image area, giving

the function c(x,y). The maximum

value(s) of c indicates the position(s)

where w best matches f

The correlation function given in 12.2-7

has disadvantages of being sensitive to

changes in the amplitude of f and w. For

example, doubling all values of f doubles

the value of c(x,y).

Digital Image Processing

20

10

Object Recognition

III.3 Matching by correlation coefficient

An other approach is to perform matching via the correlation

coefficient, which is defined as:

where x=0,1,..,M-1, y=0,1,..,N-1,

wN is average value of the pixels in w,

fN is average of f in the region coincident with the current location of

w, and

The summation are taken over the coordinates common to both f

and w.

The correlation coefficient γ(x,y) is scaled in the range -1 to 1,

independent of scale changes in amplitude of f and w

Digital Image Processing

21

III.3 Matching by correlation coefficient

Digital Image Processing

22

11

Object Recognition

III.4 Optimum Statistical Classifiers

Foundation

Denote:

p(ωj/x) is probability that a particular pattern x comes from

class ωj

Lkj is coefficient of loss when pattern x actually comes from

class ωj but classifier decides that x came from class ωk.

Then, the average loss incurred if assign x to class ωk,

rj(x):

This equantion often is called conditional average risk or

loss in decision theory terminology.

Digital Image Processing

23

III.4 Optimum Statistical Classifiers

Foundation

We know that p(A/B)=[p(A)*p(B/A)]/p(B). Using this

expression, we write 12.2-9 in the form:

where p(x /ωk) is the probability density function of the

patterns from class ωk and p(ωk) is probability of

occurrence of class ωk.

Bacause 1/p(x) is positive and common to all rj(x), so it

can be dropped from 12.2-10 then rj(x) can be:

Digital Image Processing

24

12

Object Recognition

III.4 Optimum Statistical Classifiers

Foundation

The classifier has W possible classes to choose from for

any given unknow pattern. If it computers r1(x),

r2(x),…,rW(x) for each pattern x and assigns the pattern to

class with the smallest loss, the total average loss with

respect to all decisions will be minimum.

The classifier that minimizes the total average loss is

called the Bayes classifier.

Thus Bayes classifier assigns an unknown pattern x to

class ωi if ri(x)i. In other words, x is

assigned to class ωi if

Digital Image Processing

25

III.4 Optimum Statistical Classifiers

Foundation

The “loss” for correct decision is assigned value 0

and the loss for incorrect decision is assigned

value 1. Under these conditions, the loss function

becames:

Digital Image Processing

26

13

Object Recognition

III.4 Optimum Statistical Classifiers

Foundation

Digital Image Processing

27

III.4 Optimum Statistical Classifiers

Foundation

The decision functions given in 12.2-17 are

optimal in the sense that they minimize the

average loss in misclassification.

However, we have to know:

The probability density functions of the patterns in each

class, and

The probability of occurrence of each class

Digital Image Processing

28

14

Object Recognition

III.4 Optimum Statistical Classifiers

Foundation

The second requirement is not problem. For instance, if all classes

are equally likely to occur then p(ωi) = 1/M. Even if this condition is

not true, these probabilities generally can be inferred from

knowledge of the problem.

Estimation of the probability density functions p(x/ωi) is another

matter. If the pattern vectors, x, are n-dimensional, then p(x/ωi) is a

function of n variables, which, if its form is not know, requires

methods from multivariate probability theory for its estimation.

These methods are difficult to apply in practice.

For these reasons, use for Bayes classifier generally is based on

the assumation of an analytic expression for the various density

functions and then an estimation of the necessary parameters from

samples patterns from each class. By far the most prevalent form

assumed for p(x/ωi) is Gaussian probability density function

29

Digital Image Processing

III.4 Optimum Statistical Classifiers

Bayes classifier

classes

for

Gaussian

pattern

Let consider a 1-D problem (n=1) involving two

pattern classes (W=2) governed by Gaussian

densities, with means m1 and m2 and standard

deviations σ1 and σ2, respectively. From Eq 12.217 the Bayes decision functions have the form:

Digital Image Processing

30

15

Object Recognition

III.4 Optimum Statistical Classifiers

Bayes classifier for Gaussian pattern classes

Fig 12.10 show a plot of the probability density functions for the

two classes. The boundary between the two classes is a single

point, denoted x0 suchs that d1(x0)=d2(x0).

If the two classes are equally likely to occur, then p(ω1)= p(ω2)

=1/2, and the decision boundary is the value of x0 for which

p(x0/ω1)= p(x0/ω2)

Digital Image Processing

31

III.4 Optimum Statistical Classifiers

Bayes classifier for Gaussian pattern classes

In the n-demensional case, the Gaussian density

of the vectors in the jth pattern class has the form

where, mj and Cj are approximated as

Digital Image Processing

32

16

Object Recognition

III.4 Optimum Statistical Classifiers

Bayes classifier for Gaussian pattern classes

Because of the exponential form of Gaussian density,

working with the natural logarithm of decision function is

more convenient. In other words, we can use the form:

And it infers

Digital Image Processing

33

IV.Neural Networks

The approaches discussed in the preceding is based on the

use of sample patterns to estimate statistical parameters.

The patterns used to estimate these parameters usually are

called training patterns, and a set of such patterns from

each class is called a training set.

The process by which a training set is used to obtain

decision functions is called learning or training.

The statistical properties of the pattern classes in a problem

often are unknown or cannot be estimated.

In practice, such decision-theoretic problems are best

handled by methods that yield the required decision

functions directly via training

Digital Image Processing

34

17

Object Recognition

IV.Neural Networks

An approach manage to organize some nonlinear computing

elements (called neurons) as a networks to classify a input

pattern.

The resulting models are referred to by various names:

neural networks, neurocomputers, parallel distributed

processing (PDP) modelsm neuromorphic systems, layered

self-adaptive networks, connectionist models.

Here we use the name neural networks or neural nets. We

use these networks as vehicles for adaptively developing the

coefficients of decision functions via successive

presentations of training sets of patterns.

Digital Image Processing

35

IV.1 Perceptron

The most simple of neural networks is perceptron. In its most

basic form, the perceptron learns a linear decision function that

dichotomizes two linearly separable training sets.

Digital Image Processing

36

18

Object Recognition

IV.1 Perceptron

Fig 12.14 shows schematically the perceptron model for

two pattern classes.

The response of this basic device is based on weighted

sum of its inputs; that is

which is a linear decision function with respect to the

components of the pattern vectors.

The coefficients wi, i=1,2,…, n, n+1, called weights

The function that maps the output of the summing

junction into the final output of the device sometimes is

called the activation function.

Digital Image Processing

37

IV.1 Perceptron

When d(x)>0 the activation function causes the output of perceptron to

be +1, it indecates the pattern x was recognized as belonging to class

ω1. The reverse is true when d(x)<0.

When d(x)=0, x lies on the decision surface separating the two pattern

classes. The decision boundary implemented by the perceptron is

obtained by set Eq 12.2-29 equal to zero:

which is the equation of a hyperplane in n-dimensional pattern space.

Geometrically, the first n coefficients establish the orientation of the

hyperplane, whereas the last coefficient, wn+1, is proportional to the

perdendicular distance from the orgin to the hyperplane.

Digital Image Processing

38

19

Object Recognition

IV.1 Perceptron

Denote yi=xi, i=1,2, .., n, and yn+1=1, then 12.2-29

becomes:

where

y=(y1,y2,..,yn,1)T is now an augmented pattern vector and

w=(w1,w2,..,wn,wn+1) is called the weight vector.

The problem is how to establish the weight vector ?

Digital Image Processing

39

IV.1 Perceptron

Training algorithms

Linearly separable classes: A simple, iterative algorithm for

obtaining a solution weight vector for two linearly separable training sets

follows. For two training sets of augmented pattern vectors belonging to

pattern classes ω1 and ω2, respectively.

Digital Image Processing

40

20

Object Recognition

IV.1 Perceptron

Training algorithms

The correction increment c is assumed to be positive and, for now, to be constant.

This algorithm sometimes is referred to as the fixed increment correction rule.

Digital Image Processing

41

IV.1 Perceptron

Training algorithms

Digital Image Processing

42

21

Object Recognition

IV.1 Perceptron

Training algorithms

Digital Image Processing

43

IV.1 Perceptron

Training algorithms

Nonseparable classes: One of the early methods of training

perceptron is Widrow-Hoff, or Least-Mean-Square (LMS)

delta rule, the method minimizes the error between the

actual and desired response at any time training step.

Digital Image Processing

44

22

Object Recognition

IV.1 Perceptron

Training algorithms

Digital Image Processing

45

IV.1 Perceptron

Training algorithms

Digital Image Processing

46

23

Object Recognition

IV.1 Perceptron

Training algorithms

Digital Image Processing

47

IV.2 Multilayer Neural Networks

Training algorithms

Digital Image Processing

48

24

Object Recognition

IV.2 Multilayer Neural Networks

Digital Image Processing

49

IV.2 Multilayer Neural Networks

Digital Image Processing

50

25

Digital Image Processing

Lecture 13,14,15 – Object

Regconition

Lecturer: Ha Dai Duong

Faculty of Information Technology

I. Introduction

The scope covered by out treatment of digital image

processing to include recognition of individual image

regions, which we called objects or patterns.

The approaches to pattern recognition are divided

into two principal areas:

Decision-theoretic: This catogory deals with patterns

described using quantitative descriptors, such as length,

area, texture …

Structural: Deals with patterns best described by qualitative

descriptors, such as the relational descriptors.

Digital Image Processing

2

1

Object Recognition

II. Patterns and pattern classes

A pattern is an arrangement of descriptors. The

name feature is used often in pattern recognition to

denote a descriptor.

A pattern class is a family of patterns that share

some common properties.

w1, w2, .., wK denotes pattern classes, Where K is the

number of classes

Pattern recognition by machine involves techniques for

assining patterns to their respective classes automatically

(and with as little human intervention as possible)

Digital Image Processing

3

II. Patterns and pattern classes

Three common pattern arrangements used in

practice are:

Vectors: for quantitative descriptions

Strings and trees: for qualitative descriptions

Pattern vectors are represented by bold

lowercase letters, such as z, y and z, and

take a form

or

Digital Image Processing

4

2

Object Recognition

II. Patterns and pattern classes

Example:

In

a classic paper to

recognize

three

types of iris flowers

(Setosa, virginica,

and versicolor) by

measuring

the

widths and lengths

of their petals

Digital Image Processing

5

II. Patterns and pattern classes

Another Example: We can form pattern vectors by

letting x1=r(θ1),…xn=r(θn). The vectors became

points in n-dimensions space.

Digital Image Processing

6

3

Object Recognition

II. Patterns and pattern classes

In some applications pattern characteristics are best

described by structural relationships.

For example: fingerprint recognition is based on the

interrelationships of print features. Together with

their relatives sizes and locations, these features are

primitive components that describe fingerprint ridge

properties, such as abrupt ending, branching, and

disconnected segments.

Recognition problems of this type, in which not only

quantitative mearsures about each feature but also

the spatial relationships between the features

determine class menbership, generally are best

solved by structural approachs.

Digital Image Processing

7

II. Patterns and pattern classes

Example

Digital Image Processing

8

4

Object Recognition

II. Patterns and pattern classes

Example

Digital Image Processing

9

II. Patterns and pattern classes

Example

Digital Image Processing

10

5

Object Recognition

III. Recognition Based on DecisionTheoretic Methods

Decision-theoretic appoaches to recognition

are based on the use of decision functions.

Let x=(x1, x2,.., xn)T represent an n-dimensional

pattern vector

ω1, ω2,.., ωW denote W pattern classes

The basic problem in decision-theoretic pattern

recognition is to find decision functions d1(x),

d2(x), .., dw(x) with property that, if pattern x

belongs to class ωi then:

Digital Image Processing

11

III. Recognition Based on DecisionTheoretic Methods

The decision boundary separating class ωi

from ωj is given by values of x for which

di(x)=dj(x), or equivalently, by value of x for

which:

Common practice is to identify the decision

boundary between two classes by the single

function dij(x)=di(x)-dj(x)=0

Thus dij(x)>0 for pattern of class ωi and

dij(x)<0 for pattern of class ωj

Digital Image Processing

12

6

Object Recognition

III.1 Matching by minimun distance

classifier

Suppose that we define the prototype of each

pattern class to be the mean vector of the pattern of

that class

where Nj is the number of pattern vector from class ωj

Using the Euclidean distance to determine closeness reduces

the problem to computing the distance measures:

Digital Image Processing

13

III.1 Matching by minimun distance

classifier

Assign x to class ωj if Di(x) is smallest distance

It is not difficult to show that selecting the

smallest distance is equivalent to evaluating the

functions

And assign x to class ωj if Di(x) is largest numerical

value. This formalation agrees with the concept of a

decision function as define in Eq (12.2-1)

Digital Image Processing

14

7

Object Recognition

III.1 Matching by minimun distance

classifier

Digital Image Processing

15

III. Recognition Based on DecisionTheoretic Methods

Matching by minimun distance classifier

Digital Image Processing

16

8

Object Recognition

III.1 Matching by minimun distance

classifier

Digital Image Processing

17

III.1 Matching by minimun distance

classifier

Digital Image Processing

18

9

Object Recognition

III.2 Matching by Correlation

Problem is finding matches of

subimage w(s,t) of size JxK

within a image f(x,y) of size

MxN, assume that J<=M,

K<=N

In its simplest form, the

correlation between f(x,y) and

w(x,y) is

For x=0,1, .., M-1, y=0,1,.., N-1 and the summation is taken

over the image region where w and f overlap

Digital Image Processing

19

III.2 Matching by Correlation

Move w around the image area, giving

the function c(x,y). The maximum

value(s) of c indicates the position(s)

where w best matches f

The correlation function given in 12.2-7

has disadvantages of being sensitive to

changes in the amplitude of f and w. For

example, doubling all values of f doubles

the value of c(x,y).

Digital Image Processing

20

10

Object Recognition

III.3 Matching by correlation coefficient

An other approach is to perform matching via the correlation

coefficient, which is defined as:

where x=0,1,..,M-1, y=0,1,..,N-1,

wN is average value of the pixels in w,

fN is average of f in the region coincident with the current location of

w, and

The summation are taken over the coordinates common to both f

and w.

The correlation coefficient γ(x,y) is scaled in the range -1 to 1,

independent of scale changes in amplitude of f and w

Digital Image Processing

21

III.3 Matching by correlation coefficient

Digital Image Processing

22

11

Object Recognition

III.4 Optimum Statistical Classifiers

Foundation

Denote:

p(ωj/x) is probability that a particular pattern x comes from

class ωj

Lkj is coefficient of loss when pattern x actually comes from

class ωj but classifier decides that x came from class ωk.

Then, the average loss incurred if assign x to class ωk,

rj(x):

This equantion often is called conditional average risk or

loss in decision theory terminology.

Digital Image Processing

23

III.4 Optimum Statistical Classifiers

Foundation

We know that p(A/B)=[p(A)*p(B/A)]/p(B). Using this

expression, we write 12.2-9 in the form:

where p(x /ωk) is the probability density function of the

patterns from class ωk and p(ωk) is probability of

occurrence of class ωk.

Bacause 1/p(x) is positive and common to all rj(x), so it

can be dropped from 12.2-10 then rj(x) can be:

Digital Image Processing

24

12

Object Recognition

III.4 Optimum Statistical Classifiers

Foundation

The classifier has W possible classes to choose from for

any given unknow pattern. If it computers r1(x),

r2(x),…,rW(x) for each pattern x and assigns the pattern to

class with the smallest loss, the total average loss with

respect to all decisions will be minimum.

The classifier that minimizes the total average loss is

called the Bayes classifier.

Thus Bayes classifier assigns an unknown pattern x to

class ωi if ri(x)

assigned to class ωi if

Digital Image Processing

25

III.4 Optimum Statistical Classifiers

Foundation

The “loss” for correct decision is assigned value 0

and the loss for incorrect decision is assigned

value 1. Under these conditions, the loss function

becames:

Digital Image Processing

26

13

Object Recognition

III.4 Optimum Statistical Classifiers

Foundation

Digital Image Processing

27

III.4 Optimum Statistical Classifiers

Foundation

The decision functions given in 12.2-17 are

optimal in the sense that they minimize the

average loss in misclassification.

However, we have to know:

The probability density functions of the patterns in each

class, and

The probability of occurrence of each class

Digital Image Processing

28

14

Object Recognition

III.4 Optimum Statistical Classifiers

Foundation

The second requirement is not problem. For instance, if all classes

are equally likely to occur then p(ωi) = 1/M. Even if this condition is

not true, these probabilities generally can be inferred from

knowledge of the problem.

Estimation of the probability density functions p(x/ωi) is another

matter. If the pattern vectors, x, are n-dimensional, then p(x/ωi) is a

function of n variables, which, if its form is not know, requires

methods from multivariate probability theory for its estimation.

These methods are difficult to apply in practice.

For these reasons, use for Bayes classifier generally is based on

the assumation of an analytic expression for the various density

functions and then an estimation of the necessary parameters from

samples patterns from each class. By far the most prevalent form

assumed for p(x/ωi) is Gaussian probability density function

29

Digital Image Processing

III.4 Optimum Statistical Classifiers

Bayes classifier

classes

for

Gaussian

pattern

Let consider a 1-D problem (n=1) involving two

pattern classes (W=2) governed by Gaussian

densities, with means m1 and m2 and standard

deviations σ1 and σ2, respectively. From Eq 12.217 the Bayes decision functions have the form:

Digital Image Processing

30

15

Object Recognition

III.4 Optimum Statistical Classifiers

Bayes classifier for Gaussian pattern classes

Fig 12.10 show a plot of the probability density functions for the

two classes. The boundary between the two classes is a single

point, denoted x0 suchs that d1(x0)=d2(x0).

If the two classes are equally likely to occur, then p(ω1)= p(ω2)

=1/2, and the decision boundary is the value of x0 for which

p(x0/ω1)= p(x0/ω2)

Digital Image Processing

31

III.4 Optimum Statistical Classifiers

Bayes classifier for Gaussian pattern classes

In the n-demensional case, the Gaussian density

of the vectors in the jth pattern class has the form

where, mj and Cj are approximated as

Digital Image Processing

32

16

Object Recognition

III.4 Optimum Statistical Classifiers

Bayes classifier for Gaussian pattern classes

Because of the exponential form of Gaussian density,

working with the natural logarithm of decision function is

more convenient. In other words, we can use the form:

And it infers

Digital Image Processing

33

IV.Neural Networks

The approaches discussed in the preceding is based on the

use of sample patterns to estimate statistical parameters.

The patterns used to estimate these parameters usually are

called training patterns, and a set of such patterns from

each class is called a training set.

The process by which a training set is used to obtain

decision functions is called learning or training.

The statistical properties of the pattern classes in a problem

often are unknown or cannot be estimated.

In practice, such decision-theoretic problems are best

handled by methods that yield the required decision

functions directly via training

Digital Image Processing

34

17

Object Recognition

IV.Neural Networks

An approach manage to organize some nonlinear computing

elements (called neurons) as a networks to classify a input

pattern.

The resulting models are referred to by various names:

neural networks, neurocomputers, parallel distributed

processing (PDP) modelsm neuromorphic systems, layered

self-adaptive networks, connectionist models.

Here we use the name neural networks or neural nets. We

use these networks as vehicles for adaptively developing the

coefficients of decision functions via successive

presentations of training sets of patterns.

Digital Image Processing

35

IV.1 Perceptron

The most simple of neural networks is perceptron. In its most

basic form, the perceptron learns a linear decision function that

dichotomizes two linearly separable training sets.

Digital Image Processing

36

18

Object Recognition

IV.1 Perceptron

Fig 12.14 shows schematically the perceptron model for

two pattern classes.

The response of this basic device is based on weighted

sum of its inputs; that is

which is a linear decision function with respect to the

components of the pattern vectors.

The coefficients wi, i=1,2,…, n, n+1, called weights

The function that maps the output of the summing

junction into the final output of the device sometimes is

called the activation function.

Digital Image Processing

37

IV.1 Perceptron

When d(x)>0 the activation function causes the output of perceptron to

be +1, it indecates the pattern x was recognized as belonging to class

ω1. The reverse is true when d(x)<0.

When d(x)=0, x lies on the decision surface separating the two pattern

classes. The decision boundary implemented by the perceptron is

obtained by set Eq 12.2-29 equal to zero:

which is the equation of a hyperplane in n-dimensional pattern space.

Geometrically, the first n coefficients establish the orientation of the

hyperplane, whereas the last coefficient, wn+1, is proportional to the

perdendicular distance from the orgin to the hyperplane.

Digital Image Processing

38

19

Object Recognition

IV.1 Perceptron

Denote yi=xi, i=1,2, .., n, and yn+1=1, then 12.2-29

becomes:

where

y=(y1,y2,..,yn,1)T is now an augmented pattern vector and

w=(w1,w2,..,wn,wn+1) is called the weight vector.

The problem is how to establish the weight vector ?

Digital Image Processing

39

IV.1 Perceptron

Training algorithms

Linearly separable classes: A simple, iterative algorithm for

obtaining a solution weight vector for two linearly separable training sets

follows. For two training sets of augmented pattern vectors belonging to

pattern classes ω1 and ω2, respectively.

Digital Image Processing

40

20

Object Recognition

IV.1 Perceptron

Training algorithms

The correction increment c is assumed to be positive and, for now, to be constant.

This algorithm sometimes is referred to as the fixed increment correction rule.

Digital Image Processing

41

IV.1 Perceptron

Training algorithms

Digital Image Processing

42

21

Object Recognition

IV.1 Perceptron

Training algorithms

Digital Image Processing

43

IV.1 Perceptron

Training algorithms

Nonseparable classes: One of the early methods of training

perceptron is Widrow-Hoff, or Least-Mean-Square (LMS)

delta rule, the method minimizes the error between the

actual and desired response at any time training step.

Digital Image Processing

44

22

Object Recognition

IV.1 Perceptron

Training algorithms

Digital Image Processing

45

IV.1 Perceptron

Training algorithms

Digital Image Processing

46

23

Object Recognition

IV.1 Perceptron

Training algorithms

Digital Image Processing

47

IV.2 Multilayer Neural Networks

Training algorithms

Digital Image Processing

48

24

Object Recognition

IV.2 Multilayer Neural Networks

Digital Image Processing

49

IV.2 Multilayer Neural Networks

Digital Image Processing

50

25

## ORACLE OBJECT TYPE

## Tự tạo Object trong visual basic

## Tự tạo Object trong lập trình Visual Basic 6

## Tự tạo Object trong lập trình

## CHƯƠNG ORACLE OBJECT TYPE

## Lecture 2 - RF Fundamentals

## Lecture 7 - 802.11 WLAN Architecture

## Object-oriented Design

## Lecture 3 Block

## Lecture 4 DictionaryMethods

Tài liệu liên quan