Camera model & calibration

Prof. Didier Stricker

Kaiserlautern University

http://ags.cs.uni-kl.de/

DFKI – Deutsches Forschungszentrum für Künstliche Intelligenz

http://av.dfki.de

3D Computer Vision

1

2

f Z Z

fXx

X O

-x

Pinehole model - Perspective Projection.

Remember?

• Finding camera’s internal parameters that effect the

imaging process.

1. Position of the image center on the image (usually not just

(width/2, height/2).

2. Focal length

3. Scaling factors for row and column pixels.

4. Lens distortion.

3

Camera Calibration

Calibration: finding the function (defined by the camera) that maps 3D points to

2D image plane.

First thing required is to obtain pairs of corresponding 3D and 2D points,

(X1, Y1, Z1) (x1, y1), (X2, Y2, Z2) (x2, y2), ….

4

Calibration Procedure

Calibration Target: Two perpendicular planes with chessboard pattern.

1. We know the 3D positions of the corners with respect to a

coordinates system defined on the target.

2. Place a camera in front of the target and we can locate the

corresponding corners on the image. This gives us the

correspondences.

3. Recover the equation that describes imaging projection and

camera’s internal parameters. At the same time, also recover the

relative orientation between the camera and the target (pose).

5

Calibration Procedure

1. Corner detector

2. Canny Edge detector plus fitting lines to the detected edges. Find the intersections of the lines.

3. Manual input.

Matching 3D and 2D points (we know the number of corners) by counting.

This gives corresponding pairs

(world point) < --- > (image point )

(X1, Y1, Z1) (x1, y1),

6

Finding Corners

7

World Coordinates and Camera Coordinates

Rotation matrix

8

Camera Frame to World Frame

9

Linear Algebra

Definition: a matrix R is a rotation matrix if and only

if it is a orthogonal matrix with determinant +1

Orthogonal Matrix: a square matrix with real entries

whose columns and rows are orthogonal vectors

with length 1.

That means: RRT = I

Or: R-1 = RT

Definition: 3D-Rotation

10

Sequence not standardized! Many different conventions!

Let = - Rt T

(C is the camera center)

11

Homogeneous Coordinates

Use Homogeneous coordinates, the perspective projection becomes linear.

12

Perspective Projection

13

Pixel Coordinates

14

Calibration Matrix

15

Putting Everything Together

1. Estimate matrix P using scene points and their

images

2. Estimate the intrinsic and extrinsic parameters

Left 3x3 sub-matrix is the product of an upper

triangular matrix and an orthogonal matrix.

16

Calibration

17

Computing the Matrix P

18

Computing the Matrix P

Rank 2

19

Computing the Matrix P

20

Computing the Matrix P

-

-

-

21

Computing the translation component

Finding the camera orientation and internal parameters

KR (use RQ decomposition ~QR)

Q R =( )-1= -1 -1 Q R

(if only QR, invert)

A QR decomposition (also called a QR factorization) of a matrix is a

decomposition of the matrix into an orthogonal and a right triangular

matrix.

22

Camera matrix decomposition

23

Project known 3D points on the image

Compare their projections with the

corresponding pixel coordinates of the

points.

Repeat for many points and estimate “re-

projection” error!

How should we estimate the accuracy

of a calibration algorithm?

Further Improvement

Most popular non-linear minimization algorithm is the Levenberg-Marquart

minimization

LM is more robust to local minima than e.g. the Gauss–Newton algorithm

and the method of gradient descent)

24

Improvement: computing the matrix P

25

Barrel and Pincushion

Tangential

Advanced Calibration: Nonlinear

Distortions

26

tele wideangle

Barrel and Pincushion Distortion

27

To model lens distortion Use above projection operation instead of standard projection matrix

multiplication

Modeling distortion

Apply radial distortion

Apply focal length translate image center

Project to “normalized”

image coordinates

Radial and Tangential Distortion

image plane

ideal actual

ideal actual

28

Set k1k20, solve for undistorted case

Find optimal k1,k2 via nonlinear least squares

Iterate

Tends to generate good calibrations

Distorted Camera Calibration

29

Image Rectification

30

Examplary calibration set-ups

31

Advantage:

Very simple to formulate and solve

Disadvantages:

Doesn’t tell you the camera parameters

Doesn’t model radial distortion

Hard to impose constraints (e.g., known focal length)

Doesn’t minimize the right error function

For these reasons, nonlinear methods are preferred

• Define error function E between projected 3D points and image positions

– E is nonlinear function of intrinsics, extrinsics, radial distortion

• Minimize E using nonlinear optimization techniques

– e.g., variants of Newton’s method (e.g., Levenberg Marquart)

Direct linear calibration

Alternative: multi-plane calibration

32

Images courtesy Jean-Yves Bouguet, Intel Corp.

Advantage • Only requires a plane

• Don’t have to know positions/orientations

• Good code available online!

– Intel’s OpenCV library: http://www.intel.com/research/mrl/research/opencv/

– Matlab version by Jean-Yves Bouget:

http://www.vision.caltech.edu/bouguetj/calib_doc/index.html

– Zhengyou Zhang’s web site: http://research.microsoft.com/~zhang/Calib/

33

Reduce the number of interactions

Pre-compute as much parameters as

possible: Usually all intrinsic camera parameters

More stable

Less user-input required

Calibration or Pose estimation using a

CAD model

34

Calibration or Pose estimation using a CAD

model

35

THANK YOU!