3D Computer Vision - - TU Kaiserslautern · 3D Computer Vision 1 . 2 Z f Z f x X X - O x Pinehole...
Transcript of 3D Computer Vision - - TU Kaiserslautern · 3D Computer Vision 1 . 2 Z f Z f x X X - O x Pinehole...
Camera model & calibration
Prof. Didier Stricker
Kaiserlautern University
http://ags.cs.uni-kl.de/
DFKI – Deutsches Forschungszentrum für Künstliche Intelligenz
http://av.dfki.de
3D Computer Vision
1
2
f Z Z
fXx
X O
-x
Pinehole model - Perspective Projection.
Remember?
• Finding camera’s internal parameters that effect the
imaging process.
1. Position of the image center on the image (usually not just
(width/2, height/2).
2. Focal length
3. Scaling factors for row and column pixels.
4. Lens distortion.
3
Camera Calibration
Calibration: finding the function (defined by the camera) that maps 3D points to
2D image plane.
First thing required is to obtain pairs of corresponding 3D and 2D points,
(X1, Y1, Z1) (x1, y1), (X2, Y2, Z2) (x2, y2), ….
4
Calibration Procedure
Calibration Target: Two perpendicular planes with chessboard pattern.
1. We know the 3D positions of the corners with respect to a
coordinates system defined on the target.
2. Place a camera in front of the target and we can locate the
corresponding corners on the image. This gives us the
correspondences.
3. Recover the equation that describes imaging projection and
camera’s internal parameters. At the same time, also recover the
relative orientation between the camera and the target (pose).
5
Calibration Procedure
1. Corner detector
2. Canny Edge detector plus fitting lines to the detected edges. Find the intersections of the lines.
3. Manual input.
Matching 3D and 2D points (we know the number of corners) by counting.
This gives corresponding pairs
(world point) < --- > (image point )
(X1, Y1, Z1) (x1, y1),
6
Finding Corners
7
World Coordinates and Camera Coordinates
Rotation matrix
8
Camera Frame to World Frame
9
Linear Algebra
Definition: a matrix R is a rotation matrix if and only
if it is a orthogonal matrix with determinant +1
Orthogonal Matrix: a square matrix with real entries
whose columns and rows are orthogonal vectors
with length 1.
That means: RRT = I
Or: R-1 = RT
Definition: 3D-Rotation
10
Sequence not standardized! Many different conventions!
Let = - Rt T
(C is the camera center)
11
Homogeneous Coordinates
Use Homogeneous coordinates, the perspective projection becomes linear.
12
Perspective Projection
13
Pixel Coordinates
14
Calibration Matrix
15
Putting Everything Together
1. Estimate matrix P using scene points and their
images
2. Estimate the intrinsic and extrinsic parameters
Left 3x3 sub-matrix is the product of an upper
triangular matrix and an orthogonal matrix.
16
Calibration
17
Computing the Matrix P
18
Computing the Matrix P
Rank 2
19
Computing the Matrix P
20
Computing the Matrix P
-
-
-
21
Computing the translation component
Finding the camera orientation and internal parameters
KR (use RQ decomposition ~QR)
Q R =( )-1= -1 -1 Q R
(if only QR, invert)
A QR decomposition (also called a QR factorization) of a matrix is a
decomposition of the matrix into an orthogonal and a right triangular
matrix.
22
Camera matrix decomposition
23
Project known 3D points on the image
Compare their projections with the
corresponding pixel coordinates of the
points.
Repeat for many points and estimate “re-
projection” error!
How should we estimate the accuracy
of a calibration algorithm?
Further Improvement
Most popular non-linear minimization algorithm is the Levenberg-Marquart
minimization
LM is more robust to local minima than e.g. the Gauss–Newton algorithm
and the method of gradient descent)
24
Improvement: computing the matrix P
25
Barrel and Pincushion
Tangential
Advanced Calibration: Nonlinear
Distortions
26
tele wideangle
Barrel and Pincushion Distortion
27
To model lens distortion Use above projection operation instead of standard projection matrix
multiplication
Modeling distortion
Apply radial distortion
Apply focal length translate image center
Project to “normalized”
image coordinates
Radial and Tangential Distortion
image plane
ideal actual
ideal actual
28
Set k1k20, solve for undistorted case
Find optimal k1,k2 via nonlinear least squares
Iterate
Tends to generate good calibrations
Distorted Camera Calibration
29
Image Rectification
30
Examplary calibration set-ups
31
Advantage:
Very simple to formulate and solve
Disadvantages:
Doesn’t tell you the camera parameters
Doesn’t model radial distortion
Hard to impose constraints (e.g., known focal length)
Doesn’t minimize the right error function
For these reasons, nonlinear methods are preferred
• Define error function E between projected 3D points and image positions
– E is nonlinear function of intrinsics, extrinsics, radial distortion
• Minimize E using nonlinear optimization techniques
– e.g., variants of Newton’s method (e.g., Levenberg Marquart)
Direct linear calibration
Alternative: multi-plane calibration
32
Images courtesy Jean-Yves Bouguet, Intel Corp.
Advantage • Only requires a plane
• Don’t have to know positions/orientations
• Good code available online!
– Intel’s OpenCV library: http://www.intel.com/research/mrl/research/opencv/
– Matlab version by Jean-Yves Bouget:
http://www.vision.caltech.edu/bouguetj/calib_doc/index.html
– Zhengyou Zhang’s web site: http://research.microsoft.com/~zhang/Calib/
33
Reduce the number of interactions
Pre-compute as much parameters as
possible: Usually all intrinsic camera parameters
More stable
Less user-input required
Calibration or Pose estimation using a
CAD model
34
Calibration or Pose estimation using a CAD
model
35
THANK YOU!