MIN-FakultätFachbereich Informatik

Surface ReconstructionIterative Closest Point, Delaunay-Triangulation

Robin Schlünsen

Universität HamburgFakultät für Mathematik, Informatik und NaturwissenschaftenFachbereich InformatikTechnische Aspekte Multimodaler Systeme

20. November 2017

R. Schlünsen – Surface Reconstruction 1 / 54

Table of contentsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

1. BasicsApplicationsProblemsMethods

2. Iterative Closest Point (ICP)The AlgorithmICP with Partly Overlapping SurfacesCombining ICP with inertia

3. Delaunay-TriangulationBasic TriangulationDefinitionsFlip AlgorithmExampleAnalysis

4. Sources

R. Schlünsen – Surface Reconstruction 2 / 54

ApplicationsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Computer models of real world objectsI MedicineI Natural science e.g. geologyI RoboticsI Knowledge about environment needed

I Collision AvoidanceI Path PlanningI Grasp PlanningI ...

R. Schlünsen – Surface Reconstruction 3 / 54

ProblemsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Sensor data provided by RGB-D cameras or laserscannersI Unfiltered and unorganized dataI Inaccurate, redundant samples and outliersI Only a point set, no connectivity informationI Combine samples to get a modelI Mostly triangle mesh modelsI Good for further computational processing

R. Schlünsen – Surface Reconstruction 4 / 54

ProblemsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Big data setsI High frequency of new dataI In most cases online calculationsI Needs to be very fastI Should work memory efficient

R. Schlünsen – Surface Reconstruction 5 / 54

MethodsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Generate triangle mesh successivelyI Incrementally insert sensor dataI Overlapping scans lead to redundancyI Point normals are approximated with

neighborhood informationsI Vertices are filtered to detect outliers and

inaccuraciesI Selected vertices are combined to a triangle

mesh

Source:(7)

R. Schlünsen – Surface Reconstruction 6 / 54

Table of contentsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

1. BasicsApplicationsProblemsMethods

2. Iterative Closest Point (ICP)The AlgorithmICP with Partly Overlapping SurfacesCombining ICP with inertia

3. Delaunay-TriangulationBasic TriangulationDefinitionsFlip AlgorithmExampleAnalysis

4. Sources

R. Schlünsen – Surface Reconstruction 7 / 54

Iterative Closest Point (ICP)Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Combine different scansI Get relation between these scans

I more detailedI more accurate

I Scans provide point-cloudsI Find matching pointsI Calculates transformations between point-clouds

Source: (8)R. Schlünsen – Surface Reconstruction 8 / 54

The AlgorithmBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Given are two meshes / point-clouds M1,M2I Represents the same surface / objectI Rigid transformation between M1 and M2

The Algorithm (1):Repeat (until convergence):1. Calculate the closest point p2i ∈ M2 for each point

p1i ∈ M1

2. Calculate the rotation R and the translation t thatminimizes the summed up distances between thetransformed p1i and p2i .

minR,t

∑i‖(Rp1i + t)− p2i‖2

3. Apply R and t to M1.

R. Schlünsen – Surface Reconstruction 9 / 54

The AlgorithmBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Greedy algorithmI Does not always end in the global optimum

Finding the closest point:I Calculating the distances to each point result in O(n2)I Should use kD-tree or other efficient data structures

I Matching of the closest points not uniqueI Multiple points of M1 can have the same closest point in M2I Different result if M1 and M2 are exchanged (not symmetric)

Calculating the transformation:I Use a nonlinear optimization algorithmI There are more direct and faster ways for matching

point sets

R. Schlünsen – Surface Reconstruction 10 / 54

ICP with Partly Overlapping SurfacesBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Applicable if (mostly) all points have a corresponding pointI Combination of partially overlapping scansI Larger and more detailed model of the scanned object

Source: (8)

R. Schlünsen – Surface Reconstruction 11 / 54

ICP with Partly Overlapping SurfacesBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Ignore all points, which do not have a corresponding pointI Approach: Ignore points for which the corresponding point in

the other point-cloud is an edge pointI Only works if the two point-clouds are located on top of each

otherI Algorithm has to compute, which points are edge points

R. Schlünsen – Surface Reconstruction 12 / 54

Combining ICP with inertiaBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Often problems with planar surfacesI Hard to find the correspondence between two scansI Especially with fast movementsI Can not recover if they lost correspondenceI Approach to use IMU to measure movementsI Combination with standard ICP

https://youtu.be/nL77VGllJsI?t=41s

R. Schlünsen – Surface Reconstruction 13 / 54

Table of contentsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

1. BasicsApplicationsProblemsMethods

2. Iterative Closest Point (ICP)The AlgorithmICP with Partly Overlapping SurfacesCombining ICP with inertia

3. Delaunay-TriangulationBasic TriangulationDefinitionsFlip AlgorithmExampleAnalysis

4. Sources

R. Schlünsen – Surface Reconstruction 14 / 54

Basic TriangulationBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Generating triangle meshes from a point setI Given a set of pointsI Connect them with edges to form trianglesI A triangulation of a point set is a set of triangles with the

following properties:I All points are vertices in at least one triangleI The interiors of any two triangles do not intersectI All points only intersects the triangles at the triangles verticesI The union of all vertices of the triangles is the original point set

I Delaunay Triangulation has nice properties

R. Schlünsen – Surface Reconstruction 15 / 54

Basic TriangulationBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 16 / 54

DefinitionsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Empty Circle: A circle is empty if and only if its interiorcontains no points.

R. Schlünsen – Surface Reconstruction 17 / 54

DefinitionsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Circumcircle of an Edge: A circumcircle of an edge is a circlegoing through its both points .

I Delaunay Edge: An edge is Delaunay if and only it has anempty circumcircle.

R. Schlünsen – Surface Reconstruction 18 / 54

DefinitionsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Delaunay Triangulation: A triangulation is a Delaunaytriangulation if it has only Delaunay edges and all possibleDelaunay edges are in the triangulation

R. Schlünsen – Surface Reconstruction 19 / 54

DefinitionsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Local Delaunay Edge In a triangulation an edge is locallyDelaunay if and only if it is either a boundary edge or if it isDelaunay with respect to the vertices of the two triangles thatcontain the edge.

Delaunay LemmaIf all edges of a triangulation are local delaunay then all edges arealso delaunay.

R. Schlünsen – Surface Reconstruction 20 / 54

DefinitionsBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Local Delaunay Edge In a triangulation an edge is locallyDelaunay if and only if it is either a boundary edge or if it isDelaunay with respect to the vertices of the two triangles thatcontain the edge.

Delaunay LemmaIf all edges of a triangulation are local delaunay then all edges arealso delaunay.

R. Schlünsen – Surface Reconstruction 21 / 54

Flip AlgorithmBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Greedy algorithmI Starts with a given triangulationI Searches for edges, which are not local delaunayI Flips those edgesI Terminates if all edges are local delaunay

R. Schlünsen – Surface Reconstruction 22 / 54

Flip AlgorithmBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Edge Flip: Erasing old egde and inserting new edge in theother diagonal of the quadrilateral.

I Edge is flippable if the quadrilateral is convex.

(1) p.247 fig 14.7

(1) p.247 fig. 14.8

R. Schlünsen – Surface Reconstruction 23 / 54

Flip AlgorithmBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Every edge is delaunay orI Flippable and after the flip delaunayI It can be shown that after O(n2) flips the algorithm terminates

and all edges are in delaunay.

(1) p.247 fig 14.7

(1) p.247 fig. 14.8

R. Schlünsen – Surface Reconstruction 24 / 54

Flip AlgorithmBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

Problems:I Three points on a lineI Four points on a circleI Can lead to either flat/degenerated trianglesI or crossing edges/ambigous choice of edges

(1) p.248 fig 14.9

R. Schlünsen – Surface Reconstruction 25 / 54

Flip AlgorithmBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

Properties:I Basic triangulation propertiesI Maximizes the minimal angle

I No “skinny“ trianglesI Good for computations and numerical stability

I Runs in O(n2) with n = # of verticesI Well distributed on smooth surface O(n log n)

R. Schlünsen – Surface Reconstruction 26 / 54

Flip AlgorithmBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

Flip Algorithm:I Extended for a case, where we only have points (p1, ... , pn) and

not a given triangulation.I We start with adding three points pn+1, pn+2, pn+3, which

build a triangle that includes all other pointsI This triangle is in delaunayI Adding points succesive with three steps:

1. Finding triangle tj where point pi is located in2. Adding edges from corners from tj to pi3. Flip edges if necessary to get a triangulation, which is in delaunay

I After all points are inside the triangulation, remove pn+1, pn+2,pn+3

R. Schlünsen – Surface Reconstruction 27 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 28 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 29 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 30 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 31 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 32 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 33 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 34 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 35 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 36 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 37 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 38 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 39 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 40 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 41 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 42 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 43 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 44 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 45 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 46 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 47 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 48 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 49 / 54

ExampleBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

R. Schlünsen – Surface Reconstruction 50 / 54

AnalysisBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Points on a line has to be treated differentlyI Find the triangle tj that includes the current point efficiently

I Search TreesI After adding a new point and its edges, only the edges of tj

and the new edges are potentially non local delaunayI Only six edges has to be tested for each point

I There is also a fast divide and conquer algorithm

R. Schlünsen – Surface Reconstruction 51 / 54

ConclusionBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I Knowledge about the environment is importantI Reconstruction of surfaces from sensor dataI Iterative Closest Point Algorithm can combine different scansI Good results for mobile robots if combined with IMUI Delaunay triangulation constructs a triangulated meshI Maximizes the minimal angleI Good for further computational processing

R. Schlünsen – Surface Reconstruction 52 / 54

SourcesBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I (1) Jakob Andreas Bærentzen, Jens Gravesen, François Anton,Henrik Aanæs: Guide to Computational Geometry Processing -Foundations, Algorithms, and Methods

I (2) K. S. Arun, T. S. Huang, S. D. Blostein: Least-SquaresFitting of Two 3-D Point Sets

I (3) Greg Turk and Marc Levoy: Zippered Polygon Meshes fromRange Images

I (4) Frederic Cazals, Joachim Giesen: Delaunay TriangulationBased Surface Reconstruction: Ideas and Algorithms

R. Schlünsen – Surface Reconstruction 53 / 54

SourcesBasics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

I (5) M. Nießner and A. Dai and M. Fisher: Combining InertialNavigation and ICP for Real-time 3D Surface Reconstruction

I (6) Dobrina Boltcheva, Bruno Lévy: Simple and ScalableSurface Reconstruction

I (7) http://www.dlr.de/rmc/rm/en/desktopdefault.aspx/tabid-3812/

I (8) http://manufacturingscience.asmedigitalcollection.asme.org/data/journals/jmsefk/934479/manu_138_01_011014_f006.png

I (9) https://www.youtube.com/watch?v=nL77VGllJsI&feature=youtu.be

I (10) http://www.laserscannerblog.de/wp-content/uploads/2011/07/faro-laser-scanner-focus3d-infografik-entfernungsmessung.jpg

R. Schlünsen – Surface Reconstruction 54 / 54