Interactive Key Frame Motion Editorfor Humanoid Robots

Bachelor Thesis

Simon Philipp Hohberg

02/27/2012

Freie Universität BerlinFachbereich Mathematik und Informatik

AdviserDipl.-Inf. Daniel Seifert

SuperviserProf. Dr. Raúl Rojas

Eidesstattliche Erklärung

Ich versichere, die Bachelorarbeit selbstständig und lediglich unter Benutzung derangegebenen Quellen und Hilfsmittel verfasst zu haben.Ich erkläre weiterhin, dass die vorliegende Arbeit noch nicht im Rahmen eines an-deren Prüfungsverfahrens eingereicht wurde.

Berlin, 28. Februar 2012

Abstract

Static motions enable humanoid robots to solve static problems where a dynamicsolution would not provide noteworthy benefits. The most popular technique tocreate static motions for humanoid robots is keyframing. Since a key frame motiondefines many joint angles, tools for the design of key frame motions should supportthe motion designer to deal with them.In this thesis, an already existing simple motion editor is analyzed regarding itsdrawbacks and inconveniences to formulate requirements that increase the usabilityof a newly implemented editor. With respect to these requirements it is describedhow a motion editor using a 3D robot model and the concept of time-joint anglediagrams to visualize trajectories is implemented. Also, an interpolation using cubicBézier curves is realized to create smooth motions in contrast to linear interpolatedmotions. The purpose of the 3D robot model that is included in the motion editoris also to enable users to create motions without a real robot.Furthermore, the implemented motion editor is configured by a robot descriptionfile that allows the editor to work with different types of robot.

Contents

1 Introduction 11.1 FUmanoids Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 FUmanoid Platform and FUremote . . . . . . . . . . . . . . . . . . . 21.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Structure of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Theory 92.1 Motions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 Keyframing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2 Software Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.1 Observer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.2.2 Model View Controller/Adapter . . . . . . . . . . . . . . . . . 102.2.3 Blackboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3 Bézier Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3 Requirements Analysis 153.1 Analysis of the former MotionEditor . . . . . . . . . . . . . . . . . . 153.2 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4 Implementation 194.1 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.1.1 Google Protocol Buffers . . . . . . . . . . . . . . . . . . . . . 194.1.2 Eclipse Rich Client Platform and SWT . . . . . . . . . . . . . 194.1.3 Java3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.2 Robot Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.2.1 Robot Description . . . . . . . . . . . . . . . . . . . . . . . . 204.2.2 Robot Model with Java3D . . . . . . . . . . . . . . . . . . . . 23

4.3 Motions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.3.1 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.3.2 Motion Description . . . . . . . . . . . . . . . . . . . . . . . . 29

4.4 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.5 FUmanoid Motion Player . . . . . . . . . . . . . . . . . . . . . . . . 314.6 The new MotionEditor . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5 Conclusion and Future Work 35

i

Bibliography 37

List of Figures 39

ii

1 Introduction

The avail of a robot is mostly given by its ability to interact. Since motions are anessential part of interaction, they are very important for robots in general.One main problem of humanoid robotics is the complex motion model. Not onlywalking but also any other motion needs to be defined. To simplify this motionmodel, key frame motions can be used to define certain motions statically. Creatinga key frame motion is very complex in most cases because many joints have to beconsidered and defined over time. Therefore, it is mandatory for a key frame motioneditor to be intuitive, descriptive, interactive and easy to use.Based on experiences with a very simple motion editor that was frequently usedfor many years a new motion editor was implemented with the goal to fulfill thesequalities.This thesis analyzes the requirements for a motion editor and shows which designdecisions were made to achieve those requirements as well as the implementationdetails for most important aspects of an interactive key frame motion editor.

1.1 FUmanoids Team

FUmanoids is the humanoid robot soccer team of the Freie Universität (FU) Berlin.The FUmanoids were founded in 2006 and took part in the RoboCup competitionfor the first time in 2007. RoboCup is short for Robot World Cup Initiative and isthe most important competition in several fields of robot research. There is not onlya cup for soccer playing robots, but also for rescue robots (RoboCup Rescue) anddomestic robots (RoboCup @Home).RoboCup Soccer is divided into leagues for different kinds of robots. These includehumanoid robots, small size non-humanoid robots, mid size non-humanoid robots,standard platform robots as well as a simulation league. The humanoid league, thenagain, is subdivided into different sizes (kid size, teen size and adult size), each withdifferent size restrictions for the overall robot size, arm length, leg length and so on.The FUmanoid robots are human-like autonomous robots playing in the kid sizeleague.The concept of a cup for soccer playing robots was mainly formed to foster thedevelopment of robotics and artificial intelligence in an entertaining and application-oriented way. Soccer is especially reasonable for this, because it comprises the main

1

Chapter 1 Introduction

issues for humanoid robots, i.e. localization, walking and motions in general, visionand team play. To ease the development of soccer playing robots in the beginningof RoboCup, each goal was marked with a different color and colored markers wereplaced next to the soccer field. Some of these assistances were already removed sincethen, some are still used. In the future, all of them will be removed and the wholeenvironment will be adapted to a real soccer field.

1.2 FUmanoid Platform and FUremote

The FUmanoid platform is the software running on the FUmanoid robots. It in-cludes all the artificial intelligence components for vision, localization and behavior,but also walking and playing motions.

FUremote is a set of tools for the work with the FUmanoid robots. It includestools for configuration, debugging, simulation and motion generation. FUremotecan connect to robots running the FUmanoid platform and send and receive datafrom the robot to control it. FUremote is based on the Java Eclipse Rich ClientPlatform (subsection 4.1.2) and is therefore plug-in based. Figure 1.1 shows theFUremote with several views open. These views are defined by certain plug-ins ofFUremote and can be closed or moved. One of these plug-ins is the MotionEditorwhich is displayed in an additional view when opened.

Figure 1.1: FUremote application

2

1.3 Related Work

1.3 Related Work

Creating motions for robots is very similar to motion creating for animations. Thusalmost any 3D animation tool is related to an application like the MotionEditor.Blender is a free and widely used tool for 3D rendering and is therefore used asreference representing 3D animation tools in general.3D rendering tools are much more complex than an editor for robot motion designdue to the fact that they are much more general tools making it very complicatedto use them. Nevertheless, looking at the subset of functionality that is necessaryfor creating animations, concepts have been taken over by motion editors. Thekeyframing technique (subsection 2.1.1) which is used to generate motions for robotsis an animation technique.Figure 1.2 shows the 3D rendering and animation tool Blender. In the center ofthe picture there is the 3D model that will be animated. At the bottom there isthe time line for the animation. The time line defines the number of frames for theanimation. The user also specifies a frame rate, i.e frames per second, defining theactual duration of the motion. Clicking at a certain time in the time line allows theuser to define a key frame for some 3D object with its current position, rotation andscale.

Figure 1.2: Blender with 3D model of the robot 2012

3

Chapter 1 Introduction

Another concept of animation are bones. A bone defines a rigid body with a jointat one ending. By attaching bones they can be used to create a skeleton for a 3Dobject. The mesh of this object is then connected to the bones. When creatingan animation, this skeleton can be used to either rotate certain bones directly orautomatically calculate the joints rotations by translating the end of a bone chain.The mesh connected to the bones is then automatically adapted.File formats for animations either store the positions, rotations and scales of eachobject for each key frame or only the translations, rotations and scalings performed.A robot motion can possibly be recorded with a 3D animation tool like Blender.Nevertheless, it would take great effort to play a motion recorded this way, since theanimation and its key frames are defined for 3D objects and not for joints or motors.So either a plug-in for Blender is written that knows the real robot and calculatesthe joint angles during the design process, so they can be stored in a format thatis understood by the robot, or the animation format is converted to such a formatafter the design phase.Aldebaran Robotics is a company specialized in robotics. They invented the hu-manoid NAO robot for commercial purposes. The software for the NAO robotincludes the functionality for designing and capturing motions as well as for cre-ating and programming interactive behavior. This software is called Choreographe(Figure 1.3).

Figure 1.3: Aldebaran Choreograph user interface [12]

As shown in Figure 1.3 Choreographe includes a 3D model of the NAO robot whichvisualizes the current pose of it.

4

1.3 Related Work

On the left, there is a panel displaying available behavioral components and prede-fined robot poses. The behavioral components can be dragged to the panel in themiddle where they can be connected to each other to create a behavior for the robot.This has nothing to do with the creation of motions, but allows the user to connectthe behavior with motions.

When designing a motion, Choreograph initially shows only the Timeline Panel(Figure 1.4). This panel is divided in three parts: motion, behavior layer and detailsarea. For motion design only the motion area is interesting. It depicts the key framesof the motion along a time axis. The unit of this axis is deciseconds and a key frameis visualized as gray box. When the user clicks into the time line, the pose for thistime is shown in the 3D view. The user can then also set any joint angles in the3D view which inserts a new key frame at this position. This is done by clicking ona part of the robot in the 3D view. This opens a window where the user can setthe values for the joints of this part. A key frame does not have to set all but candefine only a subset of joint angles. Key frames are interpolated with Bézier curvesby default.

Figure 1.4: Choreograph Timeline Panel: 1. Motion Area, 2. Behavior Layer Area,3. Details Area [12]

The Choreographe Timeline Editor for a motion shown in Figure 1.5 can be openedup in a new window by clicking on the pen in the motion area. This editor can beused in two different modes: curve mode and worksheet mode. The curve mode isvery descriptive and therefore much more interesting. It shows the trajectory foreach joint selected on the left as a curve with a specific color along the time. They-axis of this diagram depicts the joint angle in degrees.

A Key frame is visualized as point of the joint’s trajectory. These points can bemoved along the time or joint angle axis to edit the motion but they are restrictedto multiples of 100 milliseconds (one decisecond) along the time axis.

Furthermore, there can be different interpolations used. Besides the default auto-matic Bézier interpolation, there is constant, linear and Bézier interpolation withadaptable control handles.

All in all, Aldebarans Choreographe uses some good concepts to make motion de-signing easier and more intuitive. This is mainly due to the 3D model but also the

5

Chapter 1 Introduction

Figure 1.5: Aldebaran Choreograph Timeline Editor in curve mode[12]

Timeline Editor in curve mode. Unfortunately, these two features do not share thesame window, which is inconvenient sometimes.The main drawback of the Choreographe is that it is NAO specific and cannot beused with other robots at all.Kouretes Motion Editor (KME) is the name of another editor for the design ofmotions.[5] It was originally designed for NAO robots, but can be used with differentrobots as well. A robot is defined by a XML file. This file is then used by the KMEto create motions for this robot. This is a very good approach, since it decouplesrobot and editor and allows it to work with different robots.Figure 1.6 shows the KME. The interface is very simple. It shows a slider for eachjoint, status information on the left and the actual motion at the bottom. A motionconsists of a list of key frames. Here, a key frame is again a list of joint values anda duration value that defines the time between the current and next key frame. Akey frame always defines all joint angles. When a key frame is selected from themotion, the joint values for it can be adapted with the joint sliders. The type ofinterpolation between the key frames cannot be changed.

1.4 Structure of Thesis

First the theoretical basics of motions and implementation related topics are dis-cussed. Afterwards, the existing MotionEditor is analyzed as well as problems anddrawbacks are identified, leading to the requirements for the new motion editor. Inchapter 4 the actual realization of the MotionEditor and the most important aspectsfor the implementation including design decisions and used technologies is described.Finally, the new MotionEditor is evaluated with regard to whether it fulfills the re-quirements. Also, achieved enhancements are pointed out. At last, after drawing a

6

1.4 Structure of Thesis

Figure 1.6: Kouretes Motion Editor [5]

conclusion, an outlook for possible future features and improvements is given.The interface of the KME is very simple but not descriptive. It is not possible toget a direct feedback when editing a motion. Nevertheless, the concept of a robotdescription loaded from a file is very useful.The concepts of this editor are very similar to the former FUmanoid MotionEditor(section 3.1).

7

Chapter 1 Introduction

8

2 Theory

2.1 Motions

Motions of humanoid robots can be divided into two types of motions: dynamicand static motions. For human beings it is normal to perform motions dynamically.That means, each time such a dynamic move is performed its execution may slightlyvary, because the move is always performed with respect to feedback like remainingbalanced when walking over rough terrain or not crushing a cup when grabbingit. Because developing such a dynamic motion behavior for robotic motions is verycomplex, some motions are defined in a static way where feedback can be ignored.That means that these motions are always performed in the same way and they arenot adapted to any influence from outside.

Implying that a motion is made to solve any kind of problem, dynamic motions canbe considered a generalized solution to this problem, whereas a static motion solvesonly one fixed problem. Nevertheless, a motion in general is always a mathematicalfunction mapping time on joint angles. For static motions this mapping is predefinedand immutable, whereas the mapping of dynamic motions can vary each time themotion is performed.

Static motions are also known as offline motions and dynamic motions as onlinemotions.[17]

The most common technique for creating static motions is keyframing.

2.1.1 Keyframing

The keyframing technique allows the creator of a static motion to specify a setof joint positions called key frames. Interpolating these key frames enables thedefining of the joint positions for the entire duration of the motion. Various kinds ofinterpolations are possible and eventually produce different motions as a result. Thesimplest form of interpolation is linear interpolation, although cubic Bézier curvesor B-Splines can be used to interpolate between two adjacent points resulting insmoother trajectories. However, a combination of these interpolation types can alsobe useful.

9

Chapter 2 Theory

2.2 Software Patterns

For interactive systems it is most important to separate the graphical user inter-face from the functional logic. That allows that the user interface can be changedand that the functionality can be expanded without recreating the whole applica-tion. That is why the development of front-end applications usually follows specificsoftware patterns to achieve a flexible and extensible implementation.

2.2.1 Observer

The observer pattern is a very basic behavioral software pattern for interactivesoftware development. It defines two types of software components: observer andobserved. The observed component automatically notifies all its observers about anychange of state, which enables a loosely coupled one-to-many dependency betweenthe observing and the observed software components.[2] Figure 2.1 shows a classdiagram for the use of the observer pattern in the Model View Controller pattern.Here, View and Controller assume the role of observers of the Model attached to it.When the Model is altered, it will notify all attached observers by calling the updatemethod from the Observer interface.

2.2.2 Model View Controller/Adapter

The most important aspect for fronted applications is the separation of data in-cluding core functionality – generally called model – and representation of this datafor the user – called view. The Model View Controller (MVC) and the Model ViewAdapter (MVA) (alsoModel View Mediator) architectural software patterns are mostcommon for implementing this separation. Both patterns are very similar. Yet, forreasons of completeness, they shall be discussed here.As the view is a representation of the model it needs to be synchronized with themodel by updating it every time the model changes. This is usually called Observerpattern. When the model changes, it informs all its observers about this changes.This guarantees that the view remains synchronized.Besides model and view a third type of software component is necessary which linksmodel and view so that the user input is processed and a certain functionality isprovided. In the MVC pattern, this logic is defined by the controller. The controllerlistens for user input on the view component, processes this data regarding its logicand updates the model, which will then update the view to display the changes.In the MVA pattern this third component is called adapter or mediator. UnlikeMVC, model and view are not linked directly but all communication passes theadapter. As in MVC the adapter is notified about user input by the view, processingthis input and finally updating the model. However the model now notifies the

10

2.2 Software Patterns

Figure 2.1: MVC class diagram[4]

adapter about changes instead of triggering the view directly. The adapter thennotifies the view.[1]

Figure 2.2: Schematic MVA pattern[1]

2.2.3 Blackboard

The blackboard pattern is the generalization of the observer pattern and is thereforea behavioral pattern. It is used to collect data at one central point, the blackboard.Different software subsystems provide their results to the blackboard. That way,the combination of this data can be used to solve a more complex problem.Besides the blackboard itself, two further kinds of components make up the black-board pattern: the producers which are the source of data and write this data to the

11

Chapter 2 Theory

blackboard and the consumers that read and work on the data from the blackboard.This pattern has its origin in artificial applications and is normally used to solveproblems that do not have a deterministic solution.[4] Specialized software compo-nents working on some part of data provide partial solutions to allow the calcula-tion of an approximate solution of a more complex problem. Each component isspecialized for solving a certain problem and works on this independent of the othercomponents.Although, the blackboard pattern has its origin in artificial non-deterministic ap-plications, it fits front-end applications as well, as they have many properties incommon. In front-end application there are also many different independent andspecialized components that have to work together to provide complex functional-ities. The implementation of features requires several data sources and is mostlyindependent of other features.

2.3 Bézier Curves

Bézier curves are named after Pierre Bézier who simultaneously with Paul de Castel-jau developed these curves.[6]To understand Bézier curves, it is reasonable to start with linear interpolation.Equation 2.1 defines a linear interpolation of the points a and b for 0 ≤ t ≤ 1 bymapping t to x as shown in Figure 2.3, whereas x (0) = a and x (1) = b.

x = x (t) = (1− t) · a+ t · b (2.1)

Barycentric coordinates for three points a, b, x are defined by Equation 2.2.[3] α andβ are then barycentric coordinates with respect to a and b. For linear interpolationx has the barycentric coordinates α = 1 − t and β = t. The point x can thereforebe called barycentric combination of the two points a and b.

x = αa+ βb; α + β = 1 (2.2)

Equation 2.1 is called an affine mapping of the three points 0, t, 1 in one dimensionalspace to a, x, b. [3]Linear interpolations are affine invariant[3], which means that the relative positionof t on the straight line between 0 and 1 is the same for x on the line between a andb after mapping. For example, that if t = 1

2 , then the resulting point x is the middlepoint between a and b on the straight line through these two points.

12

2.3 Bézier Curves

Figure 2.3: Linear interpolation[3]

For a set of points this linear interpolation can be used repeatedly to produce aBézier curve. For example, given three points b0, b1 and b2, the linear interpolation ofthese points creates two straight lines in the first place (Equation 2.3, Equation 2.4).Repeating linear interpolation of these lines leads to Equation 2.5 which then resultsin Equation 2.6 by inserting Equation 2.3 and Equation 2.4 into Equation 2.5. [3]

b10 (t) = (1− t) b0 + tb1 (2.3)b1

1 (t) = (1− t) b1 + tb2 (2.4)b2

0 (t) = (1− t) b10 (t) + tb1

1 (t) (2.5)b2

0 (t) = (1− t)2 b0 + 2t (1− t) b1 + t2b2 (2.6)

Equation 2.6 produces a parabolic curve shown in Figure 2.4. First t is mapped tothe straight line between b0 and b1 resulting in the point b1

0. Next, t is mapped to thesecond straight line between b1 and b2 producing the point b1

1. Finally, t is mappedto the line through b1

0 and b11 giving the point b2

0 on the parabola.This repeated linear interpolation is called de Casteljau algorithm. Its generalizedform is denoted in Equation 2.7 as described in [3] for a set of points b0, b1, . . . , bn.

bri = (1− t) br−1

i (t) + tbr−1i+1 (t) (2.7)

The most important characteristic of Bézier curves is that they always run throughthe first and last point. All the other points are called control points or controlhandles. They are mostly not on the curve.

13

Chapter 2 Theory

Figure 2.4: Second order Bézier curve[3]

For this thesis, cubic (third order) Bézier curves are most important. Besides start-ing and ending point, these curves define two more control handles. Figure 2.5 showssuch a cubic Bézier curve with starting point b0, control handles b1, b2 and endingpoint b3.

Figure 2.5: Cubic Bézier curve constructed using de Casteljau algorithm[3]

14

3 Requirements Analysis

This section analyzes the existing MotionEditor and summarizes the requirementsfor the implementation of a new one based on the results of this analysis.

3.1 Analysis of the former MotionEditor

Figure 3.1 shows a stand up motion in the old MotionEditor. The editor depicts atable where each column represents a motor. The motor ID is shown in the tablehead and each row resembles a key frame (subsection 2.1.1). The rightmost columnis a special column, defining the time in milliseconds that passes between the formerkey frame and the current key frame. Since a row represents a key frame, it definesthe rotation value for each motor and the time for reaching these values. Whenthe motion is played, the robot will go through the rows and strike every pose bycalculating the speed each motor needs to reach the desired position in the specifiedtime. The single key frames are linearly interpolated.

Because the MotionEditor needs to know the motor IDs for the robot, the oldMotionEditor uses a predefined set of robots. For example, the robot in Figure 3.1has no motor with ID one. That is why the motor cannot be set and always remainszero.

The old MotionEditor provides few basic features. It can read a single or multiplemotors and rows as well as it can set a single or all motor values of one row. Itis also able to play a set of adjacent rows or the whole motion. For working withthe table it provides features for inserting, adding, deleting and copying rows. Inaddition, the editor allows to enable or disable a single or multiple motors. Usuallya motor tries to hold its current position and therefore can not be moved. This isthe enabled state of a motor. When a motor is disabled it can be rotated freely andwhen it is enabled again, it holds the new position the motor was moved to.

There are many obvious drawbacks for this kind of editor. First of all, it is impossibleto see what the motion or the pose for a certain row looks like. Further, the userdoes not know how changing a motor value will affect the rotation of this motorbecause (depending on the motor) increasing or decreasing the value will rotate themotor which can result in a left, right or up, down movement. Therefore, the usualway of working with the old MotionEditor was to put the real robot in a certainposition, then read the motor values for this pose, put the robot in the next position,

15

Chapter 3 Requirements Analysis

Figure 3.1: Old MotionEditor

read the motors and so on. This always requires the availability of the real robotto create a motion and also required that the real robot was put in the same poseagain, if the further pose was not correct or should be adapted.

Another drawback is that the editor provides only linear interpolation. Neverthelessdifferent interpolation types using splines might be useful to create smoother andmore natural motions.

Due to the table structure of the motions it is not possible to set the value for onlya single or a subset of motors for a given time instead of all motors. This makes itdifficult to define different motor motions independently, for example rotating onemotor slowly from one position to another while rotating another motor back andforth fast. This is indeed problematic, because the motion would have to have anew row each time the fast rotating motor changes its direction. In all these rowsthe position for the slowly rotating motor would have also to be set although this isunwanted.

The motions are serialized by converting the table into a comma separated list ofmotor values and a line break after each row.

3.2 Requirements

Considering these drawbacks the main requirements for the new MotionEditor couldbe deduced. A visualization of the robot is important for displaying the motion andto directly show how changes to motor values affect the robot’s pose at a specifictime.

16

3.2 Requirements

However, the MotionEditor should be independent of the robot it is used with,instead of supporting only predefined robot types. Therefore, a robot descriptionformat is required that describes the motors, kinematics and appearance of the robotenabling the editor to visualize a model of the robot.Another requirement is that each motor or joint has to have its own trajectory inde-pendent of the other motors. These trajectories need to be displayed as curves andpoints over time mapping time to joint angles to provide an intuitive representation.In addition, the interpolation for each part of a trajectory has to be adaptable anddifferent types of interpolations should be available. Besides linear interpolation,there should be an interpolation producing round and smooth motions. This alsorequires a new motion format for storing and sending motions.The MotionEditor has to provide tools that are easy to use for working with thepoints of a trajectory. It has to provide functionality for adding and removing,copying, pasting and moving points. Because a humanoid robot is usually verysymmetric regarding its joints, it would be useful to have a mirror tool, that allowsto copy and mirror a trajectory of the left shoulder joint to the right shoulder jointfor example, so both joints perform the same move.Motions created with the old MotionEditor could further be sped up by shorteningthe period between single key frames. As this is important for motion designers,this would have to be possible in the new MotionEditor as well .Motors should not only be named with IDs but also with names describing the equiv-alent human joint. Further, when connected to a real robot the editor should provideinformation about the temperature and load of each motor to prevent overheatingof motors.Generally, the goal for the MotionEditor should be to allow the creation of motionswithout a real robot by visualizing the motion and the robot’s movement in 3D.Finally, all motions created with the old MotionEditor have to be loadable with thenew editor.

17

Chapter 3 Requirements Analysis

18

4 Implementation

4.1 Technologies

To provide new features for the motion editor, decisions for and against certaintechnologies were made. The most important technologies the motion editor usesare the Google Protocol Buffers, the Eclipse Rich Client Platform (RCP) includingthe Standard Widget Toolkit (SWT) and Java3D.

4.1.1 Google Protocol Buffers

Protocol buffers (protobuf ) are a format for the serialization of structured data.Because it is extensible, very efficient and can be applied in an object orientedmanner, it is used for all communication between MotionEditor and FUmanoidplatform. It also allows code generation from the protobuf description making itvery comfortable to use.A protocol buffer defines a message which can be thought of as a class withoutmethods or a struct in the C programming language. Such a message consists ofa set of typed single or repeated fields. The generated code of a message defines aclass for this message which can be instantiated. It also provides getter and settermethods for all fields and methods for serialization.

4.1.2 Eclipse Rich Client Platform and SWT

The Eclipse Rich Client Platform (RCP) is a framework for the development of clientapplications. It uses the Open Services Gateway initiative (OSGi) service platformimplementation Equinox. The OSGi service platform is a specification for hardwareindependent, modularized platforms on top of a Java virtual machine (JVM). EclipseRCP uses this plug-in based architecture to provide basic features commonly used forclient applications while being extensible at the same time. A plug-in is a softwarecomponent which can be inserted or removed from the framework easily. Whenusing the Eclipse RCP framework to build a rich client application it is mandatoryto use some basic plug-ins providing core features. Besides these plug-ins developerswould create their own plug-in or plug-ins implementing all the features for theirspecial application.

19

Chapter 4 Implementation

Eclipse RCP uses the Standard Widget Toolkit (SWT) for all user interface elements.SWT is a library for creating graphical user interfaces. It uses native, i.e. platformdependent, widgets which allows the creation of applications with the usual platformlook. For Java there are two more widely used user interface libraries. These arethe Abstract Window Toolkit (AWT) and Swing. Whereas Swing defines own userinterface components, AWT provides native widgets like SWT.Eclipse RCP allows the use of SWT only. SWT is generally not compatible withAWT or Swing but provides an adapter for AWT components which allows to includeAWT elements.

4.1.3 Java3D

Java3D is a high-level application programming interface (API) for rendering 3Dobjects. It provides the usual Java platform independence and is object oriented.[15]Multiple OpenGL bindings exist for SWT like the Lightweight Java Game Library(LWJGL) and the Java OpenGL bindings JOGL and gljava.[13] One advantage ofthese bindings is their seamless integration in SWT, whereas their drawbacks aretheir only low-level functionality and OpenGL narrowness. Java3D, in contrast,abstracts from the underlying system graphics library and, therefore, can be usedwith an OpenGL binding as well as Direct3D. On the other hand Java3D can be usedwith the Abstract Window Toolkit (AWT) only. But there is a simple solution for thisproblem as SWT provides an adapter which allows the integration of AWT widgetsinto SWT. Because Java3D can be used very easily and provides an abstraction fromthe real graphics library used, the decision was taken to use Java3D for renderingthe 3D model.Java3D is a scene-graph-based API. That means that all 3D objects rendered areleafs of a 3D scene tree. The inner nodes of this tree are called groups, so every 3Dobject is assigned to a group. Figure 4.1 shows this tree structure for a 3D scene.

4.2 Robot Model

One goal for the motion editor was to make it independent of the robot’s specificconstruction because there are often changes to the hardware or the whole robotconstruction. Therefore, a robot description format was needed that describes therobot and that is provided to the editor.

4.2.1 Robot Description

The Robot Description File (RDF) was originally defined by [7]. To meet the re-quirements of the newly implemented MotionEditor, this format was extended.

20

4.2 Robot Model

Figure 4.1: Java3D Scene Graph [15]

For the MotionEditor the IDs and names of the motors are most important becausethey are needed to map a motion for one joint to a certain motor. Furthermore, theeditor needs information about the motors regarding their working angle and thevalue region the motor is working on. Yet, to make the editor more user-friendlythe description contains also all data necessary for creating a 3D model of the robotand the kinematic information to solve the forward kinematic for a given set of jointangles enabling the editor to animate motions in 3D.

The root protobuf message for a robot description is figure, which is mainly madeup by a set of bodies, links and sensors. The bodies describe the appearance of therobot, whereas the links specify the kinematics.

The sensor description defines not only pressure sensors and so forth but also themotors. The sensor description of a motor contains the motor ID, the values themotor uses for specifying the motor position, the allowed voltages and temperatures.Yet, it does not carry any kinematic information or information about the motor’sappearance.

Bodies define everything that is visible: its dimensions and appearance. A body canbe a simple box or sphere or it can be defined by a 3D object stored in an externalfile. It also determines the position and rotation of the body and may containinformation about the material and density. Moreover, a body can be defined tobe an end-effector, which can be used for inverse kinematics in future versions ofthe motion editor. For creating more complex connected bodies a body may have aparent body building a tree structure of bodies. A RobotDescribtion was created forthe FUmanoid robot 2012. The 3D model for this robot is visualized in Figure 4.2.

Links define the kinematics as already mentioned but also combine all the differentparts of the robot description. A link determines a rotation axis and the positionof it. Also, a link can have multiple child links that are rotated by this link leading

21

Chapter 4 Implementation

to a tree structure of links. This link tree is sufficient for describing any linear linkcombination. Because parallel mechanics can not be described with this and thecurrent FUmanoid robot 2012 construction uses such a parallel mechanic, there wasa need for links to define optional parallel links and opponent links.

Figure 4.2: 3D model of the FUmanoid robot 2012

A parallel mechanic is a construction of four links arranged in a parallelogram andfour rigid bodies each connecting two links and representing the sides of the paral-lelogram (see Figure 4.3). This construction guarantees that opposite sides alwaysremain parallel.

When a link defines a parallel link, this parallel link also has to be rotated everytime the link rotates. An opponent link, however, also has to be rotated when thelink rotates but in the other direction. As Figure 4.3 shows, the links l1 and l2 areopponent links. When link l1 rotates clockwise, l2 has to rotate just as far in theopposite direction due to the mechanical construction.

A link can have an associated body representing it and a motor driving this link.The motor is of course optional because there are links that do not have a motor.

22

4.2 Robot Model

Figure 4.3: Parallel Mechanics straight (a) and rotated (b)

4.2.2 Robot Model with Java3D

In Java3D there are two important classes for translating, rotating and scaling ob-jects: TransformGroup and Transform3D. A TransformGroup is always associatedwith a Transform3D which is wrapping a 4 × 4 matrix defining position, rotationand scale of the TransformGroup’s children which could be TransformGroups again.Hence the final position and rotation of a 3D object is the product of all transfor-mations from the root node to this object[15].The RevoluteJoint class from the motion editor is the equivalent for a link of revo-lute type in the robot description, but also provides functions for manipulating the3D model. A RevoluteJoint can be associated with a motor and a body like a link inthe robot description. When using the Java3D API to rotate a 3D object around acertain axis the object is usually moved to the origin, rotated and afterwards movedback, because rotation is always done with respect to the origin and not to theobject’s position. To avoid this the RevoluteJoint class defines two different Trans-formGroups separating rotation and translation. Because the final transformationfor a node is the product of all transform matrices from the root node to this node,the separation has no effect on the absolute translation or rotation. The Transform-Group for translation is the parent node of the rotation TransformGroup. All nodesthat should be rotated by the joint are attached to the rotation TransformGroup.If the rotation TransformGroup is rotated, this is done as if the group would be atthe origin and results in an in-place rotation without any movement. The body ofthe joint is either attached to the movement TransformGroup, so it is not rotatedwhen the joint rotates, or to the rotation TransformGroup, resulting in a rotationof the body when the joint rotates.

23

Chapter 4 Implementation

In the robot description all positions and rotations are absolute. Therefore it is nec-essary to calculate the relative position and rotation of a node (body or link) to agiven absolute position and rotation of a parent node. This relative transformation(Nrel) can be calculated by multiplying the inverse of the parent absolute transfor-mation (Pabs) with the absolute transformation of the node (Nabs) (Equation 4.2)because the final transformation of a node is given by the product of all transfor-mations from the root node to the child node (Equation 4.1).

Pabs ·Nrel = Nabs (4.1)Nrel = P−1

abs ·Nabs (4.2)

4.3 Motions

Motions define trajectories for each joint of a robot. These joint trajectories canconsist of several joint angles, the key frames for this joint. The actual motion of ajoint is created by interpolating over these key frames resulting in a function definingjoint angles for the whole time of the joint’s motion. Because the current pose ofa robot when starting a motion is unknown, a motion determines an initializationtime that is used to bring all joints into the starting position of the motion.The interpolation used for a joint trajectory is very important as it has a big influenceon the resulting motion. Hence providing multiple interpolation types is essentialfor a motion editor.

4.3.1 Interpolation

A motion for a single joint always consists of several points defined by the user anda type of interpolation interpolating these points and creating the actual motion.The simplest kind is linear interpolation. For a smooth trajectory interpolationsusing Bézier curves, B-Splines or NURBS (non-uniform rational B-splines) are verycommon. Bézier curves have a simple mathematical background and can be eas-ily implemented. Therefore, Bézier curves were chosen as an alternative to linearinterpolation producing smooth trajectories.It is useful to combine different interpolation types in one trajectory to speed upcertain parts by using linear interpolation, for example. That is why the implemen-tation should allow to define the interpolation type for each part between two pointsin a trajectory. Figure 4.4 shows the implementation for this interpolation in thenew MotionEditor. ComplexInterpolation, here, is the resulting interpolation thatdescribes the trajectory of a joint. This interpolation consists of multiple Interpo-lationParts. These parts consist of an ObservablePoint on the one hand that is a

24

4.3 Motions

point defined by the user and on the other hand specifies the kind of interpolationthat is used for interpolating from the point of the part before to this part’s point.It is remarkable that ComplexInterpolation and InterpolationParts together form adouble-linked list. This means that an InterpolationPart references a predecessorand successor and ComplexInterpolation holds a reference of the first and last partof this list. When a new point is added to the ComplexInterpolation, a new Inter-polationPart with some interpolation is created and inserted into this list which issorted with regard to the point’s x-coordinate of the parts.

Figure 4.4: Simplified interpolation implementation class diagram

Until now, ComplexInterpolation would be a list of several parts each with its owninterpolation. Therefore each interpolation instance would interpolate between topoints only, resulting in stiff trajectories for the BezierInterpolation. Interpolation-Parts contain more logic that enables them to check their neighbors and merge theirinterpolation instances if they are of the same type. The interpolation then con-tains more than only two points. Figure 4.5 illustrates this behavior for the Bézierinterpolation. In (a) InterpolationParts are not aware of their neighbor parts and in(b) the same trajectory is interpolated by a single BezierInterpolation instance thatadjusts the handles for a Bézier point if it lies vertically between two points.The purpose of the Bézier interpolation is to produce more human-like, smoothmotions. To achieve this, the resulting mathematical function of a trajectory thatis generated by an interpolation has to be continuously differentiable. That meansthere is a speed for any time of the trajectory which can be applied to the motor.Cubic Bézier curves are useful to interpolate between two points. So, when usingcubic Bézier curves, an interpolation over a set of points consists of several cubicBézier curves connecting one point with the next. Besides the starting and endingpoint, a cubic Bézier curve also has to define two control handles deforming thecurve.A cubic Bézier curve approximates the control polygon defined by the starting, end-ing and control points. Therefore, a cubic Bézier curve with the points P0 (x0, y0),P1 (x1, y1), P2 (x2, y2), P3 (x3, y3) is continuously differentiable if x0 < x1 ≤ x2 < x1.If this is guaranteed, the whole interpolation can only violate the quality of con-tinuous differentiability at the connection points, so the points for the interpola-tion themselves. According to [8], given two cubic Bézier curves with the points

25

Chapter 4 Implementation

Figure 4.5: Bézier interpolation with multiple interpolation instances (a) and asingle instance (b)

P0 P1, P2, P3 and Q0Q1, Q2, Q3, with durations of ∆t1 and ∆t2, connected in thepoint P3, which is equal to the point Q0, the curves are continuously differentiablein the points P3 and Q0 if P2, P3 and Q1 are collinear and Equation 4.3 is fulfilled.

P3 − P2

∆t1= Q1 − P3

∆t2(4.3)

For a point that is an extremum like P2 in Figure 4.6 the BezierImplementationdefines the x-value of the left control handle as the point’s x-coordinate minus onethird of the distance between the last point’s x-value and this point’s x-value. Theright control handle is defined accordingly as the point’s x-coordinate plus one thirdof the distance between the point’s x-value and the next point’s x-value. Bothhandle’s y-value is set equal to the point’s y-coordinate. Therefore, the first criteriaof collinearity is fulfilled.

Given a point P that is a extremum with a left handle PL = P −(

0∆t1

3

)and a

right handle PR = P +(

0∆t2

3

)whereas ∆t1 is the difference of the previous point’s

x-coordinate and P ’s x-coordinate and ∆t2 is the difference of the next point’s x-coordinate and P ’s x-coordinate, the second criteria is also fulfilled as shown inEquation 4.3.1.

26

4.3 Motions

Differentiabiliy proof for extrema.

P − PL

∆t1= PR − P

∆t2(4.4)

P − P +(

0∆t1

3

)∆t1

=P +

(0

∆t23

)− P

∆t2(4.5)(

0∆t1

3

)·∆t−1

1 =(

0∆t2

3

)·∆t−1

2 (4.6)

Figure 4.6 shows a special case for P3. Since it is vertically between its predecessorP2 and its successor P4, the handles for this point are calculated differently to avoida step-like interpolation as illustrated in Figure 4.5. This type of point will be calledsaddle point consecutively. The first idea of how to calculate the handles for sucha point was to use the average slope of the linear interpolation of the predecessorand the saddle point and of the successor and the saddle point for the straight linethrough the control handles and the saddle point. This can result in a curve thatgoes vertically beyond or beneath the predecessor or successor when the saddle pointis vertically very close to one of them. That is because the average slope is alwaysthe same for a fixed predecessor and successor. Figure 4.7 shows such a case. Thethird point is a saddle point and vertically very close to the second point. Using theaverage slope results in a Bézier curve that goes vertically beyond the second point.This behavior maybe unwanted, because the joint will go beyond the maximumangle originally defined by the designer of the trajectory. Hence, this has to beavoided.

Figure 4.6: Interpolation using cubic Bézier curves

To solve this problem, the ratio between the slopes of the linear interpolationsdefined by the previous and the current point and the next and the current point

27

Chapter 4 Implementation

have to be considered when calculating the handles for a saddle point. The nearerthis saddle point is vertically to one of its neighbors, the more important the smallslope has to be. Consequently, the slope for the straight line through both handlesand the saddle point was defined as shown in Equation 4.7. This definition adjuststhe slope so that it is reaching zero when one of the slopes reaches zero and becomesthe average slope if both slopes are equal.

m = mprev +mnext

2 ·

mprev

mnextformprev < mnext

mnext

mprevformprev > mnext

(4.7)

Because a control handle of a saddle point should have the same influence on thecurve as a control handle of a point that is an extremum, the distance was alsodefined as ∆t

3 as shown in Figure 4.6 for P3 and PR3 . Since the distance of two points

on a straight line with slope m is defined by d =√

∆x2 + ∆y2 with ∆y = m · ∆xa point P on the straight line S with slope m and distance d to S is given by

P = S +(

xx ·m

)with x =

√d2

1+m. There are two solutions for P , but only the

solution with positive x-value is interesting here. Consequently, the control handles

SL and SR of a saddle point S are defined as SL = S −( ∆t1

3√

1+m∆t1·m

3√

1+m

)and SR =

S+( ∆t2

3√

1+m∆t2·m

3√

1+m

). Equation 4.3.1 shows that the criteria of continuous differentiability

for this definition is fulfilled.

Differentiability proof for saddle points.

S − SL

∆t1= SR − S

∆t2(4.8)

S − S +( 1

3√

1+m·∆t1

m3√

1+m·∆t1

)∆t1

=S +

( 13√

1+m·∆t2

m3√

1+m·∆t2

)− S

∆t2(4.9)( 1

3√

1+m·∆t1

m3√

1+m·∆t1

)·∆t−1

1 =( 1

3√

1+m·∆t2

m3√

1+m·∆t2

)·∆t−1

2 (4.10)

A Bézier curve is defined on the interval 0 ≤ t ≤ 1. When using them to interpolatekey frames, it is mandatory to get the y-value for a given x-value, i.e. the joint anglefor a given time. This is not trivial, since a Bézier curve is an affine mapping ofthe interval 0 ≤ t ≤ 1 to the actual curve (section 2.3). Therefore, a x-value cannotjust be linearly mapped to t to get the corresponding y-value. Rather, a numerical

28

4.3 Motions

Figure 4.7: Using average slope for a point vertically between its neighbors

solution to this problem would use the inverse function of the Bézier curve. Becausethe used curves are of third order, solving this is very difficult.

The BezierInterpolation, therefore, searches for the corresponding value between 0and 1 that maps to a given x-value. First, it tries to map the input x-value linearlyto t and calculates the resulting x-value that is given by this t. If this is not thesame as the input value, t is adapted adding or subtracting a certain step-length.This step-length is initially 0.1. If the calculated x-value is greater than the desiredvalue and then smaller again, the step-length is halved. This never produced morethan ten iterations for the search of t. The resulting t is then used to calculate they-value.

4.3.2 Motion Description

Motions designed with the MotionEditor have to be serialized and deserialized forsaving, loading and playing the motion on the robot. The old motion format wasself-made defining all joint positions for each time step of the motion implying lin-ear interpolation. Since different interpolations need to be considered now, the newmotion format has to take this into account. It would have been possible to usethe old format by splitting a motion created with the new MotionEditor into manykey frames with a short period and also imply linear interpolation, so Bézier in-terpolations would have been linearly approximated. Because the period of thesekey frames would have to be very short for Bézier curve approximation, this wouldproduce a large amount of joint positions. Therefore, a new motion format basedon Google Protocol Buffers (subsection 4.1.1) was created that is more abstract andalso following the already discussed interpolation implementation (subsection 4.3.1).Accordingly, a motion consists of a set of joint motions or joint trajectories. Thismotion of a joint is a list of motion parts, each part defining a point and an interpo-lation type. This interpolation is used to interpolate the part’s point and the pointof the predecessor part. The interpolation of the first part of a joint motion canbe ignored since it has no predecessor. This motion format implies that the entityusing such a motion is capable of the implementations for the interpolations used.

29

Chapter 4 Implementation

4.4 Architecture

Front-end applications have to be highly extensible, flexible and loosely coupledto allow the integration of new features and adaption of functionality due to userfeedback. Therefore, the architecture is most important and design patterns likeMVC and MVA where developed to guarantee these qualities.

The new MotionEditor uses a combination of the MVA and blackboard pattern (seeFigure 4.8). While the separation of view and model can be achieved quite easily, theadapter component has to be well structured, because it is implementing the actualfunctionality and interacts with both view and model. Therefore, a blackboardarchitecture is used for the adapter component to separate data producers andconsumers.

The blackboard is used to register producers and consumers for certain Registered-Data. When a producer informs the blackboard about new data, the blackboardforwards this notification to all consumers registered for this data. The producerthat initiated this notification is also provided to the consumers. The consumer canthen access additional data or use functionality that is implemented by the producer.

Producers are mostly listeners applied to a view component listening for user actionsand providing these events through the blackboard to any consumer that uses thisinformation implementing some functionality. All data that is produced is also stat-ically accessible via the DataRegistry and, therefore, can be produced anonymously,i.e. some component that is not a registered producer. It is very important forconsumers to provide data for other consumers, because consumers usually listenfor only one kind of data but also need further data to provide their functionality.For example, the PointDragger is registered on mouse position data provided bythe MousePositionProducer. When it receives a new mouse position, it has to checkwhether the mouse is pressed at the moment and if any points are selected. Whenall these conditions are met, it drags the selected points.

Due to the blackboard architecture and statically accessible data, all componentsare very loosely coupled and new features can be easily integrated.

A motion editor has to supply many different functionalities. Since there is only alimited set of user input possibilities, features are usually summed up in tools, eachspecialized in providing some kind of functionality. This means, depending on thecurrently activated tool, the user input results in a different behavior. This conceptwas also realized in the new MotionEditor. Consumers are not only registered forspecific data but also for a tool. All producers are providing their data as alreadydescribed independent of the tool but the blackboard checks which tool is currentlyactivated and notifies only those consumers that were registered for this tool.

30

4.5 FUmanoid Motion Player

Figure 4.8: Simplified MotionEditor architecture class diagram for selected classes

4.5 FUmanoid Motion Player

The FUmanoid motion player is the implementation on the robot side for playingmotions sent to the robot or stored on the robot. The existing implementation usedby the oldMotionEditor simply set a speed and a goal position for the defined periodfor each motor as defined by the old motion format. According to this old format,all motors were set simultaneously. Because now a motion of a motor can start anytime, this implementation had to be adapted. Furthermore, the implementation hadto be extended to play motions using Bézier interpolation, too. Since the motionformat only defines the interpolation type and not how to play it, it is up to therobot side implementation to deal with the different interpolations. Whereas theimplementation for playing motions using linear implementation is straight forward,the implementation for Bézier interpolated motions is a little bit more difficult. Theuse of Bézier interpolation instead of linear interpolation was to generate smoothermotions by creating a continuous differentiable trajectory for a joint. To realizesmooth motions, the joint motors of a robot have to be controlled so they performthe motion as defined. Unfortunately, this cannot be completely accomplished sincethe motors used as joints currently utilized in the FUmanoid robots can only beinstructed to reach a desired motor position or angle using a given speed. Conse-quently, the motor can only linearly interpolate trajectories and it is necessary toapproximate motions using Bézier interpolation by subdividing the curve into smallsteps. The smoothness of a motion therefore depends on the step-length used tosubdivide the trajectory. In the implementation for the FUmanoid robots, a valueof 10 ms was used for the step-length and delivered quite smooth results.

A static definition of the step-length may not be the best solution since the cubic

31

Chapter 4 Implementation

Bézier curves are usually almost linear in the middle and have many slope changesthe nearer to the starting or ending. Therefore, an improved solution would use adynamic step-length with regard to the slope changes.The described implementation runs on the main processing unit of the current FU-manoid robot. Besides this processing unit, the robot also comes with a motorboard. Its main task is to communicate with the motors, i.e. setting and readingmotor values or status information. It has lesser computational power than the mainprocessing unit. Anyhow, in future versions the motion player can be run on thisunit instead of the main processing unit so motions and behavior are separated. Twotypes of implementation are possible for this. Either the motor board calculates themotor positions based on the motion format that was discussed (subsection 4.3.2),or this motion format is evaluated by the main processing unit and converted into asequence of joint values that is then executed by the motor board. The first solutionwould result in a higher load of the motor board, whereas the latter would requirevery little computational time.

4.6 The new MotionEditor

Figure 4.9 shows the new MotionEditor. On the left there is a list of robots whichcan be connected and worked with. It is also possible to add dummy robots if a realrobot is not available. This list is part of FUremote and not MotionEditor specific.On the right-hand side there is a view displaying a 3D model for the robot. Thismodel is based on the robot description provided on the startup of theMotionEditor.The actual editor view is displayed in the center. It shows a diagram for each jointof the robot that is selected in the 3D model. This is done by clicking on the jointsin the 3D model. Double-clicking on a joint will select all joints that are rotated bythis joint, i.e. the kinematic chain to the end-effector. The x-axis of the diagramrepresents the time in milliseconds, the y-axis defines the joint angles in degree or theabsolute motor values. Above each digram there is a panel that gives information onthe motor’s ID, name and status information and also buttons to enable or disablethe motor and to configure the diagram.When hovering over a diagram of a motor, the motor is marked in the 3D model.The user then instantly knows which motor this is.Two different tools can be used to work on the motion. The initial tool is the Selector.It can be used to insert points by clicking in a diagram of a motor. Furthermore,multiple points can be selected, dragged, copied, pasted and removed.The second tool is the Shifter. This tool was intended to allow to speed up parts ofthe motion. It can be used in two different modes to either select all points left of thecursor or right of the cursor. When pressing the mouse, all points on the specifiedside of the cursor are selected. Keeping the mouse button pressed and moving themouse along the x-axis will then move all selected points along this axis only.

32

4.6 The new MotionEditor

Figure 4.9: New MotionEditor

The 3D model always shows the robot pose for the current mouse position. Whena point is dragged, it visualizes the rotational changes providing instant feedback tothe user. Furthermore, when moving the mouse along the x-axis, the user is able tosee how the motion will be performed.A cross-hair is displayed for the mouse position, too, which enables to align points.By double-clicking on a point, a dialog opens that can be used to set the accuratevalues of the point.All points in the diagram are connected by using the interpolation that is selectedin the drop-down menu in the panel at the top. This panel also shows the two toolsthat can be used to manipulate the points in the diagram.A diagram of a joint can also display trajectories of other joints, so the joint’smotion can be synchronized with other joints. However, the additionally displayedtrajectories cannot be modified.The joint diagrams can be zoomed in and out, to go into detail or to get an overviewof the trajectories. Additionally, it is possible to stretch or shrink the time axis, toeasily move points over great distances or set their timing precisely.Functionality for playing the whole or only selected joint motions in either real or3D is integrated as well as setting only a single pose of the motion.For compatibility with the old motion format the new MotionEditor can load theold motion format and adapt it to the new one.

33

Chapter 4 Implementation

34

5 Conclusion and Future Work

The MotionEditor that was used by the FUmanoid team until now had many draw-backs. These were analyzed to formulate requirements for a user-friendly and effi-cient motion designing tool. The most important concepts to fulfill the requirementsof intuitiveness and motion visualization are a 3D robot model and the illustrationof trajectories as time-joint angle diagrams. These were implemented in the newMotionEditor giving direct feedback when editing motions and enable very easymotion creation. Another important requirement is the independence of the motioneditor with regard to the robot that is used. It was shown that this can be achievedby using a robot description format and also how this can be defined, so the editoris able to create motions for different robots.The newly created MotionEditor was implemented using a well modularized andextensive architecture using state of the art design patters as MVC and MVA. Theconcept of tools is common for designing applications and allows the consequentseparation of functionalities for different purposes. The editor therefore also realizesthis concept and provides two tools for motion editing. These can be used to effi-ciently design motions. Also, different kinds of interpolation can be used to connectkey frames. The possibility to choose between either linear or Bézier interpolationallows the creation of fast or smooth motions.All in all, the new MotionEditor fulfills almost all analyzed requirements. It allowsfast and easy motion creation in a descriptive and user-friendly way and can be usedwith different types of robots.Although it simplifies the creation of motions in comparison with the old one inmany ways, there are still some potential enhancements. Especially, it is still notpossible to create a completely working motion without a real robot. This requiresan improved 3D model that includes physics to give feedback of the robot’s balance.Moreover, physics would also allow the integration of dynamic conditions. Thismeans that if the user wants the robot to remain balanced the robot’s pose wouldbe automatically adjusted for each key frame guaranteeing this constraint.Furthermore, the integration of inverse kinematics could be very useful, since itallows the user to directly position limbs or end-effectors in 3D space, whereas alljoint angles are automatically calculated.It is also reasonable to move the motion player logic to the motor board of the FU-manoid robots in future versions to separate behavioral logic and motion execution.

35

Chapter 5 Conclusion and Future Work

36

Bibliography

[1] Robert Eckstein. Java SE Application Design With MVC, August2008. URL http://blogs.oracle.com/JavaFundamentals/entry/java_se_application_design_with.

[2] Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides. Design Patterns,Elements of Reusable Object-Oriented Software. Addison-Wesley, 2003.

[3] Gerald E. Farin. Curves and surfaces for CAGD. Morgan Kaufmann, 5thedition, 2002.

[4] Frank Buschmann, Regine Meunier, Hans Rohnert, Peter Sommerlad, MichaelStal. A System Of Patterns, Pattern-Oriented Software Architecture. JohnWiley & Sons Ltd, 1996.

[5] Georgios Pierris, Michail G. Lagoudakis. An Interactive Tool for DesigningComplex Robot Motion Patterns. Technical report, Technical University ofCrete, 2009.

[6] Hamid Fetouaki, Emma Skopin. Bezier-Kurven, January 2009. URLhttp://pool-serv1.mathematik.uni-kassel.de/~fetouaki/seminar/presentation_handout.pdf.

[7] Steffen Heinrich. Development of a Multi-Level Sensor Emulator for HumanoidRobots. Master’s thesis, Freie Universität Berlin, January 2012.

[8] Judith Müller, Tim Laue, Thomas Röfer. Kicking a Ball - Modeling ComplexDynamic Motions for Humanoid Robots. Technical report, Universität Bremen,Fachbereich 3 - Mathematik und Informatik, 2011.

[9] Lizhuang Ma,Yan Gao, Xiaomao Wu, Zhihua Chen. From Keyframing to Mo-tion Capture: The Evolution of Human Motion Synthesis. Technical report,Shanghai Jiao Tong University.

[10] Marco Antonelli, Fabio Dalla Libera, Emanuele Menegatti, Takashi Minato,Hiroshi Ishiguro. Intuitive Humanoid Motion Generation Joining User-DefinedKey-Frames and Automatic Learning. Technical report, University of Padua,Osaka University, Osaka University.

[11] Marco Helmerich,Swen König, Franziska Wegner. Der Humanoide NAO - Be-trachtung eines neuen Robotersystems -. Technical report, Dualen HochschuleBaden-WÃŒrttemberg Karlsruhe, January 2010.

37

Bibliography

[12] Aldeberan Robotics. Choregraphe User Guide. URL http://www.aldebaran-robotics.com/documentation/software/choregraphe/index.html.

[13] Eclipse Foundation, Inc. Using OpenGL in SWT Applications. URL http://www.eclipse.org/swt/opengl/.

[14] Google Inc. Protobuf. URL http://code.google.com/p/protobuf/.[15] Sun Microsystems. Java 3D API Specification, June 1999. URL

http://docs.oracle.com/cd/E17802_01/j2se/javase/technologies/desktop/java3d/forDevelopers/j3dguide/j3dTOC.doc.html.

[16] The RoboCup Federation. RoboCup Homepage. URL http://www.robocup.org/.

[17] Stefan Czarnetzki, Sören Kerner, Daniel Klagges. Combining Key Frame basedMotion Design with Controlled Movement Execution. Technical report, Dort-mund University of Technology.

[18] Toshihiko Yanase, Hitoshi Iba. Evolutionary Motion Design for HumanoidRobots. Technical report, University of Tokyo.

All websites were last accessed on 26th of February 2012.

38

List of Figures

1.1 FUremote application . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Blender with 3D model of the robot 2012 . . . . . . . . . . . . . . . . 31.3 Aldebaran Choreograph user interface [12] . . . . . . . . . . . . . . . . 41.4 Choreograph Timeline Panel: 1. Motion Area, 2. Behavior Layer

Area, 3. Details Area [12] . . . . . . . . . . . . . . . . . . . . . . . . 51.5 Aldebaran Choreograph Timeline Editor in curve mode[12] . . . . . . 61.6 Kouretes Motion Editor [5] . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1 MVC class diagram[4] . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2 Schematic MVA pattern[1] . . . . . . . . . . . . . . . . . . . . . . . . 112.3 Linear interpolation[3] . . . . . . . . . . . . . . . . . . . . . . . . . . 132.4 Second order Bézier curve[3] . . . . . . . . . . . . . . . . . . . . . . . 142.5 Cubic Bézier curve constructed using de Casteljau algorithm[3] . . . . 14

3.1 Old MotionEditor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.1 Java3D Scene Graph [15] . . . . . . . . . . . . . . . . . . . . . . . . . 214.2 3D model of the FUmanoid robot 2012 . . . . . . . . . . . . . . . . . 224.3 Parallel Mechanics straight (a) and rotated (b) . . . . . . . . . . . . . 234.4 Simplified interpolation implementation class diagram . . . . . . . . . 254.5 Bézier interpolation with multiple interpolation instances (a) and a

single instance (b) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.6 Interpolation using cubic Bézier curves . . . . . . . . . . . . . . . . . 274.7 Using average slope for a point vertically between its neighbors . . . . 294.8 Simplified MotionEditor architecture class diagram for selected classes 314.9 New MotionEditor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

39