Inria Rennes HOW TO COOKopenvibe.inria.fr/openvibe/wp-content/uploads/2016/06/jl... ·...

1

J. T. Lindgren / OpenViBE 1

Jussi T. Lindgren, [email protected]

Inria Rennes

HOW TO COOK

C++ BOXES

OpenViBE workshop

Asilomar, June 2016

mailto:[email protected]

2

J. T. Lindgren / OpenViBE 2J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016

Boxes?

C++ boxes, anyone?

Inside your C++ box

Finally an example

Everything is not a box

Contents

3


Boxes?

4


Recall: signal processing is

done by boxes

Signal processing boxes

included in OpenViBE...

...potentially

allow you to do

1001 things already

OpenViBE Designer

5


But you want to do thing #1002...

• Try to use existing boxes in a clever way

• Make a wish

• Hire an engineer

• Custom order from a company...

• Implement a new box yourself

6


Whats a box?

A box can be

• A wrapper that actually runs Matlab, Python or Lua

script to do its thing (David’s talk)

• A C++ class (this talk)

7


View 1. Box as a function out = f(in)

Input streams(s)

Output streams(s)

Box

Box::process()

{

// get input

// do stuff

// send output

}

A good start!

8


View 2. Box as a processor

• OpenViBE – a streaming

architecture

• Boxes process data in chunks

• Data chunks after

elaborate transformation

4

3

2

…

1

0

Head

Tim

e pas

ses

Signal

chunks

9


View 3. A programmer’s box

Box {• Description

• User parameters declared

• Reactions to parameter changes

• Input/output handling if any

• Your algorithm

}

• A C++ box is a class with several member functions

• It needs 30+ lines of ’glue code’ !

10


C++ boxes, anyone?

11


Why C++ boxes?

Pros• C++ is the implementation language of OpenViBE

• No external dependencies needed

• Code can be as fast as you can make it

• No added python/matlab compatibility issues

Cons

• Needs a compiler (free ones are ok)

• Modifications require compilation and restart

• C++ prototyping can be slow

• Getting started with C++

http://openvibe.inria.fr/build-instructions

http://openvibe.inria.fr/build-instructions

12


Approach 1. Modify an old C++ box

Simple:

• Select an old box that resembles your new box

• Locate the box code from the source tree (’plugins/’)

• The box is usually a .h / .cpp pair

• Hack away

Your starting point has:

• Some ’interface glue’ already in place and it works

• Some old extra stuff, lacks your new stuff

13


Approach 2. Skeleton Generator tool

• Skeleton Generator gives you ’the bare bones’

• Creates a pre-filled .cpp/.h pair

• You add your meat custom processing code

14


Specify all the

properties of your box

When done:

Check, then

Generate!

SKELETON

GENERATOR

15


Skeleton Generator tutorial

http://openvibe.inria.fr/tutorial-1-implementing-a-signal-

processing-box

http://openvibe.inria.fr/tutorial-1-implementing-a-signal-processing-box

16


Inside your C++ box

17


Box description

• Each box needs a description

• Box identifier (random 64bit number)

• Box name

• Author

• Inputs & outputs

• Declaration of box settings

• ...

• These are in the box header .h file

• Used to visualize the box to the user

• The header may also define ’listeners’ : code that reacts to box

parameter changes that the user does in Designer

18


Box settings

Settings declared in the .h

file get a generated GUI

Box settings are saved/loaded

with the scenario automagically

Getting value of an uint64 param at slot 3 (index from 0):

uint64 l_ui64ThirdSettingValue

= FSettingValueAutoCast(*this->getBoxAlgorithmContext(),2);

19


1

0

Codecs

• Boxes have codecs

• Like audio/video codecs

• Codecs marshall data

chunks to/from streams

• Each stream needs its

own codec

Decode is required to

get at the raw data!

4

3

2

Stream

EBML

encoded

EBML

encoded

Decoder

Encoder Box

20


Boxes follow stream conventions

Header

Chunk 0

Chunk 1

Chunk n

End

…

TimeDefines the stream properties, e.g.

content type, chunk size (if any),

sampling rate, etc.

Actual data. Box may have several

chunks pending for process()

Just an event, no content

Stream

t=0

t=0

t=1

…

t=n

t=n

21


What time is it??

• Box code may often rely on the time of a block or sample

(e.g. ”process data for 2 secs from time t of stimulation”)

• OpenViBE uses 64bit fixed-point time• 32bits for seconds, 32bits for subseconds

• Equally precise both for ’small’ and ’large’ numbers

• class ITimeArithmetics has some convenience functions • 64bit fixed point time float64 seconds

• 64bit fixed point time sample count

• Seconds + subseconds pair 64bit fixed point

22


Finally an example

23


A signal modifying box

Box::Process() {

// Decode;

// Modify;

// Encode;

}

Signal chunks in

Modified signal chunks out

Box::Init() {

// read GUI

// settings;

// Init codecs;

}

Box::Uninit() {

// free

// resources;

}

24


Box header (pseudocode)

TSignalDecoder< myBox > m_signalDecoder;

TSignalEncoder< myBox > m_signalEncoder;

class myBoxDesc {

getName(void) { return ”MyBox"; }

getAuthorName(void) { return ”Somebody”; }

getAuthorCompanyName(void) { return ”ACME”; }

// ...

getBoxPrototype(OpenViBE::Kernel::IBoxProto& rPrototype)

{

rPrototype.addSetting(”GUI setting 1",

OV_TypeId_Float, "1.0");

rPrototype.addInput("Input 1", OV_TypeId_Signal);

rPrototype.addOutput(”Output 1", OV_TypeId_Signal);

}

}

25


Box initialization

myBox::initialize() {

m_signalDecoder.initialize(*this, 0); // In-connector 0

m_signalEncoder.initialize(*this, 0); // Out-connector 0

// make decoder and encoder share buffer & sampling rate

m_signalEncoder.getInputMatrix().

setReferenceTarget(m_signalDecoder.getOutputMatrix());

m_signalEncoder.getInputSamplingRate().

setReferenceTarget(

m_signalDecoder.getOutputSamplingRate());

}

26


Box processing

myBox::process() {

for(uint32 i=0; i<nChunks; i++) {

m_signalDecoder.decode(i); // decode chunk i

if(m_signalDecoder.isHeaderReceived()) {

// n.b. input stream properties are now

// available via m_signalDecoder getters …

m_signalEncoder.encodeHeader();

}

if(m_signalDecoder.isBufferReceived()) {

float64* l_pData

= m_signalDecoder.getOutputMatrix()->getBuffer();

// ... manipulate l_pData here ...

m_signalEncoder.encodeBuffer(); // encode chunk

}

l_rDynamicBoxContext.markOutputAsReadyToSend(0 , ...);

}

}

27


Box uninitialization

myBox::uninitialize() {

// Free resources

m_signalEncoder.uninitialize();

m_signalDecoder.uninitialize();

}

That concludes a box.

28


Everything is not a box

29


• Amplifier driver is not a box• Make it an Acquisition Server component instead

• Algorithm is perhaps not a box• If a routine is shared by several boxes, consider making it a class or an

’Algorithm’

• Stimulators may be external apps if they like• E.g. separate SSVEP, P300 flashers ...

• Right after rendering, send the event markers

to Acquisition Server using ”TCP Tagging”

30


Here we have skipped a lot of details:

Box coding is best learned by doing

More about at

http://openvibe.inria.fr

That’s it!

http://openvibe.inria.fr/

Inria Rennes HOW TO COOKopenvibe.inria.fr/openvibe/wp-content/uploads/2016/06/jl... ·...

Documents

Transcript of Inria Rennes HOW TO COOKopenvibe.inria.fr/openvibe/wp-content/uploads/2016/06/jl... ·...