AsgerKroghKjær ChristianJeppesen MaryDaphneEjsing ... - Draw a Sound.pdf · Problem Analysis 1...

Asger Krogh Kjær

Christian Jeppesen

Mary Daphne Ejsing

Henrik Dahl Nielsen

Jonas F. Christiansen

Kasper Woetmann Sahl

Pernille Elisabeth Lund

Draw

Sound

a

Department of Architecture,Design and Media Technology

Title:Draw a Sound

Theme:Sonic Interactions: Design and Evaluation

Project period:Start: 06-02-2012End: 01-06-2012

Project group:Group mea12434

Members:Asger Krogh Kjær_______________________Christian Jeppesen_______________________Mary Daphne Ejsing_______________________Henrik Dahl Nielsen_______________________Jonas Fibecker Christiansen_______________________Kasper Woetmann Sahl_______________________Pernille Elisabeth Lund_______________________

Supervisor:Jacob Boesen Madsen

Assistant Supervisor:Hans Thorhauge Dam

No. printed Copies: 4

No. of Pages: 90

No. of Appendix Pages: 45

Total no. of pages: 141

Completed: 01-06-2012

Synopsis:

This fourth semester Medialogy projectdeals with sound and music computing.This project in particular is about a gamefor a mobile tablet device. This game iscalled Draw a Sound and works as thefollowing: The first player makes a pathon the touch screen, which then activatesa series of tones that the other player hasto recreate by hearing. This is made on aSamsung Galaxy Tab Android based tabletand programmed in Java and PureData.The prototype game went through threeiterations, testing test participants’ abilityto distinguish between tones, their abilityto use the game interface and layout andfinally whether or not visual help wouldimprove the participants scores’. Thiswas done through the use of statisticalexperiments. The end results showed thatthere was a difference between visual andnon-visual feedback.

The contents of this report is freely accessible, however publication (with source references) is only allowedupon agreement with the authors.

Table Of Contents

I Introduction 11

Chapter 1 Problem Analysis 131.1 Initial Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.2 Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.3 Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181.5 Target Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221.6 Partial Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Chapter 2 Problem Statement 232.1 Prototype Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.2 Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.3 Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.4 Partial Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

II Prototype 29

Chapter 3 First Iteration 333.1 Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.2 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.3 Prototype Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.4 Prototype Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.6 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423.8 Partial Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Chapter 4 Second Iteration 474.1 Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.2 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.3 Prototype Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484.4 Prototype Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.6 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.8 Partial Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

4

Chapter 5 Third Iteration 655.1 Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655.2 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655.3 Prototype Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665.4 Prototype Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 725.6 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815.8 Partial Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

IIIReview 85

Chapter 6 Project Evaluation 876.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 886.3 Further Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Bibliography 93

A Second Iteration - Experiment Questionnaire 97

B Second Iteration - Octave Table 103

C Second Iteration - Effect Table 105

D Second Iteration - Octave Results 107

E Second Iteration - Effect Results 109

F Second Iteration - Java Code 111

G Third Iteration - Java Code 119

H Third Iteration - ANOVA 141

5

List of Figures

1.1 In-game screenshot of Guitar Hero, illustrating how the player react in responseto a graphical interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.2 Diagram of the human ear. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161.3 Visual area of the brain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171.4 Diagram of different parts of the human eye. . . . . . . . . . . . . . . . . . . . 181.5 Kaossilator and Ethereal Dialpad displays. . . . . . . . . . . . . . . . . . . . . . 191.6 Happy Note Music Composer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201.7 Reactable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.1 The iterative spiral model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.1 Complete unbalanced two-factor design. . . . . . . . . . . . . . . . . . . . . . . 353.2 Figure showing a section of the MIDI numbers in the same octave. . . . . . . . 363.3 Two sound waves. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.4 The basic tone patch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.5 PureData patch and abstraction. . . . . . . . . . . . . . . . . . . . . . . . . . . 393.6 C major scale design with sound pairs. . . . . . . . . . . . . . . . . . . . . . . . 393.7 Set Up of the experiment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.8 Dataframe showing the total, correct and incorrect answers arranged after

difference in distance of MIDI note numbers between the tone pairs. . . . . . . 423.9 The results from the experiment illustrated in a graph. . . . . . . . . . . . . . . 433.10 Table showing the statistical tests and their p-values. . . . . . . . . . . . . . . . 44

4.1 Direction of sound. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484.2 Division of the octave versions screen . . . . . . . . . . . . . . . . . . . . . . . . 494.3 Division of the effect versions screen . . . . . . . . . . . . . . . . . . . . . . . . 514.4 UML diagram of the classes in the second iteration of DaS. . . . . . . . . . . . 534.5 UML diagram of the function onTouch. . . . . . . . . . . . . . . . . . . . . . . 544.6 Figure showing division of the screen. . . . . . . . . . . . . . . . . . . . . . . . 554.7 UML diagram of the function addSound. . . . . . . . . . . . . . . . . . . . . . . 574.8 UML diagram of the function playPlayList. . . . . . . . . . . . . . . . . . . . . 584.9 Octave Version. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584.10 Effect Version. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594.11 Two test situations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5.1 Table showing the different game modes. . . . . . . . . . . . . . . . . . . . . . . 665.2 Concept illustrated in an Bill Verplank model. . . . . . . . . . . . . . . . . . . 675.3 Tablet desktop. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

6

5.4 Title screen displaying the name of the application and its logo. . . . . . . . . . 695.5 Game Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705.6 Instructions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705.7 The game starts by telling whose turn it is to draw. . . . . . . . . . . . . . . . 705.8 Drawing Screen. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715.9 Scoring Screen. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715.10 Pop-up Menu illustrated in a new game, where no playbacks have been used. . 725.11 UML diagram of the classes in the third iteration of DaS. . . . . . . . . . . . . 735.12 UML diagram of the function calculating the scores. . . . . . . . . . . . . . . . 745.13 Game Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755.14 Starting a game with no visual feedback. . . . . . . . . . . . . . . . . . . . . . . 765.15 Ending a game with no visual feedback. . . . . . . . . . . . . . . . . . . . . . . 775.16 Starting a game with start point. . . . . . . . . . . . . . . . . . . . . . . . . . . 775.17 Ending a game with start point. . . . . . . . . . . . . . . . . . . . . . . . . . . 785.18 Starting a game with end point. . . . . . . . . . . . . . . . . . . . . . . . . . . 785.19 Ending a game with end point. . . . . . . . . . . . . . . . . . . . . . . . . . . . 795.20 Starting a game with full visual feedback. . . . . . . . . . . . . . . . . . . . . . 795.21 Ending a game with full visual feedback. . . . . . . . . . . . . . . . . . . . . . . 805.22 Popup Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805.23 Participant drawing his answer and receives a score. . . . . . . . . . . . . . . . 815.24 Figure showing the results gathered from the third iteration pilot test. . . . . . 82

7

Preface

This semester is about the theme Sonic Interactions: Design and Evaluation. This reportwill contain various sections pertaining to a prototype concept, that relies on making asimple video game, that utilises sound as its central component. After the initial problemstatement, an overview of the concept will be given, to specify the following analysis ofelements included in it.

The purpose of the project is to prove or disprove the chosen hypothesis, that a videogame can be made utilising sound as its central component. To achieve this, a prototypegame will be created to test whether the hypothesis is true or not. The test will involvetest participants from the target group interacting with the prototype. The hypothesiswill be formulated after the problem analysis in order to shape the implementation of theprototype.

In order to develop a solution to the problem, the human perception and relating workwill be analysed. The related work will act as an inspiration in the development of theprototype.

The experience and knowledge of others has been utilised in this project as a foundationfor the prototype and the analysis before creation of it. When such an instance occur,it will be referenced using the Harvard method, which in the referencing list will containin order: [Authors Lastname, Initial., Year, Title, Edition, Publisher, Page]. In the textitself, the references will only contain [Authors Name, Year].

At the end of the report is the added appendix. In the appendix can be found the entireJava code and PureData patches made in this project. Likewise are the questionnairesgiven to the test participants to be found in the appendix. Everything in the appendix isalso on the attached DVD along with an A/V-production showing the prototype. On theDVD can also be found a digital copy of the report.

9

Part I

Introduction

11

Problem Analysis 1This semester is about sound and music computing, and about utilising such inapplications. This group was interested in utilising sound in the accordance of videogames and how games and gameplay could be made around sound, rather than graphics.But before such can be explored an initial problem statement follows.

1.1 Initial Problem

In order to acquire a greater understanding in the field of sound, the sense of hearing wasresearched briefly in order to formulate the initial problem. During this research, it wasdiscovered that the visual sense is considered to be superior to hearing. The Merriam-Webster dictionary shortly described the visual sense as:

"The human being possesses a highly developed visual sensitivity, far superiorto that of the other senses." [QA INTERNATIONAL 2012a]

As it also described in the Merriam-Webster dictionary, the auditory sense is able todistinguish between almost 400,000 sounds, making far superior seem like a broad term.[QA INTERNATIONAL 2012a] The broadness of the term ’far superior’ leads to theinitial problem sounding thusly:

The human visual sense is superior to other human senses, but it is so superiorthat it is not relevant to make a game based on the auditory sense?

1.2 Concept

This section is devoted to describe the project’s concept. The concept should clarify thevision behind the prototype, which later on will be analysed, designed, implemented andtested. This will happen through a series of iterations, where this concept will act as thefoundation for the iterations’ analysis and design sections.

The concept is a game application called ’Draw a Sound’ or in short DaS, which will testthe players in their auditory perception facilities, with none-, limited- or full visual feed-back. The game is played by two players who take turns drawing a path on a touch screen.

13

The path will be mapped onto a coordinate system divided into subsections, each playinga unique sound in the order they are touched. The two axes on the coordinate system,which extends the entire screen, each control a value that determines the "pattern" of thesound.

The way the game is played is similar to ’Simon Says’, where one player draws a soundand the other player has to recreate the sound as closely as possible. In order to evaluatethe results of the game, a score will be given according to how close to the original shapethe second player is.

Since the conception of video games, it has been the graphical addition to the gameexperience, that has been central.[QA INTERNATIONAL 2012b] Even music games suchas Guitar Hero [Activision 2005-2010] or SingStar [Sony Computer Entertainment Europe2010] does not use sound as a player-reactionary, but rather as a player output in responseto a graphical interface.

Figure 1.1: In-game screenshot of Guitar Hero, illustrating how the player react in responseto a graphical interface.

In this project it will be attempted to see whether it is possible to make a game withsound being the central factor, to which the player must react rather than as a simpleoutput.

1.3 Perception

The following is a description of the functional structure of the auditory and visualperception. Both senses contain relevant background material, that will be used in orderto back up the coming experimental test performed in this project. During this semesterthe auditory perception is in focus. Therefore it is relevant to research how accuratethe auditory functions are as well as what they are capable of. Since the visual part ofthe human sense is more superior, than the auditory it is necessary to research how itfunctions as well as how the two senses work together.

14

Auditory Perception

Auditory frequency is characterized as a periodic vibration with a frequency audible tothe average human. It is measured in Hertz (Hz), or numbers of repetitions of waveformsper second, meaning 1,000 Hz corresponds to 1,000 repetitions of a particular waveformper second. Its period is the time required for one complete statement of an iteratedwaveform. Humans are able to hear periodic sounds from about 20 through 16,000 Hz,which can produce a sensation of pitch in the human ear. These periodic sounds are calledtones. [Warren 1999]

Any sound that reaches the human ear can be broken down to three basic components:

• Pitch: An attribute of auditory sensation in terms of which sounds may be orderedon a scale extending from low to high. For pure tones, the frequency, or cycles persecond of the waveform, is the physical context of pitch. For complex sounds, pitchcorresponds to the lowest resonant frequency.

• Timbre: An attribute that enables humans to differentiate two sounds with thesame pitch. Timbre is a result from the interaction of spectral and temporal changesthat give the characteristic sound quality associated with a given instrument or voice.

• Timing: Refers to the major acoustic landmarks in the temporal envelope of speechand music signals. For music, timing works together with the temporal informationcontributing to timbre perception. Timing refers to the duration of sounds and theirfollowing perceptual groupings into rhythm.

[Nina Kraus 2009]

These three components pitch, timbre and timing can be used as overarching perceptualcategories for describing the shared properties of speech and music complex signals.

The Ear

The human ear is divided into three sections, the outer, middle and inner part whichcan be seen in figure 1.2. The outer ear, also called Pinna, appears to contribute tolocalization of sound sources. The sound passes through the auditory canal, which endsat the eardrum and cause it to vibrate. The vibration is transmitted by a chain of threesmall bones called Ossicles, located within the middle ear, which acts like a lever, thatconverts the airborne sound to liquid-borne sound. When sound in the air collides directlyupon a liquid, a loss of about 30 dB (99.9 percent of the power) takes place, with mostof the sound being reflected back into the air. This contraction of muscles is activated byreflexes, this lessens the intensity of very strong sound stimuli, in order to minimize thepossibility of damage to the inner ear. Few people are able to contract their middle earmuscles voluntarily, thus making it possible to capture relatively faint sounds at will. Thesound travels to the inner ear leading to stimulation of the auditory receptor cells and isregistered by the brain. [Warren 1999]

15

Figure 1.2: Diagram of the human ear.

Emotion

Stimuli from music and sound have a remarkable ability to evoke emotional impact on thehuman mind. Depending on how the brain analyses the input, the auditory perception canevoke negative or positive feelings. The brain and body can be stimulated when followingmusic with a rhythmic beat, for example mimicking the rhythm of a beating heart. A slowpace invites to a relaxed mind while a fast pace builds up activity and movement. In orderto evoke an intended suitable atmosphere, music and sound effects play an important rolein auditory media such as movies and video games. [Larsen 2010]

Visual Perception

Following the analysis of auditory perception, this part will delve into the field of visualperception.

There are many different physical as well as physiological components involved in visualperception. In order to understand how our vision works one must first understand howthe brain visual part functions, since it is involved with the photo receptors in the eyes,where the light changes into neural signals and are directed to parts of the brain.

The brain is a essential part of our central nervous system, together with the spinal cordand the brain stem, the main part of the brain is divided into the cerebrum and cerebellum.Studies have shown that over 50 percent of the cortex (the outer layer of the cerebrumand cerebellum) is occupied with vision, where only ten percent is used for hearing. Thefigure 1.3 shows the areas that are devoted to vision, as well as an overview of where theyare placed in the brain. As it can be seen in the image the active areas are spread out overdifferent areas of the brain each of them with a different function. The dominant area of

16

the brain concerned with vision is the area called V1 in the figure. [Robert Snowden &Troscianko 2006]

Figure 1.3: Visual area of the brain.

Before an image can be registered by the brain, it has to pass through the eye. Visualperception begins when light strikes the retina of the eye. The main structures of the eyeare the cornea, iris, lens, vitreous humor and the retina with the optic disk and the opticnerve, as it can be seen on the figure 1.4. A layer of photo receptor cells absorb lightand convert it to electro chemical signals. The photo receptors are divided into rods andcones. When it comes to absorbing color, it is the cones that are interesting consideringthat they perceive colors in contrast to rods, which registers grey scale. In addition to this,there are three sorts of cones absorbing different wavelength such as red, green and bluescones which together form full color image with all the colors that the human trichromaticcontains. [Robert Snowden & Troscianko 2006] [Szaflarski 2012]

The human eye can only perceive light with wavelengths between 390 and 750 nanometres.Outside this area are respectively the ultraviolet and infra red lights. This is called thevisual spectrum.

Audiovisual Perception

Comparing the two senses the visual perception is by far more superior, than the auditorywhich can be seen in figure 1.3. Despite the high difference in activity the integrationbetween hearing and vision are fundamental in human perception. In this manner thehuman being can understand the input from the environment. All perception involvessignals in the nervous system which in turn result from physical stimulation of differentsense organs such as eyes, ears and so on. [Pomerantz 2003]

"Imagine you are playing ping-pong with a friend who serves the ball. Youreceive information about where and when the ball hit the table by both vision

17

Figure 1.4: Diagram of different parts of the human eye.

and hearing. Scientists have believed that each of the senses produces anestimate relevant for the task, and then these votes get combined subconsciouslyaccording to rules that take into account which sense is more reliable. This ishow the senses interact in how we perceive the world. However, our findingsshow that the senses of hearing and vision can also interact at a more basiclevel, before they each even produce an estimate." [Shams 2011]

As stated in the quote above, studies have shown that in regards to perception of movingobjects, hearing and sight interact. Even when sound is not always relevant to the task,it still influences how humans perceive the world.

Summary

By nature the human visual perception is superior to the auditory perception, althoughit is possible to evolve the sense of hearing by intense training. When designing theprototype the facts about pitch and especially the human hearing range are essential.Furthermore, studies have shown, that there is an interaction between both visual- andauditory perception.

1.4 Related Work

The project aims to create a music application, using a touch screen as stated in section1.2 Concept. Because of this, related applications are examined for inspiration regardingdevelopment of the project’s prototype.

Ethereal Dialpad

Ethereal Dialpad is a music application for Android made by Adam M. Smith, which playsflowing synthesizer music, when the finger is dragged along the touch screen. The appli-

18

cation maps a certain music scale to the axes, which determine the tunes being played.The application is inspired by Korg’s Kaossilator, a small synthesizer, with a touch screen,mapped to a music scale, which is used to create different sounds. The Kaossilator is seenin figure 1.5a.

The application features different displays. Some of them are; a plain black screen that islit up in different colors when touched, another where a line is drawn where the finger isdragged, which can be seen in figure 1.5b, third with a grid that shows, where the touchevent is registered, as illustrated in figure 1.5c. Moreover the Ethereal Dialpad containsseveral sound altering modes. The pitch quantizer lets the user choose between differentmusic scales used in the application, such as chromatic, major and minor scales. Besidesthis the octaves present in the application can also be changed, in the octaves menu. Itallows the user to choose from one to four octaves and can, depending on the number ofoctaves, divide the grid into a bigger or smaller variation of tones. In addition to thesefunctions, it is also possible to add different effects to the notes being played, such asdelay, flange, envelope and timbre. [Kirn 2010]

(a) Kaossilator. (b) Ethereal Dialpal with line. (c) Ethereal Dialpad with grid.

Figure 1.5: Kaossilator and Ethereal Dialpad displays.

Happy Notes Music Composer

Another inspiring music application for Android is Happy Notes Music Composer. It is aMusical Instrument Digital Instrument (MIDI) music creation application, which let theuser play, improvise and compose music through piano sounds, by sliding either one or twofingers on the touch screen. The melody and harmony created by the application dependon whether the user is interacting with the touch screen, with one or two fingers. Whileusing the application, the composed music is automatically saved as a two track MIDIfile, which allows the user to listen to the composition again. Besides creating melodiesand harmonies, the application also tracks and light up the places where fingers are placedon the touch screen, thereby following the movements on the screen providing a visual

19

feedback as well. The Happy Note Music Composer display can be seen in figure 1.6.[Google Play 2011]

Figure 1.6: Happy Note Music Composer.

Reactable

The Reactable is a table that plays music. The music is created by placing objects on thetable’s surface. These objects are called pucks and the table is able to tell, which kind ofpuck it is and how it is placed on the table. In figure 1.7 the Reactable is shown, with thedifferent kinds of pucks on its surface.

Figure 1.7: Reactable.

There are four types of pucks: generators, effects and filters, controllers and global con-trollers. When placing any of these pucks on the table, it will react differently dependingon the puck’s type. The generators are used to produce the actual sounds, like an os-cillator or audio from another source. The effect and filter pucks can modify the soundsthat come from the generators. The controllers do not modify the sound like the effect

20

and filter pucks do, but they rather modify how other pucks behave. Lastly the globalcontroller pucks change the behavior of the pucks globally.

Controlling each puck can be done by rotating the puck or in some case flip it, but itcan also be done by using the GUI, that appears whenever the table recognizes the puck.Pucks will automatically begin interacting with each other when they are close. In themiddle of the table is a white dot and that is the output object, so the pucks have to beconnected to the output object in order for people to hear the sound that has been madewith the table. [Reactable Systems 2012]

Summary

Of the different related work that has been researched throughout this section, the Ethe-real Dialpad is the one that resemble the projects concept the most. The way the x-axis ofthe display have been mapped to a certain music scale, appears to be a good way of doingthis. Also the grid display being used and the way of mapping a note to a specific point,is interesting to look further into. The display part of the Ethereal Dialpad contains auseful way of displaying the path drawn by the player. The line display, graphically showsthe path drawn by the player while drawing and also indicate the current position of theplayers input. Such a showing of the path across the display, could prove a useful indicatorfor the player to navigate the sound in this project’s concept. The various mix of effectsof Ethereal Dialpad, can make it difficult to tell the notes apart. Therefore the use of sucheffects, that blend notes together at such a rate, should be avoided in the project’s proto-type. If effects will be used, these effects should make it possible to distinguish betweennotes.

Happy Note Music Composer have a different approach and focus more on creativity, let-ting the user play around with different melodies and harmonies. It maps a melody tothe x-axis, playing more notes in coherence at the same point. This is of interest for thecreative part of a music application, but it may not be useful in this particular project,since the creation of melodies, might be more difficult for the players to recreate, than aseries of notes. The display of Happy Note Music Composer, only contains an indicatorof the users current position on the screen. The line display from Ethereal Dialpad, seemsto be a better choice for this project for letting the player visualise the drawn path.

Reactable stands a little out, since it, in its essence, is an instrument. It shows anotherway of using a screen to create sound, as it uses pucks as well as the user’s fingers asinterface, which is not similar to this project. However, it does a good job of showing theprinciples of what is taught in the Sound and Music Computing course on this semester.However, the way of using pucks for input, could make it difficult to create a path of notesto be recreated.

21

1.5 Target Group

As the concept’s purpose is to evaluate the difference in acuity between the auditory andvisual senses, the first step is to restrict the target group to people who suffer no disabil-ities concerning hearing or vision. Test participants with hearing or visual impairmentswill affect the test results in favor of either the one or the other. However as it is hard toaccount for visual- and auditory impairments that is unknown to the test participant, onlypeople with diagnosed disabilities is restricted from the target group. People who havedecreased hearing or vision, without being aware of it, should also be excluded, howeverthis is near impossible since the test will not include eyesight or hearing tests. Thereforepeople with mild impairments are considered an acceptable part of the target group fortesting purposes.

1.6 Partial Conclusion

As written in the perception section 1.3, Perception, the function of the human auditoryperception is a complex mechanism, in that the human ear breaks sound into three basiccomponents namely pitch, timing and timbre. Furthermore it was discovered that thehuman ear is able to hear sounds from about 20 to 16,000 Hz.

The visual perception part of the brain is a much of the brain than the auditory one.

Even though visual perception is spread out over a large area of the brain, it is also con-nected to auditory perception in the sense that hearing and sight interact with each other,even though we do not always need to include the vision to use our auditory perception.

Throughout section 1.4, Related Work, some noteworthy things were observed. One beingthat different tones was mapped on the two axes, and another was that the "path" theplayer would draw was visible for the player, while he or she draws. Moreover, it that agrid as visual feedback feature could be interesting to look into. Lastly, if effects is goingto be present in the prototype, they have be implemented in such a way that they do notblend together.

The target group will be people with a normal functioning hearing and vision.

22

Problem Statement 2This chapter is devoted to further describe the prototype, hereby building on the initialconcept of DaS as described in section 1.2 Concept, and listing the prototypes specifica-tions such as user needs and demands of function, non-function, and technical demands.The specification leads to the project’s hypothesis, which sets the goal for the prototypeiterations that follows in part II Prototype.

Before adding to the initial concept, a problem statement is formulated based on the initialproblem and the knowledge obtained in chapter 1 Problem Analysis:

How can a sound based game be developed that focuses on recognition of tonesthrough interaction with a touch screen and how can an experiment be designedthat is able to test the success of recognition?

This statement will be revisited in the conclusion in section 6.2, but before the process ofanswering the statement begins, the project limitations below are presented.

Project Limitations

For this project the prototype that will be developed in response to the concept DaS, islimited to only being compatible with one platform. This is a result of choosing to programin Java and using PureData as the program behind the creation of the prototype’s soundinterface. The programming language and program are chosen because they are taughtto the group during the semester, as well as the group having former experience in usingPureData. In order to make Java and PureData communicate, the library ’libpd’ willbe used. libpd is available for iOS and Android, but because the programming languagebeing Java, Android has been chosen for the target platform, since iOS development isdone i Objective-C. Furthermore the library limits the prototype to only run on deviceswith version 2.3.3 (Gingerbread) or greater installed. [Brinkmann 2012]

23

2.1 Prototype Description

Expanding upon the concept that was described in section 1.2 Concept, this section willdescribe the concept more thoroughly.

As mentioned before, DaS is played by two players, who take turns drawing a path on thetouch screen. The game will play the sounds shortly after the finger lifted from the screen.Based on the sound alone the other player has to recreate the path the first player drew.The players has to rely on their ability to hear the pattern of the sound, but they willhave some tools available to help them identify the locations of the sounds’ path. Duringthe players’ turn they can playback the sounds three times and also at any time, clickon the screen to hear the sound mapped to that particular field. The player is limitedto three replays so the players do not "learn" the sound path. Letting the players hearthe individual sounds will allow players to be reminded where on the screen the varioussounds are mapped. Once the user drags their finger across more than one field, it countsas a path. If the players attempts to cross their own line, the game will stop them withan error message. This is to avoid that unsportsmanlike players will continually overwritetheir own path, making more potentially more difficult for the responding player a chance.When the players lifts their finger, the path is done, and the game compares it to the pathof the first player. Then the round ends with a score and the game starts over in reverseorder.

There are four levels of visual feedback. No graphics that does not give the guessing playerany visual feedback to where the path lies. The only feedback here is the sound the firstplayer created. The next two levels are similar to each other. One, the start point, markswhich field the first player started the path. The other marks the end point. These twowill give the second player a clue of the position, but does not provide the pattern. Thelast level of visual feedback is the full graphics level, it marks the entire path that the firstplayer drew. This last one should show how what impact it will have, when the visualsense is used fully.

The scoring system tells how accurate the individual player is at recreating the opposingplayer’s path. The points will be based on how many of the same fields were touched.

2.2 Specification

Based on the 2.1 Prototype Description, this section is devoted to identify a series ofdemands, that the game must fulfill in order to achieve a solution that fits this project.Behind these demands are some user needs, that players will have in order for them toplay the game properly. An examination of these needs and why they are essential, willfollow in this section.

User needs

Below is a list of the most relevant user needs:

24

Auditory feedback: The players will receive auditory feedback when drawing theirpath, and also when clicking on the screen. This is essential because they needto understand why and how the sounds are generated through their interaction withthe touch screen. The click feature is important because otherwise it would requirethe users to learn the auditory-interface by memorization. When confronted with asound, the player would have no clue as to where to begin, if he could not try andhear where the sounds are located within the mapping.

Visual feedback: The players also need some degree of visual feedback in order to fullyplay the game. Certain elements of the interface are easiest obtained with graphics,such as the game menu and the scoring system. If there was not any visual feedback,the game would have to operate fully through voice recognition or some other formof interaction which does not involve physical interaction.

Gameplay: In order for the players to perceive the application as a game, it needsgameplay, such a game rules, challanges and win condition.

Instructions: To fully understand the game, it need some instructions.

Interaction: The players need to be able to physically interact with the game in orderto play it.

Functional demands

Based on the above, follows an examination of the functional demands, that the gamemust fulfill in order to meet the user needs:

• The mobile device running the game must have speakers and a screen in order tofulfill the needs for auditory and visual feedback.

• In order to meet the need of gameplay, the application has to act like a game. Thisis achieved by having two players compete against each other to get the highestscore. The game should also have an instructions page, in order to meet the needto understand the game functions.

• In order to meet the need for interaction, the game will use a touch screen. Theinteractional elements on the screen must be designed to fit the size and reachof human hands, so that the game can be played with the use of only fingers. Allelements on the user-interface must obey this in order for the game to be manageableto play.

MoSCoW Method

The functional demands acquired above will in this section be divided into the MoSCoWmethod. This method sorts the demands from the most important to the least important.MoSCoW is shorthand for ’Must have’, ’Should have’, ’Could have’, and ’Won’t have’.’Must have’ contains the parts of the concept, that are essential. Everything in thiscategory plays a crucial part of the prototype and are needed in order for it to functionin its most basic state. ’Should have’ are parts that are important, but not essentiallyneeded. The prototype can function without them, though it will remain in a state that

25

is far from sufficient to reach the goals of the concept. ’Could have’ are things that couldbe a part of the prototype, but are not as important as ’Must have’ or ’Should have’.This category holds parts of the concept that are not essential and do not have as high apriority as those in the ’Should have’ category. It could be good to have as many of thethings implemented in the prototype, but they are the first to be discarded if time or skillis not sufficient enough to include them. ’Won’t have’ contains all the things that couldbe a part the prototype but will not be in it doing this project. The things are those thathave been pre-discarded doing the development of the concept. This is due to the lowerpriority than all the other stuff that have been voted more important for the core concept.These things can be brought back as design considerations, if the prototype goes throughfurther development at a later time. Below are the categories and the parts of the conceptthey contain:

Must have

• Sounds that play when the application is used.• Sounds generated from user interaction.• A scoring system that illustrates how well players are doing at identifying sounds.

Should have

• A click feature to hear sounds in the grid.• A playback button that lets players hear previous sounds three times.• All the different game modes mentioned in the concept.• Start and exit buttons.

Could have

• An instructions screen.• A game menu with access to: New game, Instructions, Highscore, Exit game.

Won’t have

• A highscore system.• Multiplayer mode with more than two players.• Multiple tone-scales to choose from.

Non-functional demands

Besides the functional demand, this section will also establish a number of non-functionaldemands, which will operate as guidelines in the design sections of the prototype iterations.These demands should contribute to a continuous design throughout the development. Thedemands are as follows:

• The interface should only contain items with a size and layout that is manageableto navigate with the use of only fingers.

26

• The overall graphical layout should have a simple look to it, so that it is the auditoryfeedback that is the most dominant.

• Two colors should be used to identify the players to make them distinguishable fromeach other.

Technical demands

Lastly follows a list of technical demands. These will ensure that the game runs asintended, meaning all the above functions work in real life. The demands are as follows:

Platform: Android with atleast version 2.3.3 (Gingerbread) and with an API level of10. This is a must because the library that connects Java with PureData is notcompatible with older versions of Android.

Screen: Must have a touch screen, but it does not necessarily need to have multi touchfeatures.

Speakers: Must able to play sounds with good enough volume and quality to berecognizable.

2.3 Hypothesis

The project focuses on participants’ ability to recognize and recreate the sound path ofthe application. The goal of this project is to carry out an experiment using the prototypeDaS in its last iteration. This experiment will be analysed using scientific methods, whichfoundation is a null-hypothesis and alternative hypothesis. In this section the hypotheseswill be presented:

Null-hypothesis: In creating a game based mainly on auditory feedback,there is no difference in the mean scores between game versions with ’Fullgraphics’, ’Start point’, ’End point’ and ’No graphics’.

Alternative hypothesis: In creating a game based mainly on auditoryfeedback, there is a significant difference in the mean scores between gameversions with ’Full graphics’, ’Start point’, ’End point’ and ’No graphics’.


In this chapter the problem statement was specified to a more specific topic, than theinitial problem. This in turn specifies how the project will progress from here on out.Additionally the concept has been specified as well as requirements for the prototype.This step is crucial in order to progress further into the development of the prototype, asit is necessary to know what the prototype needs to be able to do before it can be made.These requirements were prioritized by the use of the MoSCoW.

27

Part II

Prototype

29

Introduction

The following part of the report concentrates on the development of the prototype. Thisis carried out through the use of the iterative spiral model. The model cycles in iterations,where each iteration has four phases: "What to do?", "How to do it?", "How to show it?",and "How to test it?". This project will consist of three iterations. The first phase consistsof an analysis about what to implement from the concept in the given iteration and how tobuild it. The prototype gets designed based on the outcome of the previous phase. In thethird phase the implementation of the prototype is carried out. Lastly before beginning anew iteration, the prototype gets documented and verified through testing and analysingthe results. The model can be seen in figure 2.1. [Larsen 2011]

Figure 2.1: The iterative spiral model.

31

First Iteration 3In the first iteration of the prototype, the objective is to design an experiment that tests thefoundation of the concept described in section 2.1 Prototype Description. The most basicelement needed for the concept to work, was identified as the users ability to distinguishdifferent tones from each other. If they cannot hear the difference while interacting withthe prototype, then there is not much reason to believe that the concept as a whole will bea success. This iteration is therefore devoted to implement the prototypes sound interface,and to design an experiment that tests the users ability to distinguish between includedtones in the prototype.

3.1 Hypothesis

In order to do an experiment that tests the first iteration of the prototype, a scientific null-hypothesis and alternative hypothesis was required. Scientific hypotheses must be testableand falsifiable, meaning the hypotheses concern something observable/measurable andthat they can either be proved or disproved. If the results of the experiment can disprovethe null-hypothesis, it will be rejected and the alternative hypothesis will be believed tobe true. [Crawley 2012]

The hypotheses are as follows:

Null-hypothesis: The target group cannot distinguish between tones of thesame or different pitch.

Alternative hypothesis: The target group can distinguish between tones ofthe same or different pitch.

3.2 Experimental Design

The purpose of the experiment was to ensure that there was foundation for the describedconcept as stated in section 2.1 Prototype Description. A single question, which is whethertwo tones are the same or different, were repeated several times during the experiment ondifferent conditions for the test participant. The answers was used to conclude whetherpeople could distinguish tones and to examine what intervals of tones, were most distin-guishable. This was examined for the later design for the interface of DaS. It was also

33

examined if DaS should include, one of the two versions presented later in first iterationanalysis. Each version includes a fixed number of sound pairs instead of entire musicalscales. If for example all possible pairs should be tested, then the entire MIDI scale wouldmake over 16,000 pairs, due to the 128 different tones on the MIDI scale. [Wittmann 2005]

To make the experiment less time consuming 15 sound pairs per version were chosen. Rea-son for choosing 15 sound pairs was the belief that 30 tones would be a fairly representationof the scale. This idea were based on the rule of thumb for the size of a sample whichare 30.[Crawley 2012] The pairs consisted of three types of difference in pitch between thetones. Some were the same, some were very close to each other and some that are very far.

The experiment was designed as a ’complete unbalanced two-factor’ design, where theexplanation of these concepts are described below:

• Complete/Incomplete: A complete design is where all test conditions have beentested, and an incomplete design is where one or more conditions have been left out.It is always preferable to have a complete design, as the calculated statistics shouldbe able to compare every test condition with each other.

• Balanced/Unbalanced: A balanced design is where all test conditions have beentested by an equal amount of test participants. Opposite is where not all testparticipants have tried all test conditions. A complete design is always preferable,as the comparison between the conditions gets bias if there is an unbalanced amountof repetitions.

• Factors: A factor is an explanatory variable with a discrete and finite set of possiblevalues, called levels.

[Crawley 2012]

This experiment turned out to be unbalanced as a result of not thinking about how manyof the same and different tone pairs were added during the creation of the PureData patch.The two factors in this experiment areMusic scale with the levels ’Major’ and ’Chromatic’,and Tones with the levels ’Same’ and ’Different’. The Music scale factor’s two levels madeup the two versions that was to be tested in the execution of the experiment.

The illustration in figure 3.1, shows it all put together. What should be noticed besidesof the illustration, is that the response values of the experiment, which is whether thetones are the same or different, only takes two values: ’Correct’ and ’Incorrect’. Thismade the experiment into a binomial design, which is a type of experiment where the testparticipants have 50 percent chance of guessing correct. Furthermore this experiment wasof the kind called an AB experiment, due to the test participant must choose between Aor B based on some criterion. The criterion would be the two presented tones played andthe options would be ’Same’ and ’Different’.[Crawley 2012]

The following design principles were also used in the experiment:

34

Figure 3.1: Complete unbalanced two-factor design.

• Independence: Test participants must do the experiment individually, and becausethe sound pairs was presented in random order, the test participants could not easilytell future participants how to "beat" the experiment.

• Repetition: The experiment was carried out multiple times doing the course ofone day with different test participants.

• Randomization: The set of toness was picked randomly and noted so they onlyappear once.

• Blocking: Test participants act as blocks, because they all answered the samequestions.

[Crawley 2012]

The results of the experiment will be analysed in section 3.7 Results.

3.3 Prototype Analysis

To test whether people were capable of differentiating between tones, and which tonespeople are capable of distinguishing between, a prototype had to be created. This sectiongoes through an analysis of what the prototype should be able to do, in order to eitherprove or disprove the null-hypothesis mentioned in section 3.1 first iteration Hypothesis.Furthermore the tests secondary goal would be to examine whether the chromatic or themajor scale is preferable for a later prototype design.

PureData

A prototype capable of testing the above can be made in PureData, as stated in chapter 2Problem Statement. PureData is a free software that handles real-time graphical program-ming for audio, video and graphical processing. It is the third major branch of the Maxpatcher programming languages, which include Max/FTS, ISPW Max and Max/MSP.The core of PureData is written and maintained by Miller Puckette and includes workfrom many developers. PureData is also able to run on smartphones and tablets throughprojects like libpd and RjDj. [puredata.info 2012]

This iteration primarily focused on the sound-interface for distinguishing tones. The dif-ferent tones could be played directly from PureData making it sufficient to implement thesoftware only in PureData rather than on a device. The first iteration of the prototype had

35

to have some capabilities. Firstly it had to be able to play sounds, that the participantswere be able to compare. To have any data, it was required that the tones could be eitherread or configured by the facilitator, so that it could be determined what tones peoplecould easily distinguished, and which troubled them. The tones should also be playedin randomized order, so that learning the tones can be removed from the equation, buteveryone should still go through every tone pair at least once, so that the participants’abilities could be compared.

3.4 Prototype Design

The development of the design primarily dealt with creation of the PureData patch. Thisinvolves the sound of the tone being played according to duration. It also dealt with thecreation of different tone pair triggers, which would make it easy to a given tone pair witha single mouse click. This was made for convenience during the experiment.

PureData Patch

The sounds in the PureData patch was be made from MIDI note numbers. The MIDInumber represents a note, as for example the note A. The MIDI number addresses thesame frequency in Hz as the given tone. As seen in figure 3.2 a MIDI number of 57 havethe same frequency in Hz as the note A, which is 220 Hz.

Figure 3.2: Figure showing a section of the MIDI numbers in the same octave.

36

The waveform of the sound produced from this PureData patch will be a sine wave. Thiswas chosen because a simple waveform is wanted for the distinguishing part of the test.PureData offered two objects for waveforms. One for sine waves, another for sawtoothwaves. [Kreidler 2009] A sine wave is the most basic and simple waveform. It has noharmonics and are therefore quite pure in sound. A sawtooth wave on the other handproduces more harmonic content, both even and odd harmonics, which makes a buzzingsound. [Harris 2008] Two sound waves, a sine and a sawtooth wave, can respectively beseen in figure 3.3.

(a) Sine wave. (b) Sawtooth wave.

Figure 3.3: Two sound waves.

The tones created in the PureData patch would also have to be controlled. The toneshould stop after a certain time period. For this the fade-in, sustain and fade-out timeof the sound would have to be adjusted. Furthermore for comparing tones, the PureDatapatch would be created with different tone pairs with different tone interval such as asound pair with two identical tones, a tone pair with one MIDI note difference, a tone pairwith two MIDI note difference and so on. Moreover, two designs would be made. Onecontaining the chromatic scale, another containing the major scale. Both scales wouldinclude different octaves.

Chromatic Scale

The chromatic scale design was be made from a series of individual single MIDI notenumbers from this particular scale. The chromatic scale consist of twelve tones each ahalf step apart. These twelve tones represent all root notes.[Estrella 2012a]

Major Scale

The major scale design would, like the chromatic scale design, be made of a series ofindividual single MIDI note numbers, but this time only the MIDI note numbers from themajor scale would be used. The major scale consist of seven different notes. For examplethe C major scale consist of the tones; C, D, E, F, G, A, B. [Estrella 2012b]

3.5 Implementation

For the first prototype two PureData applications were made, even though they were verysimilar. The first patch were some software, which had every note from the chromaticscale. It contains the MIDI note numbers from 24 to 108 as buttons. The other patch

37

was every MIDI note number that is in the major scale. So the two patches were virtuallyidentical in their construction, differing only in the amounts of options available.

The basic of both patches worked the same way, which can be seen in figure 3.4. It workedby connecting a number box to a [mtof∼] object. This number represented the MIDI notenumber and was converted into frequency by the [mtof∼] object. This was then connectedto an oscillator [osc∼] object, which outputs a cosine wave. A cosine wave is a waveformwith a shape identical to that of a sine wave, except that each point on the cosine waveoccurs 1/4 cycle earlier.[Rouse 2005a] Hereafter the output from oscillator is multipliedwith a [vline∼] object, which is a envelope generator. It controls the ramp up, durationand ramp down of the sound. It was controlled by a message box which sent multiplemessages. In this case it told the [vline∼] to ramp up to 1 in 10ms. Then it tells the[vline∼] to wait 200ms and then ramp down to 0 in 500ms. [Brinkmann 2012] As the lastpart it was connected to a digital to analog converter [dac∼], which provided real-timeaudio output.

Figure 3.4: The basic tone patch.

This tone patch was used for each tone made in the two PureData patches. Because itwas used for each tone, the tone patch was made as an abstraction. An abstraction is aPureData patch which has been saved in a folder that PureData looks in every time anobject is created. This path can be defined in PureData preferences. When calling thepath in a new PureData patch, the name of the abstraction has to be typed in an objectbox. The abstraction has to be created with inlets and outlets, which enables it to beconnected with other objects in the new PureData patch. The abstraction can be seenin figure 3.5b. The abstraction in this case was named MIDI note. The use of it can beseen in figure 3.5a, this shows the major scale starting in the tone C, which in this caseis a MIDI note number of 48. Each tone created have a different MIDI note number thatrepresents one of the tones in the C major scale. A message box containing this MIDI notenumber was connected to the abstraction to the inlet leading to the [mtof∼] object. Theother inlet is connected to a bang button, which trigger the inlet leading to the [vline∼]and plays the tone. The exact same design was used for the chromatic scale.

For convenience during the experiment, the created tones were organized into pairs. Thiswas done by creating two bangs which were connected to two of the tones, as seen onfigure 3.6 showing the C major scale design. To activate the tone pair, one would simplyhave to press each bang. To make it easily get an overview, a message box with the nameof the two tones being played and message box with the tone pair number, were created.

38

(a) C major scale in design PureData. (b) The abstraction calledMIDInote used in the tonedesign.

Figure 3.5: PureData patch and abstraction.

The tone pairs, seen in the same figure, ranges from 16-30. The tone pairs ranging from1-15 was used in the chromatic scale design. Beside that exactly the same design was alsoused for the chromatic scale design.

Figure 3.6: C major scale design with sound pairs.

39

3.6 Experiment

This section describes how the experiment was handled during the course of one day’stesting.

The first step in realizing the experiment was to book a room and to organize theequipment. The equipment needed was a laptop running the PureData patch and aset of headphones connected to the laptop. The second step was to find as many testparticipants as possible, hence making efficient use of the test period. The following itemsbelow explain the different elements in selecting participants as a result of avoiding biasin the statistics later on:

• Population: The statistical population is the collection of all subjects included inthe target group as stated in section 1.5 Target Group.

• Sample of population: Is made by selecting a sample of the population that areof interest.

• Sample size: This is the size of the sample population. The sample size is animportant component of any empirical study, where the goal is to make inferencesabout a population from a sample.

• Random sample: The sample is random when all possible samples of n objects areequally likely to occur. Random sampling refers to sampling method that consistsof n objects for all the population as well as the sample must consists. [Crawley2012]

The population of this experiment should be the same as the target group described inSection 1.5 Target Group. Because it is immensely expensive to test on the entire popula-tion, a sample was chosen. The sample was chosen to be fellow Medialogy students fromthe fourth semester, due to the belief that the semester included a fair representation ofthe population. They were also chosen since the experiment were meant to be a quickexperiment. The sample size was decided to be as many as possible of out the 42 studentsand ended up with results from eleven students, which was the number of repetitions thatwas manageable during the course of one days testing. In order to avoid bias from a fixedsample size, it was made random. This was achieved by asking all students if they wouldlike to participate in the experiment, which made it sufficiently random who ended uptesting.

The second step was to carry out the experiment, which began with a repetitive routineas a test participant entered the room. First he would face the test team consisting of twogroup members, as seen on figure 3.7; one acting as facilitator/technician, and one actingas logger. The definition of the roles is as follows:

• Facilitator: The role who was primarily responsible for the course of the experimentand the only one who was in dialogue with the test participants.

• Technician: The facilitator was also assigned to the role of technician, whose taskwas to operate the laptop and play the tones for the test participants.

40

• Logger: The roles primary task was to log the data, which in this experiment wasthe answers that the test participants gave the facilitator. He would also make surethe tone pairs was not accidentally repeated to the same test participant. Lastly hewould ensure that there was maintained order and control of the experiment.

[Larsen 2011]

Figure 3.7: Set Up of the experiment.

During the experiment the facilitator shortly introduced the test participant as to whatwas going to happen and then began to play the tone pairs. If the test participant agreedto it, the session was recorded on a Dictaphone, making it possible to hear the test resultsagain later on if necessary. The order of the two versions were switched around for eachtest participant, so the first test participant tested the chromatic scale pairs first and thenext participant would start by hearing the pairs from the major scale. Should this hasbeen totally randomized, the order should have been chosen by chance for each test par-ticipant. The test participant had the option to hear the two tones again, but only onceper pair. The purpose was to collect results of the participant’s first intuition rather thanletting him memorize the tones by repeating them multiple times. Each pair were pickedin random order by the facilitator as the experiment was conducted. The order was notrandomized beforehand, but rather in the testing process.

41

The logger noted the collected data, which can be seen in figure 3.8.

Figure 3.8: Dataframe showing the total, correct and incorrect answers arranged afterdifference in distance of MIDI note numbers between the tone pairs.

The dataframe illustrates the distribution of the total, correct and incorrect answersarranged after difference in distance of MIDI note numbers between the tone pairs. Thiscovers all that the experiment’s results in, which was then ready for the next phase of theiteration.

3.7 Results

This section describes the analysis of the results gathered in section 3.6 Experiment. Thetwo factors, mentioned in 3.2 Experimental Design, was discovered to be similar duringthe experiment which causes the analysis in this section to be calculated as if being onefactor instead.

Following the experiments, certain tendencies could be seen among the participant’sresults. Primarily most of the test participants were able to differentiate between most ofthe total of 30 tone pairs, which was made of the 15 tone pairs in each version. Even theleast successful participant got 22 out of 30 answer right, while four participants got allanswers right.

42

Figure 3.9: The results from the experiment illustrated in a graph.

The results plotted in the dataframe in figure 3.8 shows the number of incorrect answersthat were made in tone pairs with different MIDI note number interval. To determine whatMIDI note number interval to use later on, the data from the dataframe were analysed.Along with the data from the dataframe, a graph was made to show the success in percent,for each of the different MIDI note number intervals. This can be seen in figure 3.9, wherethe x-axis shows the distance of MIDI note numbers between the tone pairs and the y-axis shows the success in percent. The percentage from this figure together with the totalanswers and incorrect from figure 3.8, were used to analyse the data.

• Out of 77 answers, one incorrect answer was made at a MIDI note number intervalof zero: 98 percent success.

• Out of 44 answers, twelve incorrect answers were made at a MIDI note numberinterval of one: 72 percent success.

• Out of 66 answers, seven incorrect answers were made at a MIDI note numberinterval of two: 89 percent success.

• Out of 11 answers, one incorrect answer was made at a MIDI note number intervalof three: 90 percent success.

• Out of 22 answers, no incorrect answers were made at a MIDI note number intervalof four: 100 percent success.

• The rest of the MIDI note number intervals following four, had no errors.

Of all the answers given, a total of 24 were incorrect, however all the incorrect answersseemed to have certain elements in common. With only two exceptions all correct an-swers were due to the participants believing two different tones to be identical. Almostall instances where the tones were perceived as identical but were not, only differed fromeach other with a single or two MIDI note numbers, whereas most incorrect answers werepresent at a single MIDI note number in difference. Only two cases differed by more; onewith a MIDI note number difference of three, and one with a MIDI note number differenceof five. These results would indicate that sounds used in future PureData patches should

43

have two or more MIDI note numbers of distance between them in order for a highersuccess rate.

Together with the above and the data presented in figure 3.8, it can be seen that the ex-periment participants were able to distinguish between tones. The next step is to examinewhether the result occurred by chance or not. The result of interest is to find the p-value.A p-value is an estimate of the probability that a particular result, could have occurredby chance. In short the p-value is a measure of the credibility of the null-hypothesis,which is compared with a statistically significant level of typical 0.05 (five percent). Thesignificance level is the amount of evidence required to accept that an event is unlikely tohave arisen by chance. In comparison of the two, a low p-value (lower than 0.05) meansthat the null-hypothesis is unlikely to be true and the difference is statistically significant.A large p-value (larger than 0.05) means that there is no compelling evidence on which toreject the null-hypothesis. [Crawley 2012]

The probability of test participants successfully guessing the right answer was 50 percent,due to the experiment design stated in section 3.2 Experimental Design. The significancelevel was set to the standard of five percent, meaning the event is unlikely if it occurs lessthan five percent of the time. To conclude that test participants did not give the correctanswer simply by guessing, the p-value has to be below the stated significance level.

The statistical test needed to calculate this was found to be a ’binom.test()’ function in’R’, which is a program used for statistical computing. The program was chosen becauseit was presented during on of the semester courses. The ’binom.test()’ function takes thefollowing values: (number of successes, number of trials, and probability of success). Thiswas done for each of the different types of tone pairs, as presented in the dataframe infigure 3.8. The table illustrated in figure 3.10 shows tone pairs and their p-values andwere calculated as the following:

Figure 3.10: Table showing the statistical tests and their p-values.

As seen in figure 3.10 the p-values from the calculated tone pairs are below the significant

44

level of five percent. It is therefore unlikely that the results acquired from this experimentoccurred by chance. The conclusion of this statistical analysis follows in the iteration’spartial conclusion.


This section serves as a partial conclusion of the first iteration. The conclusion is basedon the knowledge gathered throughout the iteration.

The purpose of the iteration was to design and implement a part of the concept mentionedin section 2.1, Prototype, and to design and execute an experiment that could eitherprove or disprove the null-hypothesis stated in section 3.1, Hypothesis. To brush up, thehypotheses were formulated as follows:

Null-hypothesis: The target group cannot distinguish between tones of thesame or different pitch.

Alternative hypothesis: The target group can distinguish between tones ofthe same or different pitch.

During this iteration the sound interface of the prototype was implemented in PureDataas a patch. The patch was able to play MIDI tones in pairs with the same or different pitchfrom the major and chromatic musical scales. The following experiment led to results,which purpose was to either prove or disprove the null-hypothesis. If the null-hypothesiscould be disproved, it would be rejected and the alternative hypothesis would be believedto be true. Due to this statement, and the analysis gathered in figure 3.10, it can be con-cluded that the p-values of the different samples of tone pairs were lower than the chosen 5percent significance level. When comparing the two, a lower p-value means that the null-hypothesis is unlikely to be true and the difference is statistically significant. This leadsto the rejection of the null-hypothesis, causing the alternative hypothesis to be believed tobe true, which then conclusively means the target group are able to hear difference in tones.

From the analysed results in section 3.7, Results, it was chosen to use a MIDI note intervalof two. This was chosen because it has the second largest pool of answers and had anacceptable amount of incorrect answers of only seven out of a total of 66 answers, whichgave a success of 89 percent. Of the MIDI note intervals with 100 percent success, theinterval with the largest pool of answers had 22 answers in total. If each sound pair hadan equal amount of answers, the results for the tone pairs with less answers, might havelooked different. Furthermore the MIDI note with an interval of two would enable theprototype to have a bigger range of notes, if this was to be needed. It is planned in furtherdevelopment to use octaves as intervals on the y-axis. An octave has an interval of 12MIDI note numbers. A tone interval of two on the x-axis could make it easier to distin-guish the two axes from each other, than a tone interval that was closer to 12. Since this

45

is the plan, a MIDI note interval of four would provide less ways of creating the interface.

Regarding the two scales it could not be concluded if any of them were preferred overthe other. The experiment focused on the distinguishing of tones in tone pairs. Since thechromatic scale consist of all root notes the major scale is just a section of the chromaticscale, both scales have the the same notes in them. Since the test was to examine thedistinguishing of pairs the full extend of the scales was not heard and therefore was notcompared. From this and together with the chosen tone interval of two, it is decided thata certain scale will not be used in later development.

46

Second Iteration 4The second iteration will be a further development of DaS, based on the observation anddata gathering from the first iteration. This will add another iteration to the prototypedevelopment, focussing on the simplest gameplay design and the development of gameversions of DaS.

4.1 Hypothesis

In the first iteration it was discovered that the general test participant is capable ofdistinguishing between different tones played side by side. Now it would need to be testedwhether or not people can recognised direction of the sounds played on the interface. Thegame requires the players to recreate the pattern the other player made in the game,therefore it is important that the players can navigate the interface by noticing what wayone should move the finger in order to recreate the melody given.

Null-hypothesis: The target group will not be able to identify the directionof sound, that is played for them.

Alternative hypothesis: The target group will be able to identify thedirection of sound, that is played for them.


The purpose of the test was to determine, whether test participants could distinguish thedirection of the sound that was drawn. This was in order to examine whether it could bepossible for the player in a later version to hear a certain pattern that the other playerdrew. On the x-axis the notes would move from the low to the high end of the scale. They-axis changes depending on version. In one version the sound would move up and downin octaves and in another version it would increase with effects.

The design of this experiment was a complete balanced binomial design. The table infigure 4.1 illustrates the design with the two factors as well as the associated levels. Thefactor Scale had two levels, octave and effect. The factor Direction of sound had eight

47

levels, one for each possible direction. In order to meet a balanced design an even amountof samples for each sound direction were used. Three different samples of each of the eightdirection represented in the table in figure 4.1 were prepared for the test. All these sampleswere randomized in order to eliminate any bias concerning the order of presentation. AnAlternative Forced Choice (AFC) test was preformed, with eight options. [Stenholt 2012]

An AFC test is a test that forces the participants to choose one of the options given tothem. For instance if it is a 2-AFC then there’s two options, which is a kind of binomialexperiment or if it is a 3-AFC then there’s three options.

Figure 4.1: Direction of sound.


As explained in section 4.1, second iteration hypothesis, this iteration focused on creatinga sense of direction in the prototype. The interface was made in Java and PureData, usingthe tone patch created in PureData in the first iteration.

Java

Java is a programming language and computing platform first released by Sun Microsys-tems in 1995. The software is free to download and is well known throughout the world.Lots of applications including websites, mobile and TV devices have been compiled inJava, making it one of the currently most popular programming languages in use. [Java2012]

To validate the hypothesis an interface has to be made. The interface should providethe user with a sense of direction. Different versions of the interface may give differentoutcome. Therefore it could interesting in looking into different sound layouts andcomparing them to each other.


This section deals with the design development of DaS during the second iteration. Basedon the results gathered from the first iteration, further design development and changeswere made whereupon they were implemented and tested.

48

Introduction

The design consists of three parts that has been made in Java and PureData. Two of theparts were made in Java and the third in PureData. The first part in Java registers afinger on the touch screen. This is then registered on a playlist handler and is saved ina queue. Finally, the information is sent to a PureData patch, which will play a sounddepending on where the touch screen is being activated.

Octave Version

In the first design it was decided that the steps of the x-axis, should be designed withan interval of two MIDI note numbers. This was chosen from the results from the testin the first iteration. The y-axis should represent the same notes from the x-axis, but indifferent octaves. This was chosen to keep a sense of relation between the two axes, butstill making them sound different.

The first part of this design was implemented in Java, creating areas on the screen wheredifferent tones will play. These areas were arranged in a six by six pattern, covering theentire screen. This was done by dividing the length and height of the screen with six.Every square was then made to be one sixth of the screen’s height and length. A sketchof the division of the screen can be seen in figure 4.2

Figure 4.2: Division of the octave versions screen

Effect Version

The second design of the prototype dealt with an effect scale. Similar to the octave designthis had the same tone interval of two MIDI note numbers on the x-axis. However, whenmoving along the y-axis an effect would be added or removed from the tone. Three effects

49

namely chorus, flange, frequency modulator were researched to decide which effect to usein this design.

Chorus

Chorus is an effect that gives an impression that multiple voices or instruments are beingplayed in unison. The effect delays the original signal and modulate it by at least onelow frequency oscillator (LFO). The delayed and modulated signal is then mixed with theoriginal signal. The delay add a time offset whereas the LFO changes the pitch. Thequality of the chorus effect can be changes through varying the number of delays in thedesign. By using several individual delay lines and LFOs, the effect will become thickerand more complex. [Rouse 2008]

Flange

Flange is an effect that gives a swirling or sweeping sound. It combines two copies ofthe same signal. The second signal is delayed by a small amount and gradually changesperiod, which produces a swirling effect. A LFO is used to vary the speed of the copy.The oscillator moves in a range of 1-20 cycles per seconds to obtain the effect. [Rouse2005b]

Frequency Modulator

Frequency modulation is a kind of audio synthesis, where the output of one oscillator isused to modulate the pitch of another oscillator. These oscillators are called Modulatorsand Carriers. The output that is heard in a frequency modulator synthesis, is the Carrier,which is modified by the Modulator. The Modulator itself is like a LFO. It is not actuallyheard, but the effect it has on the Carrier is. [Abdullah 2012] [Wiffen 1997]

The Chosen Effect

To determine what effect to use the effects were made in PureData to hear what it wouldsound like. It was difficult to hear changes for each step on the y-axis with the chorusand flange effect. This might be because the difference in the swirling effect from a flangemay be hard to hear in a short tone like the one being produced from the tone patch inthe prototype. Frequency modulation was the effect that gave the most significant changein sound for each step on the y-axis. Frequency modulation was therefore chosen for theeffect version.

Furthermore the screen in the effect version, was arranged into a six by four pattern. Thescreen height in this design was different from the octave design, because of the effectbeing used on the y-axis. Effect values exceeding a value of 45, generated a piercing andshrill sound, that was uncomfortable to listen to. Especially MIDI note numbers greaterthan 66 with an effect value greater than 45, was unpleasant. Therefore it was decidedto keep the effect value at 45 and below with intervals of ten. This provided an amountof effect distinguishable compared to smaller intervals. Moreover it was decided to keepthe MIDI note number at 66 and below. Seen in figure 4.3 each square is associated with

50

a MIDI note number, but instead of each square having a different MIDI note number asin the octave design, this design only uses different MIDI notes in each column. Furtherdown the y-axis will add more effect to the MIDI note.

Figure 4.3: Division of the effect versions screen

Playlist Design

The second part of the prototype consists of the playlist and the functions concerning it.This part will check whether the number being added to the list is already on it or not.As it was described earlier in section 1.2, Concept, the game ends when the drawn linecrosses itself. The playlist also takes care of sending the required numbers from Java tothe PureData patch.

PureData Patch

The last part is the actual PureData patch where the tone generation occurs. The tonegeneration is based on the numbers that PureData receives from the playlist. In the octaveversion of the PureData patch, PureData needs to receive the MIDI note number of thedifferent tones it needs to play and then a bang command to play it. The effect version ofthe PureData patch, also needs to receive a MIDI note number, in addition to the bangand a number for center frequency, which control the amount of effect.

4.5 Implementation

This section is dedicated to the implementation of the second iteration of the prototype.The concept of which was described in section 2.1, Prototype Description. During thisiteration the most important task is to make the two Java applications and PureDatapatches. The Android application needs to be coded in Java, which then tells the Pure-Data patch which tones to play when the users interact with the handheld Android mobiledevice.

51

The first step in making the application, was to identify the tasks needed to be done inorder for the application to work properly. The tasks were identified as the following:

1. Create a coordinate-system that fills out the entire screen of the Android mobiledevice.

2. Divide the screen into subsections and map a unique tone to each of them.

3. Track the users finger movement across the screen and register the coordinates.

4. Check which subsections were crossed by the registered coordinates and add theirunique tones to a playlist.

5. Play all the tones added to the playlist when the finger is lifted from the screen.

6. Clear the playlist and wait for new input.

The second step was to create an Unified Modelling Language (UML) diagram, to ensurean overview of what functions were needed in order to solve the identified tasks. Figure4.4 shows how the UML diagram ended up looking.

UML

UML is a graphical notation system for designing and describing component-based andobject-oriented software. It is an open standard developed by the Object ManagementGroup (OMG). A systems’ model is graphically expressed as a diagram, which provides agood communication design. The quantity of details is depending on which mode is used.Sketch mode presents a sketch of the basic idea of the structure and behaviour of a piece ofsoftware without lots of detail. Blueprint mode emphasises on completeness and involvesdetailed illustration of design for programmers to implement. Programming languagemode provides the necessary functions for UML diagrams to automatically compile intoexecutable code to other representations like Java. [Fowler 2004]

One of the most common and useful types of UML diagram is the UML class diagram,which is used to document the static structure of the system. It describes what classesthere are and how they are related, though not how they interact to achieve particularbehaviours. In an object-oriented application, classes have attributes (member variables),operations (member functions) and relationships with other classes. The classes arerepresented with boxes which contain three parts: The upper part holds the name ofthe class The middle part contains the attributes of the class The bottom part gives themethods or operation the class can take or undertake [Fowler 2004]

To illustrate a relationship between two classes, the property can be notated as anassociation, a solid line linked between the two classes. An arrow head attached to theline shows the direction from the source class to the target class. If the arrow heard isopen-headed it indicates generalization to show that the classes have differences but alsomany similarities. The similarities can be placed in a supertype class and the differencesin subclasses [Fowler 2004].

52

Figure 4.4: UML diagram of the classes in the second iteration of DaS.

The two versions of the prototype, share a lot of functionality. The whole interface andwhich tone that should be played were primarily the same, therefore it should be put intoa super class. This however is impossible since in Java, classes only can inherit from superclass. It could be fixed by making the intended superclass as an interface, but that waythe class only inherits the method’s input parameters and return value, but everythinginside the method would have to be written both places. So convenience would not bigenough to matter. The same goes for the part that loads the PureData patch and makeseverything ready to play the patch. Not including the method that actually plays thetones.

Java

The third part in developing the application, was to write the Java Code. This was im-plemented in the program Eclipse and tested continuously on a Samsung Galaxy Tabletconnected the laptop running the program during development.

The following part below serves as a documentation of the most essential functions, but

53

also the entire code can be found in appendix F.

The following will explain parts of the code either through a flowchart or the actual Javacodes. Flowcharts has been used for explaining some parts of the code. The completecode can be seen in the appendix second iteration code.

onTouch

The OnTouch function, as seen in figure 4.5, is from the OnTouchListener interface that ispart of the Android API, which means its input parameters are fixed. It is a switch wherethree of the available cases are used. They respond to a different kind of motion event.One is when the touch is just touched, another is when the finger is dragged and the lastone is when finger is lifted. When the screen is just touched the hasDrawn a boolean thattells whether or not player is has drawn is set to false. The grands are then found. If thefinger is slided the grands will likewise be found. If the finger is lifted from the screenhasDrawn is set to true, since now the player has done drawing.Out of the switch it will check if hasDrawn is true, if it is the playlist will be played. If itis not, the sound will be added using the grands.

Figure 4.5: UML diagram of the function onTouch.

getGrands

The function getGrands takes an event MotionEvent as its input. Next it checks if the ifstatement event.getX is less than the screenWidth divided by six. If this is the case, thegrandX will be one. If this is not the case it continues down to the else if statement. Thisstates if the event.getX is less then screenWidth divided with six times two it will printout grandX is two. The reason for dividing it with six is so that the screen are dividedinto six subsections. This will be done until it reaches the last else statement, where itwill print out grandX as six if it is not one of the above. Now the exact same is done, but

54

this time with the y-axis instead. Thus converting the screen’s resolution into a six-by-sixcoordinate system. This can be seen on figure 4.6.

Figure 4.6: Figure showing division of the screen.

1 private void getGrands ( MotionEvent event ) {23 //The x−va lue s are transformed in to the grand x−a x i s .4 if ( event . getX ( ) < screenWidth /6)5 grandX = 1 ;6 else if ( event . getX ( ) < ( screenWidth /6) ∗2)7 grandX = 2 ;8 else if ( event . getX ( ) < ( screenWidth /6) ∗3)9 grandX = 3 ;

10 else if ( event . getX ( ) < ( screenWidth /6) ∗4)11 grandX = 4 ;12 else if ( event . getX ( ) < ( screenWidth /6) ∗5)13 grandX = 5 ;14 else15 grandX = 6 ;1617 // Now the same i s done , but with the y−a x i s i n s t ead .18 if ( event . getY ( ) < screenHeight /6)19 grandY = 1 ;20 else if ( event . getY ( ) < ( screenHeight /6) ∗2)21 grandY = 2 ;22 else if ( event . getY ( ) < ( screenHeight /6) ∗3)23 grandY = 3 ;24 else if ( event . getY ( ) < ( screenHeight /6) ∗4)25 grandY = 4 ;26 else if ( event . getY ( ) < ( screenHeight /6) ∗5)27 grandY = 5 ;28 else29 grandY = 6 ;30 }

55

calcSounds

The private integer calcSounds contains an if-else structure that figures out which MIDInote number is added. The different XY coordinates gives a returnInt with a specificMIDI note number attached. The grandX states a certain value and the returnInt isdetermined after the grandY value. Thus it calculates the coordinates to find the toneand returns the MIDI note number.

1 private int calcSounds ( ) {2 int returnInt = 0 ;34 // This mess f i g u r e s out which note i s added5 if ( grandX == 1) {6 if ( grandY == 1)7 returnInt = 96 ;8 else if ( grandY == 2)9 returnInt = 84 ;

10 else if ( grandY == 3)11 returnInt = 72 ;12 else if ( grandY == 4)13 returnInt = 60 ;14 else if ( grandY == 5)15 returnInt = 48 ;16 else17 returnInt = 36 ;1819 } else if ( grandX == 2) {20 if ( grandY == 1)21 returnInt = 98 ;22 else if ( grandY == 2)23 returnInt = 86 ;24 else if ( grandY == 3)25 returnInt = 74 ;26 else if ( grandY == 4)27 returnInt = 62 ;28 else if ( grandY == 5)29 returnInt = 50 ;30 else31 returnInt = 38 ;3233 . . .3435 } else {36 if ( grandY == 1)37 returnInt = 106 ;38 else if ( grandY == 2)39 returnInt = 94 ;40 else if ( grandY == 3)41 returnInt = 82 ;42 else if ( grandY == 4)43 returnInt = 70 ;44 else if ( grandY == 5)45 returnInt = 58 ;46 else47 returnInt = 46 ;

56

48 }4950 return returnInt ; // The midi note i s returned .51 }

addSound

The function addSound, as seen in figure 4.7, has the purpose of checking whether thecurrent number was just added, meaning that the current area of the touch screen hasalready been registered (e.g. the last number on the playlist being the same as the onethat is trying to get added). If that number was not just added, then it checks if it hadbeen added earlier, meaning that the drawn path has crossed over itself. If both theseturn out to be false, that is, if the number had not been added before, then is is addedthe current tone number to the playlist. The way this works is that a new touch inputis registered, checking first whether the number has just been added. If it turns out thatthe tone has not been added, the statement is considered true and the next part of thecode is run through, checking whether the tone number has ever been added, and if thisturns out to have never happened, the statement is true and the tone number is added tothe playlist.

Figure 4.7: UML diagram of the function addSound.

playPlayList

The function playPlayList, as seen in figure 4.8, runs through a for-loop for each entry inthe playList, meaning the number of tones that the list contains. During the loop a tonegets triggered and then the function sleeps for 750ms before playing the next. This makesall the tones play with some pause between each of them, otherwise they will be playedalmost at the same time, making it too fast to tell them apart. When the loop is done,meaning every note in the playList have been accounted for, the playList gets cleared sothat it is empty for the next user input.

57

Figure 4.8: UML diagram of the function playPlayList.

PureData

The fourth and last step in developing the application was to make the PureData patchthat could receive messages from the implemented android application.

Octave Version

The octave version was similar to the tone patch described in section 3.5 Implementationfor the first iteration. Instead of defining the MIDI note number in the PureData patch,this design has an object called [r trigger] for receiving the bang from Java called trigger.The octave design also has an object called [r midinote] for receiving the MIDI note numberactivated in the given Java square and send to PureData. 4.9

Figure 4.9: Octave Version.

Effect Version

The effect version used a PureData audio example called sweepable frequency modulator.This example uses a two cosine carrier signal, which cross-fades between harmonics andcreates continuously variable center frequencies. The following explanation of PureDataobjects can be followed on figure 4.10.The quotient for the center frequency is as the output of a [line∼] object. The line isthen connected to [samphold∼] object which control the center frequency and makes itpossible to avoid discontinuities in the output signal even if the desired center frequencychanges discontinuously. The [line∼] object is then separated into its fractional part bythe [wrap∼] object and into its integer part by subtracting the fractional part from the

58

original. The fractional part is labelled q and the integer is labelled k, which also can beseen in the figure. The fundamental frequency is made from a phaser, which is a sawtoothwave that is multiplied by k and k 1. Hereafter the modulator is added to both phases andthen the cosines of both phases are taken, c1 and c2. After this the next couple of objectsends up computing the weighted sum, pc1 qc2. c1 and c2 is the two sinusoids mentionedbefore and q is the fractional part as mentioned in the top, p is p=1 − q. This gives themovable frequency carrier signal seen at the end q∗c2 (1−q)∗c1. Similar to octave designthis was also implemented with [r trigger] and [r midinote] objects, but as an extension italso has an object called [r effect], which control the amount of effect on the tone whenmoving on the y-axis. [Puckette 2006]

Figure 4.10: Effect Version.

4.6 Experiment

The tests for the second iteration of DaS were carried out on a sample of the students atAalborg University. In order to work with as many participants as possible the tests forsecond iteration were performed in two similar test groups in separate rooms, each witha tablet with early version of the prototype installed. It was also of importance that theenvironment was a quiet space for better concentration. This time the sample size wasas many as possible during the three days testing. In order to end up with a completerandom sample any student near the testing location were asked if they would like to

59

participate.

Two separate iteration tests were designed, one using octaves on the tablet and anotherusing effect scale. Each shared the same procedure during testing and following is adescription of how a test section was performed. This test ended up with 14 randomlyselected test participants from different study backgrounds. First, every participant got ashort introduction to the prototype’s functions and was then questioned about their musicexperience. Each participant got a few minutes to become familiar with the interface onthe tablet by tracing their finger against the screen. Hereafter the test participant waspresented with some tasks wherein 24 pre-recorded sound samples were played from acomputer. The test participants were then supposed to determine the direction of thesound. The participants were allowed to listen to the melodies after the initial hearingif needed. In order to acquire feedback from the test participants, a questionnaire wereused. This questionnaire can be seen in appendix A. The answers were written down on aquestion sheet by the participants themselves during the test. The figure 4.11 illustratesthe test situations.

(a) Getting familiar with the tones. (b) Test in continual melodies and writing down answers.

Figure 4.11: Two test situations.

4.7 Results

This section focus on the results of the second iteration test with the use of the octavesscale and the effect scale.

Octave scale

A complete list of the octave scale results can be seen in appendix B. It shows the uniquedirections with arrows pointing in the eight directions the sounds could be moving. Eachdirection is identified with a unique PureData number. The total answers together withthe correct and incorrect answers are also stated. The test results from the octave designshowed that, out of the 336 possible right answers, a total of 146 questions were answered

60

correctly. The true probability of success is decided by the chance of answering the ques-tion correct, which is 1/8 and is therefore 0.125. The significance level is at five percent.Plotting the data in binom.test in ’R’, following results occurred:

binom.test(146,336,0.125)p-value = 2.2e −16sample estimates probability of success is 0.434538

The total of correct answers was 43.45 percent. The p-value was calculated as 2.2e−16.The p-value, which is below five percent, states that this result was unlikely to haveoccurred by chance.

A full list of the binom.tests and their p-values can be seen in appendix D. As long asthe p-value for the associated number of successes stays below five percent the likelihoodfor correct answer by chance is low. One sound task needs at least five out of 14 correctanswers to be considered unlikely to have occurred by chance. Looking at the amount ofincorrect answers and high p-values, incorrect answers often occurred when the sound wentin a diagonally direction, though with the exception of diagonally up-right. This suggestthat the participants felt greater insecurity when deciding on a diagonally direction anddid somewhat well with vertical and horizontal lines.

Effect scale

The other test results from the effect design can be seen in appendix C and show that, outof the 336 possible right answers, a total of 70 questions were correctly answered. Again’R’ was used to analyse the data in a binom.test with a true probability of success at 0.125and significance level at five percent. Following results occurred:

binom.test (70,336,0.125)p-value = 1.578e−05sample estimates probability of success is 0.2083333

The total of correct answers for this design was 20,83 percent. The p-value calculatedwas 1.578e − 05, which is also below five percent. This result was also unlikely to haveoccurred by chance, though an acceptable p-value under five percent only occurred fourtimes out of 24. A full list of the binom.tests and every p-value for the effect scale test isshown in appendix E.

A great majority of the answers given were incorrect and this occurred in any directions.An example would be tasks number 16, 22, 23 and 24 were only one or no answered cor-rectly. This experiment suggests that DaS with an effect scale is too difficult to interactwith as it brings up a lot of insecurity.

61

When comparing the test results from the two second iteration tests, a difference can beseen between the total result of the correct answers. With 43,45 percent up against 20,83percent the design using octaves had a better success rate, than the design using effects.Tones in a diagonally direction turned out to be difficult to define for many test partic-ipants. An error could be uncertainty about whether the combination of tones soundedlike moving up scale and then be mistaken for a diagonally up scale direction. None of thetest participants were aware about the randomized order or that every direction occurredan equal amount of times due to the complete design. This could be a factor in how someof the answers were pure guesses.

Since the test results of both designs were unlikely to have occurred by chance, the designwith the highest probability of success was chosen. The octave design is therefore chosenfor further use in the prototype.


Following is the partial conclusion of the second iteration. The following conclusion willhave focus on the test result and examine if the hypothesis will be reject or not. Thepurpose was to test whether people were able to differentiate between sounds played in acontinual melody.

The hypothesis were as follows:

Null-hypothesis: The target group will not be able to identify the directionof sound, that is played for them on a pc.

Alternative hypothesis: The target group will be able to identify thedirection of sound, that is played for them on a pc.

The sound interface of the prototype was again implemented in PureData. It was of in-terest to examine whether an interface with use of octaves or one with the use of effectswas the most successful. In both cases test participants were able to distinguish some ofthe directions played for them. It turned out that both design used in the two tests hada p-value lower than the significance level of five percent and therefore unlikely to haveoccurred by chance. Since the results gathered from the test was unlikely to have occurredby chance, it can be concluded that the test participants did not get the right directionby guessing alone. The null-hypothesis was thereby rejected as the target group were ableto identify the direction of sound.

The octave version was chosen for further development due to a higher success rate of43.45 percent compared to the effect versions 20.83 percent. Even though the test par-ticipants could not distinguish the direction 50 percent of the time, the octave version isused for the next prototype. Keeping in mind that the non-visual help options mentioned

62

in section 2.1 Prototype Description, might improve the success rate.

As the test was performed it was discovered that the lowest row in the octave version,were too deep a note to be heard through the tablets speakers. The row was, as seen insection 4.4 Prototype Design figure 4.2, made up from the MIDI note numbers 36-46. Forthe following iteration this row will be removed.

63

Third Iteration 5This chapter concentrates on the third and final iteration of DaS. Explanations on howthe design of the different game modes and the experiment work will be elaborated. Newmethods namely ANOVA, will be used to calculate the results of the third test. Finallythe null-hypothesis will be either confirmed or rejected.

5.1 Hypothesis

This experiment focuses on the difference between the prototypes game versions andtherefore uses the main hypotheses from section 2.3 Hypothesis.

Null-hypothesis: In creating a game based mainly on auditory feedback,there is no difference in the mean scores between game versions with ’Fullgraphics’, ’Start point’, ’End point’ and ’No graphics’.

Alternative hypothesis: In creating a game based mainly on auditoryfeedback, there is a significant difference in the mean scores between gameversions with ’Full graphics’, ’Start point’, ’End point’ and ’No graphics’.


The purpose of the experiment was to examine the target group’s ability to use the tabletin the four different game modes. The results should make it clear how well the taskswere done with or without the help of visual aids. Like the previous test, this iterationwas performed as complete balanced design, but this time with a single factor. In thatway the individual test participants tried every combination in an equal amount of times.The single factor was described as ’Game Mode’. This factor could further be divided intofour modes which were ’full graphics’, ’start point’, ’end point’ and ’no graphics’ which isillustrated in figure 5.1. The number 30 displayed in the figure are the amount of repetitionfor each game mode. In statistics, 30 are a commonly used amount of participants forscientific test.[Crawley 2012] At the end of the game round, a score was calculated in theimplemented score system. All score info were to be collected by the facilitator, at theend of each round.

65

Figure 5.1: Table showing the different game modes.


As mentioned in section 1.3 Perception, the function of sight is a highly superior humansense and is often working together with the auditory sense, for example when observinga moving object. This experiment do not involve a moving object, but a visual drawnpath on a screen which gives a unique output with the respective sound the path touches.The experiment is made to check the participant’s capabilities of relying on audio only bycomparing the average game scores for the tries with various level help and without.

The experiment featured an octave scale with visual help and feedback. The game taskwas to listen to a string of tones and then recreate the tones by using the sound as guid-ance. The game levels increased in difficulty with in one level visually showing the wholedrawn path, another level showed the beginning or the end point and finally no visualhelp at all.

The following are visual features that must, as mentioned in the hypothesis, beimplemented:

• Score system keeping track of the participant’s score.• Replay function to make it possible for the participant to hear the tones again

but with a limit of three times.• Starting/ending point would give the participant the possibility to see where the

path begins or ends.• No graphics is a that hides the whole path.• Full graphics is a function that showed the whole path.

The reason for having a scoring system is not only for the entertainment value of compe-tition in the game. It also serves as a means for rating the acuity of the players hearing,as it rates the player on how close to the original tones they get. The player should notbe able to draw a single path that hits all tones on the screen, and thereby also hittingall the right tones ending up with 100 percent accuracy.

The replay function is an additional tool in order to help the player recreate the originaltones, this will, as mentioned before, not be available for an unlimited amount of tries.This option must be available while playing so the players do not have to stop their gamein order to replay the tones.

66

The different game modes are as mentioned in section 2.1 Prototype Description no graph-ics, with the start point given, with the ending point and full path. Each of these modesare designed in order to create a prototype capable of falsifying the null-hypothesis.


This section builds on the analysis in section 5.3 Prototype Analysis and the conceptdescribed in section 2.1 Prototype Description. The section is divided into two smallerparts: Bill Verplank model and Interface Design, which together makes up the designphase of the iteration.

Bill Verplank model

Starting off, the concept from section 2.1 Prototype Description is to be extended, sothat all the game elements needed for this iteration are accounted for. This is a resultof developing the second iteration of DaS into a more finished version of a game, whichis to be implemented during this third and for now, final iteration for this project. Theextended concept will be explained below through the use of a Bill Verplank model, asseen in figure 5.2. The model should be read from left to right, where the columns wouldbe called motivations, meanings, modes and mappings. The process from left to rightmight involve observation, invention, engineering and appearance. [Verplank 2009]

Figure 5.2: Concept illustrated in an Bill Verplank model.

The model acts as a template for the design process, and provides an overview throughthe use of the eight sketches. An explanation of what the sketches represent follows below:

67

Idea: The idea represents the concept described in section 2.1 Prototype Description.

Error: The main goal is to give the players a rating based on how "good" they are athearing and recreating unique tones.

Metaphor: The game symbolises an instrument in the way it plays tones as a result ofphysical interaction.

Scenario: A scenario could be that two players want to challenge each other to see how"good" they are at identifying tones.

Model: The game will run as an Android application on a tablet device, so that playerscan hide the touch screen from each other, when drawing the path the opposingplayer has to guess. A tablet also offers great freedom to play the game in manyenvironments, which further increases the overall value of the platform.

Task: The game task is to translate finger movements across the touch screen into aunique sound pattern specific to that path of movement. It should also be easy andsimple to play the game, in such way that no complicated interface management isrequired in order to do the tasks needed to play the game.

Control: The interface should be finger friendly, such that interactive elements havean appropriate size compared to human fingertips. The interface will be furtherexplained in the next part of this section.

Display: The visual feedback will be displayed on the tablet’s touch screen, which shouldmake it intuitive to draw paths directly onto the screen in the same dimension asthe game plays the corresponding tones. The game should have an interface thatdisplays the necessary buttons in order to access common game features, such asgame modes.

Interface Design

Based on the concept explained in the Bill Verplank model, this part serves as a moredetailed design description of the game’s interface. Before documenting the needed gameinterfaces, a walkthrough follows of how to access the game.

The first step in playing DaS, is to install it on a device compatible with the specificationslisted in section 2.2 Specification. This is done executing the .apk file (Android applicationpackage), which is created from the Java Code implemented in Eclipse. When the DaSapplication is installed, it can be launched from the tablets application tab, desktop orsimilar, depending on where it was installed. Figure 5.3 illustrates how it might look ona Samsung Galaxy Tablet, with the DaS application icon placed onto the desktop.

68

Figure 5.3: Tablet desktop.

When the icon is clicked, the game launches, and this is where the interface design begins.In order for the game to be fully playable by players, also those not familiar with theconcept from section 2.1 Prototype Description, there has to be interfaces illustrating thegames goal and mechanics. The following is a walkthrough of the designed interfaces forDaS:

Title Screen: When the game icon is clicked, this screen pops up. The screen stayson for a few seconds and then ’Game Menu’ replaces the title screen. Figure 5.4illustrates the title screen.

Figure 5.4: Title screen displaying the name of the application and its logo.

Game Menu: This is the main menu of the game, containing the title at the top andbuttons below. The buttons are: ’Start, ’Instructions’, and ’Exit’. Figure 5.5 showsan illustration of the game menu. The ’Start’ button opens up to a choice of ’Singleplayer’ or ’Multiplayer’. When either one is clicked the menu asks: ’Play withgraphics’ or ’Play without graphics’. If ’Play with graphics’ is selected, three gamemodes pops up to choose from: ’Full graphics’, ’Start point’, and ’End point’. Eachof the buttons starts a game. When ’Play without graphics’ is selected the gamemode: No graphics, is launched.

69

Figure 5.5: Game Menu.

Instructions: The instructions screen is accessed from either from the ’Game Menu’or the ’Pop-up Menu’. The screen explains all there is to know about the gamemechanics. Figure 5.6 shows a section of the instructions screen, which can bescrolled down in the right side of the screen.

Figure 5.6: Instructions.

Game Starts (Blue/Red): When a game mode has been selected from the ’GameMenu’ this screen is active, see figure 5.7. This screen acts like a pop-up whichpurpose is to let the players know whose turn it is to draw. The blue player alwaysstarts the first round, and the red player always guesses in the first round. Thescreen is active for a few seconds or until the screen is clicked.

(a) Blue players turn. (b) Red players turn.

Figure 5.7: The game starts by telling whose turn it is to draw.

70

Drawing Screen (Blue/Red): After the ’Game Starts’ screen is done, this screenbegins. Here the players have a clean canvas to draw their paths. The borderof the canvas can either be blue or red, ensuring that it is always clear whose turn itis. First the blue player should draw a path that the red player has to guess. Whenthe blue player has drawn his path, the ’Game Starts (Red)’ opens, and it is now redplayers turn when the ’Drawing Screen (Red)’ opens to guess how the blues pathlooked like. See figure 5.8.

(a) Blue player. (b) Red player.

Figure 5.8: Drawing Screen.

Scoring Screen: When the red player has drawn a path he thinks creates the sametones as the blue player, this screen is active. First the blue player’s path fadesonto the screen. When the blue path is fully visible, the red player’s path fadesonto the screen in red, covering the blue path if they are in the same locations.This illustrates where the, until now invisible paths, were drawn during the drawingstates of the game round. When the two paths are fully visible, a window pops upcalculating the score, see figure 5.9. The score is based on how many of the originaltones are hit, versus how many incorrect tones are hit. When the window is clicked,the ’Game Starts (Red)’ state gets active. This changes the order, so the red playerstarts in the next game. This pattern continues as long as the players desire to playthe game.

Figure 5.9: Scoring Screen.

71

Pop-up Menu: When the option button on the tablet is clicked, this menu pops up.From this menu, four buttons are accessible: ’Exit’, ’Game Menu’, ’Instructions’,and ’Playback (3,2,1,0)’; see figure 5.10. ’Exit’ closes the game application, ’GameMenu’ opens the ’Game Menu’ ending the current game, ’Instructions’ opens theinstructions page, and ’Playback (3,2,1,0)’ plays the sound to be recreated. Eachtime the button is clicked the number gets subtracted by 1 until it reaches 0, thenthe button greys out and gets inactive until the next round.

Figure 5.10: Pop-up Menu illustrated in a new game, where no playbacks have been used.

5.5 Implementation

This section is devoted to document the final implementation of the developed prototype,DaS. The section is divided into the following smaller parts: Game Code, and Game In-terfaces. First the Java code presented.

The code for this iteration was an expansion of the code from the second iteration. Almostevery line has been kept. Some of it is, however, has been moved around, because of thegraphics requiring a new view. Instead of the TextView used in iteration two, a new classthat inherits from View was made. Which is seen in figure 5.11 on the facing page. Alsothis uses more activities since there is the main activity, showing the main menu and thegame activity, containing the game. The game activity initializes the draw view were allthe "magic" happens. Most of the methods from before, except the ones needed to playPureData tones, are present here along with some new methods to show graphics andcalculate the score. Two of them will be mentioned here under, one method that showshow the graphics are drawn and the method that calculates the score.

72

Game Code

Figure 5.11: UML diagram of the classes in the third iteration of DaS.

73

drawPoint

1 private void drawPoint ( float cx , float cy , String color , Canvas canvas ) {23 pointPaint . setAntiAlias ( true ) ;4 pointPaint . setColor ( chosenColor ( color ) ) ;56 canvas . drawCircle ( cx , cy , 10 , pointPaint ) ;7 }

This method is used when drawing the start- and the end point. It takes the position, acolor as a string and the canvas as input parameters. It is worth noting that the pointPaintof the type Paint has been declared and instantiated prior to the method is called. Thepaint receives some attributes changes to whether it will feature anti aliasing or not andwhat color it should be. Then the canvas draws a circle using the position, a fixed diameterand the paint that has just been configured.

CalcScore

Figure 5.12: UML diagram of the function calculating the scores.

This code block, as seen in figure 5.12, is the method in charge of calculating how close thesecond player got at replicating the first player’s path. It begins with checking whether ornot the second player’s playlist is bigger than the first player’s. If it is, then the difference

74

is added to the misses counter.

Then through two for loops and an if statement, the method checks for each entry on thefirst player’s list with each entry on the second player’s list. If one of them is a "hit" thehits counter will be counted up and the loop will move on to the next entry. On the otherhand if none of them hits then it is counted as a miss. Then if the number of hits is abovezero, the result will be calculated by dividing the number of hits with the sum of bothhits and misses so it shows the percentage of correct fields touched. The result is thendisplayed on the screen.

The implemented code can be seen in appendix G.

Game Interface

Following the documentation of the Java code implemented, is a presentation of how DaS’sinterfaces was created. The interfaces follow the design descriptions mentioned in InterfaceDesign, but with some differences. The explanation of these differences follows below, ac-companied with in-game screenshots edited onto a drawing of the ’Samsung Galaxy Tablet’.

The ’Game Menu’, as seen in figure 5.13, illustrates the implemented menu for DaS. Themenu served more as a practical tool rather than a menu with graphical contribution tothe aesthetics of the game. The use of composition between the graphical elements andthe buttons, as well as different font usage for the title, is something which was left outin this early version of the game.

Figure 5.13: Game Menu.

The game mode ’Playground’ was introduced as a tool for the experiment following insection 5.6 Experiment, giving the test participants a breather to get familiar with the

75

game before playing for scores.

When clicking on ’Start Game with No Visual Feedback’ the game mode with no graphicslaunches. At this stage of the game, there was no indication of which player should start,so this would be up for a facilitator to state for future test participants if needed. Figures5.14 and 5.15 shows the different game states involving this game mode with no graphicsto indicate where the path was drawn. The first state is the blue player drawing his pathand hearing the tones making of that specific pattern in the grid.

An example on a finished path can be seen in figure 5.14a, where the player starteddrawing from right to left. The white rectangle positioned at the end of the path,illustrates a highlight of the grid mapping the lastly added MIDI tone to the playlist.Throughout the drawing of the path, each new section of the grid was highlightedwhen the path was being drawn within its section. When crossing from one sectionin the grid to another, the rectangle would disappear from the first and appear at thenext section. This visual feedback was added to help make a crossbridge between thelocation of the MIDI tones added to the playlist and the organic shapes being drawn.It was believed to be unnecessarily confusing for new players to understand wherethe crossover between tones were located. This problem was decided to be not causedby lack of auditory acuity, but more the lack of knowledge about where the sound changed.

The second state, as seen in figure 5.14b, represents the handover of the tablet from theblue player to the red player. When the red player is presented with the tablet, he cannotsee anything on the screen, and therefore has to rely solely on his ability to identify thetones played to him through his auditory perception. The pop-up menu lets him playbackthe tones three times, but the explanation and illustration of the menu follows later infigure 5.22 on page 80.

(a) Blue player drew his path. (b) Red player gets the tablet and is presentedwith no visual feedback.

Figure 5.14: Starting a game with no visual feedback.

76

The third state, as seen in figure 5.15a, illustrates the red player drawing of where itis believed that the blue player drew. When the red player lifts the finger, the pathis fully drawn and the tones get played. When the tones are finished playing, thefourth and final state of the round is active. This state illustrates where both pathswere drawn and a score in yellow text appears in the top left corner, as seen in figure 5.15b.

(a) Red player draws his path, where he thinks theblue player drew.

(b) Red player gets his score.

Figure 5.15: Ending a game with no visual feedback.

Figures 5.16 and 5.17 illustrates the game mode selected with ’Start Game with StartingPoint’ from the menu. The states in this game mode are similar to the ones describedabove, but here the red player is assisted with a blue dot, representing the start point ofthe blue players path. This can be seen in figure 5.16b.

(a) Blue player drew his path. (b) Red player gets the tablet and is presentedwith a start point.

Figure 5.16: Starting a game with start point.

77

When he tries to recreate the blue players path, he knows where to begin as seen in figure5.17a. In this hypothetical example, it helped him in such an extent that he, as seen infigure 5.17b, got a score of 77. This is simulated example.

(a) Red player draws his path, where he thinks theblue player drew, but in this game mode he knowswhere to start.


Figure 5.17: Ending a game with start point.

Figures 5.18 and 5.19 illustrates a game with an end point, accessed from the menuthrough the button ’Start Game with Ending Point’. Once again the game states aresimilar to the above, only changing the dot to red. This indicates that it is the end pointof the blue players path.

(a) Blue player drew his path. (b) Red player gets the tablet and is presentedwith an end point.

Figure 5.18: Starting a game with end point.

78

(a) Red player draws his path, where he thinks theblue player drew.


Figure 5.19: Ending a game with end point.

Figures 5.20 and 5.21 illustrates a game with full visual feedback. This final game modeis accessed through the ’Start Game with Full Path’ button in the menu. When the redplayer is presented with the tablet, he has a full visual feedback showing where the blueplayer drew, as illustrated in figures 5.20a and 5.20b.

(a) Blue player drew his path. (b) Red player gets the tablet and is presentedwith a full path.

Figure 5.20: Starting a game with full visual feedback.

79

This should make it easy to recreate the path, as illustrated in figure 5.21a, making itpossible to compare the difference in scores achieved with with no or limited feedback,to this form of full visual feedback. The result of this game mode is reflected in figure5.21b, where the score was calculated to 100 in the simulated example.

(a) Red player draws his path onto the blue playerpath.


Figure 5.21: Ending a game with full visual feedback.

Across all game modes, the pop-up menu in figure 5.22, can be accesses from the optionsbottom below the touch screen as described in Interface Design. The pop-up menu givesaccess to the ’Playback’ button, also described in Interface Design, and the ’Go to MainMenu’ ends the current game and enters the menu illustrated in figure 5.13 on page 75.

Figure 5.22: Popup Menu.

With the game interfaces documented and implemented, DaS is ready for the iterationsexperiment, which follows in the next section.

5.6 Experiment

Like the previous experiments the sample population was set to be our target group. Itis a useful rule of thumb that a sample of 30 or more is a big sample, while a sampleof less than 30 is a small one. This is a frequently used amount of participants in manystatistical experiments. [Crawley 2012] Due to time concerns the experiment was dividedinto a pilot test and a future planned test. The pilot test was performed with five random

80

picked Medialogy students and the results was analysed for this rapport. For the futureplanned experiment, it has been chosen to involve a total of 30 test participants unlikethe previous experiments where the sample was as many as possible. In order to try outanother variety of age and experience, the experiment will be performed at a middle schoolwith pupils in the eight to ninth grade.

The test participants were introduced in how to use the tablet with installed version ofDaS. The task was to listen carefully to sets of continual tones designed with the usageof all four game levels, as seen in figure 5.23. The participant was allowed to listen to thetones up to three times. To submit an answer they had to draw of at least two fields.When the answer for one task was drawn the result was shown. At the end the facilitatorchecked the tablet’s output display and registered the points given.

Figure 5.23: Participant drawing his answer and receives a score.

The test progressed as follows; The test participants were given about two minutes tofamiliarize themselves with the layout of DaS, instructions here were given to put emphasison experimenting with sounds moving in different directions and familiarizing themselveswith the sounds it made. When the test participants felt they had experimented enough,or the limit of two minutes had been reached, the test participants were given theassignments, in randomized order. The order of the four different game modes variedfrom participant to participant, and the order of assignments for each game mode variedas well. The assignments consisted of a single sound drawing each, and there were threeassignments to each game mode. The assignments were three shapes, a line, an L shapeand a V shape. The test participants chosen for the small test were people from differentsemesters of the Medialogy education, as it was deemed to be sufficiently representingwhen this test was a precursor to the future planned test which will provide answers froma possible more diverse group of participants.

5.7 Results

This section describes the analysis of the results gathered in the experiment. During theexperiment three paths were tested on a Line, a path shaped like the letter L and oneshaped like the letter V. In this section all the calculations will feature the numbers forthe data from the line, the end results of the two others will be mentioned at the end. Inappendix LULZ every single calculation can be found. In appendix H all the results fromthe pilot test is shown.

81

Figure 5.24: Figure showing the results gathered from the third iteration pilot test.

Analysis of Variance

A method that will not depend on the number of pairs of factor levels is called analysisof variance (ANOVA). Since the current experiment is a single factor experiment it iscalled one-way ANOVA. It’s a method for analysing treatment means through the use ofvariance. In order to do so, the sum-of-squares is needed. [Crawley 2012]

The sum-of-squares is the sum of the mean subtracted from each observation and theyare squared before the summed up. The formula for the sum-of-squares can be seen inthe equation below.

Σ(y −y)2 (5.1)

ANOVA uses some variables that must be assigned values the variable k which is thenumber of treatments with n observations. So in this case:k = 4 and n =5.So, N , the total number of treatments must be N = k ·n= 20.

Sum-of-Squares

Now the grand mean can be found, which will be used in later calculations. The grandmean is the mean of all the observations, it can be seen in calculation 5.2.

yi=Σk

i1Σnj1yij

N =77.7 (5.2)

With the grand mean found, the total sum-of-squares (SSY ) can be found. SSY is thesum of the sum-of-squares for all the observations, using the the grand mean to calculatethe variance.

SSY =Σki1Σ

nj1(yij −yi)2=13,292.2 (5.3)

Now the sum-of-squares within groups (SSE) is found in calculation 5.4. SSE is the sumof the sum-of-squares for each observation, using their own mean value and not the grand

82

mean.

SSE=Σki1Σ

nj1(yij −yi)2=10,669.6 (5.4)

SSY is actually also equal to SSE + SST , where SST is the sum-of-squares betweengroups. SSE and SST are later on in the next part. SST can be found by subtractingSSE from SSY .

SST=SSY −SSE=13,292.2−10,669.6=2,622.6 (5.5)

With both SSE and SSY found the F -test can be undertaken.

F -test

The critical value of F from the F -distribution is in this case 2.87. This critical value ofF is a value which the F -ratios that follows is compared to.To calculate the F -ratios that is needed, both the degrees of freedom (df) for the betweengroups and within groups must be found.

dfT=n−1=4−1=3 (5.6)

dfE=N −k=20−4=16 (5.7)

Two results, where one later needs to be divided by other, is needed to find the F -ratio.These results are found by dividing the Sum-of-Squares for within- and between groups(calculation 5.5 and 5.4) by their degrees of freedom found in calculation 5.6 and 5.7.

SST

dfT=2622.6

3 =874.2 (5.8)

SSE

dfE=10669.6

16 =666.85 (5.9)

The F -ratio can then be found using the results from calculation 5.8 and 5.9 in thiscalculation:

F= 874.2666.85=1.31 (5.10)

The F -ratio from calculation 5.10 is then compared to the critical value of F that were2.87. Here for the line the F -ratio ended up being less than the critical value of F . In twoF -ratios for the L-shaped path and for the V-shaped path ended up being larger than thecritical value of F , their F -ratio being 2.953 and 3.817 respectively.

83


The third iteration of the project has now been completed and the prototype has beentested on a small group of people in order to get a hint of the final results. From the resultscalculated in the previous section, the null-hypothesis can in some cases be rejected. Whenthe test participant had to draw a simple line, the extra levels of visual feedback did notimpact the score as much as in the other two patterns. For the line the null-hypothesiscould not be rejected, since every test participant found the task relatively easy. Themore complex shapes, L and V, proved to be more difficult. With these patterns thenull-hypothesis is rejected and the visual feedback helped the test participants score ashigh as they did.

84

Part III

Review

85

Project Evaluation 6The project evaluation will evaluate the project throughout discussion, conclusion andfurther development.

6.1 Discussion

In the following section some of the choices that were made during the project will bediscussed with focus on what was done and what could have been done differently.

As the theme for this semester involved sound, the aim was to create a product wheresound was in focus rather than the visual part or any other element. As mentioned insection 3.3 PureData was used for the sound part of DaS. There would be many possi-bilities if it was experimented with other sound samples and combinations rather thanjust octave and effect scale and tried to include tones associated with music instruments.Given more time, further variety as well as other kinds of sound might have been devel-oped. Such an alternative combination of tones or another version could might prove tobe more distinguishable than the ones used in the project. Moreover another effect couldhave been chosen for the development of the interface of the effect version. Another effectbeside chorus, flange and frequency modulation, could have been a wah effect or reverb.An alternative effect could might prove to be easier to distinguish directions in the effectversion, on the other hand it could also prove to be less successful.

Additionally it can be discussed whether or not the tests were carried out properly. Smallmistakes might have occurred during the experiments, for example if the introductionswere not specific enough during the second iteration experiment. The consequence of thiswas that the participants did not fully understand the importance of the first step offamiliarizing themselves with DaS. This lead to further confusion, making it harder forthe participants later on, when they had to try to work out the different directions. Theparticipants were initially told that they had some minutes to familiarizing themselveswith DaS, it can therefore be discussed if this confusion or lack of following instructionscould have been avoided, if what they were asked to do had been more explicitly pointedout or if it would have been the same.

The project group was inexperienced, because it was the first time using statistical analy-

87

sis methods, therefore it can be discussed if more efficient ways has been overlooked. Thequestion here is; were the right tools used to analyse the data? Or should it have beenother statically tools and methods instead of ANOVA? It can further be questioned if itwas correct to block the elements that were blocked, randomize what was randomized andso forth.

Furthermore it was planned that all test participants were tested independently. It was ofimportance that the individual participants did not learn from each other, but it would beunavoidable if the experience and suggestions on task solutions went by mouth to mouthfrom a former participant to a new one and thereby influencing each other. As mentionedin section 4.2, second iteration experiment design, the solution was to play the sound pairsin random order. By doing so the results were not influenced by test participants’ knowl-edge of the task beforehand. For better individuality the test participants could have beenselected randomly from the street or other public places. Since the target group is fairlybroad, any person with intact hearing would be able to participate, but selecting thiswould be time consuming and could instead be used in later development of an iterationtest.

Wrapping up it becomes obvious that making and using test as validation of a project isnot a simple matter and that it takes both time and practice to properly present all theaspects and thus producing a reliable test that actually test all the elements as wanted ina correct manner and without errors.

6.2 Conclusion

Problem statement:

How can a sound based game be developed that focuses on recognition of tonesthrough interaction with a touch screen and how can an experiment be designedthat is able to test the success of recognition?

The project DaS revolved around developing a game based on tone recognition and withthis in mind, a prototype was developed. This prototype is an example of a game fo-cusing on recognition of tones. The application runs on an Android tablet, which has atouch screen. Multiple experiments were designed in order to develop the prototype andultimately to falsify or fail to do the same, to the overall null-hypothesis. The initiatingproblem statement was that the visual sense was far superior to the auditory, therefore theoverall null-hypothesis became that, using DaS, there would not be a significant differencein the score between levels of visual aid. However as the results of the third iteration’sexperiment turned out, there were a significant difference in score when anything else buta straight line was drawn. Based on the pilot test the null-hypothesis must be rejectedfor more intricate patterns and it can be concluded that the auditory sense is indeed infe-rior to the visual, however these results may change with the future planned experiment.However, before it can be said that the auditory sense is so inferior that it is infeasible to

88

create games for it, further testing with alternate prototypes is needed.

The requirements of the prototype were placed into the MoSCoW method in order toget a quick and easy overview of what was needed and which priority it had. Duringthe project’s three iterations everything in the MoSCoW’s ’Must have’ and ’Should have’sections were implemented, providing a functional prototype for the experiments.

MoSCoW Method:

Must have

• Sounds that play when the application is used. (Fulfilled)• Sounds generated from user interaction. (Fulfilled)• A scoring system that illustrates how well players are doing at identifying sounds.

(Fulfilled)

Should have

• A click feature to hear sounds in the grid. (Fulfilled)• A playback button that lets players hear previous sounds three times. (Fulfilled)• All the different game modes mentioned in the concept. (Fulfilled)• Start and exit buttons. (Fulfilled)

Could have

• An instructions screen. (Not fulfilled)• A game menu with access to: New game, Instructions, Highscore, Exit game. (Not

fulfilled)

Won’t have

• A highscore system. (Not included)• Multiplayer mode with more than two players. (Not included)• Multiple tone-scales to choose from. (Not included)

The items in Could have were not necessary in order for the prototype to function, howevera basic menu system was implemented as a means of changing between the different gamemodes.

6.3 Further Development

This section deals with ideas for further development of the prototype. The results ofthe project lays the groundwork for this research and further development on music-basedgames.

89

Another way of designing the prototype of this project could be to design the sound layouton the interface in another way. This alternative design could depend more on creatingtones or sounds from the coordinates from the user input, rather than having entire sec-tions dedicated to one tone. This could produce a more extensive sound catalogue, makingthe shift in tones smaller and blend the sound more together as a whole. This would re-quire a more recognizable sound system and sound mapping, so that even relatively smallcoordinate changes could be heard. For this alternative design a more extensive scoringsystem would be required. Such an alternative score system could evaluate the playeron the distance of how close the players path was to the original path, together with theorder the path was drawn.

Even further than this project, the ideas and solutions in this report could be expandedto more elaborate sound systems, such as surround-sound systems which could then beused for further gaming on other platforms than tablets, for example using the motioncontrols for home consoles.Another further development could aim the prototype for a specific age group, for exam-ple having a design for young children. A design like this could feature different animalsounds where different version of the game could have either jungle, farm or pet theme.

It was suggested during a testing of the prototype that the different levels of visual feed-back could be added to the game, depending on the player’s performance. Meaning if theplayer has trouble recreating the tones, additional levels of visual feedback will be addedstep by step.

Moreover, other alternatives to visual feedback could be implemented in a later version.One of such alternatives could be to incorporate tactile feedback. This tactile feedbackcould for an example be used to define the current grid design of the screen, letting theplayer feel the different sections of the grid in form of vibrations. Another example forthe tactile feedback could be an indication of how close the player recreating the soundpath, was to the original sound path.

In order to improve upon the design of the DaS prototype, regarding the testing of the hy-pothesis, it would be relevant to research the effect of, perhaps, more recognizable sounds.These sounds could for example be the tones, designed to sound like the strings of a guitaror the keys of a piano. This could perhaps cause the user to remember the tones easierand thus make it less difficult to reproduce the patterns given.

In order to conclusively test if the auditory acuity of players are great enough to designgames for, further testing will have to incorporate other games and game styles than DaS,as it may be that some games or game styles are more viable as sound games than others.

90

Bibliography

T. Y. Abdullah (2012). ‘F.M. Synthesis’. http://yala.freeservers.com/2fmsynth.htm.

Activision (2005-2010). ‘Guitar Hero’. http://hub.guitarhero.com/.

P. Brinkmann (2012). Making Musical Apps. O’Reilly Media, Inc. Pages 20-21.

M. J. Crawley (2012). Statistics - An introduction using R. John Wiley & Sons, Ltd.Pages 3-5, 9, 33-46, 73-83, 84-85, 156-163, 269-280.

E. Estrella (2012a). ‘Chromatic Scale’.http://musiced.about.com/od/lessonsandtips/qt/chromatic.htm.

E. Estrella (2012b). ‘Major Scales’.http://musiced.about.com/od/lessonsandtips/qt/scales.htm.

M. Fowler (2004). UML Distilled - A Brief Guide to the Standard Object ModelingLanguage, Third edition. Pearson Education, Inc Pages 1-2, 35-38.

Google Play (2011). ‘Music Composer (Lite)’.https://play.google.com/store/apps/details?id=com.happynoteslite.

B. Harris (2008). ‘Sound Waves’. http://www.thedawstudio.com/Tips/Soundwaves.html.

Java (2012). ‘What is Java technology?’.http://www.java.com/en/download/faq/whatis_java.xml.

P. Kirn (2010). ‘Development Experience on Android and Beyond’.http://createdigitalmusic.com/2010/05/ethereal-dialpad-touch-app-development-experience-on-android-and-beyond/.

J. Kreidler (2009). ‘Programming Electronic Music in Pd’.http://www.pd-tutorial.com/english/ch03s05.html.

L. B. Larsen (2011). ‘Interaction Design / HCI. Pages 20-22’.http://mea.create.aau.dk/file.php/74/IDI11_MM1.pdf.

O. S. Larsen (2010). ‘Psykologiens veje iBog’.http://ibog.psykologiensveje.systime.dk/index.php?id=1534.

A. P.-C. R. A. Nina Kraus, Erika Skoe (2009). ‘Experience-induced Malleability inNeural Encoding of Pitch, Timbre and Timing’.http://onlinelibrary.wiley.com/doi/10.1111/j.1749-6632.2009.04549.x/pdf.

93

J. R. Pomerantz (2003). Encyclopedia of Cognitive Science. London: Nature PublishingGroup. Volume 3. Page 527-537.

M. Puckette (2006). ‘Two-cosine carrier signal’.http://crca.ucsd.edu/ msp/techniques/latest/book-html/node100.html.

puredata.info (2012). ‘Pure Data’. http://puredata.info/.

QA INTERNATIONAL (2012a). ‘Human being - Sense organs’.http://visual.merriam-webster.com/human-being/sense-organs.php.

QA INTERNATIONAL (2012b). ‘Video game’.http://www.merriam-webster.com/dictionary/video

Reactable Systems (2012). ‘Reactable’. http://www.reactable.com.

P. T. Robert Snowden & T. Troscianko (2006). Basic Vision ’An Introduction to theVisual Perception’. Oxford University press. First Edition. Pages 13-14, 20-32.

M. Rouse (2005a). ‘Cosine wave’. http://whatis.techtarget.com/definition/cosine-wave.

M. Rouse (2005b). ‘Flanging’. http://whatis.techtarget.com/definition/flanging.

M. Rouse (2008). ‘Chorus’. http://whatis.techtarget.com/definition/chorus.

L. Shams (2011). ‘Even Unconsciously, Sound Helps Us See, UCLA psychologists report’.http://www.psychologicalscience.org/index.php/news/releases/even-unconsciously-sound-helps-us-see.html.

Sony Computer Entertainment Europe (2010). ‘SingStar’. http://www.singstar.com/.

R. Stenholt (2012). ‘Design and Analysis of Experiments: Binomial and CountingExperiments. Page 17’.http://sict.moodle.aau.dk/file.php/694/Course_Material_subject_11/MM11_-_Binomial_Counting_Experiments.pdf.

D. M. Szaflarski (2012). ‘How We See: The First Steps of Human Vision’.http://www.accessexcellence.org/AE/AEC/CC/vision_background.php.

B. Verplank (2009). ‘Interaction Design Sketchbook’.http://www.billverplank.com/IxDSketchBook.pdf.

R. M. Warren (1999). Auditory Perception. Cambridge University Press.

P. Wiffen (1997). ‘Synth School, Part 3: Digital Synthesis (FM, PD & VPM)’.http://www.soundonsound.com/sos/1997_articles/sep97/synthschool3.html.

C. Wittmann (2005). ‘MIDI Note Number, Frequency Table’.http://tonalsoft.com/pub/news/pitch-bend.aspx.

94

List of Figure References

If no figure reference is noted here, the figure was created by the project group.

Figure 1.1: http://electronics.howstuffworks.com/guitar-hero2.htm

Figure 1.2: [Warren 1999]

Figure 1.3: [Robert Snowden & Troscianko 2006]

Figure 1.4: http://kidshealth.org/kid/htbw/eyes.html

Figure 1.5: http://www.zzounds.com/item--KORKO1http://octoba.net/archives/20100429-ethereal-dialpad-android-587.htmlhttps://lh4.ggpht.com/GOY-q34HHCohm0UI2FfLYVe5GxNyktsa3XJdgGHBwcovICRyPFZiqrZeVsqe63JGmUuH

Figure 1.6: https://play.google.com/store/apps/details?id=com.happynoteslite

Figure 1.7: http://www.reactable.com/images/live2_cover2.jpg

Figure 2.1: Slides from the Interaction Design course by Lars Bo Larsen AssociateProfessor at Aalborg University.

Figure 3.2: http://tonalsoft.com/pub/news/pitch-bend.aspx

Figure 3.3: http://www.thedawstudio.com/Tips/Soundwaves.html

95

http://electronics.howstuffworks.com/guitar-hero2.htm

http://kidshealth.org/kid/htbw/eyes.html

http://www.zzounds.com/item--KORKO1

http://octoba.net/archives/20100429-ethereal-dialpad-android-587.html

https://lh4.ggpht.com/GOY-q34HHCohm0UI2FfLYVe5GxNyktsa3XJdgGHBwcovICRyPFZiqrZeVsqe63JGmUuH

https://play.google.com/store/apps/details?id=com.happynoteslite

http://www.reactable.com/images/live2_cover2.jpg

http://tonalsoft.com/pub/news/pitch-bend.aspx

http://www.thedawstudio.com/Tips/Soundwaves.html

Second Iteration -Experiment Questionnaire A

97

Music experience

We would like you to state how much music experience you have.

Little/no musical experience: I have never or only played musical instruments during primary and lower

secondary school or high school.

Some musical experience: I play or used to play a musical instrument for a longer period.

Very musical experienced: I have participated in music classes and play one or several musical instruments

on a regular basis.

Little/no musical experience Some musical experience Very musical experienced

Sound direction questionnaire

We would like you to hear some sound and then write which direction you think that they may come from.

The direction can come from following directions:

Thank you for your time and participation :)

Second Iteration - OctaveTable B

103

Second Iteration - EffectTable C

105

Second Iteration - OctaveResults D

107

Second Iteration - EffectResults E

109

Second Iteration - JavaCode F

1 package com . sound ;23 import java . io . File ;4 import java . io . IOException ;5 import java . util . ArrayList ;67 import org . puredata . android . io . AudioParameters ;8 import org . puredata . android . io . PdAudio ;9 import org . puredata . android . utils . PdUiDispatcher ;

10 import org . puredata . core . PdBase ;11 import org . puredata . core . utils . IoUtils ;1213 import android . app . Activity ;14 import android . content . pm . ActivityInfo ;15 import android . os . Bundle ;16 import android . util . DisplayMetrics ;17 import android . util . Log ;18 import android . view . MotionEvent ;19 import android . view . View ;20 import android . view . View . OnTouchListener ;21 import android . view . Window ;22 import android . view . WindowManager ;23 import android . widget . TextView ;2425 public class SoundSomethingActivity extends Activity implements ←↩

OnTouchListener {2627 private static final String TAG = "Draw a Sound" ;28 private PdUiDispatcher dispatcher ;2930 private static ArrayList<Integer> playList = new ArrayList<Integer >() ;31 boolean hasDrawn = false ;3233 TextView textView ;3435 // These o b j e c t s are f o r c a l c u l a t i n g the scveen s i z e36 int grandX = 0 , grandY = 0 , screenWidth , screenHeight ;37 DisplayMetrics dm = new DisplayMetrics ( ) ;38

111

3940 /∗∗ Cal led when the a c t i v i t y i s f i r s t c r ea ted . ∗/41 @Override42 public void onCreate ( Bundle savedInstanceState ) {43 super . onCreate ( savedInstanceState ) ;4445 // F u l l s c r e e n46 requestWindowFeature ( Window . FEATURE_NO_TITLE ) ;47 getWindow ( ) . setFlags ( WindowManager . LayoutParams . FLAG_FULLSCREEN ,48 WindowManager . LayoutParams . FLAG_FULLSCREEN ) ;495051 //Lock landscape o r i e n t a t i o n52 super . setRequestedOrientation ( ActivityInfo .←↩

SCREEN_ORIENTATION_LANDSCAPE ) ;5354 initGui ( ) ;5556 try {57 initPd ( ) ;58 loadPatch ( ) ;59 } catch ( IOException e ) {60 Log . e ( TAG , e . toString ( ) ) ;61 finish ( ) ;62 }63 }6465 private void initGui ( ) {6667 // Here the scveen dimensions are found .68 getWindowManager ( ) . getDefaultDisplay ( ) . getMetrics ( dm ) ;69 screenHeight = dm . heightPixels ;70 screenWidth = dm . widthPixels ;7172 textView = new TextView ( this ) ;73 textView . setText ( "" ) ;74 textView . setOnTouchListener ( this ) ;75 setContentView ( textView ) ;7677 }7879 private void initPd ( ) throws IOException {8081 // Conf igure the audio g lue82 int sampleRate = AudioParameters . suggestSampleRate ( ) ;83 PdAudio . initAudio ( sampleRate , 0 , 2 , 8 , true ) ;8485 // Create and i n s t a l l the d i spa t che r86 dispatcher = new PdUiDispatcher ( ) ;87 PdBase . setReceiver ( dispatcher ) ;88 }8990 private void loadPatch ( ) throws IOException {91 // Loads the PureData patch c a l l e d tone . pd92 File dir = getFilesDir ( ) ;93 IoUtils . extractZipResource ( getResources ( ) . openRawResource ( R . raw .←↩

tone ) , dir , true ) ;

112

94 File patchFile = new File ( dir , "tone.pd" ) ;95 PdBase . openPatch ( patchFile . getAbsolutePath ( ) ) ;96 }9798 private void triggerNote ( int n ) {99 // Sends the midi note and a bang to the pure data patch .

100 PdBase . sendFloat ( "midinote" , n ) ;101 PdBase . sendBang ( "trigger" ) ;102 }103104 public void playPlayList ( ) throws InterruptedException{105 // Plays the tones on the play l i s t one at a time106 for ( int i = 0 ; i < playList . size ( ) ; i++){107 triggerNote ( playList . get ( i ) ) ;108 Thread . sleep (750) ;109 }110 playList . clear ( ) ;111112 }113114 public static void addSound ( int sound ) {115 // Adds the sound to the play l i s t i f i t e l i g i b l e116 if ( ! isLastSound ( sound ) )117 if ( ! isSoundAdded ( sound ) )118 playList . add ( sound ) ;119120 }121122 private static boolean isSoundAdded ( int sound ) {123 // Checks i f the note i s a l r eady on the play l i s t124 boolean rtrn = false ;125 for ( int i = 0 ; i < playList . size ( ) ; i++){126 if ( playList . get ( i ) == sound )127 rtrn = true ;128 else129 break ;130 }131 return rtrn ;132 }133134 private static boolean isLastSound ( int sound ) {135 // Checks i f the l a s t sound i s the same as the cur rent one136 if ( playList . size ( ) > 0) {137 int previous = playList . get ( playList . size ( )−1) ;138139 if ( sound == previous )140 return true ;141 else142 return false ;143 } else144 return false ;145 }146147 // PureData funct i ons , j u s t l e ave them be .148 @Override149 protected void onResume ( ) {150 super . onResume ( ) ;

113

151 PdAudio . startAudio ( this ) ;152 }153154 @Override155 protected void onPause ( ) {156 super . onPause ( ) ;157 PdAudio . stopAudio ( ) ;158 }159160 @Override161 public void onDestroy ( ) {162 super . onDestroy ( ) ;163 PdAudio . release ( ) ;164 PdBase . release ( ) ;165 }166167 public boolean onTouch ( View v , MotionEvent event ) {168169 switch ( event . getAction ( ) ) {170 case MotionEvent . ACTION_DOWN :171 hasDrawn = false ;172 getGrands ( event ) ;173 break ;174 case MotionEvent . ACTION_MOVE :175 getGrands ( event ) ;176 break ;177 case MotionEvent . ACTION_CANCEL :178 break ;179 case MotionEvent . ACTION_UP :180 hasDrawn = true ;181 break ;182 }183184 if ( hasDrawn ) {185186 // The play l i s t w i l l be played i f the f i n g e r i s l i f t e d187 try {188 playPlayList ( ) ;189 } catch ( InterruptedException e ) {190 e . printStackTrace ( ) ;191 }192 }193 else {194 addSound ( calcSounds ( ) ) ;195 }196197 return true ;198 }199200 private void getGrands ( MotionEvent event ) {201 //The x−va lue s are transformed in to the grand x−a x i s .202 if ( event . getX ( ) < screenWidth /6)203 grandX = 1 ;204 else if ( event . getX ( ) < ( screenWidth /6) ∗2)205 grandX = 2 ;206 else if ( event . getX ( ) < ( screenWidth /6) ∗3)207 grandX = 3 ;

114

208 else if ( event . getX ( ) < ( screenWidth /6) ∗4)209 grandX = 4 ;210 else if ( event . getX ( ) < ( screenWidth /6) ∗5)211 grandX = 5 ;212 else213 grandX = 6 ;214215 // Now the same i s done , but with the y−a x i s i n s t ead .216 if ( event . getY ( ) < screenHeight /6)217 grandY = 1 ;218 else if ( event . getY ( ) < ( screenHeight /6) ∗2)219 grandY = 2 ;220 else if ( event . getY ( ) < ( screenHeight /6) ∗3)221 grandY = 3 ;222 else if ( event . getY ( ) < ( screenHeight /6) ∗4)223 grandY = 4 ;224 else if ( event . getY ( ) < ( screenHeight /6) ∗5)225 grandY = 5 ;226 else227 grandY = 6 ;228229 }230231 private int calcSounds ( ) {232 int returnInt = 0 ;233234 // This mess f i g u r e s out which note i s added235 if ( grandX == 1) {236 if ( grandY == 1)237 returnInt = 96 ;238 else if ( grandY == 2)239 returnInt = 84 ;240 else if ( grandY == 3)241 returnInt = 72 ;242 else if ( grandY == 4)243 returnInt = 60 ;244 else if ( grandY == 5)245 returnInt = 48 ;246 else247 returnInt = 36 ;248249250 } else if ( grandX == 2) {251 if ( grandY == 1)252 returnInt = 98 ;253 else if ( grandY == 2)254 returnInt = 86 ;255 else if ( grandY == 3)256 returnInt = 74 ;257 else if ( grandY == 4)258 returnInt = 62 ;259 else if ( grandY == 5)260 returnInt = 50 ;261 else262 returnInt = 38 ;263264

115

265266 } else if ( grandX == 3) {267 if ( grandY == 1)268 returnInt = 100 ;269 else if ( grandY == 2)270 returnInt = 88 ;271 else if ( grandY == 3)272 returnInt = 76 ;273 else if ( grandY == 4)274 returnInt = 64 ;275 else if ( grandY == 5)276 returnInt = 52 ;277 else278 returnInt = 40 ;279280281282 } else if ( grandX == 4) {283 if ( grandY == 1)284 returnInt = 102 ;285 else if ( grandY == 2)286 returnInt = 90 ;287 else if ( grandY == 3)288 returnInt = 78 ;289 else if ( grandY == 4)290 returnInt = 66 ;291 else if ( grandY == 5)292 returnInt = 54 ;293 else294 returnInt = 42 ;295296297298 } else if ( grandX == 5) {299 if ( grandY == 1)300 returnInt = 104 ;301 else if ( grandY == 2)302 returnInt = 92 ;303 else if ( grandY == 3)304 returnInt = 80 ;305 else if ( grandY == 4)306 returnInt = 68 ;307 else if ( grandY == 5)308 returnInt = 56 ;309 else310 returnInt = 44 ;311312313314 } else {315 if ( grandY == 1)316 returnInt = 106 ;317 else if ( grandY == 2)318 returnInt = 94 ;319 else if ( grandY == 3)320 returnInt = 82 ;321 else if ( grandY == 4)

116

322 returnInt = 70 ;323 else if ( grandY == 5)324 returnInt = 58 ;325 else326 returnInt = 46 ;327 }328329 return returnInt ; // The midi note i s returned .330 }331332 }

117

Third Iteration - JavaCode G

DrawtestActivity.java

1 package com . draw ;23 import android . app . Activity ;4 import android . content . Intent ;5 import android . content . pm . ActivityInfo ;6 import android . os . Bundle ;7 import android . view . View ;8 import android . widget . Button ;9

10 public class DrawtestActivity extends Activity {111213 /∗∗ Cal led when the a c t i v i t y i s f i r s t c r ea ted . ∗/14 @Override15 public void onCreate ( Bundle savedInstanceState ) {16 super . onCreate ( savedInstanceState ) ;171819 super . setRequestedOrientation ( ActivityInfo .←↩

SCREEN_ORIENTATION_PORTRAIT ) ;20 setContentView ( R . layout . main ) ;2122 Button bNo = ( Button ) findViewById ( R . id . StartNO ) ;2324 bNo . setOnClickListener ( new View . OnClickListener ( ) {2526 public void onClick ( View v ) {27 startActivity ( new Intent ( "com.draw.GAMENO" ) ) ;2829 }30 }) ;3132 Button bStart = ( Button ) findViewById ( R . id . StartStart ) ;33 bStart . setOnClickListener ( new View . OnClickListener ( ) {3435 public void onClick ( View v ) {36 startActivity ( new Intent ( "com.draw.GAMESTART" ) ) ;

119

3738 }39 }) ;4041 Button bEnd = ( Button ) findViewById ( R . id . StartEnd ) ;42 bEnd . setOnClickListener ( new View . OnClickListener ( ) {4344 public void onClick ( View v ) {45 startActivity ( new Intent ( "com.draw.GAMEEND" ) ) ;4647 }48 }) ;4950 Button bFull = ( Button ) findViewById ( R . id . StartFull ) ;51 bFull . setOnClickListener ( new View . OnClickListener ( ) {5253 public void onClick ( View v ) {54 startActivity ( new Intent ( "com.draw.GAMEFULL" ) ) ;5556 }57 }) ;5859 }6061 }

drawView.java

1 package com . draw ;23 import java . util . ArrayList ;45 import android . content . Context ;6 import android . graphics . Canvas ;7 import android . graphics . Color ;8 import android . graphics . Paint ;9 import android . graphics . Path ;

10 import android . graphics . RectF ;11 import android . view . MotionEvent ;12 import android . view . View ;1314 public class drawView extends View{1516 static float rectX1 , rectX2 , rectY1 , rectY2 ;17 static float centX , centY ;181920 // Paints and paths21 Paint pointPaint = new Paint ( ) ;22 Paint rectPaint = new Paint ( ) ;23 Paint paint = new Paint ( ) ;24 Paint p1Paint = new Paint ( ) ;25 Paint p2Paint = new Paint ( ) ;26 private Path path = new Path ( ) ;27 private Path p1Path = new Path ( ) ;28 private Path p2Path = new Path ( ) ;

120

2930 static boolean drawBox = false ;3132 String str = "" ;3334 static boolean draw , start = false , end = false , full = false ;3536 static int turn , playbacks ;3738 static float boxX1 = 0 , boxX2 = 100 , boxY1 = 0 , boxY2 = 100 ;3940 int grandX = 0 , grandY = 0 , screenWidth , screenHeight ;41 static int fryser = 0 ;4243 private static ArrayList<Integer> playList = new ArrayList<Integer >() ;44 private static ArrayList<Integer> currentPlayList = new ArrayList<←↩

Integer >() ;45 private static ArrayList<Integer> lastPlayList = new ArrayList<Integer←↩

>() ;46 private static ArrayList<Point> lolList = new ArrayList<Point >() ;47 private static ArrayList<Point> lolList2 = new ArrayList<Point >() ;4849 @SuppressWarnings ( "static -access" )50 public drawView ( Context context , int x , int y , boolean start , boolean ←↩

end , boolean full ) {51 super ( context ) ;5253 screenHeight = x ;54 screenWidth = y ;5556 this . start = start ;57 this . end = end ;58 this . full = full ;5960 turn = 1 ;61 playbacks = 0 ;6263 lolList . clear ( ) ;64 lolList2 . clear ( ) ;6566 }6768 // GUI S t u f f Goes here69 @Override70 protected void onDraw ( Canvas canvas ) {71 if ( turn == 3) {72 debug ( str , canvas ) ;7374 }7576 if ( drawBox )77 drawBox ( boxX1 , boxX2 , boxY1 , boxY2 , canvas ) ;7879 paint . setAntiAlias ( true ) ;80 paint . setStrokeWidth (6 f ) ;81 paint . setColor ( Color . WHITE ) ;82 paint . setStyle ( Paint . Style . STROKE ) ;

121

83 paint . setStrokeJoin ( Paint . Join . ROUND ) ;84 canvas . drawPath ( path , paint ) ;8586 if ( turn == 1) {87 p1Path . rewind ( ) ;88 p2Path . rewind ( ) ;8990 } else if ( turn == 2 ) {91 if ( start )92 drawPoint ( lolList . get (0 ) . getX ( ) , lolList . get (0 ) . getY ( ) , "BLUE" , ←↩

canvas ) ;93 else if ( end )94 drawPoint ( lolList . get ( lolList . size ( )−1) . getX ( ) , lolList . get (←↩

lolList . size ( )−1) . getY ( ) , "RED" , canvas ) ;95 else if ( full ) {9697 p1Path . moveTo ( lolList . get (0 ) . getX ( ) , lolList . get (0 ) . getY ( ) ) ;98 for ( int i = 1 ; i < lolList . size ( ) ; i++){99 p1Path . lineTo ( lolList . get ( i ) . getX ( ) , lolList . get ( i ) . getY ( ) ) ;

100 }101 p1Paint . setAntiAlias ( true ) ;102 p1Paint . setStrokeWidth (6 f ) ;103 p1Paint . setColor ( Color . GRAY ) ;104 p1Paint . setStyle ( Paint . Style . STROKE ) ;105 p1Paint . setStrokeJoin ( Paint . Join . ROUND ) ;106 canvas . drawPath ( p1Path , p1Paint ) ;107 }108109 } else if ( turn == 3) {110111 p1Path . moveTo ( lolList . get (0 ) . getX ( ) , lolList . get (0 ) . getY ( ) ) ;112 for ( int i = 1 ; i < lolList . size ( ) ; i++){113 p1Path . lineTo ( lolList . get ( i ) . getX ( ) , lolList . get ( i ) . getY ( ) ) ;114 }115 p1Paint . setAntiAlias ( true ) ;116 p1Paint . setStrokeWidth (6 f ) ;117 p1Paint . setColor ( Color . BLUE ) ;118 p1Paint . setStyle ( Paint . Style . STROKE ) ;119 p1Paint . setStrokeJoin ( Paint . Join . ROUND ) ;120 canvas . drawPath ( p1Path , p1Paint ) ;121122 p2Path . moveTo ( lolList2 . get (0 ) . getX ( ) , lolList2 . get (0 ) . getY ( ) ) ;123 for ( int i = 1 ; i < lolList2 . size ( ) ; i++){124 p2Path . lineTo ( lolList2 . get ( i ) . getX ( ) , lolList2 . get ( i ) . getY ( ) ) ;125 }126 p2Paint . setAntiAlias ( true ) ;127 p2Paint . setStrokeWidth (6 f ) ;128 p2Paint . setColor ( Color . RED ) ;129 p2Paint . setStyle ( Paint . Style . STROKE ) ;130 p2Paint . setStrokeJoin ( Paint . Join . ROUND ) ;131 canvas . drawPath ( p2Path , p2Paint ) ;132133 }134135 }136137 public int chosenColor ( String inputString ) {

122

138139 if ( inputString == "BLUE" ) {140 return Color . BLUE ;141 }142 else if ( inputString == "RED" ) {143 return Color . RED ;144 }145 else146 return Color . TRANSPARENT ;147148 }149150 public void drawPoint ( float cx , float cy , String color , Canvas canvas ) {151152 pointPaint . setAntiAlias ( true ) ;153 pointPaint . setColor ( chosenColor ( color ) ) ;154155 canvas . drawCircle ( cx , cy , 10 , pointPaint ) ;156 }157158 public void drawBox ( float x1 , float x2 , float y1 , float y2 , Canvas ←↩

canvas ) {159160 RectF rect = new RectF ( x1 , y1 , x2 , y2 ) ;161162 rectPaint . setStyle ( Paint . Style . STROKE ) ;163 rectPaint . setAntiAlias ( true ) ;164 rectPaint . setColor ( Color . WHITE ) ;165 rectPaint . setStrokeWidth (3 f ) ;166167 canvas . drawRect ( rect , rectPaint ) ;168 }169170 private void debug ( String text , Canvas canvas ) {171 Paint textPaint = new Paint ( ) ;172 textPaint . setColor ( Color . YELLOW ) ;173 canvas . drawText ( text , 0 , 50 , textPaint ) ;174 }175176 public void moveTo ( float x , float y ) {177 path . moveTo (x , y ) ;178 }179180 public void lineTo ( float x , float y ) {181 path . lineTo (x , y ) ;182 }183184 // Touch Event i s here185 @Override186 public boolean onTouchEvent ( MotionEvent event ) {187 float eventX = event . getX ( ) ;188 float eventY = event . getY ( ) ;189190 switch ( event . getAction ( ) ) {191 case MotionEvent . ACTION_DOWN :192 drawBox = true ;193 draw = true ;

123

194 if ( draw && turn != 3) {195 path . moveTo ( eventX , eventY ) ;196 getGrands ( event ) ;197 addSound ( calcSounds ( ) ) ;198 if ( turn == 1) {199 fryser++;200 lolList . add ( new Point ( eventX , eventY ) ) ;201 } else if ( turn == 2) {202 fryser++;203 lolList2 . add ( new Point ( eventX , eventY ) ) ;204 }205 }206207 return true ;208 case MotionEvent . ACTION_MOVE :209 drawBox = true ;210 if ( draw ) {211 path . lineTo ( eventX , eventY ) ;212 getGrands ( event ) ;213 addSound ( calcSounds ( ) ) ;214 if ( turn == 1) {215 fryser++;216 lolList . add ( new Point ( eventX , eventY ) ) ;217 } else if ( turn == 2) {218 fryser++;219 lolList2 . add ( new Point ( eventX , eventY ) ) ;220 }221 }222 break ;223 case MotionEvent . ACTION_UP :224 draw = false ;225 drawBox = false ;226 path . rewind ( ) ;227 break ;228 default :229 return false ;230231 }232233 if ( draw == false ) {234235 try {236 playPlayList ( ) ;237 } catch ( InterruptedException e ) {238 e . printStackTrace ( ) ;239 }240 }241242 // Schedules a r epa in t .243 invalidate ( ) ;244 return true ;245 }246247 // Grid S t u f f248 private void getGrands ( MotionEvent event ) {249 //The x−va lue s are transformed in to the grand x−a x i s .250 if ( event . getX ( ) < screenHeight /6) {

124

251 grandX = 1 ;252 boxX1 = 0 ;253 boxX2 = ( screenHeight /6) ;254 } else if ( event . getX ( ) < ( screenHeight /6) ∗2) {255 grandX = 2 ;256 boxX1 = ( screenHeight /6) ;257 boxX2 = ( screenHeight /6) ∗2 ;258 } else if ( event . getX ( ) < ( screenHeight /6) ∗3) {259 grandX = 3 ;260 boxX1 = ( screenHeight /6) ∗2 ;261 boxX2 = ( screenHeight /6) ∗3 ;262 } else if ( event . getX ( ) < ( screenHeight /6) ∗4) {263 grandX = 4 ;264 boxX1 = ( screenHeight /6) ∗3 ;265 boxX2 = ( screenHeight /6) ∗4 ;266 } else if ( event . getX ( ) < ( screenHeight /6) ∗5) {267 grandX = 5 ;268 boxX1 = ( screenHeight /6) ∗4 ;269 boxX2 = ( screenHeight /6) ∗5 ;270 } else{271 grandX = 6 ;272 boxX1 = ( screenHeight /6) ∗5 ;273 boxX2 = ( screenHeight ) ;274 }275 // Now the same i s done , but with the y−a x i s i n s t ead .276 if ( event . getY ( ) < screenWidth /5) {277 grandY = 1 ;278 boxY1 = 0 ;279 boxY2 = ( screenWidth /5) ;280 } else if ( event . getY ( ) < ( screenWidth /5) ∗2) {281 grandY = 2 ;282 boxY1 = ( screenWidth /5) ;283 boxY2 = ( screenWidth /5) ∗2 ;284 }else if ( event . getY ( ) < ( screenWidth /5) ∗3) {285 grandY = 3 ;286 boxY1 = ( screenWidth /5) ∗2 ;287 boxY2 = ( screenWidth /5) ∗3 ;288 }else if ( event . getY ( ) < ( screenWidth /5) ∗4) {289 grandY = 4 ;290 boxY1 = ( screenWidth /5) ∗3 ;291 boxY2 = ( screenWidth /5) ∗4 ;292 }else{293 grandY = 5 ;294 boxY1 = ( screenWidth /5) ∗4 ;295 boxY2 = ( screenWidth ) ;296 }297 }298299 // L i s t handl ing300 private void addSound ( int sound ) {301 if ( ! isLastSound ( sound ) ) {302 if ( ! isSoundAdded ( sound ) ) {303 playList . add ( sound ) ;304 }305 }306 }307

125

308 private static boolean isLastSound ( int sound ) {309 // Checks i f the l a s t sound i s the same as the cur rent one310 if ( playList . size ( ) > 0) {311 int previous = playList . get ( playList . size ( )−1) ;312313 if ( sound == previous )314 return true ;315 else316 return false ;317 } else318 return false ;319 }320321 private static boolean isSoundAdded ( int sound ) {322 // Checks i f the note i s a l r eady on the play l i s t323 boolean rtrn = false ;324 for ( int i = 0 ; i < playList . size ( ) ; i++){325 if ( playList . get ( i ) == sound ) {326 draw = false ;327 rtrn = true ;328 }329 else330 break ;331 }332 return rtrn ;333 }334335 private int calcSounds ( ) {336 int returnInt = 0 ;337338 // This mess f i g u r e s out which note i s added339 if ( grandX == 1) {340 if ( grandY == 1)341 returnInt = 96 ;342 else if ( grandY == 2)343 returnInt = 84 ;344 else if ( grandY == 3)345 returnInt = 72 ;346 else if ( grandY == 4)347 returnInt = 60 ;348 else if ( grandY == 5)349 returnInt = 48 ;350351352 } else if ( grandX == 2) {353 if ( grandY == 1)354 returnInt = 98 ;355 else if ( grandY == 2)356 returnInt = 86 ;357 else if ( grandY == 3)358 returnInt = 74 ;359 else if ( grandY == 4)360 returnInt = 62 ;361 else if ( grandY == 5)362 returnInt = 50 ;363364

126

365366 } else if ( grandX == 3) {367 if ( grandY == 1)368 returnInt = 100 ;369 else if ( grandY == 2)370 returnInt = 88 ;371 else if ( grandY == 3)372 returnInt = 76 ;373 else if ( grandY == 4)374 returnInt = 64 ;375 else if ( grandY == 5)376 returnInt = 52 ;377378379380 } else if ( grandX == 4) {381 if ( grandY == 1)382 returnInt = 102 ;383 else if ( grandY == 2)384 returnInt = 90 ;385 else if ( grandY == 3)386 returnInt = 78 ;387 else if ( grandY == 4)388 returnInt = 66 ;389 else if ( grandY == 5)390 returnInt = 54 ;391392393394 } else if ( grandX == 5) {395 if ( grandY == 1)396 returnInt = 104 ;397 else if ( grandY == 2)398 returnInt = 92 ;399 else if ( grandY == 3)400 returnInt = 80 ;401 else if ( grandY == 4)402 returnInt = 68 ;403 else if ( grandY == 5)404 returnInt = 56 ;405406407408 } else {409 if ( grandY == 1)410 returnInt = 106 ;411 else if ( grandY == 2)412 returnInt = 94 ;413 else if ( grandY == 3)414 returnInt = 82 ;415 else if ( grandY == 4)416 returnInt = 70 ;417 else if ( grandY == 5)418 returnInt = 58 ;419 }420421 return returnInt ; // The midi note i s returned .

127

422 }423424 public void playPlayList ( ) throws InterruptedException{425426 if ( playList . size ( ) == 1) {427 new Sound ( playList . get (0 ) ) ; // Plays the one item on the p l a y L i s t428 Thread . sleep (750) ; // Waits f o r some m i l l i s e c o n d s b e f o r e cont inu ing .429 playList . clear ( ) ; // Clears the play l i s t so i t i s empty .430 if ( turn == 1) {431432 } else if ( turn == 2) {433 for ( int i = 0 ; i < fryser ; i++){434 lolList2 . remove ( lolList2 . size ( )−1) ;435 }436 fryser = 0 ;437 }438439 } else {440 fryser = 0 ;441 for ( int i = 0 ; i < playList . size ( ) ; i++){442 if ( turn !=1 && ! full ) {443 full = true ;444 }445 new Sound ( playList . get ( i ) ) ;446 Thread . sleep (750) ;447 }448449 changeList ( playList , currentPlayList ) ;450451 playList . clear ( ) ;452453 if ( turn != 1) {454455 calcScore ( ) ;456 } else if ( turn == 1) {457 lastPlayList . clear ( ) ;458 changeList ( currentPlayList , lastPlayList ) ;459 }460461 currentPlayList . clear ( ) ;462463 changeTurn ( ) ;464465 }466 }467468 public void changeList ( ArrayList<Integer> a , ArrayList<Integer> b ) {469470 for ( int i = 0 ; i < a . size ( ) ; i++){471 b . add ( a . get ( i ) ) ;472 }473 }474475 public void changePointList ( ArrayList<Point> a , ArrayList<Point> b ) {476477 for ( int i = 0 ; i < a . size ( ) ; i++){478 b . add ( a . get ( i ) ) ;

128

479 }480 }481482 // Game s t u f f483 public void changeTurn ( ) {484 if ( turn == 1) {485 turn = 2 ;486 } else if ( turn == 2) {487 turn = 3 ;488 } else if ( turn == 3) {489 turn = 1 ;490 lolList . clear ( ) ;491 lolList2 . clear ( ) ;492 playbacks = 0 ;493 }494 }495496 public void playback ( ) throws InterruptedException{497 if ( playbacks < 3 && turn == 2) {498 for ( int i = 0 ; i < lastPlayList . size ( ) ; i++ ) {499 new Sound ( lastPlayList . get ( i ) ) ;500 Thread . sleep (750) ;501 }502 playbacks++;503 }504 }505506 private void calcScore ( ) {507 int hits = 0 , misses = 0 ;508 boolean hasHit = false ;509510 if ( lastPlayList . size ( ) < currentPlayList . size ( ) ) {511 misses += currentPlayList . size ( ) − lastPlayList . size ( ) ;512 }513514 for ( int i = 0 ; i < lastPlayList . size ( ) ; i++ ) {515 for ( int n = 0 ; n < currentPlayList . size ( ) ; n++){516 if ( lastPlayList . get ( i ) == currentPlayList . get ( n ) ) {517 hits++;518 hasHit = true ;519 }520 }521522 if ( hasHit == false ) {523 misses++;524 } else525 hasHit = false ;526 }527 int result = 0 ;528 if ( hits != 0) {529 result = ( hits ∗100) /( hits+misses ) ;530 }531532533 str = "Your Score : "+result ;534535 }

129

536537 }

GameEnd.java

1 package com . draw ;23 import java . io . File ;4 import java . io . IOException ;56 import org . puredata . android . io . AudioParameters ;7 import org . puredata . android . io . PdAudio ;8 import org . puredata . android . utils . PdUiDispatcher ;9 import org . puredata . core . PdBase ;

10 import org . puredata . core . utils . IoUtils ;1112 import android . app . Activity ;13 import android . content . pm . ActivityInfo ;14 import android . os . Bundle ;15 import android . util . DisplayMetrics ;16 import android . util . Log ;17 import android . view . Menu ;18 import android . view . MenuInflater ;19 import android . view . MenuItem ;20 import android . view . Window ;21 import android . view . WindowManager ;2223 public class GameEnd extends Activity{24 drawView view ;25 DisplayMetrics dm = new DisplayMetrics ( ) ;2627 Sound s ;2829 private static final String TAG = "Draw a Sound" ;30 private PdUiDispatcher dispatcher ;3132 public void onCreate ( Bundle savedInstanceState ) {33 super . onCreate ( savedInstanceState ) ;3435 // F u l l s c r e e n36 requestWindowFeature ( Window . FEATURE_NO_TITLE ) ;37 getWindow ( ) . setFlags ( WindowManager . LayoutParams . FLAG_FULLSCREEN ,38 WindowManager . LayoutParams . FLAG_FULLSCREEN ) ;3940 super . setRequestedOrientation ( ActivityInfo .←↩

SCREEN_ORIENTATION_LANDSCAPE ) ;4142 // Here the scveen dimensions are found .43 getWindowManager ( ) . getDefaultDisplay ( ) . getMetrics ( dm ) ;4445 try {46 initPd ( ) ;47 loadPatch ( ) ;48 } catch ( IOException e ) {49 Log . e ( TAG , e . toString ( ) ) ;50 }

130

5152 view = new drawView ( this , dm . widthPixels , dm . heightPixels , false , ←↩

true , false ) ;53 setContentView ( view ) ;54 }55565758 public void triggerNote ( int n ) {59 // Sends the midi note and a bang to the pure data patch .60 PdBase . sendFloat ( "midinote" , n ) ;61 PdBase . sendBang ( "trigger" ) ;62 }6364 // PureData funct i ons , j u s t l e ave them be .65 @Override66 protected void onResume ( ) {67 super . onResume ( ) ;68 PdAudio . startAudio ( this ) ;69 }7071 @Override72 protected void onPause ( ) {73 super . onPause ( ) ;74 PdAudio . stopAudio ( ) ;75 }7677 @Override78 public void onDestroy ( ) {79 super . onDestroy ( ) ;80 PdAudio . release ( ) ;81 PdBase . release ( ) ;82 }8384 private void initPd ( ) throws IOException {8586 // Conf igure the audio g lue87 int sampleRate = AudioParameters . suggestSampleRate ( ) ;88 PdAudio . initAudio ( sampleRate , 0 , 2 , 8 , true ) ;8990 // Create and i n s t a l l the d i spa t che r91 dispatcher = new PdUiDispatcher ( ) ;92 PdBase . setReceiver ( dispatcher ) ;93 }9495 private void loadPatch ( ) throws IOException {96 // Loads the PureData patch c a l l e d tone . pd97 File dir = getFilesDir ( ) ;98 IoUtils . extractZipResource ( getResources ( ) . openRawResource ( R . raw .←↩

tone ) , dir , true ) ;99 File patchFile = new File ( dir , "tone.pd" ) ;

100 PdBase . openPatch ( patchFile . getAbsolutePath ( ) ) ;101 }102103 //GUI104 public boolean onCreateOptionsMenu ( Menu menu ) {105 super . onCreateOptionsMenu ( menu ) ;

131

106 MenuInflater mInflate = getMenuInflater ( ) ;107 mInflate . inflate ( R . menu . game_menu , menu ) ;108109 return true ; // So something i s returned110 }111112 public boolean onOptionsItemSelected ( MenuItem item ) {113 switch ( item . getItemId ( ) ) {114 case R . id . playback :115116 try {117 view . playback ( ) ;118119 } catch ( InterruptedException e ) {120 e . printStackTrace ( ) ;121 }122123 return true ;124125 case R . id . mainMenu :126 // Wil l change to the main menu a c t i v i t y .127 finish ( ) ;128 return true ;129 }130 return false ;131132 }133134135 }

GameFull.java


10 import org . puredata . core . utils . IoUtils ;1112 import android . app . Activity ;13 import android . content . pm . ActivityInfo ;14 import android . os . Bundle ;15 import android . util . DisplayMetrics ;16 import android . util . Log ;17 import android . view . Menu ;18 import android . view . MenuInflater ;19 import android . view . MenuItem ;20 import android . view . Window ;21 import android . view . WindowManager ;2223 public class GameFull extends Activity{

132

24 drawView view ;25 DisplayMetrics dm = new DisplayMetrics ( ) ;2627 Sound s ;2829 private static final String TAG = "Draw a Sound" ;30 private PdUiDispatcher dispatcher ;3132 public void onCreate ( Bundle savedInstanceState ) {33 super . onCreate ( savedInstanceState ) ;3435 // F u l l s c r e e n36 requestWindowFeature ( Window . FEATURE_NO_TITLE ) ;37 getWindow ( ) . setFlags ( WindowManager . LayoutParams . FLAG_FULLSCREEN ,38 WindowManager . LayoutParams . FLAG_FULLSCREEN ) ;3940 super . setRequestedOrientation ( ActivityInfo .←↩

SCREEN_ORIENTATION_LANDSCAPE ) ;4142 // Here the scveen dimensions are found .43 getWindowManager ( ) . getDefaultDisplay ( ) . getMetrics ( dm ) ;4445 try {46 initPd ( ) ;47 loadPatch ( ) ;48 } catch ( IOException e ) {49 Log . e ( TAG , e . toString ( ) ) ;50 }5152 view = new drawView ( this , dm . widthPixels , dm . heightPixels , false , ←↩

false , true ) ;53 setContentView ( view ) ;54 }5556 // PureData funct i ons , j u s t l e ave them be .57 @Override58 protected void onResume ( ) {59 super . onResume ( ) ;60 PdAudio . startAudio ( this ) ;61 }6263 @Override64 protected void onPause ( ) {65 super . onPause ( ) ;66 PdAudio . stopAudio ( ) ;67 }6869 @Override70 public void onDestroy ( ) {71 super . onDestroy ( ) ;72 PdAudio . release ( ) ;73 PdBase . release ( ) ;74 }7576 private void initPd ( ) throws IOException {7778 // Conf igure the audio g lue

133

79 int sampleRate = AudioParameters . suggestSampleRate ( ) ;80 PdAudio . initAudio ( sampleRate , 0 , 2 , 8 , true ) ;8182 // Create and i n s t a l l the d i spa t che r83 dispatcher = new PdUiDispatcher ( ) ;84 PdBase . setReceiver ( dispatcher ) ;85 }8687 private void loadPatch ( ) throws IOException {88 // Loads the PureData patch c a l l e d tone . pd89 File dir = getFilesDir ( ) ;90 IoUtils . extractZipResource ( getResources ( ) . openRawResource ( R . raw .←↩

tone ) , dir , true ) ;91 File patchFile = new File ( dir , "tone.pd" ) ;92 PdBase . openPatch ( patchFile . getAbsolutePath ( ) ) ;93 }9495 //GUI96 public boolean onCreateOptionsMenu ( Menu menu ) {97 super . onCreateOptionsMenu ( menu ) ;98 MenuInflater mInflate = getMenuInflater ( ) ;99 mInflate . inflate ( R . menu . game_menu , menu ) ;

100101 return true ; // So something i s returned102 }103104 public boolean onOptionsItemSelected ( MenuItem item ) {105 switch ( item . getItemId ( ) ) {106 case R . id . playback :107108 try {109 view . playback ( ) ;110111 } catch ( InterruptedException e ) {112 e . printStackTrace ( ) ;113 }114115 return true ;116117 case R . id . mainMenu :118 // Wil l change to the main menu a c t i v i t y .119 finish ( ) ;120 return true ;121 }122 return false ;123124 }125126127 }

GameNo.java

1 package com . draw ;23 import java . io . File ;

134

4 import java . io . IOException ;56 import org . puredata . android . io . AudioParameters ;7 import org . puredata . android . io . PdAudio ;8 import org . puredata . android . utils . PdUiDispatcher ;9 import org . puredata . core . PdBase ;

10 import org . puredata . core . utils . IoUtils ;1112 import android . app . Activity ;13 import android . content . pm . ActivityInfo ;14 import android . os . Bundle ;15 import android . util . DisplayMetrics ;16 import android . util . Log ;17 import android . view . Menu ;18 import android . view . MenuInflater ;19 import android . view . MenuItem ;20 import android . view . Window ;21 import android . view . WindowManager ;2223 public class GameNo extends Activity{24 drawView view ;25 DisplayMetrics dm = new DisplayMetrics ( ) ;2627 Sound s ;2829 private static final String TAG = "Draw a Sound" ;30 private PdUiDispatcher dispatcher ;3132 public void onCreate ( Bundle savedInstanceState ) {33 super . onCreate ( savedInstanceState ) ;3435 // F u l l s c r e e n36 requestWindowFeature ( Window . FEATURE_NO_TITLE ) ;37 getWindow ( ) . setFlags ( WindowManager . LayoutParams . FLAG_FULLSCREEN ,38 WindowManager . LayoutParams . FLAG_FULLSCREEN ) ;3940 super . setRequestedOrientation ( ActivityInfo .←↩

SCREEN_ORIENTATION_LANDSCAPE ) ;4142 // Here the scveen dimensions are found .43 getWindowManager ( ) . getDefaultDisplay ( ) . getMetrics ( dm ) ;4445 try {46 initPd ( ) ;47 loadPatch ( ) ;48 } catch ( IOException e ) {49 Log . e ( TAG , e . toString ( ) ) ;50 }5152 view = new drawView ( this , dm . widthPixels , dm . heightPixels , false , ←↩

false , false ) ;53 setContentView ( view ) ;54 }55565758 public void triggerNote ( int n ) {

135

59 // Sends the midi note and a bang to the pure data patch .60 PdBase . sendFloat ( "midinote" , n ) ;61 PdBase . sendBang ( "trigger" ) ;62 }6364 // PureData funct i ons , j u s t l e ave them be .65 @Override66 protected void onResume ( ) {67 super . onResume ( ) ;68 PdAudio . startAudio ( this ) ;69 }7071 @Override72 protected void onPause ( ) {73 super . onPause ( ) ;74 PdAudio . stopAudio ( ) ;75 }7677 @Override78 public void onDestroy ( ) {79 super . onDestroy ( ) ;80 PdAudio . release ( ) ;81 PdBase . release ( ) ;82 }8384 private void initPd ( ) throws IOException {8586 // Conf igure the audio g lue87 int sampleRate = AudioParameters . suggestSampleRate ( ) ;88 PdAudio . initAudio ( sampleRate , 0 , 2 , 8 , true ) ;8990 // Create and i n s t a l l the d i spa t che r91 dispatcher = new PdUiDispatcher ( ) ;92 PdBase . setReceiver ( dispatcher ) ;93 }9495 private void loadPatch ( ) throws IOException {96 // Loads the PureData patch c a l l e d tone . pd97 File dir = getFilesDir ( ) ;98 IoUtils . extractZipResource ( getResources ( ) . openRawResource ( R . raw .←↩


100 PdBase . openPatch ( patchFile . getAbsolutePath ( ) ) ;101 }102103 //GUI104 public boolean onCreateOptionsMenu ( Menu menu ) {105 super . onCreateOptionsMenu ( menu ) ;106 MenuInflater mInflate = getMenuInflater ( ) ;107 mInflate . inflate ( R . menu . game_menu , menu ) ;108109 return true ; // So something i s returned110 }111112 public boolean onOptionsItemSelected ( MenuItem item ) {113 switch ( item . getItemId ( ) ) {114 case R . id . playback :

136

115116 try {117 view . playback ( ) ;118119 } catch ( InterruptedException e ) {120 e . printStackTrace ( ) ;121 }122123 return true ;124125 case R . id . mainMenu :126 // Wil l change to the main menu a c t i v i t y .127 finish ( ) ;128 return true ;129 }130 return false ;131132 }133134135 }

GameStart.java


10 import org . puredata . core . utils . IoUtils ;1112 import android . app . Activity ;13 import android . content . pm . ActivityInfo ;14 import android . os . Bundle ;15 import android . util . DisplayMetrics ;16 import android . util . Log ;17 import android . view . Menu ;18 import android . view . MenuInflater ;19 import android . view . MenuItem ;20 import android . view . Window ;21 import android . view . WindowManager ;2223 public class GameStart extends Activity{24 drawView view ;25 DisplayMetrics dm = new DisplayMetrics ( ) ;2627 Sound s ;2829 private static final String TAG = "Draw a Sound" ;30 private PdUiDispatcher dispatcher ;3132 public void onCreate ( Bundle savedInstanceState ) {

137

33 super . onCreate ( savedInstanceState ) ;3435 // F u l l s c r e e n36 requestWindowFeature ( Window . FEATURE_NO_TITLE ) ;37 getWindow ( ) . setFlags ( WindowManager . LayoutParams . FLAG_FULLSCREEN ,38 WindowManager . LayoutParams . FLAG_FULLSCREEN ) ;3940 super . setRequestedOrientation ( ActivityInfo .←↩

SCREEN_ORIENTATION_LANDSCAPE ) ;4142 // Here the scveen dimensions are found .43 getWindowManager ( ) . getDefaultDisplay ( ) . getMetrics ( dm ) ;4445 try {46 initPd ( ) ;47 loadPatch ( ) ;48 } catch ( IOException e ) {49 Log . e ( TAG , e . toString ( ) ) ;50 }5152 view = new drawView ( this , dm . widthPixels , dm . heightPixels , true , ←↩

false , false ) ;53 setContentView ( view ) ;54 }55565758 public void triggerNote ( int n ) {59 // Sends the midi note and a bang to the pure data patch .60 PdBase . sendFloat ( "midinote" , n ) ;61 PdBase . sendBang ( "trigger" ) ;62 }6364 // PureData funct i ons , j u s t l e ave them be .65 @Override66 protected void onResume ( ) {67 super . onResume ( ) ;68 PdAudio . startAudio ( this ) ;69 }7071 @Override72 protected void onPause ( ) {73 super . onPause ( ) ;74 PdAudio . stopAudio ( ) ;75 }7677 @Override78 public void onDestroy ( ) {79 super . onDestroy ( ) ;80 PdAudio . release ( ) ;81 PdBase . release ( ) ;82 }8384 private void initPd ( ) throws IOException {8586 // Conf igure the audio g lue87 int sampleRate = AudioParameters . suggestSampleRate ( ) ;

138

88 PdAudio . initAudio ( sampleRate , 0 , 2 , 8 , true ) ;8990 // Create and i n s t a l l the d i spa t che r91 dispatcher = new PdUiDispatcher ( ) ;92 PdBase . setReceiver ( dispatcher ) ;93 }9495 private void loadPatch ( ) throws IOException {96 // Loads the PureData patch c a l l e d tone . pd97 File dir = getFilesDir ( ) ;98 IoUtils . extractZipResource ( getResources ( ) . openRawResource ( R . raw .←↩


100 PdBase . openPatch ( patchFile . getAbsolutePath ( ) ) ;101 }102103 //GUI104 public boolean onCreateOptionsMenu ( Menu menu ) {105 super . onCreateOptionsMenu ( menu ) ;106 MenuInflater mInflate = getMenuInflater ( ) ;107 mInflate . inflate ( R . menu . game_menu , menu ) ;108109 return true ; // So something i s returned110 }111112 public boolean onOptionsItemSelected ( MenuItem item ) {113 switch ( item . getItemId ( ) ) {114 case R . id . playback :115116 try {117 view . playback ( ) ;118119 } catch ( InterruptedException e ) {120 e . printStackTrace ( ) ;121 }122123 return true ;124125 case R . id . mainMenu :126 // Wil l change to the main menu a c t i v i t y .127 finish ( ) ;128 return true ;129 }130 return false ;131132 }133134135 }

Point.java

1 package com . draw ;23 public class Point {4 private float x , y ;

139

56 public Point ( float x , float y ) {7 this . x = x ;8 this . y = y ;9 }

1011 public float getX ( ) {12 return x ;13 }1415 public float getY ( ) {16 return y ;17 }18 }

Sound.java

1 package com . draw ;23 import org . puredata . core . PdBase ;456 public class Sound {78 public Sound ( int midinote ) {9

10 triggerNote ( midinote ) ;11 }1213 public void triggerNote ( int n ) {14 // Sends the MIDI note and a bang to the pure data patch .15 PdBase . sendFloat ( "midinote" , n ) ;16 PdBase . sendBang ( "trigger" ) ;17 }181920 }

140

Third Iteration - ANOVA H

141

AsgerKroghKjær ChristianJeppesen MaryDaphneEjsing ... - Draw a Sound.pdf · Problem Analysis 1...

Documents

Transcript of AsgerKroghKjær ChristianJeppesen MaryDaphneEjsing ... - Draw a Sound.pdf · Problem Analysis 1...