Tag: gesture

How many papers use the word ‘gesture’?
Gesture can mean many different things. (my battery is dying.)
Gesture is defined as movement of the body in dictionaries. (59 slides, 4 minutes of battery)
Research deifnitions of gesture: communication, control, metaphor (movement of sound or notation).
Who knows what gesture even means??
He downloaded nime papers and ran searches in them. 62% of all nime papers have mentioned gesture. (Only 90% of 2009 papers use the word ‘music’)
Only 32% of SMC papers mention gesture. 17% of ICMC
He checked what words ‘gesture’ came next to – collocation analysis.

NIME papers are good meta-research material
He suggests people define the term when they use it.
Data is available/

Questions

I can’t tell if this question is a joke or not….. oh, no, we’re on semiotics…. Maybe the pairong of the word ‘gesture’ with ‘recognition’ says something fundamental about why we care about gesture.
The word ‘gesture’ goes in and out of fashion.
Maybe ‘movement’ is a more meaningful word sometimes.
how often is gesture defined?
He should have checked that, he says.

they want to turn movement data into music.
This has come out of a collaboration with a dancer, using kinect. It was an exploration. He added visualisation to his interface. And eventually 240 parameters. The interface ended up taking over compared to the sound design.
They did a user survey to find out what other people were doing. So they wanted to write something that people could use for prototyping, that’s easy, extensible, and re-usable.
They wanted something stable, fast, free and complementary, so you could use your prototype in production. Not GPL, so you can sell stuff.
A patch-based system, because MAX is awesome all of the time.
This system is easily modifiable. He’s making it sound extremely powerful. Paramters are easy to tweak and saved with the patch, because parameters are important.
Has a simple SDK. Save it as a library, so you can run it in your project without the gui. this really does sound very cool.
Still in alpha.http://harmonicmotion.timmb.com

Questions

CNMAT is doing something he should look at, says a CNMAT guy.

Leap Motion is cool, he says.
rubato composer is software that allows people to do stuff with music and maths structures and transforms. It’s MAXish, but with maths.
The maths are forms and denotators, which is based on category theory and something about vector spaces. You can define vectors and do map stuff with them. He’s giving some examples, which I’m sure are meaningful to a lot of people in this room. Alas, both the music AND the math terms are outside of my experience. …. Oh no wait, you just define things and make associations between them. …. Or maybe not…..
It sounds complicated, but you can learning while doing it. They want to make it intuitive to enter matrixes via a visual interface, by drawing stuff.
This is built on ontological levels of embodiment. Facts, processes, gestures (and perhaps jargon). Fortunately, he has provided a helpful diagram of triangular planed in different colours, with little hand icons and wavy lines in a different set of colours, all floating in front of a star field.
Now we are looking at graphs that have many colours, which we could interact with.

Leap Motion

A cheap, small device that track hands above the device. More embodied than mouse or multitouch, as it’s in 3d and you cna use all your fingers.

Rubato

Is built in java, as all excellent music software is. You can grab many possible spaces. Here is a straightfoward one in a five dimensional space, which we can draw with a mouse, but sadly, not in five dimensions. Intuitively, his gui plays audio from right to left. The undo interface is actually kind of interesting. This also sends midi….
The demo seems fun.
Now he’s showing a demo of waving his hands over a MIDI piano.

Questions

Is the software available?
Yes, on sourceforge, but that’s crashy. And there will be an andriod version.
Are there strategies to use it without looking at the screen?
That’s what was in the video, apparently.
Can you use all 3 dimensions?
Yes