MATAN BERKOWITZ: Hi, everybody.
So we're going to go on a little journey from music--
[DIGITAL MUSICAL NOTES ACCENTUATE HIS SPEECH]
--to tech and impact.
I'm going to go for different projects
and explore the evolution and ideas behind them.
But first, I'd like to start from the end.
Because for the past year and a half
or so, my team and I have been working on a device
and not really showing it to anybody.
And I'm super excited to be able to unveil it here
for the first time and share it with all of you.
So basically, I want to start with a question-- what
is the craziest thing that you can do with your tongue?
We're all thinking about the same thing, right--
controlling a wheelchair.
So-- [LAUGHS] What we did was basically to create this
device, Smart Braces, that fit in the mouth and enable you
to control digital devices around you--
like a phone, a computer, a smart home, and even
like this Medical Sunrise wheelchair right next to me.
[MURMURING FROM CROWD]
So how did we even get to the point
of trying to control wheelchairs with our tongue--
successfully or not?
It all starts for me with music.
I was performing live ever since I was 14 years old.
And after over a decade of playing shows and being
in the studio, I started getting really
frustrated with this notion of dudes in t-shirts
And I felt like there has to be more,
both as an audience member and as a performer.
I wanted to expand the spectrum of what it means to play music.
Also, I feel like on some level, spending so long pursuing
your childhood dreams without really
trying to affect anybody else's lives in a really
specific or meaningful way starts
to become kind of a burden.
There's this emptiness to doing something only for yourself
for so long.
And these thoughts for me basically led me to experiment
And specifically I was very drawn to sensors
and the idea of using the body to make music.
And this kind of became my main question at the time--
how can we translate the human body
into music in new and interesting and innovative
And this led me to do these conceptual tech infused shows,
like this one that was actually for Google
during the anniversary event in Israel,
where everything on stage was translating physical signals
So this woman, for example, was running,
and her heartbeat was creating the rhythm.
This guy was meditating, and his brainwaves
were generating the melodies.
My movements were tracked by a camera
so the motion would orchestrate the entire show.
And none of this was using any traditional instruments.
It was all the biological symphony of the human body.
And when we're thinking about the biological symphony
of the human body, I like to arbitrarily think about it
as two different kinds of signals.
The first one would be the physical, voluntary signals
of the body--
things like your movements and your voice.
These are the things you can actually control.
These are the things that you can
use to express yourselves like you would
with any musical instrument.
But then on the other hand, you have other kinds
of signals which are more like biological involuntary
your heartbeat, your brain activity.
So just like in the video before, you could influence
these kinds of signals.
By running, you could accelerate your heartbeat.
By meditating, you might be able to calm down
your brain activity.
But you can't really control them.
You can't decide what you want to play and just play it.
And therefore, the first one has more
to do with you playing the music,
while the second one has more to do with the music playing you--
like musical biofeedback.
The first time I ever saw this graph
by the Israeli start-up Neurosteer,
I was really inspired.
This is basically a guy who has been meditating
for many, many years connected to EEG, which
is a technology that measures and tracks
the electrical activity in the brain.
And using this kind of technology,
Neurosteer was able to show how at a certain point, when
he went deep into trance, his brain signal completely
Because you could see-- it's basically alpha brainwaves
showing you how focused or how tired
or how engaged someone is, and at a certain point,
when he became deep in trance, it completely
changed that graph.
And when I saw this, I thought about this as musical notation
It looked to me as if this was playing music back at me.
And I started to think about, how
can we use this to make music?
And so I collaborated with this company
and with a really inspiring individual named
Sefi Udi, who is a paraplegic.
He's paralyzed from the neck down, but before his accident,
he used to play the guitar.
And he wanted to play again and didn't know how.
And so what we did was we created a musical interface
using the EEG device.
And we enabled him to start controlling the music.
And this is kind of an important differentiator between the idea
of, I hear a melody in my head and my thoughts
are being played back--
like, [HUMS],, and the music just plays--
that doesn't really work.
What does work is by influencing your alpha brainwaves--
by calming yourself down or getting yourself worked up--
you could trigger different things in the music.
And Sefi was actually probably the most proficient
at this that I've ever seen.
After doing this with him, I actually
used this technology myself on stage,
and I gave it to other people.
But nobody has been able to replicate his level of control
over the music, which I think has a lot
to do with the sharpness of his mind
and the condition of his body.
And this person would actually come back and spiral back
in this talk, so try and remember him.
I then started playing around with this idea
of wearable devices.
The first thing I've ever created in this realm
was actually using the Google Glass--
the first edition back in the day.
And we used the head tracking feature of Google Glass
to enable you to play music using your head movements,
which led to some really bizarre shows.
And I then started thinking about,
can we take the two signals and combine them
into one creating this musical hat?
So the hat had this idea of head tracking,
where you can use those motions to consciously control
the music, and you could use these buttons to influence
the music at will.
But in the inside of the hat, there
was also the EEG device, which measured
the electrical activity in the brain.
And so the show was this weird combination
between you controlling the music and the music
playing back your biofeedback.
It then started becoming more and more
grandiose and ambitious in terms of shows.
This show for Microsoft was just pouring everything into it.
We had actual instruments-- like the aboriginal ancient
instrument of the didgeridoo, a drum kit, a guitar--
but we also had the EEG device and the heartbeat sensor
all playing together.
The different signals were being sent from the instruments
into these plates of sand and water
that would create these geometric shapes
and form according to the music.
And this is a really interesting phenomenon called [INAUDIBLE]..
So this idea of [INAUDIBLE] is really just the forms
and shapes that different frequencies and vibrations
get when they run through physical matter.
You normally have this kind of plate set up
with water sand in it, and as you
send the vibration or frequency into the plate,
it changes the form.
And it forms a very specific shape every time.
And if you do this right, it will form the same shape
every time, which is fascinating.
So for example, in this video of a demonstration
we did of this, we were sending 440 Hertz, which is basically
the note A. So the note A had this shape.
Interestingly enough, this wasn't really
a complete shape in my mind.
I felt like it was somewhat incomplete.
And so we tried other shapes.
We sent 563 Hertz into the same plate of sand,
and it became this shape.
And then, we realized we might try and send the golden ratio--
963-- which is of great significance in math
and in nature--
and see what this becomes.
And surely enough, this was a really interesting result.
AUDIENCE: Is the whole plate vibrating,
or it just at a point?
MATAN BERKOWITZ: The whole plate.
It has a speaker beneath it, and it
receives the entire vibration.
It then started occurring to me that I
have to literally fly to all of these places and be on stage
and do this myself, where it could also
be just objects or sculptures or interactive art
installations doing the same for many more people.
So we created different types of installations.
This one, for example, is a sculpture of a heart.
And when you touch your hand at the center of the heart, when
you put your palm at the center of it,
it plays back your pulse.
This next installation is called Rirrom,
which is the word mirror spelled backwards,
and it mirrors your movements with lights and with sounds.
But it gets really interesting when two people use it
at the same time, because then it
starts reflecting or mirroring how in tune
and how harmonious they are with one another.
Only when you move at the exact same way
and the exact same time does the piece really come to life.
And a third example of these interactive installations
is a thing I created with my friends
near [INAUDIBLE] called Portals, which are different trees that
are interconnected and are communicating with one another
when you touch them.
So when you touch the first tree, you start charging it up.
[MUSICAL NOTE SUSTAINING]
When it's fully charged, it activates the second tree.
Then someone at the other side could answer your call.
We created all the different versions of this interaction.
This one was in Sweden and had two trees
right next to each other.
But we also created one where the first tree was
Tel Aviv in Israel and the second tree
was in Stockholm in Sweden, and people
could communicate not only with the trees
but also with these screens and cameras connecting the two
countries-- which was basically a very elaborate way
to get Israeli men to hit on Swedish women.
They were just, like, touching the tree, you know, going like,
is it cold there?
What's your name?
So doing these kinds of things was
kind of starting to answer the first question asked
in the beginning, which was how to, I guess,
make it more interesting and more innovative
as far as art goes.
But the second question I was asking myself
about the impact or meaning of this work was still unanswered.
I felt like these things were cool and they were interesting,
but they weren't really creating real impact in the world.
And so the next question became how can we
create real impact using this kind of work?
And I started understanding the beginning of my answer
when working with Raheli, who was at the time 9 years old
and living in Jerusalem.
And she was both physically and cognitively impaired
to the point where she couldn't really do anything.
She couldn't speak, and she couldn't really
move other than moving her right hand
in a very kind of general way.
But she had this amazing smile and kind of infectious energy,
and we really wanted to do something for her.
My friend Araz, who knew her, asked me,
how can we enable this person to play music?
And this was quite a riddle.
I framed it as one button machine,
because you only have one button--
in this case, this one movement of the hand--
and you have to use this one button to create music.
And so what we did was we created this glove
and gave it three different modes.
And the first mode was drum mode,
so every time Raheli would do this motion, one time would go,
The second time would go, [CLICK]..
And so when she'd go--
[USES THE TWO TO IMITATE A DRUM] She could start playing music.
And she could listen and kind of feel that sense of rhythm
in her arm.
The second mode was DJ mode, where
we took her favorite song, which is
a super happy song about the Messiah,
and we broke it into pieces.
And basically, every time she would trigger the next part
with her hand, she had to really listen.
Because it could be too early or too late in the song--
basically, really making her musical hearing develop.
And then the third mode wasn't even musical.
We invited Raheli's mother to my studio,
and we recorded her reading Raheli's favorite children's
And then I cut that book into different sentences
and again gave her that active experience
of controlling her own storytelling experience.
So instead of her just being a passive listener,
she could actually trigger the next sentence
whenever she wanted to.
Doing this kind of work with Raheli and with Sefi--
the person I mentioned earlier--
kind of framed the next question that
was forming in my mind, which was how
can we make music accessible?
Maybe that is the path to the kind of impact
I was trying to find.
And so, this theory called the music accessibility
pyramid was forming in my mind.
At the top of this pyramid, there
are professional musicians.
These are the people who spend their time, energy, and money
on music and then make money back and are basically
in this position to do whatever they want to, if it goes well.
Then you have the hobbyists, who might spend all their energy,
time, and money on music, and they're not looking to receive
anything but their own satisfaction--
their own interest--
but they also could do whatever they
wanted to express themselves.
Then you have nonmusicians, who often
think, oh, I should have started playing
when I was much younger.
Now it's too late.
And that's obviously wrong.
Most of us can right now pick up an instrument secondhand,
borrow from a friend, we can go online, on YouTube,
or we can just get a teacher, and we
can learn to play in no time.
But there is a whole other layer of people with special needs
that might have all the time and energy and ambition and even
talent to make amazing music, but they're
hindered by their physical or cognitive condition.
And I know that last layer, by the way,
is not the biggest layer, but bear with me.
When we connect the two parts of the pyramid,
these two ends combined could create
some kind of transformation.
And trying to make that happen, I co-created an event called
Dis Co Tech--
which stands for Disability, Community, Technology.
And the idea behind it was creating the first framework
that was dedicated only to create music technology
for people with special needs--
creating the first framework for music technology
for people with special needs.
And so what we did was we had four different individuals
with four different challenges.
And around each of them, we had a team of makers, developers,
musicians, and therapists that were developing
a solution for that one person.
[PIANO MUSIC PLAYING]
[INAUDIBLE] suffers a neurological disorder,
and he used to play and compose and produce music.
But now he couldn't do that anymore,
so his team created an eye tracking
system that enabled him to compose music using his eyes.
[INAUDIBLE] was born autistic and blind,
and he could play really well with one hand--
but the other hand, he could only play with one finger.
So his team created a pedalboard that
enabled him to make up for the chords he
couldn't play with his left finger using his feet.
[PIANO MUSIC PLAYING]
[INAUDIBLE] was a professional musician,
and he was injured in a snowboarding accident
and ended up in a wheelchair.
So his team turned the wheelchair into a setup,
enabling him to plug in all of his instruments
in looping cells-- basically turning it into a one man band.
And last but not least, [INAUDIBLE]
is a professional singer that was born with one hand.
And so she could never really accompany her vocals
until her team designed and 3D printed a prosthetic arm that
enabled her to play the guitar for the first time in her life.
[MUSIC - "THE SCIENTIST"]
MATAN BERKOWITZ: So there are two main insights or lessons
learned from this experience.
The first one had to do with the pyramid.
So I realized while working on this that what we were actually
doing in a way was flipping the pyramid on its head.
Because in a way, when you're focusing
on the person that has the biggest challenge,
you actually kind of trickling down and helping everybody else
that is part of your pyramid.
In this example, we can take the eye tracking system.
So a person who cannot play music or cannot compose music
without the eye tracking system obviously needs it most.
But once you have this device that solves his condition,
you actually can help the professional musician explore
new ways to create, the hobbyist to have a new toy to play with,
and the nonmusician to have a new and intuitive way to try
and learn music and play music.
The second thing that I've learned
from doing this was that none of the things we created
was actually scalable.
All of these solutions were meant for a specific individual
with a specific condition and required
some kind of specific setup.
So in a way, it wasn't really going
to affect the lives of millions of people
like we had hoped for.
So this got me thinking about, how
can we really make that difference?
And I realized we are actually talking
about two different challenges that would require
two different solutions.
The first challenge is about music.
How can we make music accessible to as many people as possible,
regardless of their condition?
Regardless if they are healthier or disabled, if they're
young or old, if they're a professional musician or just
getting started, how can we make music
as accessible and intuitive as it could be?
And the answer to that question for me
was the Airstrument, which is the device on my hand.
The idea behind this wearable device
is that it takes that notion of voluntary signals of the thing
you can really control--
which is hand movements-- and turns it
into a super intuitive instrument.
So playing drums, for example, would look like this.
Or you could just grab a frequency.
[FREQUENCY SOUNDS BEING MANIPULATED]
Or you can try and make a whole song.
I'm going to loop myself and add layers to this.
[MUSIC DEVELOPS AND CONTINUES]
Now we could use a little bit of a chorus vocal type thing.
[DELICATE MELODY PLAYING]
VOCALS: Love will always win.
MATAN BERKOWITZ: [CHUCKLES]
The second question-- if the first question was really
about how to focus on music regardless of disabilities,
the second question became how to focus on disabilities
regardless of music.
How can we really make the world accessible for those
who need it--
for those who need this kind of accessibility--
regardless or beyond just playing music?
And the answer to that question became
this device called Tongo.
And the way it started was with a biomedical engineer called
[INAUDIBLE],, who came to meet me because she
wanted to create a musical instrument for people with ALS.
And the idea was to enable them to play music
using their tongue.
And I thought it was an amazing idea,
but being disillusioned with creating music technology
for people with special needs and nothing more,
I immediately told her, let's use the tongue
to enable them to do a lot of different things way
And this is what we've been doing ever since.
We've been working on this for about a year and a half.
Now I want to tell you a few things about the tongue
so you will get the idea.
First, I'd love you to just take the tongue right now
from the inside of your teeth, just touch one tooth.
Now touch another one.
Now slide between them.
We all have this really incredible orientation
within our mouth using our tongue.
It's almost like we already have a keyboard in our mouth
that we're not using.
Secondly, the tongue is a very strong muscle.
It can keep going for hours without getting exhausted.
Unlike eye tracking technology or other things
that end up becoming very exhausting over time,
the tongue can just keep going.
And thirdly, it survives really severe conditions
like neurological disorders and spinal cord injuries.
And so we created these Smart Braces.
This is the prototype of these Smart Braces
that are already patent pending, and they're wirelessly
communicating with different devices
like smartphones, computers, wheelchairs, smart homes--
and they're very intuitive and accessible already,
making it kind of easy to use and learn for almost anybody.
We created different apps for this,
and I'm going to show you one of them very soon.
The thing about this project is that even though we
were doing this in stealth, we had the privilege
to get funded by the Israeli innovation authority
and to have partnerships with IBM and the Odessa Medical
We even won awards from the Prize4Life ALS Foundation
and MedTech Accelerator in Israel--
all of this before coming out of stealth, which
is what we're doing right now.
We're just right now coming out of stealth.
So it's very exciting for us.
[INAUDIBLE] is right here-- you can raise your arm.
And so more than any of these partners
or more than any of these things,
the thing that touched us and helped us the most
was collaborating with individuals
who actually need this.
One of them you might recognize from earlier in this talk.
MATAN BERKOWITZ: The thing is, as we were working on this,
and we were working with people that actually needed it,
we realized the potential of this technology
to disrupt other industries-- like security, gaming, sports.
Just think about scuba diving, for example,
where you're already putting something in your mouth.
Why not make it a smart device?
And this notion is now leading us to first focus
on assistive technology and rehabilitation,
while we're exploring this new space of mouth
based controllers and what it could do.
And out of all of the apps that we've created so far,
I'd love to show you the musical one.
So this is Tongo.
This is the operating system we've built.
It has different apps, and they're all
controlled by the tongue.
[VARIOUS MUSICAL SOUNDS FROM TONGO]
TONGO: Hi, how are you?
I am hungry.
[SAMPLE OF INIGO MONTOYA FROM THE PRINCESS BRIDE]
MATAN BERKOWITZ: Recognize this one?
I'm going to try and play a song with it.
[TONGO PLAYS THE RIFF FROM "EYE OF THE TIGER"]
MATAN BERKOWITZ: Thank you, guys.
I'd like to finish this talk with kind of the main message
from me, which is not really about music, technology,
disabilities-- it's about something else.
I feel like the world right now is
at this point where we actually have a lot of responsibility.
Our generation-- the generation after us--
we're inheriting a world with a lot of challenges
and a lot of potential.
And we have to start figuring out
what we can each do to help.
And in that sense, what I have found
in my own journey is that there is a sweet spot between what
you love doing--
what you would do if no one would
pay you or ask you to do--
and between what you feel like the world needs--
the community around you, people you know,
or the world at large.
And where these two things meet--
where what you love doing and what the world
needs become one thing--
is that sweet spot where doors open
and things start happening.
And I encourage all of us to search for that,
and once we find it, to go with all
of what we have to offer and direct
our energy and our intention to that place.
Thank you for your time and attention.