I wanted to understand the emotional drivers that motivate them to use products. What do they expect and want to accomplish from different tools? What feelings do the instruments and sounds generate? What is it about them that does this?
Alex is non-verbal; she doesn't use words to communicate.
She doesn't know what to do when she's shown a microphone. Its color and shape make it look like a stick or ice cream, so she often bites and licks it.
Her dad finds it hard to show her what to do because microphones can be intimidating.
Meet Joshua – a curious and energetic 4-year old who has autism.
He loves exploring things that have texture and quickly loses interest if he doesn't get enough sensory feedback.
Josh struggles to produce sounds from instruments because he plays very lightly. This means he can't be part of the group.
Natalie loves being independent and trying new things.
She has a small range of movement.
It's hard for her to do what she wants because many tools made for kids with disabilities assume everyone makes large movements. And rely on someone being there to help.
Chloe is a bright, energetic 8-year-old. She's a visual learner who loves creating music.
Like many children, she loves expressing herself through movement.
Many accessible instruments can't translate movement into meaningful music.
We're still making assumptions about disability that lead to designs that don't work and reinforce stereotypes that disability limits creativity.
Many of the kids have multiple disabilities. Instead of assuming what their disability was, I focussed on understanding behavior, motivation, and what hindered or helped achieve their goals.
1
Gesture
People use movement and gesture in personal, nuanced, and sophisticated ways, including big and small presses, taps, thumps, slides, and scratches.
2
Gesture recognition
Many tools won't recognize and translate these different gestures into meaningful, expressive music.
3
Adaptable interfaces
Changing an instrument or interface's position improves accessibility, interaction, and expression. However, many tools can't be positioned where they're the most comfortable and accessible.
4
Sensory feedback
Sophisticated devices like iPads don't give enough tactile and sensory feedback, making them hard to use and explore.
Observing music therapy sessions showed us how the children instinctually used interfaces and instruments. And, crucially, the different techniques they'd try to make interfaces work for them.
I wanted to know why "Josh" kept running over and touching the big pole in the middle of the room. It's because he loves its texture. He really loves texture.
How people learn.
The kids showed us the nuanced, versatile ways they play and react to different forms, surfaces, materials, and sounds. And how they learn through tactile, visual, and auditory feedback.
The way people respond to traditional instruments' sonic and material properties tells us how to design expressive music interfaces.
Design . Prototype . Test
Don't change people. Adapt.
Designs have to serve the kids' emotional needs. Their motivation. Their curiosity. Their humor. The things kids love to do.
Design guidelines.
1
Adaptive
The design should adapt to physical & sensory needs.
2
Fun
Instruments should be fun and easy for parents and carers to manipulate so they can help kids use them.
3
Instructive
Shape and surfaces should be instructive and suggest affordance.
4
Translate intent
The software should scale so sonic, and sensory responses reflect the user's intent.
5
Stories
Sounds should be rich, have character, and tell a story.
6
Sonic affordance
Sounds should reflect the interface's mental model, gestures, and affordance.
Prototype sketches and designs.
Results
Kids can teach us how to design better.
Results — Prototypes and user testing.
ADAPTIVE INTERFACES
Textile interface for communication, home control, and music therapy that adapts to people's preferred gestures and position.
Insights & opportunities.
To help people learn what an interface does, use distinct gestures that create noticeable changes in sound.
Changes in frequency and pitch can be hard to recognize, so use obvious changes in rhythm and timbre instead.
Use shape, size, and texture/tactility to encourage/instruct behavior and gestures.
Think about the expectations that an object's history creates - for example, drums and ribbons- and how this can guide what people think they can do with it.
Build a platform, not just a product.
Use open design and code so other people can modify and iterate your designs to suit the preference of the people they're working with.
Think about the nuances of disability. For example, some people can only hear specific frequencies.
Make your interface's sensitivity and sensory responses scalable to suit preferences and abilities.
Parents show their kids how to use things.
Tools like microphones and public performances like dancing can be intimidating. So think about making them fun and approachable for the big kids too.
Music therapists, disability workers, and parents are the interface.
They need to easily manipulate and position your tools so that their kids can use them.
Speaking, workshops, press
World Music Therapy Congress
Sound Design — Designing Interactive Music Interfaces for Disabilities
Microsoft — The Garage
Curiosity, Empathy & Meaningfulness in Human Centred Sound Design Workshop
Special World: for special education teachers, therapists, and schools everywhere
Note Perfect
Contact
Use sound and interaction to build products everyone can use. Equally.