Audiovisual art through tangible and embodied interaction
Pompom. is an interactive system designed to aid children in musical and visual expression. It combines an on-screen interface and a physical artifact offering children an opportunity to express themselves creatively. The intent with this project was to integrate tangible and embodied interactions within music to explore what role these interactions play for musical and visual expression for children.
Brief: analyze existing educational tools and suggest the creation of a set of tools aiming at transforming the current status quo in the field of educational tools into a much more embodied form. Target group: Children of age between five and eight Team: Elena Hsu, Nathalie Strindlund, Adam Terne, Elias Widqvist My roles: research, UX design, visual design, user testing, academic writing Duration: 5 weeks Methods/tools: Arduino, HTML, CSS, JavaScript, physical material (wood, foam, fabric)
Pompom. was my final project in the course Tangible and Embodied Interaction. I was mainly a part of designing the visual appearance and evaluating the user experience. The project resulted in a physical board, a digital user interface and a final paper.
THE DIGITAL INTERFACE
When connecting the physical controller to a computer, a start screen is displayed where the user can choose from two modes. Those two modes have different themes with corresponding animations and sounds.
The Ocean Octave
The Space Sine
THE PHYSICAL BOARD
When a mode is chosen, the user can press the four buttons on the physical board to create real time visual and auditory representations of the button presses on the screen. By that, the interaction with Pompom. becomes a way for children to create audiovisual art. The physical board is made from wood and foam. Underneath the fabric cover there are four Arduino buttons.
The triangle shape of the board invites interaction from all directions and collaborative play between multiple users. To make the controller´s size fit the average size of a child´s hand, Dreyfuss´ chart on Human Factors in Design (1966) was taken into consideration when designing it.
DESIGN PROCESS
We early decided to choose music as our topic of interest and got inspired by interactions with existing midi boards and sequencers. In the early ideation phase we also played around a lot with Apple´s Garageband. We got intrigued by how fun it to play music when made easy, which inspired us to work with music as a vehicle to learn something else.
The first iteration of the physical board explored the interaction and the size of the the buttons. We made cotton balls covered with latex gloves to explore the feeling of soft buttons.
The next step was to implement the technical parts in the buttons and connect them to the visual animations.
The visual animations were made as SVG files in Illustrator which were then animated and triggered by the button presses in JavaScript. The first iteration consisted of abstract representations of known environments. The Seashell Sine (to the left) was inspired by the ocean and had piano-like sounds while the Desert Drum (to the right) was inspired by the desert and had more drum-like sounds. The sounds were made with the html5 Web Audio API.
We then made a second iteration of the physical controller to make it more visually refined and durable for play. The base was made from wood that we laser cut and placed the buttons in. The buttons were then covered with foam and textile to invite a soft and pleasurable interaction when pressing on the board as well as mute the harsh sound of the click on the buttons.
USER TESTING
We conducted user testing with children that played with Pompom. both individually and collaboratively for about 20 minutes while they were observed and interviewed. Our user testing showed that the collaborative aspect of playing with Pompom. was appreciated. We also understood that the children preferred more concrete visual animations because they want to be able to relate them to the world that they know.
After the user testing we iterated the abstract animations into the Ocean Octave and the Space Sine which shows objects relatable to known environments.
REFLECTION
One challenge with Pompom. was to keep the user´s interest continuing the interaction.
We approached this by making
it unable to play an incoherent sequence or out of key, as we hoped it would be
motivating to express creativity if one can´t do wrong. We also made the animations
reverse when finished so they play in an unending loop.
For the future, I would like the animations to be able to vary in shape, colour and/or size
to bring aspects of surprise into the interaction that would work as an additional motivation.
I had really fun making this project and gained insights into how subjects such as music can be integrated
in new contexts to create different interactions. Pompom. is easy to use compared to a traditional instrument
which I think is an advantage when introducing and encouraging children to create audiovisual art.
One of Pompom.´s strengths is that it allows for more modes to be created and
shared, which increases the chance of it eventually becoming a platform for children to express their creativity.
