Embedded Files
SYNTHESIZER CONCEPT
This project explores creative ways to bridge the gap between physical and digital data. The goal is to create a link between physical gestures and audio-visual feedback. Using MediaPipe and TouchDesigner, hand gestures are translated into data that is then used as metrics that control sound, that sound is then used to control visual interaction.
The final project showcases the art of audiovisualization, pushing conceptual interaction through generative art and sound design.
TouchDesigner programming + design for Sheridan College Interaction Design Program.
Details:
Sheridan College | Oakville, ON | 2026
Project Type:
Interaction + Experience
Role:
Interaction Design Student
Process
The process involved researching both MediaPipe as a motion-tracking tool and TouchDesigner’s interactive audiovisual capabilities.
Through online documentation and consultation with musicians and sound designers,the relationship between hand gestures and responsive audio-visual feedback was explored and refined.
Once gesture data from MediaPipe was accurately captured and interpreted, it was used to control and manipulate sound in real time. The resulting audio data was then repurposed to generate visual compositions, creating a system in which sound directly informed the generative visual output.
The resulting audio data was then repurposed to generate visual compositions, creating a system in which sound directly informed the generative visual output.
Google Sites
Report abuse