Modem collaborated with Teenage Engineering and Bureau Cool to develop a digital add-on for the OP–Z synthesizer that transforms live music into AI-generated imagery. Inspired by synesthesia, where sound can be perceived as visual sensation, the interface translates musical structure into prompts that drive an evolving visual output during performance.
Synesthesia describes a perceptual condition in which one sensory input consistently triggers another. OP–Z Synesthesia adopts this principle as a design framework, treating sound as a source of visual instruction. Musical elements such as melody, rhythm, and harmonic structure are translated into prompts that guide an image generation model. The resulting visuals shift alongside the music, allowing performers to read cause and effect as they play.
The system is developed as a modular toolchain combining the OP–Z, MIDI signals, and a custom interface connected to Stable Diffusion. Individual tracks and parameters are mapped to visual variables, establishing a clear relationship between musical input and visual response.
OP–Z Synesthesia
Partner: Teenage Engineering
Creative Development: Bureau Cool