About the Project
A real-time audiovisualizer that can be controlled by the user for live performances. By analyzing audio data from whatever music is playing, it shifts different elements of the visuals, which when combined with the user interactivity, creates a unique experience that blends VJ-ing with DJ-ing.
Concept
I've always wanted to create something that ties sound and visuals together, so for my Game Audio elective I put together an interactive audiovisualizer. The visual elements change in real-time through audio analysis and processing, while also featuring interactivity with a DualShock 4 controller, allowing the user to shift the music around, such as DJ-style killswitches for different frequency bands, or low and high-pass filters that utilize the DS4 controller's haptic R2/L2 shoulder buttons.
Design Blueprint
Technical Details
- Tools: Unity, TouchDesigner, Wwise
- Input: DualShock 4 Controller
Wwise collects and analyzes audio information using a multiband meter, which it then passes to TouchDesigner as numerical data to process changes in the visuals. The meter picks up frequencies of highs/mids/lows, splitting them up to affect different visual elements, such as the beat rings or the EKG-style lines. Unity acts as a bridge between Wwise and TouchDesigner.
By using a node to split each DualShock 4 input into individual channels, TouchDesigner processes inputs for visual effects, and calls on Wwise through different Events to trigger a variety of audio interactivity effects.