Acceleration Noise Sound Synthesis + Houdini
CS348C Final Project. Houdini, Python. (2018)
For this final project I implement an audiovisual sound synthesis system using rigidbody acceleration noise. The implementation is largely based on a previous assignment for CS448Z, describing the generation of acceleration noise using a simplified analytical model for contacting spheres. This specific simulation involves multiple steel marbles on a thick ceramic surface.
The simulation is largely done as a two-part process.
First, the video and the physics simulation are rendered through Houdini. The impact information is then imported to a separate python script, which renders the audio in a second pass.
1. Houdini simulation
The scene is composed of multiple spheres assumed to be made of steel falling onto a table assumed to be ceramic. The density and material of each object was tuned accordingly.
An Impact Analysis node within the Houdini’s DOP stores the information of each impact. This includes impact time, the two objects involved in a collision, velocity of the two bodies, impact normal, and so on. This impact information geometry exported by this node, along with the geometry of each ‘marble’ in the scene including their mass, is then exported into separate csv files.
2. Sound synthesis
A python script generates a sound file from the aforementioned two csv files. It follows the acceleration noise model for a spherical object, as mentioned in Howe 2002,
And the simplified model of it as described in the CS448Z assignment.
The script is written so that each impact information contributes to the overall audio individually. The audio is written and exported as wav using a simple numpy array assuming a frame rate of 44,100 Hz. For each impact, we generate the time scale according to the parameters, and then write its audio contribution to the appropriate frames in the numpy audio array.
Since the houdini simulation runs only at 30 frames per hour, the audio onset of each impact is randomly selected across the 44100/30 frames before each houdini frame in order to prevent all collisions happening in 30 fps.
While the ceramic board is not a spherical object, and thus would realistically have more complex rigidbody contributions to the overall sound, I assumed each collision with the board to be with a ceramic ball with a fixed mass of 25 and the density of ceramic.