vid2midi: converts videos to midi files
Posted: Sun Jun 28, 2020 4:35 am
I just released this Python script which generates midi files based on brightness values in video frames.
https://github.com/lucidbeaming/vid2midi
The idea came from a need to sync music themes and notes to scene changes in art videos I make. It is also useful for soundtracking videos in a DAW setup that doesn't offer video playback. In the past two days I have been using for more creative outlets.
It grabs the center portion of a video, averages the pixels, and returns a brightness value. That gets mapped to one of 12 notes on the chromatic scale. Time is synced to the frame duration and tick per quarternote value of the midi file.
Future plans include using hue instead of brightness and a wider range of notes/octaves.
In this example, the notes are driving the low synth in the background of the audio track for this video.
https://www.instagram.com/p/CB71eNejN9w/
https://github.com/lucidbeaming/vid2midi
The idea came from a need to sync music themes and notes to scene changes in art videos I make. It is also useful for soundtracking videos in a DAW setup that doesn't offer video playback. In the past two days I have been using for more creative outlets.
It grabs the center portion of a video, averages the pixels, and returns a brightness value. That gets mapped to one of 12 notes on the chromatic scale. Time is synced to the frame duration and tick per quarternote value of the midi file.
Future plans include using hue instead of brightness and a wider range of notes/octaves.
In this example, the notes are driving the low synth in the background of the audio track for this video.
https://www.instagram.com/p/CB71eNejN9w/
