Video effects controlled by music / tempo?

Beyond just audio!

Moderators: MattKingUSA, khz

Post Reply
tavasti
Established Member
Posts: 2115
Joined: Tue Feb 16, 2016 6:56 am
Location: Kangasala, Finland
Has thanked: 399 times
Been thanked: 215 times
Contact:

Video effects controlled by music / tempo?

Post by tavasti »

I would like to have video luminosity (and maybe saturation) to react on music tempo. And sure, possibly also some other similar effects. Is there some software for making such? I don't need real-time reaction, it is ok to do with batch-processing. Anc because music is self-made, it would be perfectly ok to define tempo, and have 'cuts' on given places. But not willing to do all those by hand, for example in kdenlive it would mean adding 3 keyframes to effect per one 'dip': normal before peak, peak, and back to normal.

Visualization software lebiniou would be one possibility, but it is not handling video with real colors, but instead working in 256 color space, and even there not 'normal colors' but mapping them to something else. And I think it does not have suitable plugin, but sure that is minor problem, I could code such :-)

Linux veteran & Novice musician

Latest track: https://www.youtube.com/watch?v=ycVrgGtrBmM

User avatar
LAM
Established Member
Posts: 996
Joined: Thu Oct 08, 2020 3:16 pm
Has thanked: 141 times
Been thanked: 352 times

Re: Video effects controlled by music / tempo?

Post by LAM »

Last edited by LAM on Sun Nov 19, 2023 8:39 pm, edited 1 time in total.

in mix, nobody can hear your screen

tavasti
Established Member
Posts: 2115
Joined: Tue Feb 16, 2016 6:56 am
Location: Kangasala, Finland
Has thanked: 399 times
Been thanked: 215 times
Contact:

Re: Video effects controlled by music / tempo?

Post by tavasti »

LAM wrote: Sun Nov 19, 2023 6:10 pm

@tavasti Ossia Score could be a candidate for job: https://ossia.io/

Looks very promising, have to check it out!

Linux veteran & Novice musician

Latest track: https://www.youtube.com/watch?v=ycVrgGtrBmM

tavasti
Established Member
Posts: 2115
Joined: Tue Feb 16, 2016 6:56 am
Location: Kangasala, Finland
Has thanked: 399 times
Been thanked: 215 times
Contact:

Re: Video effects controlled by music / tempo?

Post by tavasti »

Linux veteran & Novice musician

Latest track: https://www.youtube.com/watch?v=ycVrgGtrBmM

User avatar
Daniele71
Established Member
Posts: 79
Joined: Mon Aug 07, 2023 3:02 pm
Location: Italy
Has thanked: 29 times
Been thanked: 16 times
Contact:

Re: Video effects controlled by music / tempo?

Post by Daniele71 »

Kdenlive has some effects that react to music, not much...
Probably not what you want, I don't know..

My music: Youtube
GeekOsDaw: GeekOsDaw

tavasti
Established Member
Posts: 2115
Joined: Tue Feb 16, 2016 6:56 am
Location: Kangasala, Finland
Has thanked: 399 times
Been thanked: 215 times
Contact:

Re: Video effects controlled by music / tempo?

Post by tavasti »

Daniele71 wrote: Fri Nov 24, 2023 5:08 pm

Kdenlive has some effects that react to music, not much...
Probably not what you want, I don't know..

Which effects? I have used kdenlive making my videos, so if it has something it would be great benefit.

Linux veteran & Novice musician

Latest track: https://www.youtube.com/watch?v=ycVrgGtrBmM

User avatar
Daniele71
Established Member
Posts: 79
Joined: Mon Aug 07, 2023 3:02 pm
Location: Italy
Has thanked: 29 times
Been thanked: 16 times
Contact:

Re: Video effects controlled by music / tempo?

Post by Daniele71 »

tavasti wrote: Sat Nov 25, 2023 6:07 am
Daniele71 wrote: Fri Nov 24, 2023 5:08 pm

Kdenlive has some effects that react to music, not much...
Probably not what you want, I don't know..

Which effects? I have used kdenlive making my videos, so if it has something it would be great benefit.

Recently discovered... they are in "On Master" effect list, there are few analyzer then Lightshow and Dance. Sadly a very limited choice.

kdenlive-fx.jpg
kdenlive-fx.jpg (58.82 KiB) Viewed 8994 times

My music: Youtube
GeekOsDaw: GeekOsDaw

tavasti
Established Member
Posts: 2115
Joined: Tue Feb 16, 2016 6:56 am
Location: Kangasala, Finland
Has thanked: 399 times
Been thanked: 215 times
Contact:

Re: Video effects controlled by music / tempo?

Post by tavasti »

tavasti wrote: Mon Nov 20, 2023 7:51 am

Other potentially interesting thing, sound node for Blender https://github.com/negdo/Sound_Nodes

That Sound_nodes (at least free version) did not work for me when I tested it, but today found out this video: https://www.youtube.com/watch?v=l8RHRNZQEuE

Linux veteran & Novice musician

Latest track: https://www.youtube.com/watch?v=ycVrgGtrBmM

Gps
Established Member
Posts: 1164
Joined: Mon Mar 09, 2015 3:09 pm
Has thanked: 358 times
Been thanked: 115 times

Re: Video effects controlled by music / tempo?

Post by Gps »

tavasti
Established Member
Posts: 2115
Joined: Tue Feb 16, 2016 6:56 am
Location: Kangasala, Finland
Has thanked: 399 times
Been thanked: 215 times
Contact:

Re: Video effects controlled by music / tempo?

Post by tavasti »

Yeah, familiar with them, specially with lebiniou, even coded some stuff for it :-) There has been some discussion about lebiniou earlier viewtopic.php?t=19267

With lebiniou it is possible to process video, but it is not something 'subtle', colors are replaced fully and possibilities to control effects are limited. And because it is targeted for live performances, it does not have timeline editor where you can time your effects, etc. Possibilities are using static timings (change to next set of effects after X seconds) or control them live. And changing them live, accuracy is limited by operator, and I am not at my best on such.
Lebiniou performance is limited by cpu (mostly single thread), so to run it live you either take hit in framerate or have to use low resolution. And because main developer got his final rest, most likely there won't be any major changes to it (unless someone adopts it).

ProjectM just generates its own visuals, not giving possibility to process video input.

Gps wrote: Mon Apr 22, 2024 4:57 pm

I used a plugin in Blender that makes things move to the sound.

https://www.youtube.com/watch?v=RSzceg_ ... nnel=Gps4L

Which plugin?

It looks like DaVinci Resolve might be the thing I've been looking for. Unfortunately could not run that yet in my linux system (devuan) because it needs libsystemd0, and that causes conflict in my system. I have seen some libsystemd0 depency earlier computer, related to lebiniou build depencies. Then I did some dummy package and everything worked, maybe same thing works this time.

I did test video with DaVinci in windows: https://www.youtube.com/watch?v=8Kbpf5dbZqE

Linux veteran & Novice musician

Latest track: https://www.youtube.com/watch?v=ycVrgGtrBmM

Gps
Established Member
Posts: 1164
Joined: Mon Mar 09, 2015 3:09 pm
Has thanked: 358 times
Been thanked: 115 times

Re: Video effects controlled by music / tempo?

Post by Gps »

I wonder if Ducky 3D has removed some of his older tutorial vids.
It was his vids mostly that got me started.

The thing I do remember, bake sound to F curve, and then you select your song.

I lost the blender file, so I cant look it up anymore.

Its not something extra you have to download.

https://www.youtube.com/watch?v=2TGwcQG ... =CGTOPTIPS

https://www.youtube.com/watch?v=GS2NBrh ... tesBlender

User avatar
Daniele71
Established Member
Posts: 79
Joined: Mon Aug 07, 2023 3:02 pm
Location: Italy
Has thanked: 29 times
Been thanked: 16 times
Contact:

Re: Video effects controlled by music / tempo?

Post by Daniele71 »

ycollette wrote: Mon Apr 22, 2024 2:40 pm

ProjectM:
https://github.com/projectM-visualizer/projectm

For me, ugly like a hell also they don't follow music too much.

My music: Youtube
GeekOsDaw: GeekOsDaw

tavasti
Established Member
Posts: 2115
Joined: Tue Feb 16, 2016 6:56 am
Location: Kangasala, Finland
Has thanked: 399 times
Been thanked: 215 times
Contact:

Re: Video effects controlled by music / tempo?

Post by tavasti »

Linux veteran & Novice musician

Latest track: https://www.youtube.com/watch?v=ycVrgGtrBmM

Post Reply