CLAP - CLever Audio Plugin

All your LV2 and LADSPA goodness and more.

Moderators: MattKingUSA, khz

User avatar
rncbc
Established Member
Posts: 1060
Joined: Mon Apr 19, 2010 12:20 pm
Has thanked: 45 times
Been thanked: 256 times
Contact:

Re: CLAP - CLever Audio Plugin

Post by rncbc »

falkTX wrote:
rncbc wrote:but my plugins won't drop instance-access any day soon. why? sorry for my french, for crying out loud, i don't give a damn. it would make a whole lotta PITA to code it all over through lv2 atom, worker, schedule and what not. as is, currently, it simply works. and it is not broken. so i don't feel the urge to fix it :)
I kinda expected that.
What if I make a patch for you? Will that be ok?
be my guest. :)

i only think you just don't want to go that way--you have way more interesting and fun things to do than this---the isolation/separation process will sum to an immense refactoring escalade. although synthv1 might be dead easy, samplv1 will give you the shivers; and if when you ever get to drumkv1 then you'll be pulling hair in no time :D

thanks for the offer, anyway.
cheers
Drumfix
Established Member
Posts: 299
Joined: Mon Jan 26, 2009 5:15 pm
Been thanked: 11 times

Re: CLAP - CLever Audio Plugin

Post by Drumfix »

@ssj71: I would be mad going back from external UIs to "standard" UIs. This does not mean that the standard way should not be used anymore.
So, sit back and relax :)
falkTX wrote:simply showing the UI is not enough, we need ways to set its window title, icon, window hints (like tool window or always-on-top), and save&restore position. and most importantly it needs to be done in a separate process
Which is exactly how it should be done. The host is only required to supply the window title e.g. plugin->openEditor(title).
Icon, window hints, and save/restore position is easily done by the toolkit of the UI process itself (in my case Qt).

I think the big difference between me an falkTX is that he want all the configuration and communication between the plugin and its external UI process to be handled by the host, while i prefer to do it in the plugin.
ssj71
Established Member
Posts: 1294
Joined: Tue Sep 25, 2012 6:36 pm
Has thanked: 1 time

Re: CLAP - CLever Audio Plugin

Post by ssj71 »

Drumfix wrote:@ssj71: I would be mad going back from external UIs to "standard" UIs. This does not mean that the standard way should not be used anymore.
So, sit back and relax :)
fair enough. I now understand better your meaning. No offense taken or meant.

Drumfix wrote: I think the big difference between me an falkTX is that he want all the configuration and communication between the plugin and its external UI process to be handled by the host, while i prefer to do it in the plugin.
I think I have to side with falktx on this. It will give a much cleaner, polished result. Personally as a dev I don't want to worry about all that, I'm much more concerned about the actual interface and signal processing.
_ssj71

music: https://soundcloud.com/ssj71
My plugins are Infamous! http://ssj71.github.io/infamousPlugins
I just want to get back to making music!
User avatar
linuxdsp
Established Member
Posts: 147
Joined: Sun Mar 01, 2009 12:40 pm
Location: Oxford, England
Contact:

Re: CLAP - CLever Audio Plugin

Post by linuxdsp »

Separating UI and DSP so that you can run the UI in a different process is not inherently safer or better - when you load a plug-in, which is just a shared library, you may as well consider it (simplisticly) to be part of the host application. You wouldn't normally architect an application which has a GUI in such a way that it runs its GUI in a different process. I'm ignoring the case of creating a graphical front-end for an existing command line applicaion in this example.

The only case where splitting the UI might make sense is if you need to run the UI on a different machine. The point about it is, that you are not achieving anything inherently better or more reliable by doing so. The only reason for doing so is as a workaround for a specific use case, or becaue the linux graphics stack is rubbish.

As a design, there's something fundamentally more concise and reliable about the UI being run in the same process, once you split it from the main application / plugin you then have to resort to some kind of IPC and it all gets really horrible very quickly. Controlling a plugin via a UI connected via shared memory or pipes to / from another process is about as elegant a design as fixing the drivers seat to the roof of the car and attempting to drive using ropes attached to the controls. It might be innovative, in the sense that no-one else is doing it that way, but as soon as you try to use it for its intended purpose it becomes painfully apparent why no-one is doing it that way...

On more than one occasion I have (unwisely) attempted to use plug-ins for linux which (for whatever reason) have been designed this way, and had the UI process disconnect from the plug-in - which, to continue the analogy is like having the steering wheel detach without warning.
ssj71
Established Member
Posts: 1294
Joined: Tue Sep 25, 2012 6:36 pm
Has thanked: 1 time

Re: CLAP - CLever Audio Plugin

Post by ssj71 »

linuxdsp wrote:You wouldn't normally architect an application which has a GUI in such a way that it runs its GUI in a different process. I'm ignoring the case of creating a graphical front-end for an existing command line applicaion in this example.
Perhaps this is a flaw in my development process, but I always write the complete plugin with only the host generated interface, test and vet it, then create a graphical interface (which in my case is a completely separate binary).
Also because your plugin will be controlled by the host (automation) as much as through its own GUI, why not architect it so they are interfacing the same way?
linuxdsp wrote:The only reason for doing so is as a workaround for a specific use case, or becaue the linux graphics stack is rubbish.
So, with the current state of the graphics stack, is it better or safer to have the processes separated or not?
linuxdsp wrote:...as elegant a design as fixing the drivers seat to the roof of the car and attempting to drive using ropes attached to the controls.
They've done that on Top Gear. :)
linuxdsp wrote: On more than one occasion I have (unwisely) attempted to use plug-ins for linux which (for whatever reason) have been designed this way, and had the UI process disconnect from the plug-in - which, to continue the analogy is like having the steering wheel detach without warning.
Lv2 plugins? I imagine that is possible but I believe that if the plugin followed the spec this is more a failing of the host. LV2 has made it easy to design real-time plugins this way.
_ssj71

music: https://soundcloud.com/ssj71
My plugins are Infamous! http://ssj71.github.io/infamousPlugins
I just want to get back to making music!
User avatar
linuxdsp
Established Member
Posts: 147
Joined: Sun Mar 01, 2009 12:40 pm
Location: Oxford, England
Contact:

Re: CLAP - CLever Audio Plugin

Post by linuxdsp »

Perhaps this is a flaw in my development process, but I always write the complete plugin with only the host generated interface, test and vet it, then create a graphical interface (which in my case is a completely separate binary)...
Don't confuse compiling the UI to a separate .so - that's not what I'm talking about, I'm refering to running the UI in a completely seperate process. If you compile your (LV2) to a DSP.so and UI.so they end up in the same application process when the plugin is loaded (ok that's a simplification) but, you can pass data to and from the DSP and UI, normally via a callback to the host acting as the "controller," exactly as you would pass data around within a single application process, that's fine. The problems and additional complexity start multiplying when you have the UI running in a different process (e.g. a completely separate application containing just the UI code) because you then have to couple it to the host (and the plugin DSP) via some type of IPC. This seems an un-necessarily clunky design..

As for designing the plugin with only the host generated UI - obviously you can prototype it however you want to do - there is seldom one method that's works for everyone, and / or every type of plugin. Whenever the topic of host generated UIs comes up, I always refer to something like graphical EQs (kind of fresh in my mind as I'm currently porting my AF210 EQ to Audio Unit). It would be almost impossible to use with only the host UI (I'm having to do that at the moment for debug) but while there is still appropriate separation of the UI and DSP in the architecture of the plugin, it would have been impossible to design without a working UI as so much of the functionality is tied into the UI in that case.
So, with the current state of the graphics stack, is it better or safer to have the processes separated or not?
My point really was, that its not necessarily better - in fact I would prefer not to have to separate the UI into a different process, but sometimes the nature of the linux graphics / toolkit stack imposes that upon the design. I'll qualify that by saying that while my LV2 plug-ins use the external_ui extension, they don't actually run the UI in an external process - its just a convenient "toolkit agnostic" way to get LV2 to tell my plugin code to launch its UI window. (The plug-in's UI "engine" then handles it all in its own UI thread via XLib)
ssj71
Established Member
Posts: 1294
Joined: Tue Sep 25, 2012 6:36 pm
Has thanked: 1 time

Re: CLAP - CLever Audio Plugin

Post by ssj71 »

Thanks for the explanations. I don't quite understand this:
linuxdsp wrote: they don't actually run the UI in an external process... (The plug-in's UI "engine" then handles it all in its own UI thread via XLib)
I should probably do a bit more homework on processes vs threads. The embedded systems I'm used to aren't so sophisticated. :\
Just for the record I am concerned too about the future of the graphics stack re: wayland, mir, and X. I'm guessing though that LV2 will do whatever the hard part will be be to work around it. I have no good suggestions for a solution to the larger problem.
_ssj71

music: https://soundcloud.com/ssj71
My plugins are Infamous! http://ssj71.github.io/infamousPlugins
I just want to get back to making music!
User avatar
rncbc
Established Member
Posts: 1060
Joined: Mon Apr 19, 2010 12:20 pm
Has thanked: 45 times
Been thanked: 256 times
Contact:

Re: CLAP - CLever Audio Plugin

Post by rncbc »

falkTX wrote:meters.lv2 is a good example of IPC done well.
It can be done.

Developers should just be less lazy.
meters.lv2 only deals with scalar valued input/output ports, just like any lv2 control parameter does. meaning that writing a generic wrapper (or bridge) for DSP<->UI IPC is the least of the complications one can come about--it's a just a matter of one single do-it-all RPC-method passing an int (port index) and a float (port value) as arguments around; and that should work for all lv2 plugins that implement their model/controllers as lv2 scalar ports (control or audio) exclusively.

when the data you wish to pass across the bridge is one notch beyond that simple, like having some structured data to start with, things start to get real complicated, namely for the generic bridge developer (which in fact becomes impossible, unless some strict protocol is imposed to plugin developers as to represent their plugin's model/controllers exclusively and exhaustively through scalar values and opaque atom ports (you don't think i forgot about these ones, did you? ;))

now, some gadgets out there can became really complex, both in scale or magnitude and structure. benefits and virtues of a DSP-UI separation will vanish pretty soon, as for instance, the host and plugin will spend an increasingly, non negligible, processing power for IPC rather than doing actual DSP and UI drawing -- as an aside, that kind of sort reminds me of the old monolythical-kernel (eg. linux) vs. micro-kernel (eg. mach, minix, hurd, darwin, even winnt in the early years) debate, but whatever

imo. the only "pro" for DSP/UI separation might be about GUI toolkit isolation--you really can't mix Qt4 and Qt5 in the same address-space, same with Gtk2 and Gtk3). that's true and particularly demanding.

otherwise, if it crashes it's because there's a bug somewhere lurking in it--go fix it, you dumbasses! :D scnr.

separating a plugin's DSP from its UI won't solve the bug, ever. otoh. if a plugin's GUI eventually crashes, even though the host keeps apparently up and running (as in an out-of-process DSP/UI scenario), rest assured that i (imho) would not trust the host session any longer, no matter what. i'd rather backout to a previous session and stuff (which means that a host crashing in sympathy with some plugin is the least of troubles anyhow). if the plugin keeps crashing then i, as an user, should ditch it from any production use. simple as that.

cheers
User avatar
linuxdsp
Established Member
Posts: 147
Joined: Sun Mar 01, 2009 12:40 pm
Location: Oxford, England
Contact:

Re: CLAP - CLever Audio Plugin

Post by linuxdsp »

I can think of some cases where running the UI separately might be a good thing..
@falkTX: The reasons you cite (apart from some aspects of mixing different UI toolkits - and that comes under my original point about the state of the linux graphics stack in general, imposing certain design patterns as, while not necessarily the best, the only way to even have a hope of it working) - are safety measures against entirely avoidable scenarios.
I don't like to keep pursuing my original car analogy, but, its equivalent to fitting extra seatbelts and airbags just because you know the wheels arent fixed on properly, so that when you inevitably crash you won't get hurt so much - but while extra safety measures are useful, the designer should also take responsible measures to reduce the likelyhood of the acident happening in the first place. Fix the wheels on properly first.
It really comes down to, if you add (unnecessary) complexity to a design (software, hardware) whatever - you increase the propability that it will go wrong - often in ways you haven't even thought of or can't test for, and if you're worried specifically about the UI crashing the host, I could show you plenty of ways in which you could crash the host with a simple mistake in the DSP. In fact, I've also found malformed LV2 ttl files which will crash the host - without the plug-in even bening loaded.
Like all designs its a compromise (or a set of compromises) - either you try to sandbox everything in separate processes so a plugin can never crash the host application, and accept that you will use way more CPU than necessary, and have other reliability issues, or you accept that sometimes software crashes (guess what, writing complex software, including plug-ins is difficult). Even if you exclude the possibility of plug-ins crashing your DAW, you're not immune.
User avatar
linuxdsp
Established Member
Posts: 147
Joined: Sun Mar 01, 2009 12:40 pm
Location: Oxford, England
Contact:

Re: CLAP - CLever Audio Plugin

Post by linuxdsp »

Thanks for the explanations. I don't quite understand this:

linuxdsp wrote: they don't actually run the UI in an external process... (The plug-in's UI "engine" then handles it all in its own UI thread via XLib)
@ssj71: What I meant was, my UI code, runs in its own GUI thread, not a separate process. Or more specifically when you load the plug-in, an instance of the UI "engine" is created, which runs in its own thread, until the plug-in is removed and (optionally) its library unloaded. That thread is responsible for managing the connection to the X server, receiving X11 events, and acting upon them. The UI engine itself handles all the windows created by instances of the plugin, and the low-level drawing. On linux, threads are more lightweight than processes, and threads running in the same application process can access / share common resources (that's why you need thread locking and synchronisation) but in most cases exchanging data between them is trivial. By contrast a separate process is just that - entirely separate - and you would have to setup some means of IPC to exchange data (pipes, shared memory etc). It becomes just like two or more (separate) applications communicating. I used the "external UI" extension because I just needed a way for the host to tell my plugin to launch / close its UI, and a means of passing UI parameter changes to / from the host. I devised a method which didn't require the use of a genuinely "external" UI in another process - I just wanted the host application to have minimal interference with my UI.
User avatar
rncbc
Established Member
Posts: 1060
Joined: Mon Apr 19, 2010 12:20 pm
Has thanked: 45 times
Been thanked: 256 times
Contact:

Re: CLAP - CLever Audio Plugin

Post by rncbc »

falkTX wrote:
rncbc wrote:meters.lv2 only deals with scalar valued input/output ports, just like any lv2 control parameter does.
What are you talking about?

meters.lv2 receives atom transfer events for the fft visualizations. See:
https://github.com/x42/meters.lv2/blob/ ... ur.c#L1587
https://github.com/x42/meters.lv2/blob/ ... el.c#L1330
https://github.com/x42/meters.lv2/blob/ ... ter.c#L755
https://github.com/x42/meters.lv2/blob/ ... ope.c#L749

Of course passing float values is simple, but those plugins are doing much more than that.
(You probably only checked the first one I guess)
i've mentioned atom ports as well; but again, almost of the meters.lv2 do simple stuff and passing a blob (fft data) is not what i call structured data.

meters.lv2 might be doing the right thing nevertheless. however, you are kind of imposing here, to plugin developers, for them to (re)design, ultimately marshall and un-marshall all their DSP (model) and UI (controller/view) data and function, through discrete lv2 scalar values or opaque lv2 atom blobs (which means they have to serialize and un-serialize all the atom data on both ends, on both ways, DSP and UI).

all that can be simple as in meters.lv2, ought to be way more complex and sophisticated than that. ok. remember our exchange about the evil instance-access, a few posts earlier ? ;)

falkTX wrote:
rncbc wrote:imo. the only "pro" for DSP/UI separation might be about GUI toolkit isolation--you really can't mix Qt4 and Qt5 in the same address-space, same with Gtk2 and Gtk3). that's true and particularly demanding.
That's not that simple.
Besides preventing crashes or bad behaviour (which for you is as simple as not using bad plugins, which I understand),
there's also the possibility about the actual toolkits misbehaving.

For example, when a gtk2 file browser is started it spawns a few threads for file caching, previews and other random stuff. (calling fork in the process)
Hovering an audio file might trigger audio playback.
Not to mention the main thread will lock if the host is not using glib.
FLTK/NTK also has the issue of locking when a dialog or menu is called.

Current plugin UIs have to work around these toolkit issues.
But there's a limit on what developers can do.

You should know this yourself.
When a gtk2 file browser is opened it somehow messes up the MIDI in Qtractor. (was it fixed? did you found out what was wrong?)
Since you're not in control of plugin UIs, you can't stop them from opening those file-dialogs.

BUT if you ran gtk2 UIs on separate processes this MIDI issue would go away.
Users would certainly appreciate that. :wink:
yes i know all that, and no it is not fixed. indeed, that's one old problem re. mixing Qt and Gtk event-loops. things went pretty nice when Qt started to use the glib event-loop in its own event processing foundations. meaning that you could finally have Qt and Gtk widget co-existence, without resorting to clunky X11 event marshaling and what not. but once you opt to disable glib in Qt apps (yes, you can do that anytime, just set QT_NO_GLIB=1 (env)) you'll see that the trouble of Gtk file browser just goes away. with a catch: all gtk based plugins will stop to work properly if at all.

besides, the file browser trouble exists when using the gtk+ widget style on qtractor and that is completely unrelated and orthogonal to the host vs.plugins, dsp and ui separation discussion.

byee
User avatar
rncbc
Established Member
Posts: 1060
Joined: Mon Apr 19, 2010 12:20 pm
Has thanked: 45 times
Been thanked: 256 times
Contact:

Re: CLAP - CLever Audio Plugin

Post by rncbc »

falkTX wrote:
rncbc wrote:all that can be simple as in meters.lv2, ought to be way more complex and sophisticated than that. ok. remember our exchange about the evil instance-access, a few posts earlier ? ;)
Yes, and I completely understand that not all plugins can skip instance-access.
But for simple things, say setting filenames and basic displays, the UIs should use atom events.
Can we agree on this?
yep, by all means.
falkTX wrote:
rncbc wrote:the file browser trouble exists when using the gtk+ widget style on qtractor and that is completely unrelated and orthogonal to the host vs.plugins, dsp and ui separation discussion.
The point I was trying to make is that the issue would not happen if the UI is on a separate process.
If the UI requires instance-access this is no longer possible though.
Thus I say UI developers should their best and try to avoid it.

EDIT:
Question: If you load an lv2 plugin with a gtk2 UI and it opens a gtk file-browser, does the MIDI issue still happen?
(you can try ir.lv2 which loads files via gtk)
last time i've checked on this it was more like the gnome file browser to be in culprit as native desktop environment's open file dialog. can't say whether it's a native Gtk one or else--fact was, that something takes an exclusive lock onto the glib even-loop, right after a few interactions with the file requester dialogs, with disruptive consequences to normal Qt's asynchronous signal/slot delivery, especially when across different threads (eg. MIDI input -> main GUI thread).

cheers
JeffMcClintock
Posts: 1
Joined: Fri Jan 21, 2022 1:49 am
Has thanked: 1 time

Re: CLAP - CLever Audio Plugin

Post by JeffMcClintock »

linuxdsp wrote: Sat Jan 24, 2015 6:46 pm None of this addresses the single most important problem with making any plug-ins for linux, which is that you cannot write a plug-in with an embeddable UI, unless you write it to use the particular graphical toolkit which the host uses. Unless you know that in advance (and you can't) or even if you do (and you can't) you would have to (re)write the UI to suit all possible toolkits that the (or any) host might use.
One solution to this problem is to implement a 'virtual graphics API'. How it works is the DAW provides to the plugin a set of callbacks for drawing on a virtual surface. The plugin then has no dependencies on any particular toolkit, graphics API or platform.

As an example, a long time ago my host was written using the Windows GDI graphics API. After some years I rewrote it using Direct-2D which is a completely unrelated API that uses hardware acceleration. Once I'd written a 'virtual surface' for both APIS, all my plugin binaries worked on either API without even needing to be rebuilt. After I did the same for macOS Core Graphics my plugins run unmodified (same source code) on macOS even though they were originally written on Windows. I would love to do the same for Linux one day.

One additional benefit is that I have been able to fix several graphics bugs or platform inconsistencies just by updating the DAW, without the need to release updates of any plugins.

We get pretty good fidelity. Here is some text and gradients. (macOS on left, Windows on Right)
Image

This approach would not suit everyone, but it might be a solution worth considering or adapting.
Last edited by JeffMcClintock on Fri Jan 21, 2022 5:58 am, edited 1 time in total.
User avatar
milkii
Established Member
Posts: 477
Joined: Tue Jan 05, 2016 9:08 am
Location: Edinburgh
Has thanked: 92 times
Been thanked: 91 times
Contact:

Re: CLAP - CLever Audio Plugin

Post by milkii »

Afaiu, Pugl is what many LV2 devs use these days in regards to that problem, though I'm not a dev so I might be incorrect and chatting shit.

https://drobilla.net/software/pugl.html
https://lv2.gitlab.io/pugl/c/singlehtml

they/them ta / libreav.org / wiki.thingsandstuff.org/Audio and related pages / gh

Post Reply