Ultralite AVB

Talk about your MIDI interfaces, microphones, keyboards...

Moderators: MattKingUSA, khz

User avatar
AudioNarwhal
Established Member
Posts: 39
Joined: Thu Apr 29, 2021 12:43 pm
Has thanked: 2 times
Been thanked: 35 times

Re: Ultralite AVB

Post by AudioNarwhal »

nandoll wrote: Thu Nov 24, 2022 11:50 pm

Running at @48KHz with a 256 sample buffer, jackd v2, 64 I/O channels over USB2 in so called "vendor mode" (using the motu endpoints, not the class compliant ones).

I haven't looked into the vendor mode issues yet, however the main implementation difference should be the number of channels. Hopefully any remediation to the class compliant mode will be applicable to the vendor mode as well.

nandoll wrote: Thu Nov 24, 2022 11:50 pm

Once I get the jackd server started it runs fine. Initial startup is tricky. The input channels usually start with an 8xn offset from where they should be, I think output channels are normally fine (I check inputs only initially). Restarting jackd several times eventually gets them right, and after that I do not see channel hopping (at least it has never happened while I have been performing with the system). Very very occasionally I have seen weird distortion (high pitched periodic noise) which is one of the failure modes I have seen.

This is interesting, I haven't investigated the capture side as much as the playback side. Based on this report and a couple reports from @baptiste, I think there is also a failure mode related to the capture direction. I'm not sure how much you have gone between the class mode and vendor mode, but it would be interesting to know if there is a noticeable difference regarding the channel hop offset between the two modes.

nandoll wrote: Thu Nov 24, 2022 11:50 pm

== @ ccrma (Stanford University)
...
== Stage:
...
== Listening Room:

Very neat to hear about these interesting use cases!

nandoll wrote: Thu Nov 24, 2022 11:50 pm

One more datapoint:
I encountered this in our Listening Room system. A user with a Mac laptop, updated Motu drivers, and bad problems with channel hopping and noises of all kinds. I had never witnessed this with someone running OSX. Other users (different computers) did not report problems. The exact same hardware had been running for a long time. Replacing the USB cables fixed it! So, this points, as outlined before in the thread, to problems in Motu's firmware where it cannot recover from USB error conditions on its own (ie: problems are not only experienced in Linux).

This isn't too surprising. Based on my current understanding of the issue, the main difference with the Linux platform is that the USB isochronous stream experiences more interruptions than typically seen with our driver on windows or mac or the class driver on mac. The main focus of any firmware change is to make the device recover after such an interruption and the changes to the Linux driver are to try and prevent the interruptions in the first place.

nandoll wrote: Fri Nov 25, 2022 1:31 am

a) channel hopping and other glitchy behavior in class compliant mode:
there is a potential improvement when running a kernel with the snd-usb-audio-printk-6.0.3.patch, this would just use the snd-usb-audio driver
(latest version here: https://drive.google.com/file/d/1-00_qF ... v-rNr/view). This would also limit the channel count to what the current max is for the Motu firmware (24 channels), and potentially introduce a 12mS output latency penalty - to perhaps be solved in the future.

Yes, the latency issue should be solvable, this patch is mostly to see if improving the isoc scheduling improves the situation enough or if there are additional issues at play. I think its likely that the isoc scheduling is going to make the biggest difference, but I suspect there are some other situations that this doesn't address. I need to spend some time digging into the fallout around changing sampling rate and buffer size with different ALSA clients to understand more. I am suspicious that the class driver might be okay with introducing gaps in the isoc stream in some circumstances, which could create a problem for the hardware.

nandoll wrote: Fri Nov 25, 2022 1:31 am

b) better recovery from usb error conditions inside the Motu audio interface:
there may be test firmware images we could use, I would be happy to test those, time permitting (I can test on a Motu 8M or a 16A easily, while I could test on an UltraLite AVB this is not what I currently use).

I could test a) and b), but this would not be really useful to me in "production" as...
Not addressed (yet? - cross fingers on this!) on this thread:

I started with the UltraLite and 24-Channel mode, but the general theory of the fix should apply to vendor mode as well. If the class mode changes are seeming like a step in the right direction, I can try to make a firmware that attempts to support both modes, I'll keep you posted.

nandoll wrote: Fri Nov 25, 2022 1:42 am

If a) and b) solve (or ameliorate) the problems we see in class compliant mode, I imagine it would be possible to add the proper quirks to the kernel driver to point to the alternate endpoints that allow 64 channel transfers - module the patch in the kernel so that it does not complain about duplicated endpoints (I was doing that before using the full "motu" driver by Drumfix)...

I think this is likely the case.

@nandoll Do any of your host computers you use at Stanford support thunderbolt?

jkohls wrote: Fri Nov 25, 2022 10:13 pm

shellwalker wrote: ↑Thu Nov 24, 2022 3:13 pm
But I'm wondering: does anyone have an idea how we organize this whole patching & debugging information and the test-results in a better way? I'm afraid we'll soon lose overview...
Suggestions are welcome!

I was thinking exactly the same thing. I ended up going out of town for Thanksgiving but now I'm back and still have time and motivation to contribute to this. My suggestion is we use any existing issue management system @AudioNarwhal is already using (if available to us) or whatever the Narwhal prefers.

This thread is going okay for me so far, but it may be nice to have something more issue-categorized. I think the main struggle I am having is to reproduce some of the various issues that are reported. Definitely system information (pcie devices, usb devices) and the kernel logs with dynamic debugging enabled are are super useful pieces of information.

shellwalker wrote: Sat Nov 26, 2022 6:16 pm

AudioNarwhal needs to state whether it's ok to stage the patch there and should state the license (which I am assuming to be GPL in order to be compliant with the kernel...).

Yeah, I hope for this to just result in a change for the USB audio class driver so it would maintain its current license.

baptiste
Established Member
Posts: 26
Joined: Sat Oct 06, 2018 7:18 am
Has thanked: 10 times
Been thanked: 8 times

Re: Ultralite AVB

Post by baptiste »

Some news : i've been able to witness the moment when sound begins to decimate and shift/copy +16.

Code: Select all

[63272.317876] MOTU Patch - 6.0.3
[63272.317879] usb 1-3: Delaying URB submission due to implicit fb sync
[63272.317885] usb 1-3: Starting data EP 0x87 (running 0)
[63272.317965] usb 1-3: 12 URBs submitted for EP 0x87
[63272.321568] usb 1-3: Starting data EP 0x87 (running 1)
[63272.321575] usb 1-3: 2:1 Start Capture PCM
[63272.321577] usb 1-3: 1:1 Start Playback PCM
[63535.017272] usb 1-3: frame 6 active: -71
[63598.956328] usb 1-3: frame 2 active: -71
[64016.109739] usb 1-3: frame 5 active: -71
[64227.909899] usb 1-3: frame 2 active: -71
[64227.909905] usb 1-3: frame 3 active: -71
[65229.950864] usb 1-3: frame 1 active: -71
[65342.041963] usb 1-3: frame 2 active: -71
[65342.041975] usb 1-3: frame 3 active: -71

There has been a change in sound almost each time i had a "frame x active: -71" line. Sometimes it doesn't do anything (rare), but most of the time there is a click and sound either become decimated and shifts/copy, or it becomes normal again and playing through the corrects outputs again.
I had some events of the sound becoming normal again without this line log appearing.
I was watching the whole /var/log/syslog at the same time, the event is not related in time with any other log line.

shellwalker
Established Member
Posts: 68
Joined: Sun Jan 03, 2021 1:54 pm
Has thanked: 21 times
Been thanked: 11 times

Re: Ultralite AVB

Post by shellwalker »

jkohls wrote: Fri Nov 25, 2022 10:13 pm

....
Unless someone else has already started or people are fine with what we have already I volunteer to pull something together.

And there it is: https://github.com/johnny2k/motu_patch_testing

I've contributed a Python based setup script today: motu-setup.py
Its purpose is to configure the device easily based on settings stored in a config file ~/.config/motu_patch_setup.json which it creates on first run.
Give it a try if you like. We can gradually add more stuff as we go and make the testing more automated.
Thanks to jkohls/Johnny for making the first steps!

MOTU 828mk3, MOTU 828ES, Cakewalk by Bandlab, Kubuntu 22.04.1 LTS, https://github.com/shellwalker-coder/motu_patch_testing

User avatar
AudioNarwhal
Established Member
Posts: 39
Joined: Thu Apr 29, 2021 12:43 pm
Has thanked: 2 times
Been thanked: 35 times

Re: Ultralite AVB

Post by AudioNarwhal »

baptiste wrote: Thu Dec 08, 2022 10:11 am

Some news : i've been able to witness the moment when sound begins to decimate and shift/copy +16.

Was this at 192kHz again? Which streaming direction(s) were affected?

baptiste
Established Member
Posts: 26
Joined: Sat Oct 06, 2018 7:18 am
Has thanked: 10 times
Been thanked: 8 times

Re: Ultralite AVB

Post by baptiste »

AudioNarwhal wrote: Sat Dec 10, 2022 7:42 pm

Was this at 192kHz again? Which streaming direction(s) were affected?

Yes, 192kHz, and output direction. I can't tell for input direction, i do not input anything. Maybe i should and route this to nowhere just to check...

Drkoma
Established Member
Posts: 4
Joined: Fri Jul 01, 2022 6:13 am
Been thanked: 1 time

Re: Ultralite AVB

Post by Drkoma »

jkohls wrote: Fri Nov 25, 2022 10:13 pm

And there it is: https://github.com/johnny2k/motu_patch_testing

Can anyone explain how to test/try this? Ive never patched the kernel before , all I find is zst files, downloaded linux headers for 6 but still Im far from understanding how to go about this, even with the script.

I would love to use my MOTU mk4 and unfortunately cant , Im the poster on this issue on pw which happens on pulse audio toohttps://gitlab.freedesktop.org/pipewire ... te_1686359

I saw the explanation by Baptiste but Im still far from being able to execute it without guidance . Thanks in advance

shellwalker
Established Member
Posts: 68
Joined: Sun Jan 03, 2021 1:54 pm
Has thanked: 21 times
Been thanked: 11 times

Re: Ultralite AVB

Post by shellwalker »

Drkoma wrote: Wed Dec 14, 2022 2:44 pm
jkohls wrote: Fri Nov 25, 2022 10:13 pm

And there it is: https://github.com/johnny2k/motu_patch_testing

Can anyone explain how to test/try this? Ive never patched the kernel before , all I find is zst files, downloaded linux headers for 6 but still Im far from understanding how to go about this, even with the script.

Hi Drkoma,
Currently the scrpts in the repo are just a blant capture of various steps discussed in this thread and do not even provide any error handling. I have made a first step towards generalization by creating motu-setup.py

I have not tested "patch_and_build.sh" myself to see how good it works for the patching process. Could you please provide feedback on what you observe? Best would be to submit an issue report at Github, so that we can better track it and avoid spamming this thread.

@AudioNarwhal : is the MK4 also affected by this topic and can it be handled in the same way?

MOTU 828mk3, MOTU 828ES, Cakewalk by Bandlab, Kubuntu 22.04.1 LTS, https://github.com/shellwalker-coder/motu_patch_testing

User avatar
AudioNarwhal
Established Member
Posts: 39
Joined: Thu Apr 29, 2021 12:43 pm
Has thanked: 2 times
Been thanked: 35 times

Re: Ultralite AVB

Post by AudioNarwhal »

shellwalker wrote: Thu Dec 15, 2022 6:00 am

@AudioNarwhal : is the MK4 also affected by this topic and can it be handled in the same way?

Yes, aside from being able to send it curl commands over ethernet the mk4 should be mostly the same.

I am hoping next week I will be able to post firmware for the mk4 and 828es variants. Firmware recovery in the playback direction is seeming pretty robust, but the record direction is still presenting some difficulties.

Drkoma
Established Member
Posts: 4
Joined: Fri Jul 01, 2022 6:13 am
Been thanked: 1 time

Re: Ultralite AVB

Post by Drkoma »

@shellwalker

I just saw the explanation from @baptiste , involving compiling the Kernel with that patch.
That's the part I'm not confident attempting, or knowledgeable enough to try, even if he details how he went about it, I've never done such a think and was gathering pieces of info here and there but failed to even comprehend.

I will open an issue as soon as I understand how to proceed on that step and give a proper test to your script, but so far I'm a bit lost on how to go about this.

To avoid spamming I will just open it there as you mention. Would be nice to have a friendly way to to this and your efforts in this direction are much appreciated. :D

baptiste
Established Member
Posts: 26
Joined: Sat Oct 06, 2018 7:18 am
Has thanked: 10 times
Been thanked: 8 times

Re: Ultralite AVB

Post by baptiste »

Drkoma wrote: Sat Dec 17, 2022 9:55 pm

@shellwalker

I just saw the explanation from @baptiste , involving compiling the Kernel with that patch.
That's the part I'm not confident attempting, or knowledgeable enough to try, even if he details how he went about it, I've never done such a think and was gathering pieces of info here and there but failed to even comprehend.

It is much easier to just build the patched module. I'm off now, without the UL. I'll try to come with clearer explanations in 2023... Happy holidays !

shellwalker
Established Member
Posts: 68
Joined: Sun Jan 03, 2021 1:54 pm
Has thanked: 21 times
Been thanked: 11 times

Re: Ultralite AVB

Post by shellwalker »

baptiste wrote: Sun Dec 18, 2022 5:53 pm

It is much easier to just build the patched module. I'm off now, without the UL. I'll try to come with clearer explanations in 2023... Happy holidays !

Looking forward to your updates, Baptiste :D Enjoy the season holidays!

@Drkoma Also note the comment from @AudioNarwhal about the MK4 not supporting the curl commands mechanism.

MOTU 828mk3, MOTU 828ES, Cakewalk by Bandlab, Kubuntu 22.04.1 LTS, https://github.com/shellwalker-coder/motu_patch_testing

User avatar
AudioNarwhal
Established Member
Posts: 39
Joined: Thu Apr 29, 2021 12:43 pm
Has thanked: 2 times
Been thanked: 35 times

Re: Ultralite AVB

Post by AudioNarwhal »

AudioNarwhal wrote: Sat Dec 17, 2022 12:34 am

I am hoping next week I will be able to post firmware for the mk4 and 828es variants. Firmware recovery in the playback direction is seeming pretty robust, but the record direction is still presenting some difficulties.

I'm having some difficulties with the Mk4 build job, so that may have to wait until after the holidays. In the meantime here is a slight tweak to the UltraLite AVB firmware I already posted, and also a version to try on the 828es.

MOTU AVB 1.4.4+92559 for UltraLite AVB and Stage-B16.update
MOTU AVB 1.4.4+92559 for 828ES and 8PreES.update

supereb
Established Member
Posts: 93
Joined: Fri Aug 17, 2018 10:55 pm
Has thanked: 36 times
Been thanked: 5 times

Re: Ultralite AVB

Post by supereb »

AudioNarwhal wrote: Fri Dec 23, 2022 5:07 pm
AudioNarwhal wrote: Sat Dec 17, 2022 12:34 am

I am hoping next week I will be able to post firmware for the mk4 and 828es variants. Firmware recovery in the playback direction is seeming pretty robust, but the record direction is still presenting some difficulties.

I'm having some difficulties with the Mk4 build job, so that may have to wait until after the holidays. In the meantime here is a slight tweak to the UltraLite AVB firmware I already posted, and also a version to try on the 828es.

MOTU AVB 1.4.4+92559 for UltraLite AVB and Stage-B16.update
MOTU AVB 1.4.4+92559 for 828ES and 8PreES.update

Thank you, Thank you! Thank you!

I'm currently using the Drumfix driver. Should I disable this?

User avatar
AudioNarwhal
Established Member
Posts: 39
Joined: Thu Apr 29, 2021 12:43 pm
Has thanked: 2 times
Been thanked: 35 times

Re: Ultralite AVB

Post by AudioNarwhal »

supereb wrote: Fri Dec 23, 2022 5:26 pm

I'm currently using the Drumfix driver. Should I disable this?

If the drumfix driver is working well except for occasional channel walking / distortion on playback then you could try using only the firmware. If you are getting issues in the record direction I think the normal or patched usb audio class driver would be worth trying.

Edit: The firmware fix, for now, will only effect operation if the interface is being used in “normal class compliant mode” ie. 24x24 channels. I’m only saying this because I think the drumfix driver may support using the vendor mode to increase the channel count, so I’m not sure how you’re set up with that.

stanlea
Established Member
Posts: 700
Joined: Wed Apr 25, 2012 9:49 pm
Has thanked: 41 times
Been thanked: 23 times

Re: Ultralite AVB

Post by stanlea »

AudioNarwhal wrote: Fri Dec 23, 2022 5:07 pm
AudioNarwhal wrote: Sat Dec 17, 2022 12:34 am

I am hoping next week I will be able to post firmware for the mk4 and 828es variants. Firmware recovery in the playback direction is seeming pretty robust, but the record direction is still presenting some difficulties.

I'm having some difficulties with the Mk4 build job, so that may have to wait until after the holidays. In the meantime here is a slight tweak to the UltraLite AVB firmware I already posted, and also a version to try on the 828es.

MOTU AVB 1.4.4+92559 for UltraLite AVB and Stage-B16.update
MOTU AVB 1.4.4+92559 for 828ES and 8PreES.update

Firmware for 828ES cause a continuous noise when playing.

Post Reply