mididelay not working

[booshtukka]

&mididelay= doesn't seem to be doing anything. I've tried it on the sending side and the receiving side. Here's our URLs:

The sender - using Google Chrome

https://vdo.ninja/?vd=OBS%20Virtual%20Camera&ad=obs&push=test1234&midiout=MidiPipe%20Input%201&proaudio&webcam&autostart&cleanoutput

The receiver - using elecap

https://vdo.ninja/?view=test1234&midiin=IAC%20Driver%20Bus%201&mididelay=5000&proaudio

Is there a setting to have MIDI automatically be in sync with video/audio? And is there a reason the mididelay parameter isn't working for us?

[steveseguin]

Hi booshtukka,

I'll need to setup my midi devices to test this, as overall it looks like it should work on first glance. I've been trying to rush to get an urgent update out for SSN users. I'll set something up and test to confirm it works though as I'm nearly done that urgent update.

As per sync with video/audio, that could be something that's possible for me to add.

If wanting to sync the audio and video with the midi, currently you'd be able to capture the MIDI commands yourself, and issue buffer-commands to the IFRAME API to delay the audio/video upwards of 3 seconds based on what sync you want. Beyond 3 seconds is challenging, as browsers these days don't allow for much buffering. I do have &chunked mode, which allows for much longer buffering, and some users do use it for syncing video/audio to a remote source, however chunked-mode is less stable and still a bit experimental.

If you want the MIDI playback to sync with the buffer delay of the video/audio, I can do that, however I'm not quite sure how accurate you'd want it. I currently update the buffer statistics for incoming video/audio every 3 seconds; it's not a smooth change. If I tried to do it for every frame, it might eat up a lot of CPU and the jitter for the MIDI playout might be problematic.

Getting MIDI playout exactly in sync with the video/audio's playout means I need to capture the video/audios timestamp probably at the the time of capture, AND at the time of playout, which will be a bit tricky. It would be far easier to have the playout delay for the MIDI be relative to the audio and video playout timestamps, based on a running average, which you would dial in once per stream, and then it would adapt dynamically based on network lag. I think I'd need to play with it a bit to figure out what works exactly however.

[steveseguin]

I'm still working on this. I think I found an issue with the midi-delay function; or specifically the function that handles background delayed callbacks. I'll get this sorted and pushed to alpha when a bit more awake.

[booshtukka]

Thank you! In the meantime, I think I've discovered that my method to capture the video of going out of HDMI and in through an HDMI capture card is probably introducing latency also - so there's room for me to play with this, and maybe that's creating the issue. I wish elecap could output via Syphon or NDI on Mac - that would remove this piece of the puzzle. :)

Really appreciate your efforts. In essence, what I want is everything to be in sync with each other when the signals arrive. Any closer we can get to that, the better!

[steveseguin]

vingester.app can output to NDI and it works a bit like elecap, however it does use quite a bit of CPU load in comparison.

[booshtukka]

I will have a look! But yeah, it looks pretty heavy. Our intent is to run as few things as possible. elecap + Syphoner is working well for now, and I think Syphon is less CPU intensive. This reducing the HDMI latency, but MIDI/video is still about 750ms out.