Resampler Adjustment

User avatar
w-u-2-o
Posts: 1733
Joined: Fri Mar 10, 2017 1:47 pm

Resampler Adjustment

Postby w-u-2-o » Sat May 04, 2019 1:33 pm

PowerSDR mRX 3.4.8 introduced an Adaptive Variable Resampler option for VAC audio. The implementation is a true resampler and not just a data "smoother".

Since both the radio and the PC use different clocks to obtain their nominal 48KHz audio sampling rate, the rate in the radio will not exactly match the rate in the PC. This sample rate mismatch leads inevitably to audio buffer under- and over-runs that often result in audible glitches in the VAC audio streams, both transmit and receive. The resampler acts to transform audio data across the radio clock domain and the PC clock domain, thereby substantially eliminating these glitches. The resampler also works for those who are using the VAC IQ data output at all IQ sample rates, and spur levels will be extremely low even at 192KHz primary sampling rate.

To use the resampler, first be sure to have achieved a reasonably stable and well performing VAC configuration without the resampler active. Then check the "Resampler" checkbox in Setup > Audio > VAC1 (and/or VAC2 if using VAC2). You will probably see the under- and -over-flow counters start counting. It then takes a few seconds for the resampler algorithm to begin making estimates of the sample rate mismatch. Once the initial estimate of sample rate mismatch is obtained, the Var Ratio display will begin to show the measured ratio between the PC and radio audio sample rates. This will update in a continuous fashion and usually does not remain static, as the sample clocks do drift over time, over temperature, etc.

At this point you should use the mouse to hover over the various displays and controls, read the tool-tips that pop up over each one, and thereby become more familiar with them.

After the resampler has become stable, which should occur in about ten seconds or so, you can click on the various counters to reset them to zero. This will allow you to more easily monitor resampler performance. Resampler performance depends quite heavily on the performance of your particular PC and your particular VAC configuration. Some people obtain zero under- and over-runs in both the transmit and receive directions for many hours, others see a steady but slow trickle that racks up to a few tens of them per hour.

A measure of latency can also be obtained by noting the size of the ringbuffer shown in the diagnostic display. Smaller buffer size equates to less latency. The size of the ringbuffer is determined by an algorithm that considers primary buffer size, VAC buffer size, VAC sample rate, and VAC Buffer Latency settings. The smallest possible ringbuffer on the receive side is 512, the smallest on the transmit side is 1024.

By using the monitoring features in the resampler you can work to optimize your primary and VAC buffer settings to achieve the fewest under- and over-runs, as well as the lowest latency. A general procedure is suggested as follows, although this is by no means the only method. This is also a useful procedure if you are having problems getting the resampler to converge, i.e. you are experiencing out of control under- and over-runs.

1. Start with a large primary buffer size.
2. Start with Buffer Latency set to 0mS & "Manual".
3. Start with VAC buffer size set to match your audio interface buffer size (if you know it, otherwise start with a large value).

If you get poor audio quality, try un-checking and checking the Buffer Latency Manual button a few times.

4. If you absolutely can't get it to run, let Buffer Latency go back to automatic. If that works, you can then try various values for manual buffer latency until you find the smallest one that works for you.
5. Once things seem stable, you can experiment with reducing the primary buffer size in order to obtain smaller ringbuffer sizes while still maintaining good audio quality.
User avatar
WA0VY
Posts: 83
Joined: Sun Apr 09, 2017 3:41 pm

Re: Resampler Adjustment

Postby WA0VY » Sat May 04, 2019 5:24 pm

Scott, nice summary as usual. As you know, I run exclusively with VAC audio enabled. With the Resampler enabled, I was able to run more-or-less glitch free in HPSDRmrx at 512/48k and a manual 0 ms buffer latency setting. Primary buffer was 128. I say more-or-less because I occasionally heard some minor popping on RX but no reports of any audible glitches in TX. I have switched to Thetis 2.6.5 and am able to achieve a zero glitch state at 128/48k but only if the buffer latency is left in auto. Any other manual buffer setting I have tried results in runaway over/underflows and very rough audio. There is not much documentation yet on the changes so I do not know what the PortAudio settings do and there are no flags apparent when I mouse over them. I have tried changing those settings in manual mode and have not noted any difference in the audio. Maybe its too early in the development cycle to put out a similar summary particular to Thetis, but I sure would like to at least understand what the PortAudio settings accomplish.
73 Brent WA0VY
Joe-W4WT
Posts: 81
Joined: Sun Apr 09, 2017 5:27 pm
Location: Cumming, GA

Re: Resampler Adjustment

Postby Joe-W4WT » Sat May 04, 2019 7:28 pm

Hi Brent,

I've also been working with Thetis and playing with VAC. What I have found so far that works at a setting of 192k is a buffer size of 256, port audio both set to manual and zero, and both ringbuffer settings of manual and at 25. This seems very stable with quite a lot of other things running on the system. I haven't tried going down to 128 for buffer size yet but will today as I continue to learn how Thetis reacts with VAC.

One thing I've noticed that really kills stability is VoiceMeeter Potato. It seems that VMP has a problem with my headphones used as an output device, A3 in my case, IF I have them turned off and am listening through the speakers via the UMC202hd which is assigned to A1. VMP CPU usage goes to 17% unless I turn the headphones on and do a menu restart of the VMP engine then it drops back to around 2%. Obviously a VMP bug. 17% doesn't seem like much but it really upsets VAC in Thetis for me.

More info on the port audio and ring buffer settings, interactions, etc. would be nice for our trials to getting to lower latency and better stability with VAC.

Joe W4WT
Bryan W4WMT
Posts: 49
Joined: Sun Apr 09, 2017 10:12 pm

Re: Resampler Adjustment

Postby Bryan W4WMT » Sat May 04, 2019 7:52 pm

Hi Brent,

When you say buffer latency must be in auto, do you mean the ringbuffer control or the PortAudio control?

What F/W sampling rate are you using?

What sound API are you using, ASIO?

73, Bryan W4WMT
User avatar
WA0VY
Posts: 83
Joined: Sun Apr 09, 2017 3:41 pm

Re: Resampler Adjustment

Postby WA0VY » Sat May 04, 2019 8:17 pm

Bryan, yes, I meant the ring buffer setting needed to be in auto. F/W sampling rate is 384000 and I use the ASIO drivers. Audio comes through an ASI 5211 digital sound card from outboard digital audio processing.

On further review, with all other settings the same as described in my original post, I was able to run stable and glitch free with a ring buffer manual setting of 50/50 if I brought that number down from 120/120 10 at a time. If I moved from 0 upwards, I did not get consistent results. Same with PortAudio although I could get that to 0/0 in manual. But again, I don't know what PortAudio is doing. That was not an adjustment we could make in the Protocol 1 software. In addition, I don't know from any indicator I can see whether I am running with more latency or less latency in manual mode than I would otherwise get in auto.
73 Brent WA0VY
User avatar
WA0VY
Posts: 83
Joined: Sun Apr 09, 2017 3:41 pm

Re: Resampler Adjustment

Postby WA0VY » Sat May 04, 2019 8:23 pm

Here's a screen shot of current audio settings. I noted that at these settings, not only am I getting glitch free audio with no over/under runs, but the Var ratio is as close to "0" as I have ever seen it, whatever that might indicate.
ScreenShot 8.jpg
ScreenShot 8.jpg (75.42 KiB) Viewed 4918 times
73 Brent WA0VY
Bryan W4WMT
Posts: 49
Joined: Sun Apr 09, 2017 10:12 pm

Re: Resampler Adjustment

Postby Bryan W4WMT » Sat May 04, 2019 9:50 pm

Hi Brent,

Looks like you’re making good progress there now.

The PortAudio latency setting is exactly the same thing as the old buffer latency setting in PowerSDR, only the name has changed. In Thetis we now have separate controls for input and output, whereas In PowerSDR they were ganged. Doug and I were worried that having separate controls would make things too “busy” for the user and cause unnecessary confusion. So we decided to just see how things went during the beta and change it back to the old way if it caused problems for the users.

In any case, the PortAudio latency setting has no relevance whatsoever for ASIO drivers. The ASIO API has no latency setting per se. It’s not clear what PortAudio does with that setting when running ASIO. I’ve asked about that on the PortAudio devel list and never got a meaningful answer. I can’t imagine any reasonable ASIO driver that would benefit from having that set to anything other than zero.

The Ringbuffer setting is new for Thetis. In PowerSDR that was a “one size fits all” setting that was calculated in the background, based on several other settings and some assumptions about how well behaved the thread scheduler was in the user’s computer. In Thetis, the user gets to decide how far he wants to push the envelope with respect to latency and glitch free operation. You did exactly the right thing: start at 120ms and very slowly work your way down a few ms at a time.

The user may profit from an indicator showing the actual ringbuffer size in effect. Users may also profit from indicators showing what settings are in effect when the manual boxes are unchecked. Those sorts of things can certainly be added in future releases if need be.

73, Bryan W4WMT
User avatar
WA0VY
Posts: 83
Joined: Sun Apr 09, 2017 3:41 pm

Re: Resampler Adjustment

Postby WA0VY » Sat May 04, 2019 10:40 pm

Bryan, thanks for the information. I had to reboot for unrelated reasons and when I brought Thetis up again, the audio was rough with the same settings I showed in the screenshots. I went back to full auto and everything evened out. Again, I have no idea whether the manual settings that did at one time work even provide less latency than full auto. That would be interesting to know but I assume that the auto settings could be different for everybody and vary from moment to moment. I'll keep playing with this and report back.

P.S. Because my outboard digital audio processor is an AM broadcast processor, there is so much freaking latency involved that all of this is purely academic for me. But, I'll pretend like it matters on my machine and keep experimenting and reporting.
73 Brent WA0VY
jeffreydoran
Posts: 51
Joined: Sun Oct 29, 2017 9:39 pm

Re: Resampler Adjustment

Postby jeffreydoran » Sun May 05, 2019 1:30 pm

I have found through extensive experimentation that solid resampling settings are elusive and remain a moving target. Settings I find to work great during one radio “session” only seem to fail the next time I fire up. I always listen to myself using the monitor function while wearing a headset, and this is where latency is critical (for me). Setting everything to automatic (unchecking manual) reliably eliminate clicks pops and glitches, but latency jumps to Im guessing 50 ms or more, which really screws the brain while trying to listen to yourself.

I’ve settled on manual ringbuffer settings of 20ms, with non forced resampling. My behringer umc is sampling at 96K with a 256 buffer, but this sample and buffer doesn’t seem to have much impact. The ASIO drivers have their own latency issues, but are way better than MME. PortAudio has no effect with ASIO drivers it appears. 20-25 ms seems to be right at the upper limit of my brain. But I bet, when I get into the shack this morning, even these settings won’t seem right. Restarting the software, or rebooting the PC seems to change the dynamic as well. Good days and bad days.

I realize latency means different things to different people, but for me it’s all about hearing myself without my mouth tripping over my brain. I’m 99.9% headset ops.

By the way, launching or surfing a browser totally glitches VAC regardless of settings. I’ve resorted to browsing inside a VMware virtual window for browsing, which helps a lot, but does not totally eliminate glitching.

I choose to use VAC for microphone ops because to me, the audio sounds clean and clearer than when using the front panel mic jack. That panel jack and the circuit behind it introduce a slight but noticeable hash,
or hiss which I dont think is detectable on air, but very evident in the monitor.

The struggle is real.

Jeff
NJ2US
User avatar
w-u-2-o
Posts: 1733
Joined: Fri Mar 10, 2017 1:47 pm

Re: Resampler Adjustment

Postby w-u-2-o » Sun May 05, 2019 2:41 pm

It remains a mystery why so many people are having such trouble. There is clearly something we all don't understand about how buffers should be set. Most serious audio software adjusts buffering in a completely automatic way with no significant latency. Any DAW application is a good example of this. If any developer wants to tackle this I'd suggest looking at the open source DAW app "Ardour" for which the source code is freely available.

My audio setup is quite complex: Thetis, Reaper and Presonus Studio 192 Mobile all are connected to each other via Voicemeeter Potato using ASIO. I don't really need Reaper anymore but I enjoy the control it gives me over routing audio. The Presonus driver is set to a buffer size of 128 and a sample rate of 96KHz, as is everything else. Digi mode app's, when I use them, are also connected via Voicemeeter at 48KHz and Voicemeeter happily does the rate translations. With all Thetis audio processing turned off latency is less than 30mS between audio sink/source and the antenna (not fair to count latency added by Thetis audio processing like NR2 or the leveler, etc.)

Note that the reason I use 96KHz is because I had reached the lowest available VAC buffer size of 128 at 48KHz and things remained stable at 96KHz, hence buffers take half as long to traverse at the higher sample rate. It's not a big difference, but a millisecond here and a millisecond there and pretty soon you are talking about a lot of milliseconds :D

FWIW here is a screen shot of my VAC settings:

vac.JPG
vac.JPG (103.58 KiB) Viewed 4857 times

I have two, minor stability issues. Every once in a while Reaper does not start correctly and sound is messed up. I simply restart Reaper and it will remain stable for days at a time. And I get periodic instability from the resampler, which I correct by cycling one of the "manual" checkboxes in order to reset it.

73,

Scott

P.S. Jeff: do you seek merely sidetone, or a true representation of your audio output? In either case you aren't going to get it from MON. IMHO, MON, as implemented, is useless except as a quick way to check that your VAC TX audio is not broken.
jeffreydoran
Posts: 51
Joined: Sun Oct 29, 2017 9:39 pm

Re: Resampler Adjustment

Postby jeffreydoran » Sun May 05, 2019 3:12 pm

First, thank you for everything you do in support of OpenHPSDR and the Apache equipment, I hang on your every post.

My experience with Monitor appears to be very personal, and likely different from yours I expect. I find it very helpful, as it does seem to fairly represent my TX audio before it's converted to RF, and I like that. It renders my bandwidth and EQ faithfully, at least in my headphones. And as long as I can keep the latency ahead of my brain, I would rather listen to "Monitor" than a simple sidetone like Flex does.

Your VAC settings are IDENTICAL to mine, including the 96K/128 UMC, but my ringbuffers appear to be more stable at 20ms, with just a minor latency impact.

Jeff NJ2US
User avatar
WA0VY
Posts: 83
Joined: Sun Apr 09, 2017 3:41 pm

Re: Resampler Adjustment

Postby WA0VY » Sun May 05, 2019 6:40 pm

I don't mean to beat a dead horse, but I still don't understand the following:

1. If 40 ms is as low as I can go in a manual buffer latency setting and run stable, am I better off running auto or manual? (I suspect there may be no way to answer this question, but in case there is......).

2. I have no information as to what PortAudio does. There are no flags appearing when I mouse over and I am unfamiliar with this setting. It was observed in a prior post that it may not have any effect when running VAC. This is certainly NOT true on my radio. If I try to run it manually, everything tends to fly apart.
73 Brent WA0VY
Joe-W4WT
Posts: 81
Joined: Sun Apr 09, 2017 5:27 pm
Location: Cumming, GA

Re: Resampler Adjustment

Postby Joe-W4WT » Sun May 05, 2019 9:37 pm

Scott, thanks for mentioning "cycling one of the manual checkboxes" to fix a "runaway". I have these sometimes too and it happened today after I read your post. I cycled the checkbox on the "In" box and it immediately stopped and behaved afterwards. I have been seeing this and thinking something in the system had changed and was causing a latency problem but now it seems this is a glitch in Thetis; at least it appears that way to me.

Joe W4WT
User avatar
Tony EI7BMB
Posts: 265
Joined: Sun Apr 09, 2017 2:31 pm
Location: Dublin
Contact:

Re: Resampler Adjustment

Postby Tony EI7BMB » Tue May 07, 2019 2:52 pm

Just moved to 96K VAC and seems to be still smooth. I'll Adjust the Buffer latency over the coming days so see how low it will go. Using a Ryzen 1700 Pc windows 10 and UMC204HD interface.


Screenshot 2019-05-07 15.49.04.png
Screenshot 2019-05-07 15.49.04.png (125.89 KiB) Viewed 4739 times
Helmut
Posts: 19
Joined: Sat Jul 15, 2017 3:08 pm

Re: Resampler Adjustment

Postby Helmut » Wed May 08, 2019 6:45 pm

I found today the following anomaly when running FT8 on my Anan 7000dle with Thetis 2.6.7 and Orion_MkII_Protocol_2_v2.0_pre1 with the
Setup

Image

Signal audio is sweeping with both force disabled - when force is enabled the frequency is solid as to see on the vertical spike-lines.

force = off

Image

force = on

Image

Does the resampler have impact on the frequency-stability on the audio signals ?

By the way, decoding is no problem but it is a new strange view for me - having used a Flex 6500 in the last years and trying now the 7000dleMK1.

73 Helmut oe9eri

p.s: as seen now decoding is only for some signals - not all - when -force=off_
User avatar
w-u-2-o
Posts: 1733
Joined: Fri Mar 10, 2017 1:47 pm

Re: Resampler Adjustment

Postby w-u-2-o » Wed May 08, 2019 9:59 pm

Helmut,

When force is off, are you getting under or overflow counts in a steady stream on the resampler?

73,

Scott
Helmut
Posts: 19
Joined: Sat Jul 15, 2017 3:08 pm

Re: Resampler Adjustment

Postby Helmut » Thu May 09, 2019 2:20 am

Hello Scott,

I do not have under or overflow at the time of taking the screen-prints. The counters are on 0 over long times when zeroed.
So the switching between -force on- and -force-off does only show the behave of the pictures in the waterfall.

With force-off the decoding of many stations is only for a part of them working. I did a test with the latency buffers checked to 25
and port-audio both checkt to 0 and -force-on - for a longer time no changes in audio frequency and solid decoding over longer times.

So I will use this settings for the next days and report if there are still problems:

Image


Thank you and 73

Helmut oe9eri
User avatar
w-u-2-o
Posts: 1733
Joined: Fri Mar 10, 2017 1:47 pm

Re: Resampler Adjustment

Postby w-u-2-o » Fri May 10, 2019 2:40 am

I've never been able to duplicate this problem. This is what I get with the resampler fully active in Thetis (and the same in PowerSDR):

Fast AGC, Linear Phase Filter, optimized ATT, I'm getting plenty of -24dB decodes (can you do better than -24?)

Capture.JPG
Capture.JPG (420.73 KiB) Viewed 4572 times
Helmut
Posts: 19
Joined: Sat Jul 15, 2017 3:08 pm

Re: Resampler Adjustment

Postby Helmut » Fri May 10, 2019 8:31 am

Thank you, Scott

for the test. I can repeat it here by switching the two force buttons on- and off. When on, there is no sweep, when off I have the audio-frequency-sweeping.
Fast AGC, Linear Phase Filter, optimized ATT ??, also tested, just worked J65 St.Lucia this morning, can hear also up to -24, but forces=ON.
If I switch them off the sweeping starts immediate. Counters do not move. The line at 2000 is my HP8657B GPS-locked as reference.
in the middle of the stream I switched the forces both -on- and frequency is solid. The sweeping is sometimes more and sometimes less pronounced. This screen print does show not so strong sweeps.

Image

I will give Firmware 1.9 a try as I run the 2.0beta at the moment with Thetis 2.6.7 on my Anan 7000dleMK1

73 Helmut
Helmut
Posts: 19
Joined: Sat Jul 15, 2017 3:08 pm

Re: Resampler Adjustment

Postby Helmut » Fri May 10, 2019 8:54 am

Now with Firmware 1.9 and the sweep is stronger by accident (it varys more or less).

Image


May be some user with an 7000dleMK1 (my SN 7000DLE 0121) can try that with FT8.

Thank you
Helmut oe9eri
Bryan W4WMT
Posts: 49
Joined: Sun Apr 09, 2017 10:12 pm

Re: Resampler Adjustment

Postby Bryan W4WMT » Fri May 10, 2019 10:23 am

Hi Helmut,

Am I reading your waterfall display correctly? It’s sweeping back and forth approximately 25 Hz?

I don’t have FT8 running on my 7000DLEMKI, but I know it’s not drifting 25Hz, I would hear that instantly in SSB.

Does the sweeping eventually slow down and stop? The reason I ask is because the loop filter used in the adaptive variable resampler is huge and can take over 10 minutes to settle down (at some sampling rates).

Still though, drift excursions in the realm of 25 Hz is very hard to explain.

73, Bryan W4WMT
User avatar
w-u-2-o
Posts: 1733
Joined: Fri Mar 10, 2017 1:47 pm

Re: Resampler Adjustment

Postby w-u-2-o » Fri May 10, 2019 12:44 pm

Helmut,

What happens if you uncheck all "Manual" checkboxes and adjust VAC buffer size to the minimum that will work properly?

Also, do you listen to the actual audio coming out of VAC to determine if it sounds OK, by ear? This is easy to do with Voicemeeter. Make sure you have a valid audio output device defined as A1 (speakers attached to your PC) and turn on the A1 route on whichever virtual channel you are using for Thetis (or PowerSDR). This can be VERY helpful in adjusting VAC settings rather than doing it by looking at the waterfall in WSJT-X.

73,

Scott
Helmut
Posts: 19
Joined: Sat Jul 15, 2017 3:08 pm

Re: Resampler Adjustment

Postby Helmut » Fri May 10, 2019 3:58 pm

What happens if you uncheck all "Manual" checkboxes and adjust VAC buffer size to the minimum that will work properly?


thats what I had in my first post on this thread, all manual unchecked - now I have a setup that is working solid without the sweeps:

Image

If I check the PortAudio boxes to manual with zero in - as in my first post - I can reproduce the sweep on my system.

I Thank you for your patience and your help !

Best 73 - have a nice weekend

Helmut oe9eri / dl9eri
User avatar
w-u-2-o
Posts: 1733
Joined: Fri Mar 10, 2017 1:47 pm

Re: Resampler Adjustment

Postby w-u-2-o » Fri May 10, 2019 4:29 pm

And there you go: you just have to find the combo that works! My recommendation, as in the original write up, is find the VAC buffer size that works the best with everything else in "auto" first, then start fooling with the manual settings.

73!

Scott
Helmut
Posts: 19
Joined: Sat Jul 15, 2017 3:08 pm

Re: Resampler Adjustment

Postby Helmut » Fri May 10, 2019 8:47 pm

Have to add,

the buffer-size 1024 is good on receive, but for transmit it has to be set to 2048 on my system ! With 1024 I have lots of blow-ups in duplex....

Good night from Bavaria

Helmut dl9eri / oe9eri
M0mws
Posts: 11
Joined: Wed Aug 14, 2019 9:40 pm

Re: Resampler Adjustment

Postby M0mws » Sun Sep 29, 2019 9:26 pm

w-u-2-o wrote:It remains a mystery why so many people are having such trouble. There is clearly something we all don't understand about how buffers should be set. Most serious audio software adjusts buffering in a completely automatic way with no significant latency. Any DAW application is a good example of this. If any developer wants to tackle this I'd suggest looking at the open source DAW app "Ardour" for which the source code is freely available.

My audio setup is quite complex: Thetis, Reaper and Presonus Studio 192 Mobile all are connected to each other via Voicemeeter Potato using ASIO. I don't really need Reaper anymore but I enjoy the control it gives me over routing audio. The Presonus driver is set to a buffer size of 128 and a sample rate of 96KHz, as is everything else. Digi mode app's, when I use them, are also connected via Voicemeeter at 48KHz and Voicemeeter happily does the rate translations. With all Thetis audio processing turned off latency is less than 30mS between audio sink/source and the antenna (not fair to count latency added by Thetis audio processing like NR2 or the leveler, etc.)

Note that the reason I use 96KHz is because I had reached the lowest available VAC buffer size of 128 at 48KHz and things remained stable at 96KHz, hence buffers take half as long to traverse at the higher sample rate. It's not a big difference, but a millisecond here and a millisecond there and pretty soon you are talking about a lot of milliseconds :D

FWIW here is a screen shot of my VAC settings:

vac.JPG
I have two, minor stability issues. Every once in a while Reaper does not start correctly and sound is messed up. I simply restart Reaper and it will remain stable for days at a time. And I get periodic instability from the resampler, which I correct by cycling one of the "manual" checkboxes in order to reset it.

73,

Scott

P.S. Jeff: do you seek merely sidetone, or a true representation of your audio output? In either case you aren't going to get it from MON. IMHO, MON, as implemented, is useless except as a quick way to check that your VAC TX audio is not broken.


On the 8k, using the rear headphone jack.. I get sidetone with very little latency.
This I feed into powered speakers with a headphone jack for concentrated CW listening.
This requires Mon to be on.

Return to “PowerSDR mRX”