Resampler Adjustment
Resampler Adjustment
PowerSDR mRX 3.4.8 introduced an Adaptive Variable Resampler option for VAC audio. The implementation is a true resampler and not just a data "smoother".
Since both the radio and the PC use different clocks to obtain their nominal 48KHz audio sampling rate, the rate in the radio will not exactly match the rate in the PC. This sample rate mismatch leads inevitably to audio buffer under- and over-runs that often result in audible glitches in the VAC audio streams, both transmit and receive. The resampler acts to transform audio data across the radio clock domain and the PC clock domain, thereby substantially eliminating these glitches. The resampler also works for those who are using the VAC IQ data output at all IQ sample rates, and spur levels will be extremely low even at 192KHz primary sampling rate.
To use the resampler, first be sure to have achieved a reasonably stable and well performing VAC configuration without the resampler active. Then check the "Resampler" checkbox in Setup > Audio > VAC1 (and/or VAC2 if using VAC2). You will probably see the under- and -over-flow counters start counting. It then takes a few seconds for the resampler algorithm to begin making estimates of the sample rate mismatch. Once the initial estimate of sample rate mismatch is obtained, the Var Ratio display will begin to show the measured ratio between the PC and radio audio sample rates. This will update in a continuous fashion and usually does not remain static, as the sample clocks do drift over time, over temperature, etc.
At this point you should use the mouse to hover over the various displays and controls, read the tool-tips that pop up over each one, and thereby become more familiar with them.
After the resampler has become stable, which should occur in about ten seconds or so, you can click on the various counters to reset them to zero. This will allow you to more easily monitor resampler performance. Resampler performance depends quite heavily on the performance of your particular PC and your particular VAC configuration. Some people obtain zero under- and over-runs in both the transmit and receive directions for many hours, others see a steady but slow trickle that racks up to a few tens of them per hour.
A measure of latency can also be obtained by noting the size of the ringbuffer shown in the diagnostic display. Smaller buffer size equates to less latency. The size of the ringbuffer is determined by an algorithm that considers primary buffer size, VAC buffer size, VAC sample rate, and VAC Buffer Latency settings. The smallest possible ringbuffer on the receive side is 512, the smallest on the transmit side is 1024.
By using the monitoring features in the resampler you can work to optimize your primary and VAC buffer settings to achieve the fewest under- and over-runs, as well as the lowest latency. A general procedure is suggested as follows, although this is by no means the only method. This is also a useful procedure if you are having problems getting the resampler to converge, i.e. you are experiencing out of control under- and over-runs.
1. Start with a large primary buffer size.
2. Start with Buffer Latency set to 0mS & "Manual".
3. Start with VAC buffer size set to match your audio interface buffer size (if you know it, otherwise start with a large value).
If you get poor audio quality, try un-checking and checking the Buffer Latency Manual button a few times.
4. If you absolutely can't get it to run, let Buffer Latency go back to automatic. If that works, you can then try various values for manual buffer latency until you find the smallest one that works for you.
5. Once things seem stable, you can experiment with reducing the primary buffer size in order to obtain smaller ringbuffer sizes while still maintaining good audio quality.
Since both the radio and the PC use different clocks to obtain their nominal 48KHz audio sampling rate, the rate in the radio will not exactly match the rate in the PC. This sample rate mismatch leads inevitably to audio buffer under- and over-runs that often result in audible glitches in the VAC audio streams, both transmit and receive. The resampler acts to transform audio data across the radio clock domain and the PC clock domain, thereby substantially eliminating these glitches. The resampler also works for those who are using the VAC IQ data output at all IQ sample rates, and spur levels will be extremely low even at 192KHz primary sampling rate.
To use the resampler, first be sure to have achieved a reasonably stable and well performing VAC configuration without the resampler active. Then check the "Resampler" checkbox in Setup > Audio > VAC1 (and/or VAC2 if using VAC2). You will probably see the under- and -over-flow counters start counting. It then takes a few seconds for the resampler algorithm to begin making estimates of the sample rate mismatch. Once the initial estimate of sample rate mismatch is obtained, the Var Ratio display will begin to show the measured ratio between the PC and radio audio sample rates. This will update in a continuous fashion and usually does not remain static, as the sample clocks do drift over time, over temperature, etc.
At this point you should use the mouse to hover over the various displays and controls, read the tool-tips that pop up over each one, and thereby become more familiar with them.
After the resampler has become stable, which should occur in about ten seconds or so, you can click on the various counters to reset them to zero. This will allow you to more easily monitor resampler performance. Resampler performance depends quite heavily on the performance of your particular PC and your particular VAC configuration. Some people obtain zero under- and over-runs in both the transmit and receive directions for many hours, others see a steady but slow trickle that racks up to a few tens of them per hour.
A measure of latency can also be obtained by noting the size of the ringbuffer shown in the diagnostic display. Smaller buffer size equates to less latency. The size of the ringbuffer is determined by an algorithm that considers primary buffer size, VAC buffer size, VAC sample rate, and VAC Buffer Latency settings. The smallest possible ringbuffer on the receive side is 512, the smallest on the transmit side is 1024.
By using the monitoring features in the resampler you can work to optimize your primary and VAC buffer settings to achieve the fewest under- and over-runs, as well as the lowest latency. A general procedure is suggested as follows, although this is by no means the only method. This is also a useful procedure if you are having problems getting the resampler to converge, i.e. you are experiencing out of control under- and over-runs.
1. Start with a large primary buffer size.
2. Start with Buffer Latency set to 0mS & "Manual".
3. Start with VAC buffer size set to match your audio interface buffer size (if you know it, otherwise start with a large value).
If you get poor audio quality, try un-checking and checking the Buffer Latency Manual button a few times.
4. If you absolutely can't get it to run, let Buffer Latency go back to automatic. If that works, you can then try various values for manual buffer latency until you find the smallest one that works for you.
5. Once things seem stable, you can experiment with reducing the primary buffer size in order to obtain smaller ringbuffer sizes while still maintaining good audio quality.
Re: Resampler Adjustment
Scott, nice summary as usual. As you know, I run exclusively with VAC audio enabled. With the Resampler enabled, I was able to run more-or-less glitch free in HPSDRmrx at 512/48k and a manual 0 ms buffer latency setting. Primary buffer was 128. I say more-or-less because I occasionally heard some minor popping on RX but no reports of any audible glitches in TX. I have switched to Thetis 2.6.5 and am able to achieve a zero glitch state at 128/48k but only if the buffer latency is left in auto. Any other manual buffer setting I have tried results in runaway over/underflows and very rough audio. There is not much documentation yet on the changes so I do not know what the PortAudio settings do and there are no flags apparent when I mouse over them. I have tried changing those settings in manual mode and have not noted any difference in the audio. Maybe its too early in the development cycle to put out a similar summary particular to Thetis, but I sure would like to at least understand what the PortAudio settings accomplish.
73 Brent WA0VY
Re: Resampler Adjustment
Hi Brent,
I've also been working with Thetis and playing with VAC. What I have found so far that works at a setting of 192k is a buffer size of 256, port audio both set to manual and zero, and both ringbuffer settings of manual and at 25. This seems very stable with quite a lot of other things running on the system. I haven't tried going down to 128 for buffer size yet but will today as I continue to learn how Thetis reacts with VAC.
One thing I've noticed that really kills stability is VoiceMeeter Potato. It seems that VMP has a problem with my headphones used as an output device, A3 in my case, IF I have them turned off and am listening through the speakers via the UMC202hd which is assigned to A1. VMP CPU usage goes to 17% unless I turn the headphones on and do a menu restart of the VMP engine then it drops back to around 2%. Obviously a VMP bug. 17% doesn't seem like much but it really upsets VAC in Thetis for me.
More info on the port audio and ring buffer settings, interactions, etc. would be nice for our trials to getting to lower latency and better stability with VAC.
Joe W4WT
I've also been working with Thetis and playing with VAC. What I have found so far that works at a setting of 192k is a buffer size of 256, port audio both set to manual and zero, and both ringbuffer settings of manual and at 25. This seems very stable with quite a lot of other things running on the system. I haven't tried going down to 128 for buffer size yet but will today as I continue to learn how Thetis reacts with VAC.
One thing I've noticed that really kills stability is VoiceMeeter Potato. It seems that VMP has a problem with my headphones used as an output device, A3 in my case, IF I have them turned off and am listening through the speakers via the UMC202hd which is assigned to A1. VMP CPU usage goes to 17% unless I turn the headphones on and do a menu restart of the VMP engine then it drops back to around 2%. Obviously a VMP bug. 17% doesn't seem like much but it really upsets VAC in Thetis for me.
More info on the port audio and ring buffer settings, interactions, etc. would be nice for our trials to getting to lower latency and better stability with VAC.
Joe W4WT
Re: Resampler Adjustment
Hi Brent,
When you say buffer latency must be in auto, do you mean the ringbuffer control or the PortAudio control?
What F/W sampling rate are you using?
What sound API are you using, ASIO?
73, Bryan W4WMT
When you say buffer latency must be in auto, do you mean the ringbuffer control or the PortAudio control?
What F/W sampling rate are you using?
What sound API are you using, ASIO?
73, Bryan W4WMT
Re: Resampler Adjustment
Bryan, yes, I meant the ring buffer setting needed to be in auto. F/W sampling rate is 384000 and I use the ASIO drivers. Audio comes through an ASI 5211 digital sound card from outboard digital audio processing.
On further review, with all other settings the same as described in my original post, I was able to run stable and glitch free with a ring buffer manual setting of 50/50 if I brought that number down from 120/120 10 at a time. If I moved from 0 upwards, I did not get consistent results. Same with PortAudio although I could get that to 0/0 in manual. But again, I don't know what PortAudio is doing. That was not an adjustment we could make in the Protocol 1 software. In addition, I don't know from any indicator I can see whether I am running with more latency or less latency in manual mode than I would otherwise get in auto.
On further review, with all other settings the same as described in my original post, I was able to run stable and glitch free with a ring buffer manual setting of 50/50 if I brought that number down from 120/120 10 at a time. If I moved from 0 upwards, I did not get consistent results. Same with PortAudio although I could get that to 0/0 in manual. But again, I don't know what PortAudio is doing. That was not an adjustment we could make in the Protocol 1 software. In addition, I don't know from any indicator I can see whether I am running with more latency or less latency in manual mode than I would otherwise get in auto.
73 Brent WA0VY
Re: Resampler Adjustment
Here's a screen shot of current audio settings. I noted that at these settings, not only am I getting glitch free audio with no over/under runs, but the Var ratio is as close to "0" as I have ever seen it, whatever that might indicate.
73 Brent WA0VY
Re: Resampler Adjustment
Hi Brent,
Looks like you’re making good progress there now.
The PortAudio latency setting is exactly the same thing as the old buffer latency setting in PowerSDR, only the name has changed. In Thetis we now have separate controls for input and output, whereas In PowerSDR they were ganged. Doug and I were worried that having separate controls would make things too “busy” for the user and cause unnecessary confusion. So we decided to just see how things went during the beta and change it back to the old way if it caused problems for the users.
In any case, the PortAudio latency setting has no relevance whatsoever for ASIO drivers. The ASIO API has no latency setting per se. It’s not clear what PortAudio does with that setting when running ASIO. I’ve asked about that on the PortAudio devel list and never got a meaningful answer. I can’t imagine any reasonable ASIO driver that would benefit from having that set to anything other than zero.
The Ringbuffer setting is new for Thetis. In PowerSDR that was a “one size fits all” setting that was calculated in the background, based on several other settings and some assumptions about how well behaved the thread scheduler was in the user’s computer. In Thetis, the user gets to decide how far he wants to push the envelope with respect to latency and glitch free operation. You did exactly the right thing: start at 120ms and very slowly work your way down a few ms at a time.
The user may profit from an indicator showing the actual ringbuffer size in effect. Users may also profit from indicators showing what settings are in effect when the manual boxes are unchecked. Those sorts of things can certainly be added in future releases if need be.
73, Bryan W4WMT
Looks like you’re making good progress there now.
The PortAudio latency setting is exactly the same thing as the old buffer latency setting in PowerSDR, only the name has changed. In Thetis we now have separate controls for input and output, whereas In PowerSDR they were ganged. Doug and I were worried that having separate controls would make things too “busy” for the user and cause unnecessary confusion. So we decided to just see how things went during the beta and change it back to the old way if it caused problems for the users.
In any case, the PortAudio latency setting has no relevance whatsoever for ASIO drivers. The ASIO API has no latency setting per se. It’s not clear what PortAudio does with that setting when running ASIO. I’ve asked about that on the PortAudio devel list and never got a meaningful answer. I can’t imagine any reasonable ASIO driver that would benefit from having that set to anything other than zero.
The Ringbuffer setting is new for Thetis. In PowerSDR that was a “one size fits all” setting that was calculated in the background, based on several other settings and some assumptions about how well behaved the thread scheduler was in the user’s computer. In Thetis, the user gets to decide how far he wants to push the envelope with respect to latency and glitch free operation. You did exactly the right thing: start at 120ms and very slowly work your way down a few ms at a time.
The user may profit from an indicator showing the actual ringbuffer size in effect. Users may also profit from indicators showing what settings are in effect when the manual boxes are unchecked. Those sorts of things can certainly be added in future releases if need be.
73, Bryan W4WMT
Re: Resampler Adjustment
Bryan, thanks for the information. I had to reboot for unrelated reasons and when I brought Thetis up again, the audio was rough with the same settings I showed in the screenshots. I went back to full auto and everything evened out. Again, I have no idea whether the manual settings that did at one time work even provide less latency than full auto. That would be interesting to know but I assume that the auto settings could be different for everybody and vary from moment to moment. I'll keep playing with this and report back.
P.S. Because my outboard digital audio processor is an AM broadcast processor, there is so much freaking latency involved that all of this is purely academic for me. But, I'll pretend like it matters on my machine and keep experimenting and reporting.
P.S. Because my outboard digital audio processor is an AM broadcast processor, there is so much freaking latency involved that all of this is purely academic for me. But, I'll pretend like it matters on my machine and keep experimenting and reporting.
73 Brent WA0VY
Re: Resampler Adjustment
I have found through extensive experimentation that solid resampling settings are elusive and remain a moving target. Settings I find to work great during one radio “session” only seem to fail the next time I fire up. I always listen to myself using the monitor function while wearing a headset, and this is where latency is critical (for me). Setting everything to automatic (unchecking manual) reliably eliminate clicks pops and glitches, but latency jumps to Im guessing 50 ms or more, which really screws the brain while trying to listen to yourself.
I’ve settled on manual ringbuffer settings of 20ms, with non forced resampling. My behringer umc is sampling at 96K with a 256 buffer, but this sample and buffer doesn’t seem to have much impact. The ASIO drivers have their own latency issues, but are way better than MME. PortAudio has no effect with ASIO drivers it appears. 20-25 ms seems to be right at the upper limit of my brain. But I bet, when I get into the shack this morning, even these settings won’t seem right. Restarting the software, or rebooting the PC seems to change the dynamic as well. Good days and bad days.
I realize latency means different things to different people, but for me it’s all about hearing myself without my mouth tripping over my brain. I’m 99.9% headset ops.
By the way, launching or surfing a browser totally glitches VAC regardless of settings. I’ve resorted to browsing inside a VMware virtual window for browsing, which helps a lot, but does not totally eliminate glitching.
I choose to use VAC for microphone ops because to me, the audio sounds clean and clearer than when using the front panel mic jack. That panel jack and the circuit behind it introduce a slight but noticeable hash,
or hiss which I dont think is detectable on air, but very evident in the monitor.
The struggle is real.
Jeff
NJ2US
I’ve settled on manual ringbuffer settings of 20ms, with non forced resampling. My behringer umc is sampling at 96K with a 256 buffer, but this sample and buffer doesn’t seem to have much impact. The ASIO drivers have their own latency issues, but are way better than MME. PortAudio has no effect with ASIO drivers it appears. 20-25 ms seems to be right at the upper limit of my brain. But I bet, when I get into the shack this morning, even these settings won’t seem right. Restarting the software, or rebooting the PC seems to change the dynamic as well. Good days and bad days.
I realize latency means different things to different people, but for me it’s all about hearing myself without my mouth tripping over my brain. I’m 99.9% headset ops.
By the way, launching or surfing a browser totally glitches VAC regardless of settings. I’ve resorted to browsing inside a VMware virtual window for browsing, which helps a lot, but does not totally eliminate glitching.
I choose to use VAC for microphone ops because to me, the audio sounds clean and clearer than when using the front panel mic jack. That panel jack and the circuit behind it introduce a slight but noticeable hash,
or hiss which I dont think is detectable on air, but very evident in the monitor.
The struggle is real.
Jeff
NJ2US
Re: Resampler Adjustment
It remains a mystery why so many people are having such trouble. There is clearly something we all don't understand about how buffers should be set. Most serious audio software adjusts buffering in a completely automatic way with no significant latency. Any DAW application is a good example of this. If any developer wants to tackle this I'd suggest looking at the open source DAW app "Ardour" for which the source code is freely available.
My audio setup is quite complex: Thetis, Reaper and Presonus Studio 192 Mobile all are connected to each other via Voicemeeter Potato using ASIO. I don't really need Reaper anymore but I enjoy the control it gives me over routing audio. The Presonus driver is set to a buffer size of 128 and a sample rate of 96KHz, as is everything else. Digi mode app's, when I use them, are also connected via Voicemeeter at 48KHz and Voicemeeter happily does the rate translations. With all Thetis audio processing turned off latency is less than 30mS between audio sink/source and the antenna (not fair to count latency added by Thetis audio processing like NR2 or the leveler, etc.)
Note that the reason I use 96KHz is because I had reached the lowest available VAC buffer size of 128 at 48KHz and things remained stable at 96KHz, hence buffers take half as long to traverse at the higher sample rate. It's not a big difference, but a millisecond here and a millisecond there and pretty soon you are talking about a lot of milliseconds
FWIW here is a screen shot of my VAC settings:
I have two, minor stability issues. Every once in a while Reaper does not start correctly and sound is messed up. I simply restart Reaper and it will remain stable for days at a time. And I get periodic instability from the resampler, which I correct by cycling one of the "manual" checkboxes in order to reset it.
73,
Scott
P.S. Jeff: do you seek merely sidetone, or a true representation of your audio output? In either case you aren't going to get it from MON. IMHO, MON, as implemented, is useless except as a quick way to check that your VAC TX audio is not broken.
My audio setup is quite complex: Thetis, Reaper and Presonus Studio 192 Mobile all are connected to each other via Voicemeeter Potato using ASIO. I don't really need Reaper anymore but I enjoy the control it gives me over routing audio. The Presonus driver is set to a buffer size of 128 and a sample rate of 96KHz, as is everything else. Digi mode app's, when I use them, are also connected via Voicemeeter at 48KHz and Voicemeeter happily does the rate translations. With all Thetis audio processing turned off latency is less than 30mS between audio sink/source and the antenna (not fair to count latency added by Thetis audio processing like NR2 or the leveler, etc.)
Note that the reason I use 96KHz is because I had reached the lowest available VAC buffer size of 128 at 48KHz and things remained stable at 96KHz, hence buffers take half as long to traverse at the higher sample rate. It's not a big difference, but a millisecond here and a millisecond there and pretty soon you are talking about a lot of milliseconds
![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
FWIW here is a screen shot of my VAC settings:
I have two, minor stability issues. Every once in a while Reaper does not start correctly and sound is messed up. I simply restart Reaper and it will remain stable for days at a time. And I get periodic instability from the resampler, which I correct by cycling one of the "manual" checkboxes in order to reset it.
73,
Scott
P.S. Jeff: do you seek merely sidetone, or a true representation of your audio output? In either case you aren't going to get it from MON. IMHO, MON, as implemented, is useless except as a quick way to check that your VAC TX audio is not broken.
Re: Resampler Adjustment
First, thank you for everything you do in support of OpenHPSDR and the Apache equipment, I hang on your every post.
My experience with Monitor appears to be very personal, and likely different from yours I expect. I find it very helpful, as it does seem to fairly represent my TX audio before it's converted to RF, and I like that. It renders my bandwidth and EQ faithfully, at least in my headphones. And as long as I can keep the latency ahead of my brain, I would rather listen to "Monitor" than a simple sidetone like Flex does.
Your VAC settings are IDENTICAL to mine, including the 96K/128 UMC, but my ringbuffers appear to be more stable at 20ms, with just a minor latency impact.
Jeff NJ2US
My experience with Monitor appears to be very personal, and likely different from yours I expect. I find it very helpful, as it does seem to fairly represent my TX audio before it's converted to RF, and I like that. It renders my bandwidth and EQ faithfully, at least in my headphones. And as long as I can keep the latency ahead of my brain, I would rather listen to "Monitor" than a simple sidetone like Flex does.
Your VAC settings are IDENTICAL to mine, including the 96K/128 UMC, but my ringbuffers appear to be more stable at 20ms, with just a minor latency impact.
Jeff NJ2US
Re: Resampler Adjustment
I don't mean to beat a dead horse, but I still don't understand the following:
1. If 40 ms is as low as I can go in a manual buffer latency setting and run stable, am I better off running auto or manual? (I suspect there may be no way to answer this question, but in case there is......).
2. I have no information as to what PortAudio does. There are no flags appearing when I mouse over and I am unfamiliar with this setting. It was observed in a prior post that it may not have any effect when running VAC. This is certainly NOT true on my radio. If I try to run it manually, everything tends to fly apart.
1. If 40 ms is as low as I can go in a manual buffer latency setting and run stable, am I better off running auto or manual? (I suspect there may be no way to answer this question, but in case there is......).
2. I have no information as to what PortAudio does. There are no flags appearing when I mouse over and I am unfamiliar with this setting. It was observed in a prior post that it may not have any effect when running VAC. This is certainly NOT true on my radio. If I try to run it manually, everything tends to fly apart.
73 Brent WA0VY
Re: Resampler Adjustment
Scott, thanks for mentioning "cycling one of the manual checkboxes" to fix a "runaway". I have these sometimes too and it happened today after I read your post. I cycled the checkbox on the "In" box and it immediately stopped and behaved afterwards. I have been seeing this and thinking something in the system had changed and was causing a latency problem but now it seems this is a glitch in Thetis; at least it appears that way to me.
Joe W4WT
Joe W4WT
- Tony EI7BMB
- Posts: 653
- Joined: Sun Apr 09, 2017 2:31 pm
- Location: Dublin
- Contact:
Re: Resampler Adjustment
Just moved to 96K VAC and seems to be still smooth. I'll Adjust the Buffer latency over the coming days so see how low it will go. Using a Ryzen 1700 Pc windows 10 and UMC204HD interface.
Re: Resampler Adjustment
Helmut,
When force is off, are you getting under or overflow counts in a steady stream on the resampler?
73,
Scott
When force is off, are you getting under or overflow counts in a steady stream on the resampler?
73,
Scott
Re: Resampler Adjustment
I've never been able to duplicate this problem. This is what I get with the resampler fully active in Thetis (and the same in PowerSDR):
Fast AGC, Linear Phase Filter, optimized ATT, I'm getting plenty of -24dB decodes (can you do better than -24?)
Fast AGC, Linear Phase Filter, optimized ATT, I'm getting plenty of -24dB decodes (can you do better than -24?)
Re: Resampler Adjustment
Hi Helmut,
Am I reading your waterfall display correctly? It’s sweeping back and forth approximately 25 Hz?
I don’t have FT8 running on my 7000DLEMKI, but I know it’s not drifting 25Hz, I would hear that instantly in SSB.
Does the sweeping eventually slow down and stop? The reason I ask is because the loop filter used in the adaptive variable resampler is huge and can take over 10 minutes to settle down (at some sampling rates).
Still though, drift excursions in the realm of 25 Hz is very hard to explain.
73, Bryan W4WMT
Am I reading your waterfall display correctly? It’s sweeping back and forth approximately 25 Hz?
I don’t have FT8 running on my 7000DLEMKI, but I know it’s not drifting 25Hz, I would hear that instantly in SSB.
Does the sweeping eventually slow down and stop? The reason I ask is because the loop filter used in the adaptive variable resampler is huge and can take over 10 minutes to settle down (at some sampling rates).
Still though, drift excursions in the realm of 25 Hz is very hard to explain.
73, Bryan W4WMT
Re: Resampler Adjustment
Helmut,
What happens if you uncheck all "Manual" checkboxes and adjust VAC buffer size to the minimum that will work properly?
Also, do you listen to the actual audio coming out of VAC to determine if it sounds OK, by ear? This is easy to do with Voicemeeter. Make sure you have a valid audio output device defined as A1 (speakers attached to your PC) and turn on the A1 route on whichever virtual channel you are using for Thetis (or PowerSDR). This can be VERY helpful in adjusting VAC settings rather than doing it by looking at the waterfall in WSJT-X.
73,
Scott
What happens if you uncheck all "Manual" checkboxes and adjust VAC buffer size to the minimum that will work properly?
Also, do you listen to the actual audio coming out of VAC to determine if it sounds OK, by ear? This is easy to do with Voicemeeter. Make sure you have a valid audio output device defined as A1 (speakers attached to your PC) and turn on the A1 route on whichever virtual channel you are using for Thetis (or PowerSDR). This can be VERY helpful in adjusting VAC settings rather than doing it by looking at the waterfall in WSJT-X.
73,
Scott
Re: Resampler Adjustment
And there you go: you just have to find the combo that works! My recommendation, as in the original write up, is find the VAC buffer size that works the best with everything else in "auto" first, then start fooling with the manual settings.
73!
Scott
73!
Scott
Re: Resampler Adjustment
w-u-2-o wrote:It remains a mystery why so many people are having such trouble. There is clearly something we all don't understand about how buffers should be set. Most serious audio software adjusts buffering in a completely automatic way with no significant latency. Any DAW application is a good example of this. If any developer wants to tackle this I'd suggest looking at the open source DAW app "Ardour" for which the source code is freely available.
My audio setup is quite complex: Thetis, Reaper and Presonus Studio 192 Mobile all are connected to each other via Voicemeeter Potato using ASIO. I don't really need Reaper anymore but I enjoy the control it gives me over routing audio. The Presonus driver is set to a buffer size of 128 and a sample rate of 96KHz, as is everything else. Digi mode app's, when I use them, are also connected via Voicemeeter at 48KHz and Voicemeeter happily does the rate translations. With all Thetis audio processing turned off latency is less than 30mS between audio sink/source and the antenna (not fair to count latency added by Thetis audio processing like NR2 or the leveler, etc.)
Note that the reason I use 96KHz is because I had reached the lowest available VAC buffer size of 128 at 48KHz and things remained stable at 96KHz, hence buffers take half as long to traverse at the higher sample rate. It's not a big difference, but a millisecond here and a millisecond there and pretty soon you are talking about a lot of milliseconds![]()
FWIW here is a screen shot of my VAC settings:
vac.JPG
I have two, minor stability issues. Every once in a while Reaper does not start correctly and sound is messed up. I simply restart Reaper and it will remain stable for days at a time. And I get periodic instability from the resampler, which I correct by cycling one of the "manual" checkboxes in order to reset it.
73,
Scott
P.S. Jeff: do you seek merely sidetone, or a true representation of your audio output? In either case you aren't going to get it from MON. IMHO, MON, as implemented, is useless except as a quick way to check that your VAC TX audio is not broken.
On the 8k, using the rear headphone jack.. I get sidetone with very little latency.
This I feed into powered speakers with a headphone jack for concentrated CW listening.
This requires Mon to be on.
Re: Resampler Adjustment
TL;DR? Read the bold bits.
Yesterday I got the idea to use WSJT-X’s FreqCal mode to do the frequency calibration on my 200D. WSJT’s FreqCal process is complex and goes through a variety of standard frequency stations to make a curve of needed offset vs. frequency. I wanted to just perform a simple cal at one frequency, so I deleted all the cal frequencies in its list except 15.000 MHz (WSJT-X: Settings > Frequencies > mode=FreqCal). Then I was able to run the frequency calibration continuously and observe the effect of adjusting in Thetis (General > Calibration > Freq Cal).
I was going to start tweaking the Thetis cal setting, but first I noticed something strange. The frequency error amount was continuously varying, by itself, up and down about +/- 2 Hz. The variation period also varied, but it was about 30 seconds on average. (2 Hz of variation is too small for me to identify clearly in the WSJT-X wide graph, even on the WWV carrier, but it’s easy to measure in the WSJT FreqCal mode.)
The WSJT-X measurement uses VAC, so one of my first thoughts led me to the VAC resampler in Thetis. It was turned on and seemed to be operating normally. On a lark, I disabled the resampler by checking both Force checkboxes. The frequency variation immediately became about +/- 0.2 Hz – a 10x improvement in frequency stability. I repeated this several times and confirmed the checkboxes were having this effect every time.
I have operated with both Force checkboxes checked for several hours on FT8 since making this discovery about freq. stability. There is so much randomness associated with FT8 operating (band conditions, number of station in the band segments), that I am hesitant to state this report: My success rate with making FT8 QSOs is greatly improved.
(By the way, I monitored for under- and underflows with the Force checkboxes checked. The Var Ratio numbers stayed very close to 1; i.e., >0.99 and < 1.01. Under and overflow numbers stayed at 0. No audible glitches on RX or TX.)
I Googled to try to learn what the frequency stability requirement is for FT modes. The first useful link I found was http://www.w0wtn.org/downloads/n0dl/Introduction%20to%20Ham%20Radio%20Digital%20Mode%20FT8.pdf which contains "To decode an FT8 signal we need to determine its frequency within ~1 Hz...", written by K1JT. Later I found http://laarc.weebly.com/uploads/7/3/2/9/73292865/ft8syncv8.pdf, which says "FT8 requires that transmitting stations and receiver be synchronized to better than 20 ms in time and 1 Hz in frequency. Such precision is generally not possible with amateur radio equipment using external reference locks, so the protocol must supply its own synchronization signal. The different WSJT-X modes accomplish this in a variety of ways, depending on the design requirements. FT8 uses a 7x7 Costas Array."
Hmm… so FT8 should be able to self-correct some frequency errors. Perhaps the +/- 2 Hz wobbulation caused by the resampler disrupts the FT decoder’s ability to perform its internal synchronization.
73, John
Yesterday I got the idea to use WSJT-X’s FreqCal mode to do the frequency calibration on my 200D. WSJT’s FreqCal process is complex and goes through a variety of standard frequency stations to make a curve of needed offset vs. frequency. I wanted to just perform a simple cal at one frequency, so I deleted all the cal frequencies in its list except 15.000 MHz (WSJT-X: Settings > Frequencies > mode=FreqCal). Then I was able to run the frequency calibration continuously and observe the effect of adjusting in Thetis (General > Calibration > Freq Cal).
I was going to start tweaking the Thetis cal setting, but first I noticed something strange. The frequency error amount was continuously varying, by itself, up and down about +/- 2 Hz. The variation period also varied, but it was about 30 seconds on average. (2 Hz of variation is too small for me to identify clearly in the WSJT-X wide graph, even on the WWV carrier, but it’s easy to measure in the WSJT FreqCal mode.)
The WSJT-X measurement uses VAC, so one of my first thoughts led me to the VAC resampler in Thetis. It was turned on and seemed to be operating normally. On a lark, I disabled the resampler by checking both Force checkboxes. The frequency variation immediately became about +/- 0.2 Hz – a 10x improvement in frequency stability. I repeated this several times and confirmed the checkboxes were having this effect every time.
I have operated with both Force checkboxes checked for several hours on FT8 since making this discovery about freq. stability. There is so much randomness associated with FT8 operating (band conditions, number of station in the band segments), that I am hesitant to state this report: My success rate with making FT8 QSOs is greatly improved.
(By the way, I monitored for under- and underflows with the Force checkboxes checked. The Var Ratio numbers stayed very close to 1; i.e., >0.99 and < 1.01. Under and overflow numbers stayed at 0. No audible glitches on RX or TX.)
I Googled to try to learn what the frequency stability requirement is for FT modes. The first useful link I found was http://www.w0wtn.org/downloads/n0dl/Introduction%20to%20Ham%20Radio%20Digital%20Mode%20FT8.pdf which contains "To decode an FT8 signal we need to determine its frequency within ~1 Hz...", written by K1JT. Later I found http://laarc.weebly.com/uploads/7/3/2/9/73292865/ft8syncv8.pdf, which says "FT8 requires that transmitting stations and receiver be synchronized to better than 20 ms in time and 1 Hz in frequency. Such precision is generally not possible with amateur radio equipment using external reference locks, so the protocol must supply its own synchronization signal. The different WSJT-X modes accomplish this in a variety of ways, depending on the design requirements. FT8 uses a 7x7 Costas Array."
Hmm… so FT8 should be able to self-correct some frequency errors. Perhaps the +/- 2 Hz wobbulation caused by the resampler disrupts the FT decoder’s ability to perform its internal synchronization.
73, John
Re: Resampler Adjustment
W1JA wrote:
(By the way, I monitored for under- and underflows with the Force checkboxes checked. The Var Ratio numbers stayed very close to 1; i.e., >0.99 and < 1.01. Under and overflow numbers stayed at 0. No audible glitches on RX or TX.)
73, John
Hi John,
The "silver lining" here is that your SDR and WSJT are so closely matched in frequency that you don't need the adaptive re-sampler in the first place (as evidenced by the overflow/underflow counters remaining at zero over goodly periods of time).
It was already known (ref Stu) that the re-sampler imposes a small vfo frequency error (proportional to the "Var Ratio"), but I don't remember anyone seeing an error as large as 2 Hz (or anything close). Also, I think you're the first to notice a wobble (probably the servo loop hunting around). Very interesting report.
Have you tried using a GPSDO to lock your 200D?
73 W4WMT
Re: Resampler Adjustment
Thanks, Bryan. I don't have a GPSDO. But I question what benefit it would have. If the remaining 0.2-Hz instability is coming from the reference oscillator in my 200D, it seems that it's plenty good enough for excellent FT mode performance. And, as you noted, I'm already lucky to have both oscillators matched well enough to not need the resampler. Been running for five hours so far today, and over and underflows are still at 0. (Last evening, after running all day, I did notice some nonzero numbers, but there were no audible glitches -- or I was not in the shack to hear them.)
73, John
73, John
Re: Resampler Adjustment
John, W1JA, after reading your comments re freq cal and improvements on FT8 I was quite confused - I couldn't see how there would be any connection or correction for that matter by doing this since FT8 is relative and not absolute (its not tied to an absolute frequency)... but I did it anyway - and i posted on the WSJT-X forum asking about this .... today I got a response from Joe himself, K1JT who said "not sure where you got the idea this would help - it doesn't" ... so indeed JUST for clarification Freq Cal using WSJT-X is NOT needed and has no bearing on decodes for FT8.
Of course there was a mixture of that AND resampler discussion in your comment - and it obviously is the resampler that was causing issues... I'll freely admit that I (emphasis) made the connection betweeen FRQ CAL and 'improvements' ... so my bad there... but I'm posting this so others won't go down an irrelevant path like I did.
Gary
K9RX
Of course there was a mixture of that AND resampler discussion in your comment - and it obviously is the resampler that was causing issues... I'll freely admit that I (emphasis) made the connection betweeen FRQ CAL and 'improvements' ... so my bad there... but I'm posting this so others won't go down an irrelevant path like I did.
Gary
K9RX
Re: Resampler Adjustment
Gary,
JT is of course correct - but I don't believe John (W1JA) was asserting that the WSJT-X cal function improves things (we have discussed it). He used WSJT-X cal only to observe frequency instability in Thetis as a function of the Thetis resampler setup, and saw a 10x difference in frequency stability between the two settings. All adjustments he's talking about are in Thetis, not WSJT-X. What he observed caught my interest.
The absolute variance in frequency was merely 2Hz (worst case) or less. While that small amount may have an effect on encode/decode effectiveness, what might have more pronounced effect is its rate of change during a single 15-second FT8 cycle. Maybe I'm misunderstanding something but it seems to me that this wobble rate might have a greater effect than the drift magnitude, since FTx modes use phase modulation.
At any rate, I tried it myself, checking the "force" boxes in the Thetis resampler and noticed a similar improvement. In totally unscientific, subjective observations while operating I found a noticeable increase in my ability to get responses. Interestingly, I also noticed that the lopsidedness of signal reports evened out. By this I mean I have consistently noticed that I receive lower reports than I send, many more times than the other way around (i.e. when I receive a higher report than I send). On making the change this evened out dramatically. This is as close to a measurable thing I've come across so far.
Will continue to experiment.
JT is of course correct - but I don't believe John (W1JA) was asserting that the WSJT-X cal function improves things (we have discussed it). He used WSJT-X cal only to observe frequency instability in Thetis as a function of the Thetis resampler setup, and saw a 10x difference in frequency stability between the two settings. All adjustments he's talking about are in Thetis, not WSJT-X. What he observed caught my interest.
The absolute variance in frequency was merely 2Hz (worst case) or less. While that small amount may have an effect on encode/decode effectiveness, what might have more pronounced effect is its rate of change during a single 15-second FT8 cycle. Maybe I'm misunderstanding something but it seems to me that this wobble rate might have a greater effect than the drift magnitude, since FTx modes use phase modulation.
At any rate, I tried it myself, checking the "force" boxes in the Thetis resampler and noticed a similar improvement. In totally unscientific, subjective observations while operating I found a noticeable increase in my ability to get responses. Interestingly, I also noticed that the lopsidedness of signal reports evened out. By this I mean I have consistently noticed that I receive lower reports than I send, many more times than the other way around (i.e. when I receive a higher report than I send). On making the change this evened out dramatically. This is as close to a measurable thing I've come across so far.
Will continue to experiment.
73,
Chris, W2PA
Chris, W2PA
Re: Resampler Adjustment
Gary, I never intended to mean that absolute frequency accuracy has anything to do with FT mode performance. (As long as one is in an FT sub-band.
) I was reporting what I did chronologically, and I did start out with a desire to check the absolute accuracy of my 200D. Maybe I should have left that part out.
Chris, I'm glad you notice this effect also. For me, it's a night-and-day difference. I was almost to the point of giving up on the FT-modes. I also notice the same thing you mentioned about signal reports.
Another thing: When I started with FT-8 in 2017, I didn't have this problem. It developed only in recent months. What changed, I wonder. A firmware change? A degradation in the reference oscillator in our Apache-Labs hardware? Or maybe it's just the more-crowded FT sub-bands.
![Wink ;)](./images/smilies/icon_e_wink.gif)
Chris, I'm glad you notice this effect also. For me, it's a night-and-day difference. I was almost to the point of giving up on the FT-modes. I also notice the same thing you mentioned about signal reports.
Another thing: When I started with FT-8 in 2017, I didn't have this problem. It developed only in recent months. What changed, I wonder. A firmware change? A degradation in the reference oscillator in our Apache-Labs hardware? Or maybe it's just the more-crowded FT sub-bands.
Re: Resampler Adjustment
thanks guys - I stand fully corrected. Keep up the great work.
Gary
K9RX
Gary
K9RX
Re: Resampler Adjustment
Not seeing this problem here. Running Thetis 2.7.0 (a3). I used WWV on 10MHz around 1030 local time in DIGU mode as the calibration signal. Resampler on or off, I saw some wobbulation (I love that term
) on the order of +/- 0.2Hz, and during any 15 second epoch (commensurate with FT8 transmission length) it was typically but not always much less than that.
I need to do this experiment over again with a signal generator so that I can remove ionospheric variability from the measurement.
If you are having trouble with the resampler and wish to use it try changing your VAC buffer size and/or ringbuffer settings.
Pay no attention to the WWV sidebands in the plots below, they changed like crazy depending on what was being modulated by WWV at the time. Only the carrier counts. Band conditions also changed quite a bit over the 20 minutes I played with this.
Resampler ON:
Resampler OFF:
![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
I need to do this experiment over again with a signal generator so that I can remove ionospheric variability from the measurement.
If you are having trouble with the resampler and wish to use it try changing your VAC buffer size and/or ringbuffer settings.
Pay no attention to the WWV sidebands in the plots below, they changed like crazy depending on what was being modulated by WWV at the time. Only the carrier counts. Band conditions also changed quite a bit over the 20 minutes I played with this.
Resampler ON:
Resampler OFF:
Re: Resampler Adjustment
Posting my results, using latest versions of SW and FW on ANAN-8000DLE. One is re-samper-on ("Force" unchecked) and one is re-sampler off ("Force" checked). In a one-minute time period at 15 MHz (WWV), I get 0.297 Hz variation checked, 3.4 Hz unchecked - a factor of 11.4 difference in "wobbulation".
(BTW, notice on the right, I'd just earlier worked 3D2AG on 17m with re-sampling off. I'd been unable to get a reply with re-sampling on. Off took 2 calls.)
(BTW, notice on the right, I'd just earlier worked 3D2AG on 17m with re-sampling off. I'd been unable to get a reply with re-sampling on. Off took 2 calls.)
- Attachments
-
- 2020-05-22.png (71.7 KiB) Viewed 57932 times
-
- 2020-05-22 (1).png (69.41 KiB) Viewed 57932 times
73,
Chris, W2PA
Chris, W2PA
Re: Resampler Adjustment
Chris,
How are you getting it to make any measurements in FT8 mode? I have to be in Freq Cal mode to get it work.
What happens when you put WSJT-X in Freq Cal mode and select DIGU on the radio?
I just did another test with 2.2.0RC1. No difference. It was a bit better, but I think that's because the ionosphere for this trial was less active.
I also plotted the frequency deltas versus time. No real pattern, which again suggested random, ionospheric induced propagation effects.
73,
Scott
How are you getting it to make any measurements in FT8 mode? I have to be in Freq Cal mode to get it work.
What happens when you put WSJT-X in Freq Cal mode and select DIGU on the radio?
I just did another test with 2.2.0RC1. No difference. It was a bit better, but I think that's because the ionosphere for this trial was less active.
I also plotted the frequency deltas versus time. No real pattern, which again suggested random, ionospheric induced propagation effects.
73,
Scott