Per the design of the digital filter implementation, the passband frequencies you specify represent the -6dB point on the filter response curve.
While even the smallest (really shortest, but I'll stick with the nomenclature found in Setup > DSP > Options) filter size is, relatively speaking, a brick wall compared to filtering found in typical analog rigs, it still has a definite skirt that starts well before the -6dB point.
You can sharpen up the skirt to an amazing degree and achieve a virtual brick wall filter shape by going to the largest filter size available, which is 16384, and by staying with the BH-4 filter window type.
Paraphrasing material that Warren, NR0V, the author of this code, published some time ago:
Bandpass filter response is a function of two settings: (1) choice of window function (BH-4 or BH-7) and (2) choice of filter size (1024 / 2048 / 4096 / 8192 /16384). Bandpass filter response is also a function of internal DSP sample rate but that is fixed at 48K for Receive/non-FM in current openHPSDR PowerSDR releases.
For the BH-4 window, initial rejection (at first side lobe) is ~-110dB and ultimate rejection is ~-140dB.
For the BH-7 window, initial and final rejection are ~-200dB.
The advantage of the BH-4 window is slightly sharper cutoff for a given buffer size. The advantage of the BH-7 window is the depth of the final cutoff.
In the following example, a passband of 0Hz to 1000Hz is shown. The shape of the skirts will remain the same regardless of passband definition, so these shapes will hold for any configuration, i.e. the curves shown below completely define the filter response. Note also that the upper transition region is just a mirror image of the lower transition region.
You may note that doubling the filter size halves the width of the transition region (skirt).
Again, the specified passband upper and lower frequencies (in this example, 0Hz and 1000Hz) are the -6dB frequencies, by design.
Which filter type you choose, Linear Phase or Low Latency (really minimum phase), will not affect the filter shape in amplitude. The phase response of the low latency filter type is not linear, however any phase non-linearity is generally inaudible, and the ionosphere has a much larger effect on this anyhow. The big advantage of the low latency filter is that you can run any size filter you like in order to achieve very steep skirts and not take a major penalty in receive or transmit latency (e.g. at 16384 is is <20mS in the low latency filter and >180mS in the linear phase filter).