LabRadar and Signal-to-Noise Ratio

[rsterne]

I have been corresponding with Ron Burnett, who is working with his LabRadar on trying to develop Drag Coefficient curves for pellets.... Ron has tested both .177 cal and .22 cal pellets, and after some distance downrange he is finding that the data becomes inconsistent.... He shoots over an open field with no obstructions, and a long distance behind the target before any trees.... He has determined that when the Signal-to-Noise Ratio (SNR) drops below about 20 is when the data becomes inconsistent.... In his setting, that is at about 50-55 yards for .177 pellets and at about 75 yards for .22 cal.... even though there is a radar return at much longer ranges....

I would suggest that you keep this in mind if you have a LabRadar…. The range at which this occurs will depend on your surroundings, and the presence of anything that might disrupt or reflect the radar.... The SNR does not drop smoothly, but you will see a general decrease with range.... You may get the occasional SNR below 20 while most are still above.... That data may, or may not, be good data.... but once most of the SNR levels (which are reported in the detailed shot string downloads) are below 20, your data may be questionable.... This most frequently shows up as velocity readings that are faster than the preceeding one, which of course is impossible.... and leads to negative Cd calculations (again, impossible) for single data pairs….

Bob

[JungleShooter]

Thanks for that update, Bob.
You say things the way they are. 👍🏼I've recently gotten a little tired of reviewers where anything they test is "badas$$" — or where they mostly "test" products from one brand, and similar games....
 
 
So, no .22 pellet BCs beyond 75y... — disappointing, as it may be to me and many others.... 😟 
 
 
Well, I guess it's back to two chronos, fortified with heavy acrylic glass, and lots of shots....
And that method is even within my own rather limited $-possibilities...!  👍🏼 😊 
 
Matthias

[rsterne]

I think the performance is at least as good as LabRadar claim on their website....

.177 Pellet – 30 yards;
 .177 BB – 30 yards;
 22 LR – 60 yards;
 223 – 60 yards;
 270 – 70 yards;
 308 – 80 to 100 yards;
 9mm – 130 yards;
 40 S&W – 130 yards;
 45 ACP – 130 yards;
 500 S&W – 130 yards;
 12 gauge Slug – 90 yards;
 Paint Ball – 50 yards;
 Arrows – 50 yards;

I'm not at all disappointed with the results Ron is getting.... I hope I can duplicate it.... Ron says the background and terrain is important....

Bob

[WobblyHand]

To extract meaningful parameters (range, velocity, angle) from radar data one generally needs higher signal to noise ratios (SNR).  The higher the SNR, the lower the standard deviation of the parameter.  So for low SNRs (like 20 [13dB]) the uncertainty of the measurement can be high.  The mean value of the measurement is generally correct. 

The standard deviation of the velocity is given by

V(rms) = lambda/(2T*sqrt(2*SNR)),

where lambda = c/fc, c is the speed of light, fc is the radar carrier frequency, T is the total measurement time, and the SNR is the ratio of the power of the signal to the power of the noise. 

I don't know the carrier frequency of the Labradar, but let's assume it is 10.525 GHz (X-band) for the moment.  So lets plug in some numbers.  Just guessing these parameters, but, let lambda = 3e8/10.525e9 = 2.85e-2 [meters], T=10e-3 [seconds] (10ms), SNR=20 [unitless], then

Velocity(rms) = 2.85e-3/(2*10e-3*sqrt(20)) = 0.32 [meters/sec]. 

That's not that bad, about 1.04 fps.  That's the standard deviation of the velocity for a look time of 10ms.  At a pellet velocity of 300 m/sec, as an example, the pellet travels 3m.  If one wants to know the velocity in a particular range that is finer than 3m, the dwell time is less, and the standard deviation of the measurement increases.  If we wish to know the velocity within a 98.6% certainty, than all we can say is the velocity is between plus or minus 3* Vrms.  (plus or minus 3 sigma).  If that's good enough, then great!

When I designed radars, I tried to use a minimum SNR of at least 20dB [100] as my goal.  This usually resulted in a lot of work for myself, as it is a challenge to get high SNRs for little (low radar cross section) targets.  But using 20dB allowed me single look parameter estimation at a low standard deviation.

Take away is, single low SNR measurements are not good for calculating BC or anything else.  The only way to get around this is to 1) improve the SNR of the radar (usually not easy) or 2) take multiple measurements to reduce the measurement uncertainty.  SNR improvements can be made by increasing radar transmit power, increasing the size of the antenna, reducing the range of observation, increasing the target size, improving the receiver noise figure, lower loss signal processing or increasing observation time.  Practically speaking, it's hard to improve, especially if one doesn't have internal access to the radar.  The only thing a user could do is take multiple measurements and average them.

So Bob, your colleagues observations are consistent with theory. 

One last thing, SNR declines as 1/R^4 for a point target.  SNR gets really small really quickly.  For every doubling of the range the SNR declines by 12dB (a factor of 16)  Also there is such a thing as multipath, which can be constructive, or destructive.  Multipath can make some target ranges be undetectable, while further ranges are ok.  If one is bothered by multipath (and the target should be detectable at that range) usually a small change in height of the target (or the radar) can allow one to detect at that range. 

[rsterne]

Thanks, I got a part of that, anyways.... 

Bob

[WobblyHand]

Sorry to overcomplicate that...  Hah, this is how I feel sometimes when you are explaining stuff!   

Takeaway is the measurement isn't exact, it's an estimate, which is a random number with a mean value that is extremely close to the actual value.  The standard deviation of the measurement (the random part) is normally distributed about the mean (bell curve).  Low SNR measurements have higher standard deviation (wider and lower bell curve).  To accurately estimate the mean we need to either take a lot of measurements and take the average of the measurements, or have a high SNR measurement whose standard deviation is tiny enough (narrow and tall bell curve) to ignore.

So when you mentioned the measurements showing velocity occasionally increased as the range increased what was observed was the random component of the measurement being large.  (The SNR was poor at that range.)  The random component was large enough, and on the wrong side of the distribution which made it look non-causal.  Because, as we all know, for a non-actively propelled pellet, (out of the muzzle) the velocity is always decreasing.  If you take enough measurements and take the mean, the value will converge towards the true value.  If you took 10 measurements and averaged the results, then the standard deviation of the final result would reduce by a factor of roughly 3.  Or 100 measurements, then the standard deviation would reduce by a factor of 10.  It's no fun, but that's the only option one has to get high quality (low error) measurements without modifying the radar.  Then one has to worry about the repeatability of the shots themselves. 

Random numbers and statistics can be difficult to digest.  Parameter estimation aka measurements have errors if one looks closely enough.  The errors, or more accurately, the standard deviation of (velocity) errors decrease with increasing SNR as 1/sqrt(SNR) as the original equation showed.

It's kind of like machining, nothing is really flat or smooth if one zooms in enough.  Even a surface plate.  Got a big enough magnifier, nothing is flat.  You could measure a single tiny point that was out of whack and still meet the rms smoothness specification. 

[rgb1]

Here is a composite of 10 shots generated this morning along with a sample of
extended data readout (with processing). Below Mn .63 (SNR 24, dist 200 ft) the
Cd curves begin to scatter and by Mn .6 (SNR 21, dist 240 ft) things have really
loosened up. In order to get good data further down range, the unit was moved
out 50 yards (shielded with a piece of 2x12) where it can be controlled with a
smart phone (app from LabRadar website). The average of the second set of shots
has been added to the composite, the red dashed line labeled "extended". Very
little blending (< 1%change) was required to mesh it with the first set average.


Bruce, the unit operates at 24 Ghz, actual transmitted power is only 3mW. The
readout is on a two millisecond interval but I have no idea of the sample time (your T).
Perhaps you could make an educated guess.


                                                                                     Ron

[rsterne]

Ron, thanks for posting that, it makes the problem perfectly clear.... Does the smart phone "App" require connection to a phone service, or does it work via Bluetooth?.... I have an Android tablet, but we have no cell service out here....

Bob

[rgb1]

Bluetooth, so your tablet should do the job provided
it's at least version 4.4 .

[WobblyHand]

Ron, interesting stuff.   Is the SNR in dB, or linear?  Theory predicts the SNR will reduce as 40*log₁₀(200/240) = -3dB.  (This is due to the 1/R⁴ in the radar range equation.)  The simple equation is valid, if the target is a point, and if there is no multipath (reflections off the ground or sides of buildings, etc.).  If your values for SNR are in dB, a drop of 3dB makes sense.  If not, either there is multipath happening, or something else.  Without any other knowledge of your setup (pictures of your test, and the environment) I can't even hazard a guess.

Pardon me, but I don't know the parameters that are in the graph.  Cd is the drag coefficient?  Mn is what?  It's my turn to be puzzled

Going back to the equation that I originally posted, we now have fc = 24GHz, so lambda = c/fc = 3e8/24e9 = 1.25e-2 m (1.25cm or about 1/2").  If the update time is 2ms, T has to be less than or equal to 2ms .  So let's see what T=2ms gives us.  I will use the MKS system of units to make our lives easier.  [meters, seconds, etc.]

V(rms) = lambda/(2*T*sqrt(2*SNR))      Eqn(1)

Since I don't know if SNR is in dB or not, I will use linear.  (Kind of like a worst case...)  SNR = 21 or 24

V(rms) = 1.25e-2/(2*2e-3*sqrt(2*21)) = 0.482 m/sec  for SNR = 21   (1.58fps)  1 sigma
           = 1.25e-2/(2*2e-3*sqrt(2*24)) = 0.451 m/sec  for SNR = 24   (1.48fps)  1 sigma
 
So let's assume that most of the time, the reports from the radar come back within +/-3 sigma of the mean value.  For a bell curve, that would be 99.7% of the time.  (The reports would be within +/-2 sigma, 95% of the time, and +/-1 sigma 68% of the time)  https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule  Another way of looking at this is 4.2% of the time, the report could be from 2-3 sigma from the mean.  On average, the reading could be "off" from the actual value by 3-5 fps, 4% of the time.  I'm not sure that these kinds of errors matter for your measurements or math, but it's good to be aware of these potential sources of error.  Do you have plots of the velocity vs distance that you can share?  I have no idea what the velocities are at 200 and 240 feet.  If they are within 5-20 fps of each other, the measurement errors (1 sigma, etc.) could be significant.  If they are more like 100fps different, then the errors wouldn't be significant.

If T = 0.2ms, then the errors would be 10x greater!  I'm guessing that Labradar is doing it the right way, and using the whole observation interval that is available.

Let me say that so far, from what I've heard reported about Labradar, I continue to be impressed.  It looks to be quite capable.

[rgb1]

Bruce, to help answer some of your questions.......
 - get a copy of the users manual, it has specs and general info about the unit


        mylabradar.com/download


 - click on the sample shot text file, most have overlooked it (only downloaded once)
 - Cd is drag coefficient, Mn is Mach number.


                           
                                                                              Ron

[WobblyHand]

Ron, just downloaded the User Manual and the Quick Setup Guide.  Can't seem to find the sample shot text file on the download page.  It could be right in front of my face...  Can you humor me and point it out?  I see two Firmware downloads, the User Manual & Quick Start Guide.

From what little I've looked at, Labradar outputs velocity at selected user programmable ranges.  How many ranges are available?  I presume you are doing the calculations for Cd and Mn.  Is that correct? 

Thinking out loud here; at 20C, the speed of sound is 343 m/s.  (Had to look that up.)  You have Mn = 0.6, give or take, and that corresponds to 205.8 m/s (675 fps)?  What I can't extract yet is the distance information, or the velocity vs distance. 

How are you determining Cd?  Is from a differential velocity change, or some other way?  This isn't my area of expertise, so if you can, please point me to where I can learn more about this.  Thanks. 

[WobblyHand]

Answering one of my questions:  How many ranges can Labradar report on.
From the FAQ: LabRadar will report the muzzle velocity and up to five other velocities at the distances that you select in increments of one (1). You will be able to select these distances prior to shooting. You can select feet, yards, meters, etc. Velocities can also be reported in metric and standard values.

Does this mean, you can get velocities at 200,201,202,203,204 feet ?  Is this what is used to estimate Cd?

[WobblyHand]

Found a paper on using radar for determining BC.  https://arxiv.org/ftp/arxiv/papers/1608/1608.06500.pdf  Downloaded it, now I have to digest it.  Unfortunately, all it says is that it can be done, but does not discuss the methodology.

[rgb1]

Bruce, the sample shot file is an attachment to my post, just above the graph,
easily overlooked. It's what we refer to as extended data and contains 200+
points. Bob and I don't use the five programmable readouts from the radar
screen, they're entirely inadequate for our work.
In a nutshell, Cd is determined by relating drag force to velocity loss. I use
a slightly different equation than the Courtneys.
One other thing......BC and Cd are two entirely different animals.


                                                                               Ron

[WobblyHand]

Yes!  The file was in front of my face!  I have it now.  The data is much more comprehensive than 5 values.  OK,  I need to plot some stuff to understand things.  Been trying to come up to speed on this.  Lot's to read.  Back with some plots...

Units are seconds, fps, feet, unknown, dimensionless, dimensionless  correct?

[rsterne]

The LabRadar generates two different kinds of reports and stores them on an SD card so that you can transfer them to a computer Excel file.... The "preset" distances are also reported on the screen, for example you can have it report the velocity at 0, 10,25, 50, 75 and 100 yards.... Those are also saved to one type of report.... The other report is much more detailed, and looks like this....



You can change the units in the first 3 columns to what you want (eg. m/s, yards, metres, etc)…. Note that once the bullet is acquired the data is reported every 2 mSec.… If you plot the velocity and range, you can get something like this....



The equations are for the trendlines generated by Excel, but we are finding that doing that too early in the process makes things worse, not better.... The Cd is calculated by the following equation....

Cd = -2*Mass*LN(V2/V1)/(Rho/32.174)/Area/(D2-D1)

Mass = bullet mass (slugs)
V2 = far velocity (fps)
V1 = near velocity (fps)
Rho = air density (lb/CF)
Area = bullet frontal area (sq.ft)
D2 = far range (ft.)
D1 = near range (ft.)

If you do this for each pair of data points, you can then plot the Cd vs the mach number.... The mach number is calculated by dividing the average velocity (V2+V1)/2 by the speed of sound at the ambient temperature.... You can do this for 10 shots, and then plot all the Cd/Mn pairs on a graph and let Excel find the trendline, like this....



In this particular set of data, for an 8.4 gr. pellet in .177 cal, the velocity dropped to Mach 0.65 (730 fps) at about 50 yards, and as you can see the Cd spread becomes huge below that.... Please note these three charts are all taken from different data sets, and are just examples.... Ron's plots are for each bullet, rather than one chart for all 10 shots.... There are other methods to come up with the Cd curve as well, some of which are "manual/visual" in nature....

Bob

[WobblyHand]

Bob, thanks for posting.  That's helpful.  I like the detailed data a lot better!  Ok, as a radar guy, I like to start out with some simple plots - just to validate I can think about this the right way.  Here are 2 plots, which aren't useful for you directly, but help me understand what is happening.  Velocity vs Range and SNR vs Range.  I hope the plots show up.  I can't see them in the preview.

The SNR plot tells me the 'units' or SNR are dB.  This is because the SNR (for a point target) should drop about 12 dB per doubling of the range.  If we look at 50 feet the SNR is about 43dB.  So at 100 feet the SNR should be about 43-12= 31 dB.  From the plot its about 34 dB.  That's pretty close for experimental noisy data.  Likewise if one compares 100 and 200 feet, it's about 12 dB lower.

[WobblyHand]

Bob, when you say for each pair of points, do you mean adjacent data points on the curve?  So for instance you difference 41.7 ft and 41.29 ft and then 41.55 ft and 41.29 ft? 

[rsterne]

That is correct.... one data pair is one time increment, ie the Cd is calculated every 2 mSec.… V1 and D1 are at "X" seconds, and V2 and D2 are at "X + 0.002" seconds....

Bob