Tech Notes

An Overview of Software DSP for Weak Signal Enhancement
by Roger Rehr W3SZ

Image10.gif (38324 bytes)

This article consists of two parts:

The first is a general discussion of several of the mostly software DSP solutions I've tried.  

The second part is a brief discussion of my experience using the Elecraft K2 as IF for 144 MHz EME / weak-signal work with the DSP-10 audio version and using the RF Version of the DSP-10 for the same purpose.  There are actually two sets of comments in part two; those I made after a relatively noise-free weekend during the Italian EME contest, and those made after a fairly noisy ARRL EME contest weekend.


Part 1
DSP is an acronym for "Digital Signal Processing".  There are many uses, both current and potential, for DSP in amateur radio.  The ARRL Handbook 2000 has an excellent chapter on DSP.  Anytime an analog signal is converted to digital form and processed in some way, DSP is occurring.  This article will discuss some of the DSP software that is available to enhance the receiving of  RF signals.  The day of nearly totally digital communications gear, with the entire radio, or nearly the entire radio, in software is coming fast.  But here we'll stick to software that is in use today by hams who need to pull those really weak signals out of the mud, or to separate one signal from many crowding it.

 
The Holy Grail of EME and other weak signal work for many of us is a DSP program that will provide visual evidence of very weak signals over a fairly large frequency range as well as substantial improvement in signal reception by virtue of DSP processing.  

Specifically, the ideal program would provide:

1. Waterfall display with adjustable gain, baseline, visualized bandwidth, frequency bin size, and number of averages per displayed line.  A waterfall display is basically a way of displaying the time course of signals that have been received over a period of time by having one axis (usually the horizontal) represent frequency, the second axis (usually vertical) represent time, and then using color to display signal strength.  Some examples are shown later.

 2. Spectral display with the following parameters being adjustable: gain, baseline, visualized frequency range, frequency bin size, and number of averages per displayed spectrum.  A spectrum is just the familiar plot of signal intensity vs. frequency for a single point in time.

3. DSP audio processing with
 a. variable bandwidth filtering with adjustable center frequency
 b. adjustable LMS (Least Mean Square) noise reduction algorithm
 c. binaural receive capability
 d. defeatable and adjustable AGC
 e. adjustable notch filtering


LMS noise reduction is a statistical method of using the signals received in the recent past to estimate the noise contribution in the total received signal, and use those estimates to remove the noise so that only the desired signal remains.  This technique uses the least squares method to achieve this by minimizing the mean squared error.  

Binaural receiving methods delay the arrival of part or all of the received signal going to one ear.  This 'pseudo-stereo' sometimes makes the desired signal seem to pop out of the background.  

Some digital notch filters are much sharper and deeper than analog notches.  The ideal notch would, once assigned to an offending carrier, follow that carrier as it moved in frequency so that the carrier would stay suppressed if it drifted, or if the receiver was tuned or drifted.

Programs (and accompanying hardware in some cases) to date have each provided one or more of the above capabilities, but no program to my knowledge has provided all of the above. When fully implemented, SM5BSZ's planned software may come the closest of any package to providing all of the above.  SM5BSZ's software is now in the developmental stages. (see below)

DSP-10

Of software / hardware currently available, the superb DSP-10 series http://www.proaxis.com/~boblark/dsp10.htm comes the closest to doing all of the above. (See also the DSP-10 series of articles in QST - ed.) It has an excellent waterfall display with adjustable gain, baseline, and averaging.  The visualized bandwidth can be set only to 2400 or 4800 Hz, and frequency bin size is not adjustable. It has an excellent spectral display with adjustable gain and baseline, 2400 or 4800 Hz visualized bandwidth, and adjustable averaging.  The frequency bin size is not adjustable.  It also has DSP audio processing with 200 and 450 Hz filters, fixed at a center frequency of 600 Hz.  It has adjustable LMS noise reduction, binaural receive capability, defeatable and adjustable AGC, but no notch filter.  

It is different from the other products reviewed here in that it uses an Analog Devices EZ-Kit Lite evaluation board to do the DSP rather than a sound card. It connects to a PC via a serial port and just makes use of the PC for control and display. It runs under DOS ( NOT a DOS Window).  The software is free from Bob Larkin's web site listed above.  The EZ-Kit Lite is available from a variety of vendors.  I got mine from Newark Electronics (Newark Pt No 66F3812, Mfg Pt no ADDS-21XX-EZLITE, Manufacturer ANALOG DEVICES, price $94.70).  

Setting up the audio version is a piece of cake.  For performance and versatility DSP-10 is heads and shoulders above the other currently available packages, although I believe the long-in-the-tooth DOS program DSP-Blaster, while very limited in its display capabilities, still holds the edge in terms of audio signal processing effectiveness.  In addition to the audio version, a complete computer-controlled 144 MHz transceiver using the DSP-10 software was described in the September thru November 1999 issues of QST.  I have built this and have found it to be superb as a microwave IF.  The ability to spot those weak signals that are up to 4 kHz off frequency at 2305 MHz and above is critical in the microwave contests, and this radio really does that superbly.  I have recently tried this RF Hardware Version of the DSP-10 as a 144 MHz weak signal receiver and used it on 144 MHz EME.  Below I've given a short note on my reactions to it in comparison to my usual receive system.

Other software doing some of the above desired tasks includes the following.  Additionally, there is a listing of "Sound Blaster" and  other sound-card-related ham software at http://www.muenster.de/~welp/sb.htm#analyzer .  This list is very useful, although it is somewhat dated. 

DSP-Blaster

DSP-Blaster, an early DOS SoundBlaster based DSP program by K6STI, Brian Beezley. This program has good LMS noise reduction, narrow filters with adjustable bandwidth and center frequency, a beautiful notch filter, synthetic stereo, and much more.  http://seti1.setileague.org/software/dspblast.htm gives some info.  Brian has been difficult to reach and I don't know if any of the listed email addresses are current, although you could try k6sti@n2.net.  More comments are at http://www.nitehawk.com/rasmit/dsp_soft.html and a bit of K6STI history at http://www.qth.net/archive/bitbucket/199901/19990131.html.  Brian may now be back in business now, I am not sure...the November, 2000 QST (p58) suggests that he is again selling at least some of his software.  This is a good program that works only with genuine Sound-Blaster Cards, and can be quite finicky in terms of not wanting to run unless your computer is set up 'just right'.  But when it works, its does a very good job and I find I can pull out the really weak EME signals with it a bit better than with any other program, although DSP-10 is a very close second, and provides much better display capability.  Essentially, its much easier to find a weak signal initially with DSP-10 because of the spectral and waterfall displays, but the intelligibility of the signal once found, if its very weak, is better with DSP-Blaster.

FFTDSP

FFTDSP, http://www.webcom.com/af9y which provides waterfall display capability but no audio processing, by AF9Y.  This is an excellent display program which many EMEer's swear by.  AF9Y's web page referenced here is a superb tour/reference for a variety of weak signal, DSP, and astrophysical topics.  I don't use it because I find the display of DSP-10 adequate, and I need the filtering it provides.

 

ChromaSound

ChromaSound http://www.siliconpixels.com by N7CXI and VE3EC that provides graphic filter control and spectrum display over 5000 Hz or so, along with DSP noise reduction but no waterfall.  I don't think it does as good a job as DSP Blaster, DSP-10, or the Hamview author's programs in signal processing, and mention it here just for completeness.  I don't use it at all as the other programs just mentioned do a much better job for me.

GNASP1

GNASP1, which provides selectable filtering and spectrum output but no waterfall, noise reduction, or notching.  See http://members.tripod.com/~gniephaus/gnasp1/gnasp1.html.  I've not used this at all any more as the filters didn't seem to be anything special and its lack of noise reduction is a real negative here at W3SZ.

Hamview

Hamview (by I2PHD and IK2CZL), now being replaced by Spectran (version beta4 build 127 as this is written), provides a waterfall display, spectral display, mouse-adjustable filters, an LMS type denoiser.  The filters/denoiser in Spectran do not work as well for me as those in DSP-Blaster and the DSP-10 audio package, but the waterfall display is
excellent. Links are at http://www.radiodsp.com or http://members.nbci.com/spectran/ .  I don't use this because the filters as noted just aren't quite as good as I need.

For a superb web page that has a link to an excellent DSP Resource page go to http://www.nitehawk.com/rasmit/ and click on the "DSP for Weak Signal" link at the top of the page.  OR, go directly to http://www.nitehawk.com/rasmit/dsp50.html to get to DSP directly.  If you do the latter you'll miss a nice introductory page though.

SM5BSZ's software
Leif, SM5BSZ,  has begun the migration and upgrading of his MS-DOS PC radio http://ham.te.hik.se/~sm5bsz/pcdsp/pcdroot.htm to Linux.  This is where ham radio will eventually be at for weak signal work, but it is currently a work in progress, with just the early stages under construction at this point.  This is what he envisions for the system when complete:

Start Leif's comments:
I am currently working on a new DSP radio package. This time the system is designed for flexibility so it can be used for many different combinations of computers, A/D boards and analog radio circuitry. The platform is Linux and the package will typically operate with a 486 computer together with a conventional SSB receiver as the minimum configuration. The current high end operation is with a 4-channel 96kHz A/D board and a Pentium III providing nearly 2 x 90kHz of useful signal bandwidth in a direct conversion configuration (stereo for two antennas). When the Linux package is in full operation I will interface it to a modern radio A/D chip and digital data decimation chip. The component cost is very low and there will be an exciting improvement in dynamic range, bandwidth and flexibility.

The LINUX PC-radio for Intel platforms http://ham.te.hik.se/homepage/sm5bsz/linuxdsp/linroot.htm will be continuously upgraded to show various aspects of digital radio processing and how they are implemented in the DSP package. The Linux PC-radio is not designed for VHF weak signal only. It is very flexible and designed to accommodate routines for all radio communication modes on all frequency bands.

The program can run on a 486 to process 3kHz bandwidth with almost any sound board.  It can also run on a Pentium III with a 96kHz board such as Digital River Delta44 to produce spectra covering about 90kHz bandwidth using two mixers to provide a direct conversion receiver. (For EME it may be easiest to make a direct conversion receiver for a fixed frequency such as 10.7 MHz and put some converter in front of it).

This is an ongoing project. The current version only produces spectrum graphs with user selectable resolution and averaging. More sophisticated processing will be added gradually.

The package will provide more than 30kHz bandwidth with a standard audio board and should be very useful for 10 GHz EME and any other mode where a wide spectrum range has to be searched.

I am using Red Hat 6.1 and it works fine on several computers. I am using OSS for the sound board devices and they charge USD 40 (if I remember correctly) for the Delta44 option.  For the screen I use svgalib.  For the future I will try to make the package run under several Linux distributions.

Someone tried SuSe but that did not work, I do not know why.  I have bought Caldera, Debian, SuSE and Mandrake and some time I will try to get the software run under these distributions - but right now other things have higher priority.  You may have to change sound.h and run make to select the drivers for your system<  See below--W3SZ  >. I am using two sound boards so I can run the audio output at a low speed. For CW modes there is no need for 96kHz output....


End of Leif's comments.


The goal here at W3SZ is to have something like Leif describes.  John K3PGP helped me to get started with a homebrew computer-controlled receiver using TUF-1's for mixers, a couple of computer-controlled frequency synthesizers for the LO's, and the DSP-10 as the final audio stage.  This has worked well, but the visible bandwidth with the DSP-10 at 4.8 kHz was less than the 10-30 kHz I want.  Hence my interest in Leif's software.

I had never used Linux, or any UNIX before, but found that with a little help and some quick reading I was able to get Linux installed on my Windows 95 computer and Leif's program running over a couple of evenings.  Right now, as Leif pointed out, his program just gives a visual display of the audio.  But hopefully there is more to come, and I wanted to keep up with his progress so I bit the bullet and started now.

I run a Pentium Pro 200 with a Sound Blaster AWE32.  The machine was originally set up to dual boot with NT and Win95 so I can't boot Linux from a hard disk...I just boot from floppy when I want to run Linux.  I will eventually get a separate machine for Linux.  I have 1.5 GB HD space set aside for Linux (about right), 128 MB RAM (much more than I need, I think), Number Nine Virge S3 type video card, PS2 3 button mouse.  I am running Red Hat Linux 6.2 and got the OSS sound drivers off of the web from http://www.opensound.com  ; they are not free but made the installation pretty simple.  I had never used Linux before and don't really know C but I was able to get going with the documents on the web from OSS and Red Hat ( http://www.redhat.com ).  So its not too hard to do.  Please note:  If you have Windows NT on your system and you install Linux to boot from a hard disk you will have a disaster...do not do this.  I believe you must do a CUSTOM install in order to choose the option of NOT installing LILO on the hard disk (the boot utility).  To me, even if I could have set it up to boot from the hard disk, I likely would have taken this route...when I want to boot to Linux I boot from the Linux boot floppy.  When I want to boot to Windows, I pop the floppy part way out of the drive before start-up.

Click here for details from my experience at installing all of this (Linux):


Below are some details on the look/feel of the program.

The spectral display is very nice and it's great the way the bandwidth and center frequency etc. can be varied.  The bandwidth as well as the center frequency of the display are changed using the mouse, and the floor and gain of the waterfall are changed by clicking on the  appropriate parameter field with the left mouse key and then using the keyboard to enter the new parameter.

At this point, at least here, there is no audio output from the soundboard while the program is running, but just the graphics (which are very nice, especially in the weak signal CW screen).  Currently there is a waterfall display at the top, with the spectral display below;  kind of like DSP-10 but reversed, and with the ability to change the width of the display in  terms of Hz (within the range permitted by your prior choice of FFT parameters) as well as physically on the screen, the center frequency, the zero point of the spectral display as well as its amplitude range using the left mouse button in real time.

You can create .gif files of the screen by just typing "g" and following the instructions. Read the files settings.txt and mouse_on_graph.txt in the DSP directory to see how to control things once you have the program running.

But anyway, RIGHT NOW the spectral display is the best there is that I've seen in terms of its on-screen adjustability, although I don't have it all figured out yet and there IS still room for improvement.

GOOD SHOW, Leif!!

As noted above, I've used FFTDSP, DSP-Blaster, Spectran, HamView, and DSP-10 here over the past several years;  If Leif continues with this it should obviate the need for the others...



Part 2.        

Comments on K2 and DSP, etc.

September 2000.  I got things working again, at least receive-wise, last night/this morning for the Italian EME Contest, and was able to compare in an admittedly unscientific way some of the receive combinations we've been talking about in an EME environment.  This was the first time I had the 2 x 2 MXP20 array 'on the moon' since it went up this past February.  I didn't get the calls on any small stations, but I did hear and copy the usual big guns:

WB9UWA, OZ1HNE, F3VS, I3DLI, EA3DXU, SM2CEW, K2GAL (both ways, tropo and EME), RU1AA, W0EP (probably a bad call ??W0HP??), I2FAK, and several others I was not able to get full calls on. The big guns were mostly 10-14 dB out of the noise as measured by the spectral display on the DSP-10.

What I found was:

The new version of Spectran does a better job than DSP-10 (and of course better than DSP-Blaster) of visually DISPLAYING the signals in both the spectrum display and the waterfall.  But the AUDIO processing (what you listen to, as opposed to what you see on the screen) is much better with DSP-Blaster and DSP-10 than it is with Spectran.  So you can run two computers; one to watch the band with Spectran, and one to hear the signals with DSP-Blaster or DSP-10 ;)

It was not a very noisy night from a pulse noise/noise blanker standpoint, but I found that the noise blanker in the Elecraft K2 WAS VERY helpful and there were a couple of times when I felt that the lack of a noise blanker in the DSP-10 hardware was a disadvantage, even with the LMS noise reduction it has turned on and cranked up.

Nevertheless, to my surprise I felt I could find the stations more easily and found listening to them/copying them more comfortable on the DSP-10 hardware than on the SSB LT2S/K2 combination, whether I was running the LT2S/K2 with no DSP, with DSP-blaster, with the AUDIO version of DSP-10, or with Spectran.  I think this relates primarily to the fact that the binaural implementation on the DSP-10 is very much to my liking, and I like it far better than the DSP-Blaster's, even though I previously liked the DSP-Blaster's version very much.  There were signals I essentially couldn't appreciate with the binaural 'off' on the DSP-10, that just popped out when I turned it on (I normally run with it on, and turned it off just to try to figure out why the DSP-10 receiver seemed so 'hot', so what actually happened was that signals that were easy copy became lost in the mud when I turned off the binaural).  Also, the quality of the signal through the RF DSP-10 just seems better than when the audio DSP-10 is tacked onto the end of the audio chain after the K2.  Thirdly, the very slow tuning rate of the DSP-10 (10 Hz at a time by keystroke, not with an infinitesimal rotation of a knob) was a real advantage with the weak signals, especially since they could be seen on the display and 'dialed in' till they were centered at 600 Hz on the screen, right at the center of the DSP filter.  Finally, I think the mast mounted preamps likely eliminated any effect of the better MDS the xvrtr/K2 have compared with the DSP-10, and evened the score in this regard.  

I had previously compared the K2 with the IC-735 as IF, and the K2 won hands down; I think the K2 is a very, very good IF rig, but liked the DSP-10 better today.  Please note that it may take a bit of fiddling with the delay parameter on the DSP-10 for optimal effect. I have the delay set at 1024.

So I guess the ideal setup for me would be the RF hardware DSP-10 with a spectral display and waterfall more like that of Spectran, and with a noise blanker in addition to the DSP functions it has.

All of this is subjective, so take it all with a grain of salt...

I wait for a really noisy night to see what the noise blanker on the K2 does under those conditions.  

I did not try the spectral display of Leif, SM5BSZ, in this comparison as I was sure it wouldn't be as good as the others yet under these extreme conditions, and the DSP audio processing isn't available yet...

K2 vs. DSP-10, again..

Well, for the first night of the first leg of the ARRL EME Contest 2000 I got my wish: a noisy night to test the K2's noise blanker.  I also got a night with a lot of local activity, and this combination made some things very clear.

The antenna system was again a 2 x 2 stack of M2 2MXP20's with separate receive lines for Horizontal and Vertical Polarity.  There were dual ARR 144 MHz preamps on the tower, close to the power dividers.  The feed lines (LMR600UF to the bottom of the tower, and then 7/8 inch hardline to the operating position) connect to an EME grade relay in the shack so I can switch polarity with a flick of a switch, or monitor both H and V on two different receivers.  For the comparison I kept the relay in place, and hooked a two way power divider after it to split the receive signal into two paths, one path to the SSB Electronics LT2S Mk II / K2 combination, and one to the RF version of the DSP-10.  As I have more than 20 dB of preamp gain on the tower, and significantly less feedline loss, this signal splitting didn't produce any noticeable degradation.  I had the audio from both combinations fed into a professional audio mixer (Sampson) with good RF shielding, and then fed the output from the mixer into a computer running DSP Blaster, and to another computer running Spectran for video purposes only (spectrum and waterfall displays).

What I found was significantly different in some respects than what I reported for the Italian EME contest, as that was a rather quiet night (though even then the K2 Noise Blanker was helpful).  

First of all, there are some 1.5 KW stations within a few miles of me, and the DSP-10 kind of folded up and died when they were transmitting at times when I was in the line of sight to the moon and it was low.  In contrast, the K2 / Transverter was really only bothered close in, frequency-wise, and then I could still work through the noise.  Having the noise blanker on did increase the problems with local QRM, as is the case with any noise blanker I've ever tried, but I could still work with the NB on.  In contrast, I just had to shut the DSP-10 down when the locals were transmitting.  Score one for the K2 / Transverter combination.

Second, the first night of the first leg of the ARRL EME Contest was rather noisy here, and the LMS noise reduction on the DSP-10 just didn't take out the noise sufficiently.  This is not surprising, as when the pulse noise is very bad here, NO LMS algorithm I've ever used is sufficient.  It takes a good noise blanker plus LMS to do the job!  During the periods where the pulse noise was absent, the DSP-10 was a joy to use.  As I noted before, it is very comfortable to listen to and with the binaural implementation Bob Larkin has placed on it, it really pulls the signals out of the mud.  During the quiet periods I really preferred it to the K2/Transverter combination, but again it couldn't be used when the pulse noise or local QRM was present.

Thirdly, I have a lot of birdies here at W3SZ.  Frequently they were close in to the station I was trying to copy.  Here the use of the K2/Transverter fed into the computer running DSP Blaster was again superior, as I could either make the filter very narrow, or turn on the adaptive notch DSP Blaster provides (its VERY sharp and narrow), or both and eliminate the offending carrier.  No similar option with the DSP-10, although you could run one computer with DSP-10 and feed the output to another computer with DSP Blaster running;  I did this and then could accomplish the same thing as long as the pulse noise and local QRM was absent.

Fourth, on the VERY weak stations, it was helpful to be able to play with the filter parameters on DSP Blaster, and I found that for the really weak stations I could only copy them with the KK2/Transverter/DSP Blaster combination.  With the DSP-10 the center frequency of the filter is limited to 600 Hz, and the only bandwidths are 200 and 450 Hz as noted above.

By the end of the night I had essentially put the DSP-10 away and was running with just the K2 / Transverter / DSP Blaster combination, and very happy.


In terms of the displays, I ended up liking the spectrum display of the DSP-10 best, and the waterfall of  Spectran best.  I don't have a means of capturing the output of Spectran to a file, but I do for DSP-10.  The following  Screen Shots consist of : 


This first screen capture is of the W3CCX beacon here on a night with a lot of pulse noise, and the K2 Noise Blanker turned on.  

Just focus on the spectrum display at the top of the graph, not at the waterfall, below.  The waterfall is essentially a history of all of the playing around I had done for the prior 10 minutes (time lines on the left).  Note that the noise floor is about 2-3 dB on the scale and the W3CCX beacon at about 665 Hz reaches the 40 dB mark.  

Image

 


This screen capture was actually obtained just before the first one on this page, and is identical except that the K2 Noise Blanker has been turned off.  

Notice on the spectral display at the top of the display that the W3CCX beacon is still peaking at 40 dB, but the noise floor has come up to 20 dB, rising by about 17-18 dB.  This is impressive!  Now, if you flip back and forth between the first and second displays, and focus on the bottom of  the waterfall displays, you will see how the waterfall is white/yellowed out at the bottom of the second display, below, but that the signal is in the clear on a black background on the first display, above.  There's no doubt here at W3SZ when the pulse noise starts!  But its fun to play these electronic tricks and eliminate it...I'd actually been hoping for a night like last night when I took these pix to test everything out!

Image


This DSP-10 display was taken the next day, of W5UN at 144.040 off the moon.  

The audio gains for the entire audio chain have been changed, so the displays are not directly comparable (Dave has a great signal, but its not within 10 dB of the local beacon, although it sometimes seems like it!!)  The signal at 600 Hz is Dave, peaking at just under 30 dB (28.6 dB, to be exact) on the scale, or 15 dB above the noise.  The other signals are birdies.

Image


This last screen shot shows the same beacon as the first DSP-10 screen shot, but using Spectran.

 Note that the spectrum is missing details like dB markers that DSP-10 has.  In addition, the spectrum cannot be scaled nicely like the DS10 spectra in terms of amplitude range (DSP-10 amplitude range can be set anywhere from 70 dB full scale range all the way down to 7 dB full scale range, and the baseline is independently adjustable).  There is some scaling possible with Spectran, but it is rudimentary by comparison. 

Image

Click on image for a larger view.

One advantage of Spectran's spectral display is that it does a good job of automatically compensating for baseline/filter roll that might be present in the signal coming out of the analog rig.  With DSP-10, this compensation can be done, but its not automatic and sometimes the software seems to get confused when one changes scales after normalizing in this manner.   On the other hand, Spectran's waterfall display is far more to my liking than that of the DSP-10; details and nuances of weak signals are more easily seen, without the need for careful, sometimes tedious adjustments to optimize the waterfall as with DSP-10.    

I hope this was helpful...any questions/comments, etc...fire away...

73,    Roger Rehr
W3SZ   (ex AA3QK, WA3JYM)  FN20ah
Reading, PA 

rrehr@epix.net 
http://www.epix.net/~rrehr/w3sz.htm

[Elecraft Home Page]