Single Pulse Source Observations

FRBs and RRats are highly transient by their nature. As a result, we could see a single peak in a 10 hour observation. While keeping the raw voltages on hand can help with re-processing observations to avoid missing features due to issues in the current methodology, we no longer have the storage space on the UCC processing nodes or NUIG1.

Proposed methodology for processing single pulse observations observations:

• Process the observation with CDMT. Produce both a 0-DM and N-DM filterbank at the nominal time resolution (currently 655.36us, 16x ts, 8x chan)
• Perform RFI detection on the 0DM filterbank
  • We currently do not have a strong methodology here, apart from bandpass analysis and rfifind for detecting DM=0 features
• Search this output after an 8-bit decimation from digifil
  • Log the heimdall commands used, RFI channels flagged 
  • Investigate the optimal scale timescale for digifil (-I, default 10s) for FRBs; some are expected to last up-to or over 1 second at our frequencies due to scattering.
• After a search is complete, archive the CDMT filterbanks
  • Digifil: 2x ts for further space savings if needed, mostly a layover from previous 8x tsch 
    • -I 0 : No scale changes, raw 2:1 conversion
    • -b-32 : Float32 output, no change from raw filterbank
    • -t 2 : Down sample to 655us resolution
  • Compressed with zstandard: Further 10-20% compound storage saved

There are a few ways that this methodology could be changed to make the resulting filterbanks easier to search + store, or improve SNR

• Future changes
  • Chop bandwidth? Top 5MHz / Bottom 7 are Nyquist suppressed + RFI contaminated
    • Removing these could save us 15% of storage and speed up processing as searching the last 10MHz introduces an addition delay of 25 seconds @ R3's DM
    • No easy way to do this with the current voltage extraction/processing method, would need to be after the filterbanks are formed
  • Investigate having CDMT split filterbanks every N samples
    • Consider overlap requirements to not miss signals on the boundaries
    • Duplicated data, but higher theoretical SNR when we can include more channels by more selective RFI flagging
    • Or just find a decent RFI flagging algo...

We note that for RRats, we do not recommend forming a 0DM filterbank as those sources often do not need validation as they should be bright enough to be obvious with/without coherent dedispersion.