Skip to main content

FRB Observations

FRBs are highly transient by their nature. As a result, we could see a single peak in a 10 hour observation. While keeping the raw voltages on hand can help with re-processing observations to avoid missing features due to issues in the current methodology, we no longer have the storage space on the UCC processing nodes or NUIG1.

Proposed methodology for processing FRB observations:

  • Process the observation with CDMT. Produce both a 0-DM and N-DM filterbank at the nominal time resolution (currently 327us, 8x ts, 8x chan)
  • Perform RFI detection on the 0DM filterbank
  • Search this output after an 8-bit decimation from digifil
    • Log the heimdall commands used, RFI channels flagged 
    • Investigate the optimal scale timescale (-I) for FRBs; some are expected to last up-to or over 1 second at our frequencies due to scattering. Digifil default is definitely lower than this. Wow, it's apparently actually 10 seconds by default
  • After a search is complete, archive the CDMT filterbanks
    • Digifil: 2x ts for further space savings
      • -I 0 : No scale changes, raw 2:1 conversion
      • -b-32 : Float32 output, no change from raw filterbank
      • -t 2 : Down sample to 655us resolution
    • Compressed with zstandard: Further 20-40% compound storage saved
  • Future changes
    • Chop bandwidth? Top 5MHz / Bottom 7 are Nyquist suppressed + RFI contaminated
      • Removing these could save us 15% of storage and speed up processing as searching the last 10MHz introduces an addition delay of 25 seconds @ R3's DM
      • No easy way to do this with the current voltage extraction/processing method, would need to be after the filterbanks are formed
    • Investigate having CDMT split filterbanks every N samples
      • Consider overlap requirements to not miss signals on the boundaries
      • Duplicated data, but higher theoretical SNR when we can include more channels by more selective RFI flagging
      • Or just find a decent RFI flagging algo...
Step Method Storage Used Product Overall on Disk
Generate Voltages Observer 1 1  
Compressed zstandard, Olaf's recorder ~0.6-0.8 0.6 0.6
CDMT -a -b 8 -d 0,DM,2 0.25 0.25 0.85
Digifil (Search) -b 8 -I <DECIDE> 0.0625 0.0625 0.9125
Cleanup: Digifil (search) rm -0.0625 -0.0625 0.85
Digifil (Compress) -b-32 -t 2 -I <DECIDE> 0.125 0.125 0.975
Compress Digifil (compress) zstandard <0.09 0.09 1.065
Cleanup: Voltages, CDMT rm -0.6 - 0.25 - 0.125 -0.975 0.09
Overall       ~100 GB/obs-hr
Back to top