FRB Observations

FRBs are highhighly transient by their nature. As a result, we could see a single peak in a 2010 hour observation. While keeping the raw voltages on hand can help with re-processing observations to avoid missing features due to issues in the current methodology, we no longer have the storage space toon performthe suchUCC actions.processing nodes or NUIG1.

 Proposed methodology for processing FRB observations:

• Process the observation atwith CDMT.CDMT. Produce both a 0-DM and N-DM filterbank at the nominal time resolution (currently 327us, 8x ts, 8x chan)chan)
• Perform RFI detection on the 0DM filterbank
• Search this output after an 8-bit decimation from digifil
  • Log the heimdall commands used, RFI channels flagged 
  • Investigate the optimal scale timescale (-I) for FRBs; some are expected to last up-to or over 1 second at our frequencies due to scattering. Digifil default is definitely lower than this. Wow, it's apparently actually 10 seconds by default
• After a search is complete, archive the CDMT filterbanks
  • Digifil: 2x ts for further space saving compared to raw filterbanks, output savings
    • -I 0 : No scale changeschanges, raw 2:1 conversion
    • -b-32 : Float32 output, no change from raw filterbank
    • -t 2 : Down sample to 655us resolution
  • Compressed with zstandard: Further 20-40% compound storage saving
• Future changes
  • Reduce bandwidth?bandwidth? Top 5MHz / Bottom 10 are noise + RFI contaminated
    • Removing these could save us 15% of storage and speed up processing as searching the last 10MHz introduces an addition delay of 25 seconds @ R3's DM
    • No easy way to do this with the current voltage extraction/processing method, would need to be after the filterbanks are formed
  • Investigate having CDMT split filterbanks every N samples
    • Consider overlap requirements to not miss signals on the boundaries
    • Duplicated data, but higher theoretical SNR when we can include more channels

Step	Method	Storage Used	Product	Overall on Disk
Generate Voltages	Observer	1	1
Compressed	zstandardzstandard, Olaf's recorder	~0.6-0.8	0.6	0.6
CDMT	-a -b 8 -d 0,DM,2	0.25	0.25	0.85
Digifil (Search)	-b 8 -I <DECIDE>	0.0625	0.0625	0.9125
Cleanup: Digifil (search)	rm	-0.0625	-0.0625	0.85
Digifil (Compress)	-b-32 -t 2 -I <DECIDE>	0.125	0.125	0.975
Compress Digifil (compress)	zstandard	<0.09	0.09	1.065
Cleanup: Voltages, CDMT	rm	-0.6 - 0.25 - 0.125	-0.85975	0.12509
Overall				~140GB/100 GB/obs-hr