The next step is to investigate the FLOPs for each network, and determine the processing time on a given device.In this section, the processing time of the deep learning filter is studied. As the filter is intended as a real-time trigger, a fast execution time is crucial. The current ARIANNA hardware is used to test and measure the execution time under realistic conditions. There are several time components that impact the physics capabilities of the ARIANNA detector: the time to transfer the data from the waveform digitizers to the microcomputer, read, the time to reformat and calibrate the raw data for the deep learning evaluation, and the time to evaluate the event with deep learning and make a decision, the time to store an event to the local SD card, and the time to transmit the event via the Iridium satellite network. The architecture of the ARIANNA pilot station cannot acquire neutrino events while the data acquisition system is processing events or during the transmission of data over Iridium satellite. It is useful to express the processing time in terms of fractional loss of operational time, or deadtime. Operational livetime, L, is the calendar time of nominal operation, T, corrected for the time losses due to event processing and transmission. As shown in this section, wholesale grow bags the latter two time scales depend on the microprocessor. It is assumed to be negligible when evaluating new platforms for future designs of the ARIANNA data acquisition system.Two microprocessors are explored for their processing time and power consumption: a Raspberry Pi compute module 3+ microcomputer and an MBED LPC1768 ARM microcontroller.
The MBED is the current device installed in ARIANNA and is implemented through custom C code. The Raspberry Pi is a microcomputer with a Raspbian operating system, which is based on Debian. As with the MBED, the neural network is implemented with a similar custom code on the Raspberry Pi. Since the optimal networks found in the previous section are small and shallow, a custom code is written that implements the trained neural networks in C for maximum performance. To test the prediction capabilities and the classification time in both devices, a simulated event is read in and either matrix multiplied by the array of weights and biases in the FCNN case or convolved with the weights and bias filters in the CNN case. For the Raspberry Pi, since it is not attached to the ARIANNA data acquisition system , the processing time is measured by looping over the processing code 100 times, while measuring the total time for100 loops with the clock function in C. The total time divided by 100 is the average processing time per event. For the MBED, since it is attached to the ARIANNA board, it has the ability to be probed for reset pulses. Reset pulses are used by the MBED to reset the logic of the FPGA and triggering circuitry to prepare for a new event. The time between reset pulses will provide the total deadtime. In the case that the station is triggered continuously, which would result in 0% livetime, the time between reset pulses corresponds to the processing time.
Livetime is defined as the time between the reset pulse of the previous event and the trigger of a new event. To accomplish this setup, a pulse is injected into the hardware with an amplitude large enough to trigger the system. To measure each component, different processing functions are disabled. While table 1 shows that the relationship is not completely linear, FLOPs provide a reasonable proxy to estimate the relative speeds of specific deep learning models. All models listed reach the required efficiency of 95% neutrino signal at 105 noise rejection. Therefore, the fastest network is chosen for the lab tests, which is the CNN with 100 input samples.Optimizing the network architecture and processing time are not the only factors to consider when implementing a deep learning network onto ARIANNA. Reliability in the harsh Antarctic climate must be considered as well as the limited power available in the remote location of Antarctica. The MBED was tested in the field for reliability in cold temperatures and meets the specification on power consumption, operating under one Watt. In contrast, the Raspberry Pi is rated to −25 °C and requires more power than the MBED. The Raspberry Pi was stress tested under cold conditions, running it from 20 °C to −60 °C. It ran continuously with the deep learning filter for the hour it took to cool down to −60 °C, and then it was run for an additional hour at this temperature. Once it was brought back up to room temperature, the Raspberry Pi was still operational. If chosen as the optimal device in the future, the Raspberry Pi would need to be tested further for long term operational reliability such as temperature cycling.
We compare the performance of a deep learning filter to a realistic template matching procedure using a single template, similar to what was used in a previous analysis. It is found that the deep learning method is typically faster and performs better. A neutrino template is constructed by convolving a predicted Askaryan pulse with the antenna, amplifier and filter responses of the ARIANNA signal chain as already done in previous analyses. A single template is used to minimize the computational costs, and also because of the observation that the template is dominated by the detector response; variations in the predicted Askaryan pulse have only a small influence on the resulting templates . The plot of the general simulated template waveform is found in figure 6, and for this study, the amplifier response is added to this waveform without noise. Following the same data format as the 100 input sample CNN, the template was trimmed to 100 samples around the maximum absolute value of the waveform. This template was cross-correlated with the simulated signal and noise data sets, and the maximum absolute value of the cross-correlation is used as a measure for signalness, i.e., the output is a number between 0 and 1 as in the deep learning case. To compare the performance of the template and neural network method, the signal efficiency vs. noise rejection factor is computed and compared to the CNN result which is presented in figure 7. The CNN method is found to perform significantly better. At the benchmark value of 95% signal efficiency, the template method only achieves a little more than two orders-of-magnitude noise rejection. One explanation for this is that the CNN identifies smaller 10×1 features within the training sets, which gives it more flexibility. Additionally, the CNN has 5 times the amount of features to extract compared to the template’s single waveform/feature. Another aspect of the template matching technique is to determine the processing speed. Estimating the processing speed for this method, the FLOPs are roughly 29,900, which is close to three times the amount of FLOPs of the 100 input sample CNN. Narrowing in further on the template signal pulse to 50 samples around the maximum of the waveform, the FLOPs are roughly 7,450. This is now less FLOPs compared to the 10,096 FLOPs of the 100 input sample CNN, but the efficiency of the template matching is still significantly worse. Therefore, grow bags for gardening the cross-correlation neutrino template matching method is less efficient and slower than the CNN technique.The deep learning filter uses the 100 input sample CNN described in section 3. Data taking with LPDA antennas proves a challenge in the lab due to the lingering radio frequency noise present in the environment. Without a radio quiet space, one cannot replicate the environment of the Antarctic ice since the antennas would measure local radio sources which would bias the data. Thus, for in-lab tests of the deep learning implementation, an experimental “post LPDA antenna” radio neutrino pulse is created and injected into the ARIANNA hardware, bypassing the antenna, to verify the simulated results. The expected neutrino template is generated with the standard simulation tool, NuRadioMC, which convolves the expected electric field at the detector with the LPDA antenna response. This neutrino template is then programmed into an Agilent Tech. 81160A arbitrary pulse generator to produce an analog version of the neutrino template as observed by the LPDA antenna. This waveform is then injected into the Series 300 amplifier of the ARIANNA pilot station, which adds realistic amplifier noise. By adjusting the input amplitude of the template, the SNR can be tuned to arbitrary values. Figure 8 shows a diagram of the experimental set up. The noisy signal is then routed to the input of the ARIANNA DAQ for data taking.
Figure 6 gives a comparison of the simulated neutrino template to those produced by the analog pulse generator, known as measured neutrino template. The DAQ consists of an SST chip, an FPGA, and an MBED microcontroller. Once digitized by the SST chip, the FPGA passes the digitized data to the MBED, where the channel with the largest signal is chosen and runs through the deep learning filter. Once finished, the filter gives the network output value of the event to determine its classification. Any event whose probability is below the cutoff would normally not be saved to memory. For testing purposes, all events are saved into memory along with the deep learning calculated probability. The deep learning filter performance on the ARIANNA hardware is also checked on a local computer which takes the digitized data and recalculates the probability. Both methods show equivalent results. This setup allows for the recording of neutrino signal data sets and noise data sets. To record the noise data set, the neutrino signal generator is deactivated so that the ARIANNA DAQ only sees thermal noise amplified by the amplifiers. This test focuses on low SNR events which are the most difficult to differentiate between signal and noise. Therefore, the high/low trigger threshold is set to an SNR of 3.6 and a two of four coincidence logic between channels is required. The corresponding SNR distribution is shown in figure 9, where the SNR is calculated as the maximum amplitude of any of the channels divided by the RMS noise. The measured noise distribution matches well to the SNR distribution of the simulated noise data set used in the previous sections. Recording the neutrino signal data set is more challenging because at these low thresholds, a noise fluctuation might trigger the readout. So instead of a self-trigger, the ARIANNA DAQ is externally triggered via the trigger output of the pulse generator. The time delay is adjusted so that the signal pulse is at the correct location as expected from the ARIANNA high/low trigger. The amplitude of the signal pulse is adjusted to produce a low SNR distribution just above the trigger threshold. The neutrino template in figure 6 remains constant for all of the tests, but since the noise added by the amplifier follows a Gaussian distribution, the resulting SNR follows the same distribution. As seen in figure 9, the distribution is broader and shifted to a slightly higher mean than the noise data sets. This is expected because the interference of the extended signal pulse with noise gives several chances for an upward fluctuation, and the maximum value of any channel is chosen for the SNR estimate. The resulting SNR distribution of this low-amplitude simulated neutrino signal data set matches the experimental distribution. To test the deep learning filter and verify that the simulated hardware components are comparable between measured and simulated data, a histogram of the network output values is plotted. This distribution is obtained by using the trained 100 input sample CNN to classify simulated and measured events. As seen in figure 10, the distribution of the simulated MC data set agrees well with the experimentally measured distributions. The small deviations are likely due to differences in SNR distributions and environmental effects such as strong radio pulses leaking into the cables. This gives us confidence that the MC simulation indeed describe measured data and that the inferred noise rejection factor and signal efficiency are credible.