A collection of notebooks that try to detect artificially inserted alien signals in a large data set. The challenge was taken from: https://www.kaggle.com/c/seti-breakthrough-listen. The dataset is also available on the kaggle website.
Different approaches were tried out to reliably increase the SNR of alien signals (red boxes) in noisy images. Results can be found in /source/conservative-approach/. The most promising preprocessing step: Derivative images, was also applied on the whole dataset. But the results were not useful as input for a neural networks.
If the right preprocessing steps were applied it is possible to automatically detect clusters in images with good SNR. That algorithm is implemented in /source/conservative-approach/derivatives+cluster.ipynb
It seems like the creators of the challenge shifted the frequency and reused part of the backgroind noise. That means if a perfect match is found, shifted and subtracted, the result would be an image without background noise. That process is shown in the notebook /source/remote-notebooks/magic-2-an-explanation.ipynb
Two different neural networks were trained on the task and then fine-tuned to deliver good results. A ResNet18 and an EfficientNet. The corresponding notebooks can be found in /source/remote-notebooks/. Their architectures can be viewed in /net architectures/.
ResNet delievered poor results which were not better than a no skill model..
But EfficientNet achieved good results. The initial implementation with an AUC score of 0.753 could be improved to a score of 0.800.
This was achieved by using only ON measurements and applying a 1x1 Convolutional Layer as first layer.
