Created by: axj336

First identified in Jan 2019, sometimes the listener script experiences a socket error, this can lead to Chirp missing an alert.

If you think an alert is missing please create a new issue

The listener script is derived from the LIGO alerts user guide and uses the GCN Python module.

An example of the traceback is shown below. First, the connection to "External IP" is closed. Then the listener tries to reconnect after 2 seconds. The connection is refused, so the listener waits and tries again in 4 seconds. This procedure continues with (exponential backoff)[https://en.wikipedia.org/wiki/Exponential_backoff] until the listener is waiting for 1024 seconds. The listener continues to attempt to reconnect, waiting 1024 seconds each time, until the connection is allowed.

We run two servers in duplicate. These are completely independent, located in different countries (both Europe), but running the same code. On all of the cases I have checked, these socket errors occur at the same time on each server. This leads me to conclude that the issue is with the external IP address the scripts connect to.

Chirp collaborators can access the initial issue.