Introduce double buffered HULA reader

[h3ndrk]

Introduced Changes

This PR introduces Unix socket draining when reading from the HULA socket. This is needed because our HULK process connects very early to the Unix socket and HULA/LoLA immediately starts sending state messages to HULK which fills our read buffer. After the connection, we are further initializing the HardwareInterface, e.g. doing camera initialization/resetting. This may take a long time, filling up around 1 second of historic state messages in HULKs read buffer. Once the HardwareInterface has been initialized, the cyclers begin to run and the control cycler will rush to drain the accumulated messages, because the sensor data receiver has a lot of data available to feed the cycle. This results in a lot of messages being sent to HULA/LoLA containing historic (1 second old) control messages. But since HULA/LoLA only updates its shared memory and does not communicate directly with the hardware, this is usually not noticed and it will converge quickly to nearly no latency and drained buffers. This bug was discovered because the new HULA will communicate synchronously with the hardware thus needing this fix.

Changes in HULA are not needed. No worries, there is a lot of test code which accumulates a lot of added lines. 😉

ToDo / Known Issues

None

Ideas for Next Iterations (Not This PR)

None

How to Test

The double buffering should have nearly 100% test coverage, but please test on a real NAO if everything works as intended. The chest button LED blinking frequency should be exactly one second immediately on startup which is a good indicator that everything works as intended.

[schmidma]

Some general remarks and mostly nit-picking on naming.

One general question:
You are concerned a lot about partial messages and stitching them back together afterwards. Is this actually happening? I assumed the unix socket to have extremely high memory bounds on the internal buffers? Or why are you concerned with partial messages? Please elaborate.

[h3ndrk]

I assumed the unix socket to have extremely high memory bounds on the internal buffers?

I read that Unix sockets have around 100-200 KB of buffer space. HULA sends messages of 652 bytes which means the buffer is full after around 300 messages which is approximately 3-4 seconds at a rate of 80 Hertz. When the buffer gets full, the likelihood that only a partial message still fits is very high. In my experiments, I saw around 80 KB of buffered messages to be usual during initialization. Hopefully, this explains the message stitching efforts.

[schmidma]

But did you ever observe partial messages in the socket buffer? And what is HULA doing then? Is it blocking? Crashing?!

[h3ndrk]

But did you ever observe partial messages in the socket buffer?

No, the scenario would happen if HULK pauses reading for 3-4 seconds, then reads faster than HULA being able to write the remaining message. This could happen if the scheduler decides to not schedule HULA for a short moment in time.

And what is HULA doing then? Is it blocking?

Yes

Crashing?!

No, but it will wait for everything to be written until reading more messages from LoLA.

[h3ndrk]

@schmidma Tested successfully on a NAO with legacy HULA.