Qserv core* files causing docker to fail to start

Dear Qserv developers,

I have recently taken over a test install of Qserv from Teng
(One of his previous posts - Installing and testing latest version of Qserv in Docker)
and have run into an issue involving very large (2.5G) core* files being generated locally on the master container. Which then stops the container from starting back up as it is out of disk space.

I was able to get around this by copying out the core files to somewhere local
# docker cp master.localdomain:/qserv/core* /somewhere_local

Then I was able to bring up the container and delete the core file.
docker start master.localdomain && docker exec -it master.localdomain rm -rf /qserv/core.2277

I was just wondering if there was any way of moving the location of these core files?
There are a few mounted volumes that are present on the container, so an easy solution would be to dump them there if at all possible.

Many thanks,

Hi Craig,

Since Docker containers all share the kernel of the host node, core file location is mediated by the kernel core_pattern setting for the host node and is not per-container. You can investigate how this might be set on your host node (it differs somewhat between Linux distributions, but for many may be set via /proc/sys/kernel/core_pattern.)

We could also investigate options to prevent the Qserv containers from generating core files at all, but this begs the question: how often is the system dumping core at your site, and can we assist in identifying the underlying problem?


One note: wherever the core file is dumped, you can mount a host filesystem volume (or network volume) at that spot to avoid filling the container’s filesystem. But as Fritz says, not having a core dump at all is the best solution :slight_smile:

Happy new year.

Thank you both for replying.
Thankfully it appears this has just been a once-off, but if I run into it again I can let you know if I need some assistance in narrowing down what the cause might be.

A bit more Googling led me to this: https://unix.stackexchange.com/questions/192716/how-to-set-the-core-dump-file-location-and-name
Which mentions that it is possible to set a core file size limit which might also be useful.