Everything you wanted to know about core dumps, but were afraid to ask

15 Oct 2008

Basics

The latest trend in Linux (and Mac OS X) distributions is to turn off core dumps. And since core dumps are your friend if you are developer, you'll want to fix this.

You can check the size of the core dump file with the bash command ulimit:

$ ulimit -c
0
Means, "no core dumps". To turn them back on:

$ ulimit -c unlimited
$ ulimit -c
unlimited

Core Dump Unit Test

If you are testing how and where core dumps occur, changing ulimits in system files, etc, you want some way of making core dumps "on demand" to make sure you got it right. This little C program can be compiled with gcc -o coredump coredump.c. Now you can make all core dumps you want.

#include <stdlib.h>

int main(int argc, char**argv) {
abort();
return 0;
}

Linux

Detailed notes can be found with man core. Since it's linux, it may vary for your version. The main point is that you can control the core file naming using the /proc filesystem or sysctl.

$ ls -l /proc/sys/kernel/core*
-rw-r--r-- 1 root root 0 2008-10-15 18:06 /proc/sys/kernel/core_pattern
-rw-r--r-- 1 root root 0 2008-10-15 18:06 /proc/sys/kernel/core_uses_pid

$ cat /proc/sys/kernel/core_pattern
core

The core pattern uses a printf style string (from the core man page:

%%  A single % character
%p PID of dumped process
%u real UID of dumped process
%g real GID of dumped process
%s number of signal causing dump
%t time of dump (seconds since 0:00h, 1 Jan 1970)
%h hostname (same as ’nodename’ returned by uname(2))
%e executable filename
For instance:

echo "/var/corefiles/core-%e-%p.core" > /proc/sys/kernel/core_pattern
Make sure the filename is "unique" using the PID if you have multiple instances running

Important! If you are debugging a server, note this little detail on the man page (man 2 setuid):

If uid is different from the old effective uid, the process will be forbidden from leaving core dumps.
I guess this is some security check but can make debugging difficult.

Core File Cleanup

Core files can be large. And if you are having problems, your disk can filled up fast interferring with regular operations. You'll have to have a cron process to find and remove old core files.

Another option is to make a special disk partition where core files can live. If it fills up, nothing particularly bad happens.

Forcing an application to dump core

You can make a running application dump core by send the right signal via kill. Which signal may depend your application but SIGTRAP frequently works. You can get a list of signals with kill -l

$ kill -s SIGTRAP 4321
FATAL ERROR (5) Trace/breakpoint (core dumped)

Mac OS X

Core files are in the /cores directory. I'm not sure if this can be changed. By default ulimit -c is 0