RegisterLogin
Linux

Diagnosing the Linux OOM Killer Silently Crashing MySQL/MariaDB on Your cPanel WordPress Server

A deep-dive diagnostic guide to understanding, detecting, and permanently fixing the Linux Out-of-Memory (OOM) Killer repeatedly crashing MySQL or MariaDB on cPanel/WHM servers running WordPress — covering kernel tuning, zswap, oom_score_adj, systemd overrides, and proactive memory governance.

Nextgen Team•
Diagnosing the Linux OOM Killer Silently Crashing MySQL/MariaDB on Your cPanel WordPress Server

Diagnosing the Linux OOM Killer Silently Crashing MySQL/MariaDB on Your cPanel WordPress Server

Your WordPress site goes down at 3 AM. You SSH in — MySQL is dead. You restart it, everything works fine, and there’s no obvious PHP error in sight. An hour later it happens again. You blame the plugin, the theme, the host. But the real culprit is hiding in a six-line kernel log entry that most sysadmins never look at: the Linux OOM (Out-of-Memory) Killer.

This guide walks you through the complete forensic process — from reading the raw kernel dump to permanently protecting your database with kernel-level and systemd-level controls, zswap, earlyoom, and oom_score_adj.

What Is the OOM Killer and Why Does It Target MySQL?

The Linux kernel performs memory overcommit by default — it happily hands out more virtual memory than physical RAM exists, betting that processes won’t use it all simultaneously. When that bet fails during a traffic spike, the kernel enters a panic state and invokes the OOM Killer: a scoring algorithm that selects the most “expendable” process and terminates it with SIGKILL.

MySQL and MariaDB are frequent victims because:

  1. Large resident set size (RSS) — innodb_buffer_pool_size holds the entire hot dataset in RAM, giving MySQL a high OOM “badness” score.
  2. Non-pageable memory — InnoDB buffer pool pages are locked, so the kernel sees MySQL as taking memory away from everyone else.
  3. cPanel’s Apache/PHP stack — Each LiteSpeed or Apache worker spawns a fresh PHP process. Under load, dozens of concurrent lsphp workers each allocate 32–128 MB, suddenly overwhelming available RAM.

The kernel always kills the process with the highest OOM score, calculated as:

oom_badness = (pages_used / total_pages) * 1000  +  oom_score_adj

A fresh MySQL instance with innodb_buffer_pool_size=4G on an 8 GB server will score close to 500, making it the #1 target.

Step 1: Confirm the OOM Killer Is the Culprit

Do not assume OOM. Verify it with logs first.

Check kernel ring buffer

dmesg -T | grep -E "(Out of memory|oom_kill|Killed process)" | tail -30

Expected output:

[Thu Sep 19 03:14:52 2026] Out of memory: Kill process 14821 (mysqld) score 492 or sacrifice child
[Thu Sep 19 03:14:52 2026] Killed process 14821 (mysqld) total-vm:8421536kB, anon-rss:3912448kB, file-rss:0kB, shmem-rss:0kB

Check persistent system logs

# AlmaLinux / CloudLinux / CentOS
grep -Ei "(out of memory|oom_kill|Killed process)" /var/log/messages | tail -40

# Ubuntu / Debian
grep -Ei "(out of memory|oom_kill|Killed process)" /var/log/syslog | tail -40

# systemd journal (any distro)
journalctl -k --since "7 days ago" | grep -i "out of memory"

Read the full OOM dump

The kernel logs an entire process table snapshot just before killing. Find the block:

grep -A 60 "Out of memory" /var/log/messages | head -80

Example kernel OOM dump:

Sep 19 03:14:51 srv1 kernel: mysqld invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
Sep 19 03:14:51 srv1 kernel: mysqld cpuset=/ mems_allowed=0
Sep 19 03:14:51 srv1 kernel: CPU: 2 PID: 14820 Comm: mysqld Not tainted 5.14.0-427.33.1.el9_4.x86_64
Sep 19 03:14:51 srv1 kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Sep 19 03:14:51 srv1 kernel: [  14820]     0 14820  2105384   978112   9097216        0             0 mysqld
Sep 19 03:14:51 srv1 kernel: [  14950]   501 14950   253456    80322   2031616      640             0 lsphp
Sep 19 03:14:51 srv1 kernel: [  14971]   502 14971   251088    79450   2015232      512             0 lsphp
...
Sep 19 03:14:52 srv1 kernel: Out of memory: Kill process 14821 (mysqld) score 492 or sacrifice child

The oom_score_adj=0 on mysqld confirms it has no protection at all.

Step 2: Understand Your Memory Budget

Before tuning anything, audit what is actually consuming RAM.

# Human-readable overview
free -h

# Per-process sorted by RSS
ps aux --sort=-%mem | head -20

# Full memory map of mysqld
cat /proc/$(pidof mysqld)/status | grep -E "(VmRSS|VmSwap|VmPeak)"

# Check swap usage
swapon --show
cat /proc/swaps

Sample output from a typical overloaded 8 GB cPanel server:

              total        used        free      shared  buff/cache   available
Mem:           7.8G        7.1G        112M        244M        620M        381M
Swap:          2.0G        1.9G         89M

This pattern — swap nearly exhausted, available RAM under 400 MB — is the OOM Killer’s green light.

Identify top memory consumers

# Sort by actual memory use, not virtual
smem -r -s rss | head -20

# Or without smem:
for proc in $(ls /proc | grep -E '^[0-9]+$'); do
  if [ -r /proc/$proc/status ]; then
    awk '/Name|VmRSS/{printf "%s ", $2}' /proc/$proc/status
    echo
  fi
done | sort -k2 -rn | head -20

Step 3: Audit MySQL/MariaDB Memory Configuration

The most common mistake on cPanel servers is an innodb_buffer_pool_size that was configured for a dedicated database server but now runs alongside Apache, LiteSpeed, and PHP.

# Check current InnoDB buffer pool
mysql -e "SHOW VARIABLES LIKE 'innodb_buffer_pool_size';"

# Check all memory-related variables
mysql -e "SHOW VARIABLES;" | grep -E "(buffer|cache|sort|join)_size"

# Run MySQLTuner for a full audit
curl -sL https://raw.githubusercontent.com/major/MySQLTuner-perl/master/mysqltuner.pl -o /tmp/mysqltuner.pl
perl /tmp/mysqltuner.pl --user root --pass $(cat /root/.my.cnf | grep password | cut -d= -f2)

The safe formula for shared cPanel servers

On a server that runs both a web stack and MySQL, never allocate more than 55–60% of physical RAM to innodb_buffer_pool_size. For 8 GB RAM:

innodb_buffer_pool_size = 8192 * 0.55 ≈ 4505 MB → use 4G

Edit /etc/my.cnf (or /etc/mysql/my.cnf):

[mysqld]
# --- Memory Budget (8 GB server, shared cPanel stack) ---
innodb_buffer_pool_size        = 4G
innodb_buffer_pool_instances   = 4       # one per GB of pool size
innodb_log_file_size           = 512M
innodb_log_buffer_size         = 16M

# Reduce per-connection overhead
sort_buffer_size               = 256K    # default 256K, keep it low
join_buffer_size               = 256K
read_buffer_size               = 128K
read_rnd_buffer_size           = 256K
thread_stack                   = 192K

# Connection limits
max_connections                = 100     # 150 * 1MB overhead = 150MB

# Query cache (only MariaDB, deprecated in MySQL 8)
query_cache_type               = 0
query_cache_size               = 0

Restart MariaDB:

systemctl restart mariadb
# Verify new buffer pool is live
mysql -e "SHOW ENGINE INNODB STATUS\G" | grep -A5 "BUFFER POOL"

Step 4: Protect MySQL with oom_score_adj

The kernel’s scoring system is a dial from -1000 (never kill) to +1000 (kill first). Setting MySQL to -800 makes it nearly immune without making it completely unkillable.

Immediate (runtime) adjustment

MYSQL_PID=$(pidof mysqld)
echo -800 > /proc/${MYSQL_PID}/oom_score_adj
# Verify
cat /proc/${MYSQL_PID}/oom_score_adj

This survives restarts — runtime writes to /proc/PID/ do not.

# For MySQL 8.x
systemctl edit mysqld

# For MariaDB
systemctl edit mariadb

This opens a drop-in override file. Add:

[Service]
OOMScoreAdjust=-800

Save and reload:

systemctl daemon-reload
systemctl restart mariadb   # or mysqld

# Confirm it took effect
cat /proc/$(pidof mysqld)/oom_score_adj
# Expected: -800

Important: Setting OOMScoreAdjust=-1000 makes the process completely immune and is dangerous. If MySQL itself has a memory leak, the kernel will be forced to kill other critical system processes — possibly crashing the entire server. -800 is the recommended balance.

Step 5: Kernel VM Tuning — vm.swappiness and vm.overcommit_memory

Tune swappiness

The default swappiness=60 tells the kernel to start swapping when RAM is only 60% full. For a database server, set it lower so MySQL stays in RAM:

# Check current value
cat /proc/sys/vm/swappiness

# Apply immediately
sysctl -w vm.swappiness=10

# Make permanent
echo "vm.swappiness=10" >> /etc/sysctl.d/99-nextgen-memory.conf
sysctl -p /etc/sysctl.d/99-nextgen-memory.conf

Tune overcommit mode

# Check current mode
cat /proc/sys/vm/overcommit_memory
# 0 = heuristic (default), 1 = always allow, 2 = strict

On cPanel servers, mode 2 (strict) often breaks fork()-heavy workloads like Apache. The safer choice is to leave overcommit_memory=0 but lower vm.dirty_ratio and vm.dirty_background_ratio to reduce memory pressure from dirty page buildup:

cat >> /etc/sysctl.d/99-nextgen-memory.conf <<EOF

# Reduce dirty page pressure
vm.dirty_ratio                 = 10
vm.dirty_background_ratio      = 3

# Panic on OOM instead of killing processes (optional, use on dedicated DB nodes)
# vm.panic_on_oom              = 1

# Do not overcommit beyond physical RAM + swap
# vm.overcommit_memory         = 2
# vm.overcommit_ratio          = 80
EOF

sysctl -p /etc/sysctl.d/99-nextgen-memory.conf

Step 6: Enable zswap — Compressed RAM Swap Cache

zswap intercepts swap writes and compresses the pages into a pool in RAM before they touch disk. On servers with HDDs or even SATA SSDs, this can reduce swap I/O by 60–80% and buy critical seconds during memory spikes.

Check if zswap is available

cat /sys/module/zswap/parameters/enabled
# Y = enabled, N = not active
zcat /proc/config.gz | grep ZSWAP   # if kernel config is accessible

Enable zswap

# Enable immediately
echo 1 > /sys/module/zswap/parameters/enabled

# Set pool size (% of total RAM for the compressed pool — start at 20%)
echo 20 > /sys/module/zswap/parameters/max_pool_percent

# Use the z3fold allocator (more efficient than zbud for larger pools)
echo z3fold > /sys/module/zswap/parameters/zpool

# Choose LZ4 as the compressor (best speed/ratio balance)
echo lz4 > /sys/module/zswap/parameters/compressor

Make zswap permanent via GRUB

# Edit GRUB configuration
vi /etc/default/grub

Find GRUB_CMDLINE_LINUX= and append:

GRUB_CMDLINE_LINUX="... zswap.enabled=1 zswap.compressor=lz4 zswap.max_pool_percent=20 zswap.zpool=z3fold"

Apply:

# AlmaLinux / CloudLinux / CentOS Stream
grub2-mkconfig -o /boot/grub2/grub.cfg

# Ubuntu
update-grub

Monitor zswap effectiveness

cat /sys/kernel/debug/zswap/pool_total_size
cat /sys/kernel/debug/zswap/stored_pages
cat /sys/kernel/debug/zswap/written_back_pages   # pages that missed the pool and hit disk

A healthy ratio is written_back_pages staying at 5–10% of stored_pages.

Step 7: Deploy earlyoom for Proactive Process Termination

The kernel OOM Killer acts when memory is already completely exhausted, which causes the system to thrash (swap in/out at full speed) for 30–60 seconds before killing anything — during which WordPress serves 502/503 errors. earlyoom kills the sacrificial process before the kernel panics.

Install earlyoom

# AlmaLinux / CloudLinux (enable EPEL first)
dnf install epel-release -y
dnf install earlyoom -y

# Ubuntu/Debian
apt install earlyoom -y

Configure earlyoom

Edit /etc/sysconfig/earlyoom (RPM systems) or /etc/default/earlyoom (Debian systems):

# Trigger when free RAM drops below 10% OR free swap below 5%
EARLYOOM_ARGS="-m 10 -s 5 --prefer '(lsphp|php-fpm)' --avoid '(mysqld|mariadb)' -n -d"
  • --prefer '(lsphp|php-fpm)' — kill PHP workers before anything else
  • --avoid '(mysqld|mariadb)' — never kill the database unless truly last resort
  • -n — send notifications via wall
  • -d — debug mode for logging

Enable and start

systemctl enable --now earlyoom
systemctl status earlyoom

# Watch earlyoom decisions in real time
journalctl -u earlyoom -f

Step 8: Limit PHP Worker Memory on cPanel/LiteSpeed

Preventing memory exhaustion is better than managing it after the fact. Control the maximum number of concurrent PHP workers:

In WHM → LiteSpeed (LSWS)

Navigate to WHM → LiteSpeed Web Server → External Applications and set:

  • Max Connections: 8–16 (depends on RAM, not CPU cores)
  • Initial Request Timeout: 60s
  • Memory Soft Limit: 256M
  • Memory Hard Limit: 512M

Via PHP-FPM pool configuration (if using PHP-FPM)

; /etc/php-fpm.d/www.conf or per-domain pool
pm                    = dynamic
pm.max_children       = 20
pm.start_servers      = 4
pm.min_spare_servers  = 2
pm.max_spare_servers  = 6
pm.max_requests       = 500          ; recycle workers to prevent memory leaks

Restart PHP-FPM:

systemctl restart php-fpm

Per-WordPress-site memory limit

Add to wp-config.php:

define('WP_MEMORY_LIMIT', '256M');
define('WP_MAX_MEMORY_LIMIT', '512M');

And in .htaccess for hard-capping at the server level:

php_value memory_limit 256M

Step 9: Set Up Continuous Memory Monitoring

Reactive fixes prevent one crash. Proactive monitoring prevents the next ten.

Install atop for historical memory data

dnf install atop -y   # or: apt install atop -y
systemctl enable --now atop

# Replay memory stats from 3 AM this morning
atop -m -r /var/log/atop/atop_$(date +%Y%m%d) -b 03:00 -e 03:30

Create a memory watchdog cron script

cat > /usr/local/bin/memory-watchdog.sh <<'SCRIPT'
#!/bin/bash
# Memory watchdog — alerts and optionally kills PHP workers if RAM < 15%
THRESHOLD=15
LOGFILE=/var/log/memory-watchdog.log
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')

TOTAL=$(awk '/MemTotal/{print $2}' /proc/meminfo)
AVAILABLE=$(awk '/MemAvailable/{print $2}' /proc/meminfo)
PCT=$(( AVAILABLE * 100 / TOTAL ))

if [ "$PCT" -lt "$THRESHOLD" ]; then
  echo "[$TIMESTAMP] WARNING: Available RAM is ${PCT}% (${AVAILABLE}kB free of ${TOTAL}kB)" >> "$LOGFILE"
  # Kill the highest-memory lsphp worker
  VICTIM=$(ps aux --sort=-%mem | grep lsphp | head -1 | awk '{print $2}')
  if [ -n "$VICTIM" ]; then
    echo "[$TIMESTAMP] Killing lsphp PID $VICTIM" >> "$LOGFILE"
    kill -9 "$VICTIM"
  fi
fi
SCRIPT

chmod +x /usr/local/bin/memory-watchdog.sh

Add to root’s crontab:

crontab -e
# Add:
* * * * * /usr/local/bin/memory-watchdog.sh

Step 10: Verify the Full Stack Is Stable

After applying all changes, run a 24-hour memory soak test:

# Watch real-time memory pressure every 5 seconds
watch -n5 'free -h && echo "---" && swapon --show && echo "---" && cat /proc/$(pidof mysqld)/oom_score_adj'

# Monitor earlyoom
journalctl -u earlyoom --since "1 hour ago"

# Check OOM events in kernel log
dmesg -T --level=err,crit,alert | grep -i "oom\|killed"

After 24 hours with no OOM events, lock down the configuration with:

# Snapshot current sysctl state
sysctl -a > /root/sysctl-baseline-$(date +%Y%m%d).txt

Summary: Complete Fix Checklist

Action Command / Location Impact
Confirm OOM events dmesg -T | grep oom Diagnosis
Right-size innodb_buffer_pool_size /etc/my.cnf — 55% of RAM Reduce MySQL footprint
Protect MySQL via systemd systemctl edit mariadb → OOMScoreAdjust=-800 Stop MySQL being killed
Lower vm.swappiness sysctl -w vm.swappiness=10 Keep DB in RAM longer
Enable zswap GRUB cmdline + sysfs Compressed RAM buffer
Deploy earlyoom dnf install earlyoom Kill PHP before DB
Limit PHP workers WHM → LiteSpeed or php-fpm.d/ Prevent RAM spikes
Memory watchdog cron /usr/local/bin/memory-watchdog.sh Continuous protection

When Tuning Isn’t Enough: Upgrade Your Infrastructure

All of the above are optimizations on constrained hardware. If your cPanel server hosts more than 15 active WordPress sites, or a single WooCommerce store with consistent traffic above 500 concurrent users, tuning alone cannot substitute for more RAM or a dedicated database node.

Consider running your database on a separate server, or migrating to hardware purpose-built for demanding workloads. Nextgen’s Dedicated Servers offer bare-metal configurations with up to 256 GB RAM, allowing you to provision MySQL with a generous buffer pool without competing against PHP workers. For locally deployed applications within Pakistan, Dedicated Servers in Pakistan provide the same bare-metal performance with low-latency connectivity to Pakistani ISPs — ideal for high-throughput WooCommerce or LMS deployments.

Frequently Asked Questions

Q: My MySQL restarts fine manually after every crash — is OOM definitely the cause? Yes. OOM kills with SIGKILL, which bypasses graceful shutdown. MySQL always recovers cleanly via InnoDB crash recovery on next start — which is why it appears to “work fine” after a manual systemctl restart.

Q: Should I set vm.overcommit_memory=2 to prevent OOM entirely? Not on cPanel. Setting mode 2 causes fork() to fail when virtual memory accounting exceeds the limit, which breaks Apache, cPanel daemons, and many PHP processes. It is only appropriate on dedicated single-workload servers.

Q: Does zswap work with encrypted LUKS volumes? Yes — zswap operates entirely within RAM. The compressed pages are never written to the encrypted swap device unless the pool fills (controlled by max_pool_percent).

Q: Can I set OOMScoreAdjust=-1000 for complete immunity? Technically yes, but avoid it. If MySQL has a memory leak at score -1000, the kernel will kill critical system processes or crash with a kernel panic. -800 provides strong protection while leaving the kernel a last-resort option.