Who Reset my OPcache?

Who Reset my OPcache?

Six core dumps, a PHP bug that’s been open since 2022, and why some WordPress sites were getting stuck in maintenance mode.

We run WordPress on FrankenPHP. For months, some of our sites had been failing in ways that didn’t add up: the occasional segfault in the kernel log, a container that quietly restarted, and every so often a site stuck on “Briefly unavailable for scheduled maintenance” after an auto-update. None of it reproduced on demand, and none of it pointed anywhere useful.

This is the story of tracking it down. It turned out to be one of the more satisfying bugs I’ve chased in a while. The root cause is a real PHP bug that has been open for about four years, the trigger was a perfectly reasonable line of code in a popular plugin, and the whole thing only bites because of the way FrankenPHP runs PHP.

Here’s the short version, if you just want the picture:

A bit of background

FrankenPHP is PHP embedded in the Caddy web server. PHP-FPM runs a pool of separate worker processes; FrankenPHP instead runs one process and handles each PHP request on a thread, using PHP’s thread-safe (ZTS) build. Every one of those threads shares the same OPcache, the chunk of shared memory where PHP keeps compiled scripts so it doesn’t re-parse WordPress on every request.

Our setup, roughly: one container per WordPress site, FrankenPHP 1.11.2 on PHP 8.5, OPcache on, and a page cache in front. Keep “one process, many threads, one shared cache” in mind. It’s the whole story.

The first September core: a lesson in disk space

I’d collected a few core dumps back in the spring without getting anywhere, so when another crash landed in September I went at it more carefully. That core was 6.8 GiB. Great, I thought. Then gdb couldn’t produce a single useful backtrace: Cannot access memory at address … on every stack.

The core was truncated. Its program headers declared 13.3 GiB of content, but the filesystem the kernel was writing to had simply run out of room halfway through. What makes this nasty is that truncation is silent. The ELF headers still advertise the full segment sizes, and gdb’s only hint is a single warning buried among hundreds of others. Worse, Linux writes memory segments in ascending address order, and thread stacks live at the top of the address space, so stacks are the first thing you lose. 62 of 64 thread stacks were gone, including the one that crashed.

These days, before I do anything else with a core, I check that the file is at least as big as its program headers say it should be.

Not everything was lost. The PT_NOTE segment (registers, signal info, the list of mapped files) is written first, so it survived. That was enough to pin the crash to a single instruction inside zend_fetch_debug_backtrace: PHP was incrementing a reference count on a “pointer” of 0x1ad00000006, which is obviously not a pointer. My theory at the time was that PHP had walked a stale call stack. That turned out to be close, but I couldn’t prove anything from what was left. The fix for next time was boring: make sure there’s plenty of free space (around 20 GiB for us) wherever cores land.

The second September core: complete, and it talked

A few days later, one of our hosts logged three general protection faults in a single morning. Only one of them produced a core, but this time it was complete, all 10.7 GB of it.

Our libphp.so is stripped, which usually means “good luck”. Two things made it workable anyway:

  1. Match the build exactly. A core contains the first page of every mapped ELF file, including its build-id. That matched the libphp.so in our image, so I ran docker export on the image and pointed gdb’s sysroot at the result. That alone is enough for gdb to unwind every thread, since the unwind tables (.eh_frame) survive stripping.
  2. Bring your own types. The official PHP images ship PHP’s headers, and the full source tarball sits in /usr/src/php.tar.xz. I compiled a tiny C file that does nothing except include the headers I cared about, built it with -g, and loaded it into gdb with add-symbol-file. Suddenly I could cast raw addresses to PHP’s internal structures (zend_execute_data, zend_op_array, OPcache’s shared globals) without a debug build of PHP. One gotcha: FrankenPHP’s main binary is Go, so gdb parses expressions as Go until you set language c.

Here’s the native backtrace of the crashing thread:

#0  get_function_or_method_name        <- general protection fault on 0x0400003c000001a9
#1  zend_cannot_pass_by_reference
#2  (PHP VM helpers)
#4  zend_call_function
#5  zend_call_known_function
#6  (curl extension callback)
#7  libcurl ... curl_easy_perform
#14 curl_exec
    ... php_execute_script -> FrankenPHP's php_thread

PHP was inside curl_exec(), libcurl called back into PHP, and PHP crashed reading a function’s class pointer. That pointer, 0x0400003c000001a9, isn’t even a valid x86-64 address, which is why this was a general protection fault rather than a plain segfault.

PHP’s own call stack told the rest of the plot. The interpreter keeps a pointer to the current frame in a register (r14), and each frame links to its caller:

wp-admin/update-core.php          do_core_upgrade()
class-core-upgrader.php:221       do_action('upgrader_process_complete', ...)
  ... WordPress hook plumbing ...
  a plugin's update checker -> wp_remote_post()
Requests/Transport/Curl.php       curl_exec() -> header callback     <- crash

Someone had just updated WordPress core from the dashboard. The update itself was done, and WordPress was running its post-update hooks. One of those checked for plugin updates over HTTP, and PHP died inside curl’s callback.

The really interesting part was the state of that call stack. Most of the function structures in it were garbage. Where there should have been pointers, there were fragments of text like the curr and for the . Every one of those structures lived in OPcache’s 256 MiB shared memory segment. Something had overwritten the compiled code this request was in the middle of running.

The smoking gun

OPcache keeps a block of shared globals near the start of that segment, and libphp keeps a pointer to it in its .bss. Here’s what it said:

FieldValue
manual_restarts2, and zero out-of-memory or hash-full restarts
restart_reasonACCEL_RESTART_USER, meaning someone called opcache_reset()
last_restart_timeone second before the crash
hits / misses0 / 398: the whole cache had been rebuilt in that last second

PHP’s per-thread state (reachable through its thread-local storage) filled in the rest:

  • The crashed thread’s request had started 20 seconds before the restart.
  • Another thread’s request started at the exact second of the restart. OPcache stamps last_restart_time with the start time of whichever request performs the restart, so that thread did it, and then went on to recompile WordPress into the freshly wiped memory.

And the truncated core from before? OPcache’s segment sits at a low address, so it had survived the truncation. Same story: manual_restarts was 2, and the last restart happened in the same second as the crash.

So we had an opcache_reset(), a restart carried out by one thread while another was still mid-request, and the memory reused almost immediately.

Going back through the old cores

With a fingerprint to look for, I went back to the cores from the spring. Every one of them was complete, and the same check takes seconds:

CrashRestarts from opcache_reset()Last restart before the crash
March 312same second
April 2342 seconds
May 122same second
May 150none; the container had only been up for 84 seconds
September 202same second
September 2621 second

Five of the six crashes landed within two seconds of an OPcache restart. The May 15 crash is the odd one out: no restart had ever happened in that container, and it died with a different kind of fault. I’m treating that one as a separate problem that I haven’t explained yet, rather than forcing it to fit.

Who calls opcache_reset()?

WordPress core doesn’t. It only invalidates individual files with opcache_invalidate(). So I looked at the hooks registered in the crashed request. The strings in WordPress’s hook table had been overwritten too, but PHP stores each hash table key’s hash right next to the entry, so I could hash candidate callback names myself and match them up. The upgrader_process_complete callbacks at priority 10, in execution order:

  1. WP_Textdomain_Registry::invalidate_mo_files_cache
  2. googlesitekit_opcache_reset, which had already run
  3. redirection_clear_opcache_on_upgrade
  4. a theme framework’s cache hook
  5. wp_version_check
  6. wp_update_plugins, running when we crashed
  7. wp_update_themes

Google Site Kit calls opcache_reset() after every core, plugin, or theme update. WordPress core registers its own callbacks in a file that loads after plugins, so at the same priority, Site Kit always goes first. Digging through the not-yet-overwritten part of the cache also turned up Yoast SEO, which resets on every request while its database upgrade is pending, and Redirection, which resets after its own updates.

To be clear, none of these plugins are doing anything wrong. opcache_reset() is a documented PHP API. Although Site Kit’s source does carry a comment noting that opcache_reset is prohibited on WordPress VIP “due to memory corruption”, which in hindsight is a pretty good hint.

Why it’s unsafe on FrankenPHP

opcache_reset() doesn’t wipe anything itself. It sets a “restart pending” flag, and the actual restart happens at the start of a later request, but only if OPcache decides nobody is using the cache anymore. Here’s how it decides on Linux, from ext/opcache/ZendAccelerator.c:

static inline bool accel_is_inactive(void)
{
	/* ... */
	mem_usage_check.l_type = F_WRLCK;
	/* ... */
	if (fcntl(lock_file, F_GETLK, &mem_usage_check) == -1) {
		/* ... */
		return false;
	}
	if (mem_usage_check.l_type == F_UNLCK) {
		return true;
	}
	/* ... */

Every active request holds a shared fcntl() read lock on a lock file, and the restart asks whether anyone would block a write lock. That works for PHP-FPM, where every worker is a separate process. But POSIX record locks belong to the process, and F_GETLK never reports locks held by the process that’s asking. In FrankenPHP, every request is a thread in the same process, so all the other requests’ locks are invisible. The answer is always “nobody’s using it”, and the restart goes ahead while other threads are mid-request. It clears the hash table, resets the interned strings, and rewinds the allocator, so the next scripts to be compiled land right on top of code that’s still running.

The Windows build of OPcache has long used an atomic counter for this, since Windows SAPIs are threaded. Linux thread-safe builds never got the equivalent. And it isn’t only opcache_reset(): any OPcache restart, including running out of memory or out of hash slots, has the same problem under a threaded server.

The thread that wouldn’t die

One more thing turned up in that core: a PHP thread that had been stuck for four days. Its request had started five seconds before the container’s first opcache_reset() restart. It ended up in free() during PHP’s symbol-table cleanup, glibc detected heap corruption, and glibc called abort().

Normally that kills the process. But FrankenPHP’s main program is written in Go, and Go installs its own signal handlers. As far as I can tell, when the signal lands on a non-Go thread (PHP’s threads are plain pthreads), Go’s handler re-raises it with the default action, sleeps for a millisecond, puts its own handler back, and returns. glibc’s last-resort hlt instruction faults again, and around it goes. The process stayed up, the request never finished, and one PHP thread was simply gone. Two restarts in that container, two casualties.

The maintenance-mode mystery

Once I understood the mechanism, the “stuck in maintenance mode” reports suddenly made sense. This is what a WordPress auto-update does on a site with Site Kit installed:

  1. It updates a plugin. upgrader_process_complete fires, and Site Kit calls opcache_reset().
  2. It turns maintenance mode back on, waits two seconds, then requests the site’s own homepage to make sure the update didn’t cause a fatal error.
  3. That check, or any other request that shows up at the same time, starts on another thread and carries out the pending restart, while the update request sits in curl waiting for an answer.
  4. The update request resumes on a wiped cache and crashes or wedges. It never turns maintenance mode off.

WordPress stops honoring a .maintenance file once it’s 10 minutes old, and the auto-update lock expires after an hour, so these sites “fixed themselves”. That’s a big part of why this was so hard to pin down. (One wrinkle: our page cache can answer that homepage check without running PHP at all. But something else, most likely the health check, reaches PHP every few seconds anyway, so the restart still lands while the update is running.)

Reproducing it on purpose

I didn’t want to install Site Kit and wait around for updates, so I wrote a throwaway mu-plugin that does the same dance on demand (reference). Requesting /wp-login.php?hh_opcache_test=true:

  1. calls opcache_reset(), like Site Kit,
  2. sleeps two seconds, like the updater,
  3. sends a request to /wp-admin/, a path our page cache never serves, so it always reaches PHP,
  4. keeps running WordPress code, and reports OPcache’s counters along the way.

On the old image, the first run sat there for about a minute and came back as a 502. There was nothing in the container logs, not even an access-log line. FrankenPHP’s admin API showed why: /frankenphp/threads reported one of the site’s two PHP threads as busy, and it never came back. OPcache’s counters confirmed that a restart had happened in the middle of that request.

The fun twist was that I couldn’t make it happen a second time. With one thread stuck, the site only had one working PHP thread, so requests ran strictly one at a time, and the race needs two requests running at once. Each later run’s loopback request just waited for a free thread and timed out, and the pending restart ran harmlessly once the test was done. A good reminder that “it didn’t happen again” isn’t the same as “it’s fixed”.

On the patched image, after restarting the container, the whole test came back in a couple of seconds: opcache_reset() returned false, there were zero restarts, and both threads were idle afterwards.

Where things stand upstream

As of September 29, 2026:

PHP is where this really needs fixing.

  • php-src GH-8739, open since June 2022, is the canonical issue. It was originally reported against PHP-FPM; FrankenPHP reports such as GH-18517 were closed as duplicates of it.
  • PR #22281 gives Linux thread-safe builds the same kind of atomic in-flight counter that Windows uses. That covers manual and automatic restarts. It’s open.
  • PR #21778 takes a different approach, epoch-based reclamation. Also open. An earlier version (#21775) was closed, and the original locking-based attempt, #14803, stalled after pushback about adding a lock to every cache lookup.
  • No released PHP version includes a fix yet. The latest releases at the time of writing are 8.4.26 and 8.5.11.

FrankenPHP has shipped a mitigation.

  • PR #2364, released in v1.12.5, swaps opcache_reset() for its own handler that restarts every PHP thread instead. It drains in-flight requests, force-kills anything still running after 6 seconds, then restarts the PHP engine with a cold cache.
  • PR #2564, in v1.12.7, removed the companion hook for automatic restarts because it could cascade into repeated restarts. On the latest FrankenPHP, only explicit opcache_reset() calls are covered.
  • Related reports: #1737 (crashes on WordPress sites, open), #2170 (closed by #2364, with a great reproducer), and #2568 (thread restarts misbehaving when pecl_http is loaded, open).

What we did in the meantime

1. Take opcache_reset() away, gently. The obvious move is disable_functions = opcache_reset. The catch is that since PHP 8, disabled functions are deleted outright, so a plugin that calls it without checking function_exists() first would hit a fatal error. The plugins we found all check, but we host plenty of plugins nobody has audited. Because the function is deleted, though, PHP lets you declare your own with the same name. So we disable it and load a no-op replacement with auto_prepend_file:

disable_functions = ...,opcache_reset
auto_prepend_file = /usr/local/etc/php/frankenwp-prepend.php
<?php
if (!function_exists('opcache_reset')) {
    function opcache_reset(): bool
    {
        return false;
    }
}

Plugins get false, the same answer they’d get on a host that restricts the OPcache API, and nothing crashes. We don’t lose anything functional either: changed files are still picked up within a couple of seconds through timestamp validation, and WordPress’s per-file opcache_invalidate() keeps working. It only marks scripts as stale and never reuses their memory, so it’s safe. We tested the stub on every PHP version we ship (8.2 through 8.5), and it’s easy to back out, either globally or for a single site.

2. Upgrade FrankenPHP anyway. We moved to FrankenPHP 1.12.7, which also meant Caddy 2.11.4. With the stub in place, FrankenPHP’s own override never kicks in, but we wanted the security fixes that came with 1.12.5. We deliberately didn’t rely on the override by itself. Every call would drain and restart all of a site’s PHP threads on a 6-second kill timer, and a WordPress auto-update run can trigger that after every plugin it updates. That’s a much better failure mode than memory corruption, but it would probably still leave sites stuck in maintenance mode.

3. Keep an eye on what’s left. Automatic restarts, when OPcache runs out of memory or hash slots, still have the same race until PHP ships a fix. Neither core showed any of those, so for now we’re watching the counters. On FrankenPHP 1.12, /frankenphp/threads also shows each thread’s current request and when it started, which makes a stuck thread easy to spot.

Once a PHP 8.4 or 8.5 release mentions GH-8739 in its changelog, we’ll rebuild our images on it. Whether the stub ever goes away depends on what FrankenPHP does with its override after that, but it costs us nothing to keep.

Takeaways

  • Check core dumps for truncation before you trust them. It’s silent, and the stacks go first.
  • Stripped binaries aren’t a dead end. A build-id match, an exported container filesystem, and a types-only object compiled from the package’s own headers got me readable PHP internals without a debug build.
  • In a threaded PHP server, OPcache restarts aren’t safe. That includes opcache_reset() and, for now, automatic restarts too.
  • Process-wide primitives and threads don’t mix. fcntl() locks are per-process by design, and code written for pre-fork servers can quietly assume one request per process.
  • A Go host process changes how crashes look. A fatal signal on a non-Go thread can turn into a thread that spins forever instead of a process that dies, which turns crashes into hangs.

All in all, kind of a fun one. If you run WordPress on FrankenPHP, check which of your plugins call opcache_reset(), and go subscribe to GH-8739.

Next Steps

We continue to work hard to bring you the best value for all of your WordPress hosting needs.  If you have any questions, concerns, or would generally like to chat, feel free to contact us anytime.  We want to make sure you get the most out of HonestHosting.io!