The best server maintenance is the kind your clients never hear about

On October 21, PHP released fixes for two critical remote code execution vulnerabilities. The PHP security advisory said all PHP installations serving web requests were at risk, and that "there are no complete mitigations, updating is the only safe choice."

A lot of site owners took that warning literally. The monitoring service Oh Dear reported that at 14:30 UTC that day, 18.8% of the sites it watches were offline, against about 0.9% a week earlier. Many had been switched off on purpose until they could be patched.

We run most of our clients' sites on servers we manage with Laravel Forge. Forge doesn't install PHP patch releases automatically, so we updated the servers by hand that afternoon. Servers running more than one PHP version needed each version updated separately, and a couple of the updates waited behind other package jobs for longer than we expected. By the evening every site was patched and running without errors.

One older server was on a PHP version too old to receive the fix. Patching it meant a planned migration to a newer operating system and PHP version, with a short window of downtime. We did the migration. Once a PHP version stops getting security releases, a critical fix turns into a migration, which is a good reason to keep every server on a supported version.

Patching a handful of servers in an afternoon is ordinary work. Plenty of hosting companies look after hundreds of servers, with monitoring to match. For me, the useful result was that none of our clients had to ask whether their site was affected.

I think the best experience for a client is one where they don't have to know about every vulnerability that comes out. There have also been a lot of them lately. PHP put out a security release on September 24. Laravel published a critical advisory on October 19 for apps that use its TokenGuard. Node.js shipped its own security releases the same day as PHP, and PHP followed with an advisory about session upload progress on October 23. Emailing clients every time one of these lands would get a little ridiculous after the first few.

I think the amount of maintenance a site needs has gone up a lot recently, and AI is part of the reason. Laravel's post-mortem on that week says: "This will keep happening. Disclosures like these are now frequent, and AI is shortening the time between a vulnerability becoming known and someone trying to exploit it." The Node.js Brotli bug fixed on October 21, which could allow remote code execution, was reported by OpenAI Codex Security. The number of published CVE records has also more than doubled compared with the same part of last year.

Sites that fall behind on updates get taken over and turned against their own visitors. Some examples from WordPress this year:

• On July 17, WordPress fixed a chain of core bugs, nicknamed WP2Shell, that let anyone create an administrator account and run code on the server. Wordfence saw mass exploitation start the next day and has blocked more than 11 million attempts. WordPress pushed forced automatic updates, which kept most sites safe.
• In June, Dutch police removed malware from 14,971 hacked WordPress sites that the SocGholish group had been using to push fake browser updates to visitors.
• In August, Check Point traced close to 2,000 hacked WordPress sites to one operation. The sites showed visitors a fake captcha that tricked them into running malware, and some stored data stolen from victims. One was running a version of WordPress from 2021 with nearly 40 known vulnerabilities.

More disclosures mean more patches to apply, and less time to apply them. The work itself is routine: watch the advisories, update the servers the day a fix ships, and keep each server on a version that still gets fixes. When that work is done on time, the client doesn't need to hear about it.

Isaac Earl

// Lead Developer