Apache2 HTTPD AH00112 DocumentRoot does not exist

Unlike most of the information I was able to find, this error is not always related to permissions.

Some history first…

I am running a web farm consisting of 3 servers behind a load balancing firewall.  The servers are identical.

Installed Debian Stretch (stable) with some packages from Buster (testing) due to needing newer versions.

I have all 3 machines running gluster with a replicated brick on each machine.  Overall performance is okay but not the greatest.  It is about where I expected it to be for the hardware used.

SELinux is not in use.

What changed

While doing some year end clean up (and new year prep), I found I was behind on updates.  So I fetched and installed the latest updates and rebooted just in case.  Step through my restart procedure, no errors jump out.  Later I found out that none of the web sites were working, they all said “Forbidden”.

What happened

So check the logs and the first thing I find is AH00112 DocumentRoot [xxx] does not exist. for every virtual host. WTF???

The only time I’ve seen this before is when I forgot to mount the gluster volumes.
As root I can clearly see the files and folders. I can even see them as www-data. I can browse the entire file system right back to / as both users. So it is not permissions.
I checked the change logs (apache and Debian versions) and could find nothing that jumped out.
Using apache2ctl configtest everything was OK. I even put in a bad vhost directory just to test the output from configtest where it showed the truely missing directory.

Everything worked before the upgrades.

Setting apache2’s loglevel to trace1 (also tried 2 through 8) showed nothing useful.

Fixed?

Turns out to be some interaction between apache2 and gluster.
After I upgraded gluster from 3.8.8 (stable) to 3.13.0 (testing) the sites started working again.

So something Apache changed between 2.4.25 and 2.4.29 looked at the file system a bit differently.
Or Gluster between 3.8.8 and 3.13.0 fixed an issue with file access by apache.

Even now I can not find anything in the log files to indicate what apache was choking on when trying to access the files in the gluster storage.

But I got it to work again so I am happy once more.

Bookmark the permalink.

Comments are closed.