We recently moved from CentOS to Ubuntu. We did this for a few reasons:
  • CentOS PHP is old. We have two scripts that won't run on the default CentOS PHP. Our choice was either to do a custom build of PHP or switch to Ubuntu.
  • CentOS has an annoying "undocumented feature" which limits the number of available filehandles and was limiting our ability to create Apache log files for all of our virtual hosts.
  • More and more of our team members are running Ubuntu at home, so it's easier to support just one platform.

Ahh... but it wasn't all wine and roses from there.

CentOS uses one simple PHP configuration. Ubuntu uses different PHP configurations for command line PHP and web-based PHP. I like it, but it's not immediately intuitive.

CentOS uses a very simple httpd.conf setup. Ubuntu uses a freakishly complex apache2.conf setup. I think I may like this, once I get used to it. I'm still not certain that my Apache config is "Ubuntu standard."

CentOS and Ubuntu have different modules compiled into Apache.
[root@centos]# apachectl -l
Compiled in modules:

root@ubuntu:/# apache2ctl -l
Compiled in modules:

CentOS and Apache also had very different default ideas regarding which Apache modules should be loaded. It took a bit of reading to figure out how to load modules in Ubuntu's unique Apache config system.

Our server has 24G of RAM, which was plenty of memory for CentOS. Unfortunately, it's not enough for Ubuntu. We've been running out of RAM and swap space.

Figuring out why was a little odd. The memory size of our Apache processes on Ubuntu are about 25% smaller than they were on CentOS, so it seems that less memory should be used.

The answer seems to be in the list of shared libraries linked to each build of Apache.
[root@centos /]# ldd /usr/sbin/httpd
libm.so.6 => /lib64/libm.so.6 (0x00002b621a1ba000)
libpcre.so.0 => /lib64/libpcre.so.0 (0x00002b621a43d000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00002b621a659000)
libaprutil-1.so.0 => /usr/lib64/libaprutil-1.so.0 (0x00002b621a872000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00002b621aa8f000)
libldap-2.3.so.0 => /usr/lib64/libldap-2.3.so.0 (0x00002b621acc7000)
liblber-2.3.so.0 => /usr/lib64/liblber-2.3.so.0 (0x00002b621af02000)
libdb-4.3.so => /lib64/libdb-4.3.so (0x00002b621b110000)
libexpat.so.0 => /lib64/libexpat.so.0 (0x00002b621b405000)
libapr-1.so.0 => /usr/lib64/libapr-1.so.0 (0x00002b621b628000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b621b84f000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002b621ba6a000)
libc.so.6 => /lib64/libc.so.6 (0x00002b621bc6f000)
libsepol.so.1 => /lib64/libsepol.so.1 (0x00002b621bfc5000)
/lib64/ld-linux-x86-64.so.2 (0x00002b6219f9d000)
libuuid.so.1 => /lib64/libuuid.so.1 (0x00002b621c20b000)
libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002b621c410000)
libsqlite3.so.0 => /usr/lib64/libsqlite3.so.0 (0x00002b621c632000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00002b621c88c000)
libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00002b621caa2000)
libssl.so.6 => /lib64/libssl.so.6 (0x00002b621ccbb000)
libcrypto.so.6 => /lib64/libcrypto.so.6 (0x00002b621cf05000)
libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002b621d257000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00002b621d4ec000)
libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00002b621d705000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002b621d933000)
libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00002b621db35000)
libz.so.1 => /usr/lib64/libz.so.1 (0x00002b621dd5b000)
libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00002b621df6f000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002b621e177000)
root@ubuntu:/# ldd /usr/sbin/apache2
linux-vdso.so.1 => (0x00007fff135ff000)
libpcre.so.3 => /lib/libpcre.so.3 (0x00007fcd0afc1000)
libaprutil-1.so.0 => /usr/lib/libaprutil-1.so.0 (0x00007fcd0ad9f000)
libapr-1.so.0 => /usr/lib/libapr-1.so.0 (0x00007fcd0ab6c000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007fcd0a950000)
libc.so.6 => /lib/libc.so.6 (0x00007fcd0a5de000)
libldap_r-2.4.so.2 => /usr/lib/libldap_r-2.4.so.2 (0x00007fcd0a395000)
liblber-2.4.so.2 => /usr/lib/liblber-2.4.so.2 (0x00007fcd0a186000)
libdb-4.6.so => /usr/lib/libdb-4.6.so (0x00007fcd09e49000)
libpq.so.5 => /usr/lib/libpq.so.5 (0x00007fcd09c24000)
libmysqlclient_r.so.15 => /usr/lib/libmysqlclient_r.so.15 (0x00007fcd09822000)
libsqlite3.so.0 => /usr/lib/libsqlite3.so.0 (0x00007fcd0959b000)
libexpat.so.1 => /usr/lib/libexpat.so.1 (0x00007fcd09371000)
libuuid.so.1 => /lib/libuuid.so.1 (0x00007fcd0916c000)
librt.so.1 => /lib/librt.so.1 (0x00007fcd08f64000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x00007fcd08d2c000)
libdl.so.2 => /lib/libdl.so.2 (0x00007fcd08b28000)
/lib64/ld-linux-x86-64.so.2 (0x00007fcd0b1f1000)
libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0x00007fcd0890e000)
libresolv.so.2 => /lib/libresolv.so.2 (0x00007fcd086f6000)
libgnutls.so.26 => /usr/lib/libgnutls.so.26 (0x00007fcd08449000)
libtasn1.so.3 => /usr/lib/libtasn1.so.3 (0x00007fcd08238000)
libz.so.1 => /lib/libz.so.1 (0x00007fcd08020000)
libgcrypt.so.11 => /lib/libgcrypt.so.11 (0x00007fcd07db9000)
libssl.so.0.9.8 => /lib/libssl.so.0.9.8 (0x00007fcd07b6a000)
libcrypto.so.0.9.8 => /lib/libcrypto.so.0.9.8 (0x00007fcd077e1000)
libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0x00007fcd07546000)
libcom_err.so.2 => /lib/libcom_err.so.2 (0x00007fcd07342000)
libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0x00007fcd07115000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00007fcd06efb000)
libm.so.6 => /lib/libm.so.6 (0x00007fcd06c76000)
libgpg-error.so.0 => /lib/libgpg-error.so.0 (0x00007fcd06a73000)
libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0x00007fcd0684e000)
libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 (0x00007fcd06646000)
libkeyutils.so.1 => /lib/libkeyutils.so.1 (0x00007fcd06443000)

That's not easy to decipher. The short version is that CentOS httpd links two shared libraries which Ubuntu does not link:
  • libselinux.so.1
  • libsepol.so.1

While Ubuntu apache2 links seven shared libraries which CentOS does not link:
  • libgcrypt.so.11
  • libgnutls.so.26
  • libgpg-error.so.0
  • libmysqlclient_r.so.15
  • librt.so.1
  • libtasn1.so.3
  • linux-vdso.so.1

The really short version is that, unless I want to build a custom version of Apache, I am going to have to invest in a couple grand worth of RAM.

Would I go back? No, I don't think so. I'm trying to avoid custom builds of any software. They can be a real maintenance headache. But I do certainly appreciate CentOS a bit more, now that I'm running Ubuntu.