I spent some of last week and this week getting all the Fedora koji builders updated/re-installed.

There’s quite a stable of them these days:

  • 2 bkernel servers for building only kernel, grub2, pesign, and any secure boot needs.
  • 27 buildvm servers. These are Fedora 20 kvm guests on 3 large virt servers (9 per). They do general i686/x86_64/noarch builds.
  • 12 buildhw servers. These are Fedora 20 servers on hardware. They do general i686/x86_64/noarch builds and also image builds that need hardware virt support
  • 2 buildppc servers. These are RHEL6 servers on ppc lpars. They do epel ppc64 builds.
  • 48 arm-builder servers. These are highbank arm SOC’s. (2 chassis of 24 each). They do general armv7 and noarch builds

Thats 91 servers to update/reboot/re-install. Happily we have ansible, so it’s not so bad at all. I can reinstall all the buildvm’s in about 15minutes with ansible, and updating itself is pretty simple as well, with most of the time being waiting until particular builders aren’t building so there’s no outage.

All of them (except buildppc) are now up to date on Fedora 20 updates and are running the 3.17.2 kernel.