This week in Fedora Infrastructure
A bit late this week, but was a bit too busy to get this post out the door until now. 😉
So, this last week was a busy one. I was out at our main datacenter. We have some excellent on-site folks that do things for us as needed there, but we still try and get out there a few times a year to do things that need scheduled timing or someone who knows our setup to complete. I was out monday, tuesday and wed this time. I got a number of things done out there:
- Got everything accounted for and ready to label things that were missing labels.
- We pulled 4 old decommissioned servers out and freed up their space/power/net/etc.
- I got memory added to 13 servers. This should allow us to increase density of virtual guests some. Also, I grew the memory on our main updates pushing machine, so we can hopefully have it generate deltarpms for much larger packages (right now there’s a limit we had in place due to memory.)
- Helped the gnome.org folks out some by inventorying their rack and getting stuff labeled there. We also hopefully we will get them a new switch sometime soon so they can better organize their network and get to their management interfaces.
- Worked on bringing our new blade server on line. We were missing a fabric switch, but it seems to have been located just after I left, so hopefully it will be on line soon. This is going to replace our existing bladecenter that is builders.
- Checked all machines for attention lights. This let us know about a drive that failed that we forgot about.
- Lots of other misc running around and checking things.
The mass rebuild started friday afternoon, and looks to be done now. It should land probably in rawhide tomorrow (monday). This time the rebuild was much faster. There’s a variety of reasons that could be the case: We rebuilt all our buildvm’s using virtio (they were not for the f19 mass rebuild), we now have 48 arm builders which allows the mass rebuild to spread out src.rpm building, noarch builds and tagging to a bunch more machines, and I gave our koji db a bunch more memory now that we had it available (so it could keep up better as well). In any case 2 days for a mass rebuild is great! There’s a number of failed builds this time, so do go and clean them up as soon as you can.
Next week is flock! Looking forward to it.