Rawhide notes from the trail, the 2016-12-28 edition

by on 2016/12/28 at 3:24 pm
Posted In: fedora, linux, rawhide

Just a few rawhide notes to end out the year:

  • Python 3.6 rebuild completed (mostly) and was merged back into rawhide. There were a number of things that needed cleanup, but we seem to have gotten them all handled in a few days. Next time we should really allow a day or two before we merge the side tag in and fix up things that are important for composes before we merge into rawhide.
  • Astute folks may have noticed: Builds in koji now will complete on all arches, even if one fails. In the past if one arch failed, the rest were canceled and the build came back as failed right away. Now all the arches will complete before the failure comes back. This allows you to see not only the problem that failed the build, but any other problems it would have hit on other arches as well. This should be of help especially to larger builds.
  • There is still a bit of fallout from the rawhide always signed changes we need to fix up in the new year: chain-builds no longer work, the rawhide fedora-release needs to be modified to always expect repos are signed, and we need to discuss what sort of testing we want to add in now that we have a handy place for adding it.

Have a smooth ride the rest of the year, no matter where your trail takes you.

Comments Off on Rawhide notes from the trail, the 2016-12-28 edition

2016 Holiday hacking (so far)

by on 2016/12/27 at 6:10 pm
Posted In: fedora, linux

I hope everyone is having a lovely holiday season and doing the things they find fun or recharging.

I’ve finally gotten to a few hacking projects that I thought might be interesting to share with others:

  • I finally cleaned up my local ansible repo some and set it up as a pagure project: ( https://pagure.io/scrye-ansible/ ). It still needs a bunch more cleanup (these playbooks were some of the first I ever wrote, so many years ago), but they should help anyone who wants to setup test machines like I have here or provide some more ansible examples.
  • For 4+ years I have had a CentOS6 digitial ocean droplet I have been using as a secondary nameserver for my domains. It actually predates IPV6 support at digitial ocean and had a manually setup he.net ipv6 tunnel on it. I finally looked at the ansible digitial_ocean module and setup a new shiny Fedora 25 droplet, created entirely via ansible as well as putting my secondary nsd configuration in it. This is all now in the above scrye-ansible repo. I may well be using more digital ocean droplets now that I see how easy it is via ansible.
  • I looked around for a new rom for my phone (nexus 6) and didn’t find much that filled me with joy. So, for now I just moved to the latest nightly CM version and will see if this new fork ( http://lineageos.org/ ) pans out. The other roms out there are really unprofessional looking: (homepage is a xda forum post, binaries have no checksums and download from bigfiles.com or the like, last commits to source repos was years ago, etc).
  • One big issue I had had with my phone of late is it quitting when it has 30-40% battery left and suddenly saying it has 0%. I found that you can reset the battery calibration (boot with power, down volume, select ‘bootloader logs’, press power for 10+ seconds until the phone reboots). That seems to have gotten things a good deal happier now, but time will tell. Unfortunately, the battery on the nexus 6 is not very easily replaceable.
  • I got all my home machines rebooted into the latest update kernel. I sometimes delay on my firewall/gateway machine because it means a short outage, but everything is updated now.
  • Pulled all the old backups off my phone and freed up a fair pile of space. There’s no reason to store them there, I can copy them back if I need to restore something from them (which I never really do after the initial restores).
  • Setup usbguard on my home machines. The chances of someone plugging in a evil usb device to them is small, but no harm in setting it up and denying all the non known to me devices. I’ve been using it on my laptop for a long while now.
  • Updated my drop-keys (gnome) and xscreensaver-keys (Xfce) scripts and pushed them to the pagure repo: https://pagure.io/scrye-ansible/c/37610c85918125d92bedff6673b58310605e90dd?branch=master basically these scripts can watch for the screen to lock and drop all your unlocked ssh keys. For gnome there’s an upstream bug that might make this land in upstream gnome-keyring: https://bugzilla.gnome.org/show_bug.cgi?id=735149

In the next few days I hope to catch up on some packaging backlog and perhaps get some armv7 installs done.

I hope everyone is having an enjoyable, relaxing holiday!

Comments Off on 2016 Holiday hacking (so far)

Rawhide notes from the trail, the 2016-12-18 edition

by on 2016/12/18 at 1:05 pm
Posted In: fedora, linux, rawhide

Hello from the Rawhide trail.

With the recent Flag day (on Dec 12th), we switched all rawhide builds to allow us to sign (and hopefully eventually test) all packages. Here’s how it works:

  • Your rawhide build used to just be tagged into the f26 (currently rawhide) tag. Now, it tags into the f26-pending tag instead.
  • The autosigner sees the build, signs it and moves it to the f26 tag for the next rawhide compose.

Unfortunately, there is a backlog of packages we needed to sign, many from the ppc{le} bringup, so thats why there hasn’t been any rawhide composes the last few days. The one today should be out later today however, and we should be back on track from there.

So currently we are just signing things at the $release-pending tag, but we would like to try and start doing some automated QA there at some point. Nothing that will hold up builds for too long, but something that will catch obviously broken builds from landing. Now that we have everything otherwise in place we can start figuring out what we want to run there.

Also, coming soon to rawide will be the first rebuild of python packages for the upcoming python 3.6. Hopefully that will all be smooth sailing, but a number of package updates will land for that soon.

Comments Off on Rawhide notes from the trail, the 2016-12-18 edition

Rawhide notes from the trail, the 2016-12-03 edition

by on 2016/12/03 at 2:26 pm
Posted In: fedora, linux, rawhide

Well, it’s been a while but I thought I would share news from rawhide both from the previous month or so and upcoming.

With the release of Fedora 25 a number of developers and testers are switching back to rawhide in the run up to the Fedora 26 branching event. Welcome back to the trail!

In the last month there’s been various small rawhide issues:

  • openssl 1.1.0c came out and had a number of issues. In particular it broke authentication to koji, so any package maintainers had to downgrade back. It also broke a bunch of tests in python, which has delayed the landing of python 3.6 in rawhide. There’s a patched version of 1.1.0c that landed last week, so everyone should be ok to update openssl again.
  • The latest systemd version landed, but then had to be untagged again because it broke a number of parts of compose. Hopefully those will get worked through and fixed.
  • More recently (The last week) a kf5/qt5 update is in progress and isn’t fully finished. Rawhide users will notice a number of packages dnf says it’s skipping due to broken dependencies. This should get fixed up soon, so just keep skipping those items.
  • Finally, the last few days we have run into an odd problem that looks like it might be related to RHEL7.3 squid (which is whats on kojipkgs.fedoraproject.org and used for package downloads for builds). Sometimes packages just don’t seem to download and it breaks the compose, but testing later they work fine. We downgraded squid back to the 7.2 version for now and composes seem to be working again.

Finally in exciting upcoming news: There’s a “flag day” on december 12th (2016-12-12) for a bunch of changes that release engineering has been wanting to make for a long time. One of these changes will finally allow us to do the last thing we need to do to get a 100% signed and gated rawhide. There will be a change in fedpkg (as well as some koji server changes) that will make rawhide builds land in a ‘f26-pending’ tag instead of just going directly to ‘f26’. Then, our autosign process will sign them and retag them to f26. We also can add in automated QA here to decide if a build is ready to be tagged in or not. I hope next year that we can add a bunch of tests here and make rawhide more and more stable.

Comments Off on Rawhide notes from the trail, the 2016-12-03 edition

Security Score Card For Fedora Infrastructure

by on 2016/10/24 at 11:41 am
Posted In: fedora, linux

Josh is asking folks to send him their security score card via twitter. Since I’ve been trying to blog more and like pontificating, I thought I would respond here in a blog post. 😉

There’s 4 parts to the scorecard:

  1. Number of staff
  2. Number of “systems”
  3. Lines of code
  4. Number of security people

For Fedora Infrastructure, some of these are pretty hard to answer, but here’s some attempts:

  1. Fedora Infrastructure is a Open organization. People who show up and start doing things are granted more and more permissions based on their merit. Sometimes people drift away to other things, sometimes new people show up. There’s some people employed by Fedora’s primary sponsor Red Hat, specifically to work on Fedora. Those account for 3.5 sysadmins, 5 applications developers, 2 release engineers, and 2 design folks. Specific areas will have potentially lots more community folks working on them. So, answer: 13-130?
  2. This one is easier to quantify. We have (almost) everything in ansible, so right now our ansible inventory + some misc non ansible hosts is around 616 hosts.
  3. This is another one thats difficult. We have a lot of applications (see https://apps.fedoraproject.org/) Some of them are just upstream projects we have instances of (mediawiki, askbot, etc). Others are things where we are primary developers on (fedocal, pagure, etc). It would be a fun project to look at all these and count up lines of code. Answer: dunno. ;(
  4. If this is full time security people working only on security issues, then 0. We do have a excellent security office in Patrick who is super smart and good at auditing and looking for issues before they bite us, but he’s not doing that full time. Others of the sysadmin teams do security updates and monitoring lists/errata and watching logs for out of the ordinary behavior, but thats also not full time. So, answer: 0 or 1 0r 3?

So, from this I think it would be nice to have a better idea of our applications (not lines of code), but just where to keep track of things better and who knows that application. It would be awesome to get some full time security folks, but I am not sure that will be in the cards.

I’d like to thank Josh for bringing up the discussion… it’s an interesting one for sure.

Comments Off on Security Score Card For Fedora Infrastructure