State of Fedora Infrastructure 2016

Kevin Fenzi and Pierre-Yves Chibon

CC-BY-SA-4.0

We are going to discuss things that have happened in Fedora Infrastructure since the last flock and then look forward to the next one, from both a Dev and Ops perspective.

Since last Flock

Fedora Infrastructure Private Cloud

• Added a PPC64 compute node (thanks IBM!)
• Added some storage
• Lot of things going on in Copr
• Jenkins.fedorainfracloud.org

Since last Flock

Ansible move

• Completed our move to ansible management
• Have been helping test upstream releases
• Fixing issues and reducing check reports
• Workshop friday morning at 10am, come help us!

Since last Flock

RHEL7 migration

• Moved a large majority of our RHEL6 to RHEL7
• Breakdown is currently:

  • RHEL7.2: 266
  • Fedora24: 144
  • RHEL6.8: 24
  • Fedora23: 74
  • Fedora22: 17

• ~525 ansible instances
• bodhi2 and Hyperkitty/Mailman3 helped retire older OSes

Since last Flock

• Brought secondary arches into main infrastructure
• Using the same ansible playbooks and installed the same way

Looking forward

Fedora Infrastructure Private Cloud

• Move to RHOSP8 or 9
• Add aarch64 and armv7 instances
• Ipsilon auth / projects for all contributors
• Open shift instance

Looking forward

Moving fedorahosted.org

• Hope to sunset next year
• pagure trac issue migrator almost done
• File pagure issues for missing functionality

Looking forward

• Containters in production
• mirrorlists first "app"
• Leverage container build system
• Others to come

Looking forward

• Database replication / HA
• Remove Single point of failure
• Allow update cycles with no/minimal outages

Looking forward

• "Secondary" arches move into primary koji
• Working with releng, proposal is in to fesco
• Still needs more discussion
• aarch64 likely the first one in

Looking forward

Monitoring config automation

• currently manual/slipshod
• leverage ansible variables
• get close to 100% coverage

2015-2016 in review

• Hyperkitty
• mdapi
• fedora-packages
• mote
• MirrorManager2
• basset
• pagure

HyperKitty

• New archive interface for the new Mailman 3
• Leaded by Aurélien Bompard (Fedora Engineering)
• Deployed at https://lists.fedoraproject.org/archives/
• Bridges forums and mailing lists
• Mailman2 retired
• Go see the talk!

mdapi

• Meta-Data API
• Query the metadata from the yum/dnf repositories

fedora-packages

• Rewrite
• Ported to mdapi as source of info
• More stable
• fedmsg-based updater

Møte

• New interface for the (irc) meeting logs
• JS Based
• Lead by a community member! (Cydrobolt)
• Deployed at https://meetbot.fedoraproject.org

MirrorManager2

• Re-write of our old mirrormanager with a newer framework
• Integrates stats about the different mirrors
• Integrates map of the distribution of the mirrors
• Integrates stats of the propagation of the updates among the mirrors
• Deployed at: https://admin.fedoraproject.org/mirrormanager
• Go see the talk!

Basset

• anti spam application
• registration process changes
• account statuses
• Go see the talk!

Pagure

• Python-based, home-made git forge
• Large face-lift (credits to Ryan Lerch)
• Deployed at https://pagure.io
• Go see the talk!

Coming soon

• Ipsilon – OpenID Connect
• FAS3
• OSBS / Pkgdb2 – Namespacing
• Fedora-hubs
• Pagure
• Pkgs.fedoraproject.org
• Fedora-quick-mirror
• FMN
• Wiki upgrade

Ipsilon

OpenID Connect

• OAuth2 on steroids
• Will allow cross-app auth
• Will allow easier CLI integration

FAS3

• New UI
• New code-base

• New framework
• Easier to extend/maintain
• Easier to customize for 3rd party

• Support token-based auth
• More modern OS, will allow 5 more RHEL6 retirements

OSBS / Pkgdb2

• Namespacing
• Required to allow building other artifacts than rpms

• OSBS layered image build system
• Modules

Fedora-hubs

• Online dashboard
• Easier on-boarding for new-comers
• Hackfest Friday Aug 5th, 13h30

• Room: OpenQA / Rembrandt

Pagure

• Private repos
• Mirroring code to external repo (Github, gitlab, fedorahosted…)
• Pagure-importer

• Supports github and fedorahosted (trac)

• Mailing lists for projects

pkgs.fp.o

• pagure @ pkgs.fp.o

• Fork
• Pull-Request
• No tickets
• ACLs remain in pkgdb
• Hooks

• Deployed at https://pkgs.stg.fedoraproject.org/pagure/

quick-fedora-mmirror

• Better way to mirror
• easier way to report to mirrormanager

FMN

• New backend architecture
• at least 3x faster processing messages

Wiki upgrade

• Will use OpenID
• postgresql in the background
• Will allow us to retire RHEL6 instances
• better mobile device support?

Questions?