Vancouver in May 2018
I attended Openstack Summit
in Vancouver, BC, Canada on May 21st - 23rd. Mostly my interest was in
attending the colocated Opendev event that was
focused on CI/CD.
As a contributer to the CentOS and
Fedora projects and as someone who maintains CI
systems for my job, I was interested in
hearing about how large scale projects like Openstack coordinate test
strategies, and communicate results across all of the different subprojects.
OpenStack Summit Day 1 (May 21st)
I spent most of the first day wandering around Openstack Summit, meeting some
of the infra team members and attending sessions. Some highlights from Summit
overall, since I spend the rest of the week at OpenDev:
made its debut as a toplevel project in the Openstack Foundation
There was a rather controversial
Keynote sponsored by Canonical, that sparked some discussion about the relationship amongst vendors at project-focused events.
OpenDev Day 2 (May 22nd)
Using CI in an Ecosystem link
This was an open discussion about how to do tests on components of a "release"
but which are themselves individual projects. This is a common pattern for
Linux distributions that take in software projects from 1000s of sources and
package them individually for inclusion in the distribution. Openstack has
similar problems because of its dependence on upstream python projects, that
have separate lifecycles and goals.
We found that lots of projects are tackling this in a similar pattern:
- Generate a graph of dependencies
- Run as many tests as you can from the component projects whenever something
- Have integration tests that run separately
- Promote the components from testing -> known good
Cross-Community CI/CD link
This session was mostly focused on helping individual projects under the
Openstack foundation talk to each other. Openstack, and the peripheral projects
didn't have a good forum to talk to one another, and share best practices
between folks who are on Openstack's Zuul, and folks using Jenkins-based
solutions from the Linux Foundation and other providers.
We decided to collaborate using the OpenCI "Community of Practice".
Ansible Community Meetup link
We began the Ansible community meetup with a demo from dmsimard about
ARA, which is an ansible plugin that
presents a friendly web interface describing ansible runs. ARA is something
I've been meaning to investigate ARA for my own projects and seeing a live demo
makes it that much easier.
We spent the last half of the meetup talking about scaling up, many of the
participants were managing 10s of thousands of of machines and looking for
management strategies. The most common way to deal with this was to use
ansible-pull, but that requires careful management of secrets.
Day 2 Summary
If I had to summarize day 1 in a sentence it would be this: Openstack acts like
a Linux Distribution, we can learn from one another.
Participants in each of the discussion sessions, and in the hallway track, were
primarily interested in testing and delivering Openstack, but they all used
experience taken from their roles as participants in the distribution
ecosystems (Debian, Ubuntu, Fedora, CentOS, Mageia, etc.) as lessons.
OpenDev Day 3 (May 23rd)
OpenCI: What, Why, How? link
OpenCI is a non-governed, non-product community of people working generically
on the CI/CD problem for integration projects. Key members come from the NFV,
and Openstack communities and are looking for a forum to discuss best practices
and collaborate/agree on standard methods of communication between projects
that care about each-others' CI results.
Operationally the folks in the room were looking at methods and a specification
for triggering CI runs in a separate infrastructure based on their
dependencies. For example, if the OpenDaylight project wants to run its tests
once a 'good' test of a component in OPNFV completes, how do we do that?
One solution that we brought from the distro space is to wrap an existing
and deliver it via Fedmsg.
Fedmsg is used in the Fedora and Debian projects, and it would make sense for
openci to occupy the io.openci namespace so that others can subscribe to topics
from various infrastructures they're interested in.
We talked about strategies to include and rebroadcast event streams from
existing infrastructures as well. For example, we have a Fedmsg structure
emitted from the Fedora CI Pipelines; it could be interesting to also
rebroadcast those events using Fedmsg so projects that rely on Fedora or CentOS
as distributions could consume those events in their own format.
Productizing Open Source Through CI link
This was a good discussion about strategies that teams (even outside openstack,
and OSS projects) use to test products based on information from the people
that consume the product. I'll admit the subject matter was interesting to
listen in on, but this was not part of my focus for attending the event.
Day 3 Summary
1 Sentence: Because the Openstack family and distributions are so similar (see
day 2's summary), we can collaborate on tools too!
One action I'd like to take from day 3 is to continue participating in the
OpenCI discussions on the mailing lists. It seemed that Fedmsg was interesting
as a touchpoint for a group of engineers who are desperate for a single event
stream that they can act on in their own infrastructures. Luckily there aren't
many changes/updates/features needed in Fedmsg to get started quickly
My "To Watch" List
It seems like no matter what conference you go to, there's always a choice you
have to make between 2 sessions scheduled at the same time. Luckily we have the
Internet for this, here's what I would have gone to if I was able to clone