Automating the Vox Pupuli Yak Shaving

# Automating the Vox Pupuli Yak Shaving

## Automate all the yak shaving!

???

---

## $ whoami
<img src="bastelfreak2.jpg" alt="drawing" style="float: right;width: 400px;"/>

* Tim 'bastelfreak' Meusel

* Puppet Contributor since 2012

* Merging stuff on [Vox Pupuli](https://voxpupuli.org/)  (Puppet Community) since 2015

* Vox Pupuli Project Management Committee member

???

---

## $ whoami
<img src="robert.jpg" alt="drawing" style="float: right;width: 400px;"/>

* Robert 'flipez' Müller

* Ruby Developer since 2013

* Builds clouds at Hetzner

* Also builds Rails apps and tunes football tables

???

---

# What's this Vox Pupuli thingy?

???

* lets have a look on the daily tasks a maintainer can do to keep modules in shape

---

![img](contributors.svg)
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* based on github org
* who of you is part of that group?
* who uses vox pupuli modules?
* who doesn't know what vox is?
---
class: center, middle, inverse

# The path of a Open Source Maintainer...

## What do we do?

???

* lets have a look on the daily tasks a maintainer can do to keep modules in shape

---

???

* lets start with yak shaving
---

![imagefoo](stdlib7.png)

]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

Puppetlabs releases a new major version of stdlib
---

![imagefoo](deps.png)

]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* This leads to an interesting challenge
* All puppet modules have an upper version boundary
* At least in acceptance tests those boundaries are enforced, we need to update it
* vox pupuli has 105 modules that depend on stdlib
---

![imagefoo](puppet.png)

]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* of course the same applies for the puppet version
* Dear puppet: a headsup info for a new release would be dope

---

![imagefoo](rhel.png)

]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* and it applies for all the operating systems

---

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* We've a basic set of labels that are identical across all modules
* sometimes we add / remove labels
* sometimes we get repositories migrated to us

---

* CONTRIBUTING.md, rspec configuration

* Many files are very similar

* .travis.yml, Gemfile, Rakefile
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

* modulesync was initially developed by Puppet Inc.

* Now maintained by Vox Pupuli

* a template directory + configuration file for every module
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* We've currently 124 modules on the forge
* we need to release modules on a regular basis

---
class: center, middle, inverse

# The path of a Open Source Maintainer...

## What do we actually do?

???

---

* We have many pull requests

![img](prs.png)

* [All open PRs](https://github.com/pulls?q=is%3Aopen+is%3Apr+user%3Avoxpupuli+archived%3Afalse+sort%3Acreated-desc)

* [All open PRs with green CI](https://github.com/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr+user%3Avoxpupuli+archived%3Afalse+sort%3Acreated-desc+status%3Asuccess)

* [All open PRs with green CI and no merge conflicts](https://github.com/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr+user%3Avoxpupuli+archived%3Afalse+status%3Asuccess+sort%3Aupdated-desc+-label%3Amerge-conflicts)
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* `with no merge conflicts` is handled by our labels
* github search doesn't allow
---

* New feature? `enhancement` label
* Bug / Bugfix? `bug` label
* Breaking change? `backwards-incompatible` label
* Additional work required? `needs-work` label

]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

* labels like `needs-feedback`, `needs-work`, `tests-fail`, `needs-docs`, `beginner-friendly`, `needs-rebase` help us to sort pull requests and issues by grouping them

Changelog:

* labels like `bug`, `enhancement`, `backwards-incompatible` help us to generate a CHANGELOG.md

]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* show the CHANGELOG.md
* or https://github.com/voxpupuli/puppet-zabbix/blob/master/CHANGELOG.md
---

* Otherwise GitHub fails to account the contribution properly

![img](email.png)
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

.right-column[
* Ensure that new templates are always [epp](https://puppet.com/docs/puppet/latest/lang_template_epp.html), not [erb](https://puppet.com/docs/puppet/latest/lang_template_erb.html)

* Epp templates have proper scopes and datatypes

![img](epp.png)
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

# Let us solve all our problems!

???

---

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* explain VPT means Vox Pupuli Tasks
---

.right-column[
<img src="getallthediffs.png" alt="drawing" style="float: right;width: 130px; margin-right: 50px"/>

### "There is this script. Do you think we can improve it?"

* Checks supported OS ranges
* Checks supported Puppet range

* Sort repos into categories based on
  * managed modules files
  * voxpupuli/plumbing repo
  * forge releases
  * information in files in every repository
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???
---

.right-column[
<img src="getallthediffs.png" alt="drawing" style="float: right;width: 130px; margin-right: 50px"/>

### "There is this script. Do you think we can improve it?"

* Checks supported OS ranges
* Checks supported Puppet range

* Sort repos into categories based on
  * managed modules files
  * voxpupuli/plumbing repo
  * forge releases
  * information in files in every repository

## Uhm, I guess
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???
---

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

* Refreshed every 6 hours
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

* Refreshed every 6 hours
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

# What else did we do?

???

---

class: center, middle, inverse
<img src="csaswh.jpeg" alt="drawing" style="float: center;width: 400px"/>

???
---
.left-column[
## VPT
### Automate daily tasks
### Architecture
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???
---

```ruby
case type
    when 'pull_request'
      # Ignore events which are triggered by the bot
      return if payload.dig('sender', 'id').to_i == 53_702_691

if Repository.notably? payload['repository']['name']
        @processor = GithubEvent::PullRequest.new(payload)
      end
    when *known_but_ignoreable_events
      true
    else
      Raven.capture_message("Unknown Hook Received: #{type}", extra: payload)
    end
```
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

```ruby
def validate(saved_changes)
  # Don't run through the validaten, if only the eligable_for_comment attribute got updated
  return if saved_changes.keys.sort == %w[eligable_for_comment].sort && !mergeable.nil?

# if the pull request is now closed, dont attach/remove labels/comments
  return if closed?

label = Label.needs_rebase
  if mergeable == true
    ensure_label_is_detached(label)
  elsif mergeable == false
    repository.ensure_label_exists(label)

##
    # We only add a comment if we addded a label. If the label already is present
    # we also already added the comment. So no need for a new one.
    add_comment(I18n.t('c.needs_rebase', author: author)) if ensure_label_is_attached(label)
  elsif mergeable.nil?
    UpdateMergeableWorker.perform_in(1.minute.from_now,
                                     repository.name,
                                     number,
                                     id,
                                     saved_changes)
  end
end
```
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---
.left-column[
## VPT
### Automate daily tasks
### Architecture
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---
.left-column[
## VPT
### Automate daily tasks
### Architecture
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---
.left-column[
## VPT
### Automate daily tasks
### Architecture
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---
.left-column[
## VPT
### Automate daily tasks
### Architecture
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---
.left-column[
## VPT
### Automate daily tasks
### Architecture
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

# What else can we do?

## What do you want us to do? What are your usecases?

???

---
.left-column[
## VPT
### Automate daily tasks
### Architecture
### Lookout
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* All examples we showed earlier could be automated

---

* Implement GitLab API support
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* We want that other communities and organisations use this as well
* many of them use GitLab

---

* Implement GitLab API support

* Help companies that want to host this on premise for internal repositories
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---

* We've great tooling to scale module handling

* GitHub/GitLab API enables us to automate even more and to focus on code reviewing

* Let us know which parts in the communities you would like to see automated

* Checkout the current dashboard at [voxpupu.li](https://voxpupu.li) or our docs at [github.com/voxpupuli/vox-pupuli-tasks](https://github.com/voxpupuli/vox-pupuli-tasks#vox-pupuli-tasks---the-webapp-for-community-management)

### Thanks for your attention!
]

.footnote[[@Was1NicerDude](https://twitter.com/Was1NicerDude) & [@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* We've some open issues with collected ideas. We need feedback!