Vox Pupuli - Behind every successful open source project is a vibrant community

# Vox Pupuli - Behind every successful open source project is a vibrant community

---

# Vox Pupuli - Behind every successful open source project is a vibrant community
## How to grow an online community to over 180 people managing over 250 projects

---

## $ whoami
<img src="bastelfreak2.jpg" alt="drawing" style="float: right;width: 400px;"/>

* Tim 'bastelfreak' Meusel

* Puppet Contributor since 2012

* Merging stuff on [Vox Pupuli](https://voxpupuli.org/) (Puppet Community) since 2015

* Vox Pupuli Project Management Committee member

* Living in Cologne, Germany

* Senior IT Automation Consultant at [betadots](https://betadots.de/)

<a href='https://betadots.de' alt='company website betadots'><img src="logo-1181x1181.png" alt="drawing" style="position: fixed; bottom: 12px; left: 20px; width: 100px;"/></a>
???

* who has seen this picture before because I reviewed/merged your pull request?

---

# What is Puppet?

???

* Who knows Puppet or worked with it?
---

]
<a href='https://betadots.de' alt='company website betadots'><img src="logo-1181x1181.png" alt="drawing" style="position: fixed; bottom: 12px; left: 20px; width: 100px;"/></a>
.footnote[[@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

---
count: false
.left-column[
## Puppet
### What?
]

* Brings its own declarative Language, Puppet DSL

???

* Alternatives like Salt/Ansible don't, they are mostly YAML
---
count: false
.left-column[
## Puppet
### What?
]

* Brings its own declarative Language, Puppet DSL

* Invented/Owned by Puppet Inc (acquired by Perforce)

???

---
count: false
.left-column[
## Puppet
### What?
]

* Brings its own declarative Language, Puppet DSL

* Invented/Owned by Puppet Inc (acquired by Perforce)

* Apache 2 license (with CLA)

???

---
.left-column[
## Puppet
### What?
### How?
]

.right-column[
example class to manage nginx:
```puppet
class nginx (
  Sensitive[String[1]] $content,
) {
  package { 'nginx':
    ensure => 'installed',
  }
  -> file { '/etc/nginx/nginx.conf':
    ensure => 'file',
    content => $content,
  }
  ~> service { 'nginx':
    ensure => 'running',
    enable => true,
  }
}
```

???

* example class of three resources. A resources describes one specific thing on a system

---
.left-column[
## Puppet
### What?
### How?
]

* glorified .tar.gz archive

* similar to a Ruby gem or Python Package

* Usually each Puppet Modules lives in its own Git(Hub) repository

* A module is focussed on managing a specific software component

???

* grouping resources based on software component makes modules exchangeable and composable!
---
class: center, middle, inverse

# Who uses open source software?

???

* easy question as a warm up
---
class: center, middle, inverse

# Who wanted to report a bug?

???

* Among this group, who wanted to report a bug or provide a patch?
---
class: center, middle, inverse

# Who uses dead open source software?

???

* While trying to report a bug, who noticed the project is maintained by a single person and dead since 3 years?

You might guess where this is going
---
class: center, middle, inverse

# What's this Vox Pupuli thingy?

???
Now a question I will try to answer for you, what is vox pupuli

* Who heard about it before?
---

???

Some stats. A group that:
* 150 Puppet modules in 2021. 179 *active* modules at the moment
* 43 Ruby Gems in 2021, 54 *active* gems at the moment
---

* And Rubygems!
]
<a href='https://betadots.de' alt='company website betadots'><img src="logo-1181x1181.png" alt="drawing" style="position: fixed; bottom: 12px; left: 20px; width: 100px;"/></a>
.footnote[[@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* 150 Puppet modules in 2021. 175 *active* modules at the moment
* 43 Ruby Gems in 2021, 54 *active* gems at the moment
---
count: false

* And Rubygems!

* Be helpful and polite to each other

]

<a href='https://betadots.de' alt='company website betadots'><img src="logo-1181x1181.png" alt="drawing" style="position: fixed; bottom: 12px; left: 20px; width: 100px;"/></a>
.footnote[[@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

* We've a code of conduct at least since 2016 and had a single violation

* A person wasn't polite to a few others

---

* And Rubygems!

* Be helpful and polite to each other

* Be open-minded to people and technology

]

???
* new people are always welcome and it's easy to participate
* usually interact via IRC or via GitHub issues and pull requests
* Puppet/Perforce has a slack community. Our slack room is bridged with our IRC channel
* technology: example conversation: Hey can I add gentoo support to a module? Or Arch Linux support?

* domain specific experts

* Any fellow Arch Linux friends here?
---

* And Rubygems!

* Be helpful and polite to each other

* Be open-minded to people and technology

* Provide a home for orphaned Puppet modules

]

???

---

* And Rubygems!

* Be helpful and polite to each other

* Be open-minded to people and technology

* Provide a home for orphaned Puppet modules

* Unite lonely module and tooling authors to a collective

]

???

---
count: false

* And Rubygems!

* Be helpful and polite to each other

* Be open-minded to people and technology

* Provide a home for orphaned Puppet modules

* Unite lonely module and tooling authors to a collective

* Ensure continued development for our code
]

???

* We want to work together on modules
* together we develop tooling to test our modules
---
.left-column[
## Vox Pupuli
### What?
### Who?
]

![img](contributors.svg)
]

???

* 2021: 161 members in the GitHub org with merge access
* now: 187

* every dot marks a presentation about or related to vox pupuli
---

*  Rebasing a feature branch against upstream master is hard
]

???

---

*  Rebasing a feature branch against upstream master is hard

* Many newcomers to test frameworks

* Interested in learing RuboCop/puppet-lint/rspec-puppet/serverspec
]

???

* Those are the tools we use in our CI
* talk about them later
* CI runs automatically
* CI does code annotations
* People want to fix/enhance their code because CI feedback is quick and nice
---

*  Rebasing a feature branch against upstream master is hard

* Many newcomers to test frameworks

* Interested in learing RuboCop/puppet-lint/rspec-puppet/serverspec

* SRE/DevOps/Platform/Systems Engineers in a small/tiny team

* Use Puppet at work and want to use public modules
]

???

* a team so tiny, often a single person

---

*  Rebasing a feature branch against upstream master is hard

* Many newcomers to test frameworks

* Interested in learing RuboCop/puppet-lint/rspec-puppet/serverspec

* SRE/DevOps/Platform/Systems Engineers in a small/tiny team

* Use Puppet at work and want to use public modules

* Consultants that contribute to one/a few modules for a period of time
]

???

* We provide letter of recognitions or kinda an employment reference

---

*  Rebasing a feature branch against upstream master is hard

* Many newcomers to test frameworks

* Interested in learing RuboCop/puppet-lint/rspec-puppet/serverspec

* SRE/DevOps/Platform/Systems Engineers in a small/tiny team

* Use Puppet at work and want to use public modules

* Consultants that contribute to one/a few modules for a period of time

* ~~Many administrators of legacy infrastructures~~

]

???

* people running outdated software/hardware is still a problem, but
* At least in the Infrastructure as Code bubble it's getting better
* Less requests to support RHEL 5/6 or ancient ruby versions

---

???

* Who of you maintains a open source project

---

* successful implies documentation

* More users → more bugreports and feature requests
]

???

---

* successful implies documentation

* More users → more bugreports and feature requests

* People loose interest, lack time, burnout
]

???

* if a single person maintains a project: the more users you have the more likely it is to burn out
* remember earlier when I asked about abandoned software?

---

* successful implies documentation

* More users → more bugreports and feature requests

* People loose interest, lack time, burnout

* Vox Pupuli wants to prevent this

* with a big group of collaborators and efficient tooling
]

???

* if a single person maintains a project: the more users you have the more likely it is to burn out
* remember earlier when I asked about abandoned software?

---

# How does Vox Pupuli work?

???

* now you might as yourself. this sounds amazing, how does it work? Glad you're asking

---
.left-column[
## Vox Pupuli
### What?
### Who?
### Why?
### How?
]

???

* who worked in a company with a similar complex org chart?
* Who liked it?
* Who prefers small/flat hierarchies?
---

???

* there is no real hierarchy
* Vox Pupuli is one big group
* we work with lazy consensus
* If a decision needs to be made and there is no majority for any of the options, PMC will choose
* people love to work in such an environment, it enables people

---

???

* what else do people love about vox pupuli? Our styleguide

* We are distributed around the world
* From the beginning, we were in different timezones and people with different backgrounds

---

* Write down every discussion about it, document it!
]

???

* Often it's possible to implement something in different ways
* Or to format the code in different ways
* we notice this during code reviews, we use PRs for every change/contribution
* if something is unclear, we talk about it and document it afterwards
---

* Write down every discussion about it, document it!

* It's okay to update a style guide or revert a decission
]

???

* This is a living document. new collaborators should read it. those people are awesome to spot (my) spelling issues, enhancements, stuff that doesnt make sense
* review it for each new puppet version
---

* Write down every discussion about it, document it!

* It's okay to update a style guide or revert a decission

* Our styleguide: [voxpupuli.org/docs/reviewing_pr](https://voxpupuli.org/docs/reviewing_pr/)
]

???

* If you've a development team within your company, I recommend following this pattern
* Write down your guidelines, publish them on the internet, interact with others
---

* Write down every discussion about it, document it!

* It's okay to update a style guide or revert a decission

* Our styleguide: [voxpupuli.org/docs/reviewing_pr](https://voxpupuli.org/docs/reviewing_pr/)

* Over time our guidelines got adopted by many companies and Puppet itself

* It's the defacto standard for the Puppet language
]

???
* code best practice and style guide
* Puppet/Perforce also has a styleguide, but it's lacking many details. Ours is an enhancement or addition
---

* Write down every discussion about it, document it!

* It's okay to update a style guide or revert a decission

* Our styleguide: [voxpupuli.org/docs/reviewing_pr](https://voxpupuli.org/docs/reviewing_pr/)

* Over time our guidelines got adopted by many companies and Puppet itself

* It's the defacto standard for the Puppet language

* A styleguide brings more value if you can enforce it
]

???

* Wouldnt it be nice if we could enforce this somehow? if we could verify if our code looks like the styleguide recommends it?

---

* Write down every discussion about it, document it!

* It's okay to update a style guide or revert a decission

* Our styleguide: [voxpupuli.org/docs/reviewing_pr](https://voxpupuli.org/docs/reviewing_pr/)

* Over time our guidelines got adopted by many companies and Puppet itself

* It's the defacto standard for the Puppet language

* A styleguide brings more value if you can enforce it

* puppet-lint, rubocop, metadata-json-lint
]

???

* linters!
* metadata-json-lint for json, rubocop for ruby files, puppet-lint for puppet DSL
* puppet-lint can be extended with plugins. one plugin for each specific entry in the styleguide

* this makes reviews easier. as a contributor, you don't need to wait for feedback from us. there are tools that can tell you what's wrong. and in most cases how to fix it (autofix). People love that.

---

* Provide feedback fast, reduce required review time
]

???

* I still fix bugs in other projects and notice that they don't have any tests or linting
---

* Provide feedback fast, reduce required review time

* Use code annotations!
]

???

* GitHub and GitLab support this. A linting/test tool can tell the CI system which line, which character violated a styleguide or broke a test
---

* Provide feedback fast, reduce required review time

* Use code annotations (RuboCop)!

![img](rubocop.png)
]

???

* example for rubocop. tells you the line, what kind of violation and a link to documentation
* turns out people love that!
* this enables people to work on their own most of the time, with a very short feedback loop

---

* Provide feedback fast, reduce required review time

* Use code annotations (RuboCop, puppet-lint)!

![img](puppetlint.png)
]

???

* we have the same feature for puppet-lint
* what does it say? missing docs?
* Remember when I earlier talked about documentation in a project?
* In the puppet ecosystem you can add some inline comments and a tool called puppet-strings can generate documentation
* We check during each CI run if the REFERENCE.md is up2date
* many languages support it to generate documentation from code+inline docs. if yours does it, use it!
* puppet forge will display the REFERENCE.md for each release
---

* Provide feedback fast, reduce required review time

* Use code annotations (RuboCop, puppet-lint, rspec-puppet)!

![img](rspec.png)
]

???

* rspec-puppet is used for unit testing
* note the operating system? keep that in mind?

---

* A CI system should provide feedback as soon as possible
]

???

* who likes to wait 20 minutes for CI to get feedback? Only people that drink a lot of coffee and need a refill

---

]

???

We implemented this in github actions.
* we start from quick checks to slower checks
* first validate the code. puppet code, ruby code, if the syntax is valid
* the linting and formatting. that's rubocop/puppet-lint mentioned earlier
* then we generate a CI matrix
---

* `jq .requirements[0] metadata.json`
```json
{
 "name": "puppet",
 "version_requirement": ">= 6.1.0 < 8.0.0"
}
```

]

???

* each puppet module contains a metadata.json
* it lists the supported puppet versions
* we get each major version and can generate CI jobs for each major version
* makes implementing support for newer versions easy. bump metadata.json and watch CI
* same pattern works in other languages, like python or ruby projects
* dont hardcode this in your CI config.
---

* `jq .requirements[0] metadata.json`
```json
{
 "name": "puppet",
 "version_requirement": ">= 6.1.0 < 8.0.0"
}
```

* `jq .operatingsystem_support[0] metadata.json`
```json
{
  "operatingsystem": "RedHat",
  "operatingsystemrelease": [
    "6",
    "7"
  ]
}
```
]

???

* We have a tool called beaker to spin up VMs/docker instances
* generate one job per OS version per puppet version
* remember earlier when we talked about being open to new technology? someone wanting to add support for a new operating system? add it to metadata.json and you get a new CI job
---

* It is documented! [github.com/voxpupuli/gha-puppet](https://github.com/voxpupuli/gha-puppet#puppet-github-actions)
]

???

As mentioned, documentation is important, the whole setup is FOSS and runs on GitHub actions, can be ported to gitlab
---

* It is documented! [github.com/voxpupuli/gha-puppet](https://github.com/voxpupuli/gha-puppet#puppet-github-actions)

* It's a shared worflow and others can use it!

```yaml
name: CI

on: pull_request

concurrency:
  group: ${{ github.ref_name }}
  cancel-in-progress: true

jobs:
  puppet:
    name: Puppet
    uses: voxpupuli/gha-puppet/.github/workflows/basic.yml@v1
```
]

???

just in case you develop modules on github, you can reuse our workflow
---

* It is documented! [github.com/voxpupuli/gha-puppet](https://github.com/voxpupuli/gha-puppet#puppet-github-actions)

* It's a shared worflow and others can use it!

* Everything can be run locally: [voxpupuli.org/docs/how_to_run_tests](https://voxpupuli.org/docs/how_to_run_tests/)
]

---
count: false

* It is documented! [github.com/voxpupuli/gha-puppet](https://github.com/voxpupuli/gha-puppet#puppet-github-actions)

* It's a shared worflow and others can use it!

* Everything can be run locally: [voxpupuli.org/docs/how_to_run_tests](https://voxpupuli.org/docs/how_to_run_tests/)

* Add a CONTRIBUTING.md to a module
]

???

Thats a github feature. when you open a PR and the file is present, github will tell you about it

---

* Rakefile, directory structure, CONTRIBUTING.md
]

---

* Rakefile, directory structure, CONTRIBUTING.md

* modulesync was developed by Puppet Inc, moved to Vox Pupuli

* It takes a config file + templates, throws it into every modules git repo

* Source Code: [github.com/voxpupuli/modulesync](https://github.com/voxpupuli/modulesync#modulesync)
]

???

* this is not puppet-specific, it can be used for every group of git repos with similar boilerplate code
* it can automatically create GitLab/GitHub/GitHub Enterprise PRs, or just push branches
* distributed as ruby gem

---

.right-column[
* Our config repo: [github.com/voxpupuli/modulesync_config](https://github.com/voxpupuli/modulesync_config#modulesync-configs)
![img](modulesync.png)
]

???
---

# The Conclusion

---
.left-column[
## Vox Pupuli
### Conclusion
]
.right-column[
* Document how your team works

* Try to develop publicly, even corporate code

* Document your style guide

* Document your projects

* Try to reduce boilerplate code

* Manage boilerplate code in a central place

]

---

* We have great people

* You need help with Puppet?

* Let us know, we help out!

* You have an orphaned module or know one?

* Ping us, migrate it to us

* You have domain specific knowledge?

* Ruby, Python, Rspec, Beaker, $software we automate

* Sponsor us through [GitHub sponsors](https://github.com/sponsors/voxpupuli) or [OpenCollective](https://opencollective.com/vox-pupuli/)
]

???
or need a consultant? want to talk about community or scaling?
---

* Mailinglist: voxpupuli@groups.io

* For feedback: `bastelfreak` on <a href="https://slack.puppet.com/">slack.puppet.com</a>/Libera.Chat IRC or [tim@bastelfreak.de](mailto:tim@bastelfreak.de)

* This talk and previous ones: <a href='https://github.com/bastelfreak/talks'>github.com/bastelfreak/talks</a>

### Thanks for your attention!
]
<a href='https://betadots.de' alt='company website betadots'><img src="logo-1181x1181.png" alt="drawing" style="position: fixed; bottom: 12px; left: 20px; width: 100px;"/></a>
.footnote[[@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

---

.left-column[
## Vox Pupuli
### Conclusion
### Link collection
]
.right-column[
* Our Website! [voxpupuli.org/docs/](https://voxpupuli.org/docs/)

* CI Setup: [github.com/voxpupuli/gha-puppet](https://github.com/voxpupuli/gha-puppet#puppet-github-actions)

* unit tests: [github.com/voxpupuli/voxpupuli-test](https://github.com/voxpupuli/voxpupuli-test#voxpupuli-test-gem)

* Acceptance tests: [github.com/voxpupuli/voxpupuli-acceptance](https://github.com/voxpupuli/voxpupuli-acceptance#voxpupuli-acceptance-gem)

* Release tooling: [github.com/voxpupuli/voxpupuli-release](https://github.com/voxpupuli/voxpupuli-release#vox-pupuli-release-gem)

* Our modulesync_config repo: [github.com/voxpupuli/modulesync_config](https://github.com/voxpupuli/modulesync_config#modulesync-configs)
]
<a href='https://betadots.de' alt='company website betadots'><img src="logo-1181x1181.png" alt="drawing" style="position: fixed; bottom: 12px; left: 20px; width: 100px;"/></a>
.footnote[[@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]

???

???
CI setup

---

.right-column[
<img src="car.jpg" style="width:85%;">
]
<a href='https://betadots.de' alt='company website betadots'><img src="logo-1181x1181.png" alt="drawing" style="position: fixed; bottom: 12px; left: 20px; width: 100px;"/></a>
.footnote[[@bastelsblog](https://twitter.com/bastelsblog) for [@voxpupuliorg](https://twitter.com/voxpupuliorg)]