At my new position at Plemi, I want to optimize the continuous integration flow. Last week, I had a pretty interesting conversation with Mathieu Nebra, we discussed about our same quest: best ROI through the development flow. I also had an awesome experience with Antoine Guiral (the man behind twoolr), we shared a “lean startup” friday night around a very interesting project.

In my previous article, I introduced a pragmatic Git flow. A first step in the flow focused on the local environment of the developer. I like to start this post with a personal, yet simple consideration : “the trust in your development team is reflected by your CI flow”.

It’s very reductive but the github flow is a perfect example. It’s composed of only one master stable branch and several feature branches, the production flow is deadly simple. However Github uses Github (dog fooding for the win) and this flow strategically relies on it own awesome feature called pull request. Sadly, I can’t afford to use github everywhere for many -good and bad- reasons. So I tried to set up a flow with a self-hosted git repository coupled to our continuous integration server

Overview of the development flow

1. A Developer works on their local git repository in a dedicated feature branch
2. He pushes often as he can (min 2 in a day) to the remote feature branch
3. The CI builds on every push, merging the trunk and his feature branch
4. When a feature is over, the developer merge it back to the trunk

The context and the tools

For the new comer, a quick summary of the tools used in this flow

• Source code management with git
• Branch management is based on the git flow template
• Continuous integration rely on Jenkins
• Code quality rely on Sonar
• I work with PHP, yet the flow can be used in other langage

The flow starts on the developer side

I already wrote on that topic in my previous article. In short, the developer works into a feature branch named against this pattern: “feature.ticketNumber_description”. He writes code without worrying about the SCM. Then, he dedicates some time to nicely split his work into several independent commits. If it’s needed, he can rebase his local branch. Finally he pushes the commit pack.

On the continuous integration plateform side

We have two jobs for each repository: one for the develop branch and another one for all feature branches. The feature job is configured to build every branch with the pattern “feature.*”. Jenkins is notified after each push by a git post-receive hook.

Tip: this hook doesn’t start a build, it trigger Jenkins for a SCM polling. In this way Jenkins will start to build only if there is relevant change(s) in the repository. Read the official documentation: “Build by source changes”

Continuous merge

I configured Jenkins to merge locally the lastest pushed branch with the develop branch. This allow an early and cheap detection about merge conflicts.

If Jenkins can’t merge automatically, it fails the build and the developer is already aware that he needs to manually merge back the develop branch into his feature branch. He can’t delay this conflict resolution because it blocks the CI process: checkstyle, unit tests and coverage won’t be available to him until he resolves the merge.

This continuous merge process has the huge advantage to avoid the nightmare of a messy merge while your developer responsible of a shitty conflict is unreachable for an undefined period.”

Don’t trust Jenkins too much

I don’t use “push after a successful build” option provided by Jenkins. Basically, This allows you to automate the integration process: if a build is stable it will push the new modification to a remote branch. Yet, it don’t fit our needs since I ask the developer to push everyday. Many features need more than one day to be achieve and I won’t let Jenkins push some kind of work in progress.

Don’t trust a single developer

I don’t rely too much on Jenkins and this is also related to my first quote: I work with a pretty nice team but “I don’t trust a single developer” (and even myself!) about building the perfect git log (i.e. the perfect code) in a one shot. We are all new in the company and we need to practice a little bit more with the processes or the new coding standards for example.

Trust your team

Moreover, when I say: “I don’t trust a single developer”, it’s a way to highlight the code review phase. When a feature is closed, it must be fully reviewed and discussed by a technical leader before being merged into the develop branch. This has 2 majors consequences: First I can’t let Jenkins automatically merge into the develop branch; second, the technical leader is free to make a final rebase of the feature branch before the merge. This ensures a stable and clean develop branch.

Finally, we have to distinct the 2 usages of develop. Jenkins use this branch locally as an integration branch for each feature while the remote branch reflects the lastest stable version ready to be checked by Q&A and then deployed.

What’s next ?

• Everything go to the production.
• You launch some stress test on a staging server and then deploy to prod
• You notify your Q&A and expect a final feedback

At our office Q&A check is “asynchronous”, it can be delayed from 1 to 3 days. That’s why I’m still using the release branch from the git flow template. I freeze develop into release and wait for a Q&A feedback. If it’s a green light, I’ll deploy the release branch.

I customized the git flow for my need, we don’t use the “release” concept because we deploy ASAP each feature. This flow fits perfectly to the AGILE methods. You collect technical metrics sooner and it’s simpler to detect and to fix potential problems on a small iteration. Your project manager/product owner collects user feedbacks sooner, easing the business decisions for the next sprints/features

I mentioned 2 jobs for each repository, one job is limited to the develop branch and serves the code quality purpose, metrics from this job are exported to Sonar