Practical Rules for Software Development — Part 1
I was planning to write a series of blog posts of my experiences in the design principles of software development. I see that even senior developers do not follow these principles. I will avoid design terms and explain how to apply these in practice. These are not related to my current or previous employer but my general experience over a general period.
Keep It Simple Stupid. Aka KISS principle.
Keep your application as simple as possible. If your developers start to complain about “Too many moving parts”, you should stop what you are doing and think about how you can simplify it. Split your application into simple parts that can be completed by one developer in a maximum of two weeks. Two weeks is generally accepted as one sprint in Scrum-based development.
Every service should be simple enough to be re-written in two weeks. You are going to re-write that service multiple times anyway.
Split your work into services and libraries.
Services should have business logic, one git repo, one deployable Docker image, a README file explaining how to deploy, and use, a CI/CD pipeline connected to it. CD may be excluded if it is a product and deployed externally. Currently, the best practice is to loosely couple CD and CI.
CI stands for Continuous Integration, testing, and merging of code.
CD stands for Continuous Deployment, deployment of the service automatically after tests.
Libraries should have general logic, helper methods, loggers, any reusable logic. You should split libraries logically into separate repositories, they should have a README explaining how to use them, a CI pipeline, a CD pipeline if the library will be pushed to an artifact repository.
Libraries are more long term investments. I usually use my free time to write libraries that I may need in the future. It is a good practice to encourage developers to use some of their development time to write libraries and tooling. Usually, Scrum masters and product managers ignore this since they are not directly bringing new features, but it is needed to decrease future work. Having libraries also allows teams to predict future work better.
Services should be composed of libraries. A service is a glue between libraries. If your service gets too complicated, move complicated parts into separate well tested libraries.
Use semantic versioning everywhere.
This is very important, set this up on day one. I usually follow the “v1.2.3” format. I don’t like to use build and pre-release tags because you can not follow iteration. It makes harder to compare the iteration and creates a feeling of stagnation. I also use the “v” prefix since there can be other non-versioned tags like “milestone-1”, “release-1”, “MVP”.
This version should be your git tag, docker tag, and any tag you use in your artifact repository. Automatically incrementing version tags using special tags in your commit message has become a trend. Many people still do this by hand. Here is the shell script, I wrote in my previous job:
Trunk Based Development is your friend.
Split your work into smaller pieces that can be merged into the master branch without making any compile or build failures. It can be just one method or new constant values. It is hard to review large commits so if you can split your feature into smaller commits it is better. Large commits, usually pass without proper review and testing so bugs go by. It is hard to pinpoint the time which changes caused a bug that you are observing.
For more explanation:
Some product managers do not allow pull-requests/merges that are not directly bringing a feature, which makes trunk-based development impossible. They are simply wrong.
In larger companies, we used the long-running feature branches like“release-24”, “release-for-acme”. Merging them back to master was a big struggle. If you can avoid it, avoid it.
This post is going too long so I will finish with some flows as a summary:
Pull-request workflow:
Pull-request workflow works only if you have good tests and reviews are in place. Assuming you have 5–9 people teams:
Reviews:
There should be one code owner for the part of the code that is being changed. You should have at least 2 reviewers and one of them should be code owner. Gerrit has a voting based coding review system. Github doesn’t have this but if your team is small enough you can handle this by common sense. Github has a setting to define the code owner and there are bots to check the number of reviewers. Having 2 reviewers means, your developers need to spend 30% of development time on reviewing. Some of the product managers and the scrum masters plan sprints ignoring code review duration and developers end up overestimating tickets to compensate for this or they settle with not reviewing code at all.
Code coverage and tests:
Set up code coverage as a test step, there are so many tools out there eg.: codecov or coveralls. As a good rule, keep your test coverage over 70%. Do not try to achieve 100% code coverage from day one, if you are not a bank or security provider. High test coverage usually decreases flexibility. When developers are forced to write more tests, they settle with writing unnecessary tests or tests which are testing the wrong logic which will hurt you more than having low coverage.
Start with choosing your code repo provider (eg: Github, Bitbucket or Gitlab), then choose a CI provider (eg.: Jenkins, CircleCI, Travis, )
I have luck with Github + CircleCI + Coveralls for Golang development.
The most important rule to achieve this is that a developer should be able to develop, test, and build an individual service or library locally without having to set up a complicated build environment.
Without following this rule, you can not apply any of the principles explained here and you can not have small build iterations.
CI Pipeline:
This is a generic CI pipeline, details depend on language and platform.
Golang, Python with Pip and Javascript with NPM can use git as a library repository so for these languages, the library development flow looks like this:
CD pipeline:
I will not explain the CD pipeline here. Previously we were including CD pipeline as a last stage of the CI pipeline but recently FluxCD and ArgoCD have become CD standards and it is loosely coupled with CI.
Conclusion:
To sum up, these are the key concepts of this post:
- Keep it simple stupid.
- Split your work into smaller iterations.
- Do versioning and have stable dependencies.
- These development practices are a commodity, so don’t spend too much time on re-inventing this kind of work.
Follow me on Github:
Or Linkedin:
