DevOps is about knowing! So at this stage you have kicked off your DevOps initiative. You have Developers and Operations staff working together in a PoC team or even already managed to roll-out several teams. But how do you know if the initiative is successful and giving your organisation the desired benefits?
One of the key things of a DevOps initiative is to measure how well your initiative is performing and if you really achieve the efficiency you are looking for. This is why a very important part of DevOps is to measure your performance and this is also an integral and built-in part of all real DevOps implementations. In some cases the DevOps guys have even been known as ”the guys that measure everything”. This is why the ”M” has its place as an important part of the CAMS model.
M for Measurement
The overarching objective when implementing a DevOps initiative is of course to increase efficiency. However, this can come in different flavours and mean different things for different individuals or organisations. Efficiency can also be measured in different ways. It can be to optimize a specific part of the flow or to optimize a complete process. If we weigh in other aspects and objectives of a DevOps initiative, like taking care of security concerns and decrease incident and error handling times, it further increases the scope of parameters that need to be measured in order to know how your DevOps initiative is performing. As with most things, taking it one step at a time is a sound approach.
Normally when implementing a DevOps initiative, you would look at one or a few key processes or projects to start with (we also promote this approach with the ”Think Big – Act Small – Scale Fast” approach – see previous blog posts). This gives you an idea of our recommendation when it comes how to measure DevOps performance. Start by measuring something you can control as a short process or a specific task where you have comparable data. Your measurement and control can be built out from there to cover whatever processes or functions you would like to measure and compare.
Maybe the initiative to create new measurements will soon not come from management, but rather from the ”techies” that are doing the DevOps work. The nature of a true technician is often that they want to know what they are doing and that specific functions, servers, processes or whatever it might be, is performing as expected. This is for instance the true nature of a serious operations person.
A successful DevOps implementation, where everybody agrees and is part of the initiative, will probably feed a lot of interesting data that you probably wouldn’t even have thought of yourself.
3 common DevOps metrics
However, it may be useful to get an idea on where to start and what can kind of metrics is common to measure in a DevOps initiative. Here are a few examples:
- Deployment frequency: One common objective with DevOps is to increase the ability to do deploys from development to production. This is often paired with a microservices initiative, where a large application is split into much smaller parts to enable more frequent deployments through avoided dependencies.
- Mean time to recover: How quickly can we rectify and deliver fixes to production problems/errors?
- From commit to deploy: How quickly can we deliver upgrades to the application in production from our version control solution? If one wants to expand this to a larger organisational perspective it might also be interesting to measure the time from when a new requirement is described until that new feature is deployed. This last one can be tricky to measure since it is hard to say when that process is initiated, but one popular metric in this context is the Change lead time that measures the time from when the the first new code is committed until the feature is in production.
Another objective of the DevOps initiative is to be able to release often to put new features into play quickly. At the same time that we do this, we do not want to introduce a lot of errors or malfunctioning pieces of code. In order to avoid this, one popular metric is Change Failure Rate. This metric follows how many of the committed deploys that are successful. Of course any DevOps initiative is interested in keeping the Change Failure Rate low, as a high number indicates that something isn’t working as expected in the process and the desired effects of the investment or project are unlikely to be reached.
Measurements - a natural part in DevOps
The above mentioned metrics are of course only a few examples on what can be measured in a DevOps initiative. Which metrics are important to your organisation can of course only be decided by yourselves. Within a successful DevOps initiative, measurement are a natural part and a ”self playing piano”, where operators themselves feed management with interesting data and work to improve performance wherever possible.
With a little imagination, important measures and follow up can be presented in real time dashboards that are available to everyone and updated in real time. This is when DevOps operators really have the initiative up and running, are proud of it and neither can nor wishes to hide. We are a team and we are in this together. Isn’t that a feeling we all would like to get from work?
DID YOU LIKE THIS? THERE IS MUCH MORE TO LEARN
