Testing in production using Feature Flags

Gustavo Matsunaga
3 min readJun 20, 2021

--

In the software industry, we always agree that testing should be always be carried on earlier stages than production environments, typically over testing/staging environment, as part of the iterative development cycle. But what if on some businesses this stage conditions are not seemless as production? For this cases we should be also able tu run “Testing in production”.

Iterative development cycle with Testing in production

The questions that arise is: How to deploy product delivery only visible to a “few users” in order to carry on testing, without affecting others users?

The main purpouse of this article is to share a software practice that enable us to continously deploy product updates, based on API and Frontend/ Mobile App, being only “visible” to “some users”. There exists other defacto practices that helps addressing “Testing in production” such as Canary-Released which will not be covered within this article.

Everytime we need to deploy product updates in a “silent” way, we need to take in consideration some Hot Deploy or Hot Swap environment techinque.

Note: In some cases, when deploying breaking changes, Hot Deploy is not always a good idea, but for other cases, mostly incremental feature releasing, I just wrote this article.

Hot Deploy

In the software projects I used to collaborate, we used to build APIs with a Micro-Services architecture. Hot Deploy feature is being provided by the Core Backend masterpiece: WSO2 Enterprise Integrator, whose Application Server supports hot deploy of Synpase artifacts for: Api, Sequences and Task. For those who want to know more about WSO EI take a look at these articles: https://medium.com/wso2-learning.

Other backend technologies based on docker and kubernetes infrastructure, can also deploy in a “silent” way and conmute when ready container in a Hot Swap stratergy, being totally transparent to end-user.

Not only backend should be deployed in a “silent” way but also Frontend or Mobiles App, which are no big deal. But as you may noticed, the “new features” should only we “visible” not to everyone and this introduces another important concept we will cover: Feature Flags.

Feature Flags

Feature Flags or Feature Toggles it’s a technique that allow us to “turn on” some “features” on runtime and it’s a key enabler for doing “testing in production”. There exists a lot services which implements Feature Flags for many programming languages

Feature Flag example: Turn ON/OFF Chart

When addressing Feature Flags, it´s desirable to be able to controlled these “Feature Flags” using a Backend Service in relation to end-users( so we are able to control in runtime who can use/test each feature of the application)

Testing in production

Now putting the puzzle pieces together, we are able to deploy “silently” and “turn on” these new feature to some “testing users” that will conduct “testing in production” without affecting others.

If “testing in production” succeed, then we could enable these new feature to the other users or groups of user, without additional deploy steps, juts by turning “on” on the service.

If “testing in production” failed, then the feature is “turn off”, and the cycle repeats until succeed without any fail.

--

--

Gustavo Matsunaga

Electronic Engineer specialized in Software Development with cross-profile skills product owner/scrum master/developer driven by Agile