“The outcome of the execution of E-type software entails a degree of uncertainty, the outcome of execution cannot be absolutely predicted”, or more fully, “Even if the outcome of past execution of an E-type program has previously been admissible, the outcome of further execution is inherently uncertain; that is, the program may display inadmissible behaviour or invalid results”.
The above statement is an extract from Meir Lehman’s “Software Uncertainty Principles”. The E-type systems are the systems that almost all of us work on developing. These systems are prone to many bugs, many performance issues and production incidents.
Most of us work on these types of systems every day and every one of us tries to optimise, this or that thing to reduce the number of bugs, to make the systems less prone to defects and to make each successive execution of the system predictable.
What has changed what kind of new principles we have now in the industry that would contribute to make the systems ’products’ performance more predictable?
A handful of principles and tools come together to solve this issue.
My suggestion is to use the combination of:
SCRUM, Test Driven Development, Selenium, Continuous Integration, cfEngine
Let’s look at the points in separate then discuss the whole together.
Agile software development methodologies, such as SCRUM, increase the transparency of the whole environment. The issues and problems are quickly highlighted, and it is encouraged in lean working environments to try to concentrate on the ways of preventing problems like that, rather than on the problem that resulted because of it. Meaning, if you have a bug that occurred on production, do not blame the person who caused the bug during the implementation but try to see what in the whole release process went wrong that the bug was missed.
The short iterating cycles of the Agile development methodologies and the emphasis to have Shippable product after each iteration (Sprint, as in SCRUM) makes the development teams to approach the testing more seriously, to make sure the product is tested completely and make sure the future executions of the system are predictable.
The Test Driven Development is a relatively new approach to software development, that encourages developers to write Tests first, and then start coding their functionality. Let’s take a closer look.
Imagine I am implementing functionality that when I hit a button, a light bulb illuminates. I write a Test first, check that the light bulb is illuminated. At first the test(s) will be failing, but as soon as the functionality is implemented, the assertions will be true and the tests will work as intended.
This will ensure that every piece of functionality developed is wrapped with tests, resulting in an increased Test Coverage. This in the end means that the system is being tested more thoroughly and in a more detail.
Moving on to another way of testing, that makes everything even better. Selenium is a tool that allows browser based UI testing. The development team members prepare and maintain Selenium tests according to the Acceptance Criteria of the User Stories (features) they implement. A nice collection of Selenium tests is soon collected, with at least the most critical flows of the system being checked by the automated tests, instead of using testers.
The heart of all of this is the Continuous Integration tool that checks out the code from the repository, runs it against the Unit Tests, runs static code analysis checks, deploys the system and runs Selenium acceptance tests.
Cfengine is a policy-based configuration management system that provides automated configuration and maintenance of servers, from a policy specification. In our case and in our process it is required to make sure that the parameters of the environment, where our E-Type system operates are correct, and are the same on all environments that our system runs on. The more confident we want to be in the executions of our system, the more the external factors affecting our system need to be stable.
All of these bring us to an environment, where in every stage of the software development process the code/system/product is checked to make sure it is robust. Many feedback points are available that give a lot of info on the status of the product:
After the execution of the build, static code analysis, unit tests, deployment, selenium tests
If followed carefully and when the development team feels comfortable working with this approach then the risk factor from the Uncertainty Principle decreases, because in every stage of the process the integrity and robustness of the system is ensured, and the doubt about the outcome of further execution of the E-Type system is becoming to diminish.