Bad Behaviour: Automating System-level Safety Testing Using Open Source Tools - Paul Albertella

Bad Behaviour: Automating System-level Safety Testing Using Open Source Tools - Paul Albertella, Codethink

Testing is a key aspect of any safety project and test automation is highly recommended. Open source projects lead the way in this, but is FOSS really suitable for testing safety critical software systems? Combining tests with a continuous integration workflow lets us verify at each commit to our software, simplifying impact analysis. Storing tests and configurations in repositories with our code lets us control and coordinate changes to both; infrastructure-as-code means we can control our test environments too. Organising tests into pipelines lets us run stages in parallel; using a configurable pool of runners lets us run them on an industrial scale. But how can we derive tests to verify safety requirements as well as functional requirements, in a way that lets both safety and developer teams understand the relationships between them? Can we then run those tests on the target system? On the target hardware? On multiple target hardware platforms? Can we validate tests and safety mechanisms using fault injection? And trace from safety analysis to requirement to test to results? And integrate all this into our CI environment? We describe how Codethink is using open source tools such as LAVA and subplot to solve these and other hard problems for safety critical systems testing.