The practice of testing applications for security defects, whether they manifest as logic flaws or bugs in your code, can be a controversial subject for the Application Security (AppSec) community.

With the advent of DevSecOps and the introduction of automated tests is the CICD pipeline, testing has increased in popularity (if not frequency). Moreover, testing solution vendors are extremely keen to provide API driven functionality to automate and integrate security testing earlier and deeper in the SDLC.

What’s the value of testing?

But while we can all agree that testing is better than no testing at all, what is the real value of testing software without a true understanding of what we’re testing for?

Sure, most scanners and source code analysis tools have an ever-growing list of OWASP … security rules, policies and signatures which make them tempting to “point and shoot” (certainly guilty as charged in the early years of my career). After scrolling through thousands of false positives though, one starts to wonder if that process actually brings real value to the identification of security defects which can seriously impact your business?

The answer is not so simple. It mainly depends on the maturity level of your organisation. For instance, let’s say you’ve never run a SAST tool on your code – many organisations still run 6 – 7-year-old code with deprecated libraries.  Out of the mountain of findings (over half of which are likely false positives), you might find some real vulnerabilities. Those include hardcoded passwords, legacy methods, maybe a few insecure redirections, and a bunch of CRFs findings. While this is all useful stuff to fix and address in due time, it’s unlikely to find logic flaws in your signup flow.

The same is true for DAST tools. Without a deep-ish understanding of the application and knowing what you’re looking for, as long as you’re using modern development techniques, MVCs and use currently maintained libraries, these tools are not great at finding anything really CRITICAL when run in “auto” mode.

I think there’s definitely value in a robust testing regimen especially at the beginning of an AppSec program, but over time and after you’ve sorted all the low hanging fruits the meaty stuff tends to dry out …as it should do.

Find the right way!

So, what’s the right approach, should we stop testing our apps altogether after we got rid of all our CRITICALs and HIGHs? Not so fast!

Testing has a very important place in a mature AppSec program, but – over time – the focus shift from trying to find low-hanging fruits to a more structured approach. A mature AppSec program runs tests to verify that the security requirements defined during the design stage, have been properly followed and that they continue to be do release after release. Think of it as security regression testing; a test exists to tell us if things work as they should. But how do we know how things should work in the first place?

Enter Behaviour Driven Development (BDD) Security…

BDD Security is a security testing framework that uses natural language in a Gherkin syntax (in a Given, When, Then …) to describe security requirements as test features. Those same requirements are also executable as standard unit or e2e integration tests which means they can run as part of the build process in a fully automated way.

The Process   

1. Identify threats

Entire books have been written about threat modelling and it’s certainly a bit of an art to get right. Fundamentally, a good threat model should produce a list of threats i.e. ways in which your applications and systems can be misused or abused.

 Let’s say you’re designing a new signup flow; likely threats could be account enumeration (i.e. information disclosure) or lack of availably (i.e. denial of service)

2. Define Security requirements

Once we know what can go wrong is time to define what can be done about it. In case of our sign-up flow we should require that our application is indeed protected from potential brute force and DoS attacks.

3. Select Countermeasures

Countermeasure (or security controls) are then introduced to mitigate these threats. In case of our signup flow, valid countermeasures may include implementation of an account lockout mechanism or captcha like solutions.

4. Test countermeasure

Here where we test that our controls working effectively! There’s where testing come in. If we understand the problem and have a clear plan of action is much easier to test the plan actually works. The purpose of a test then becomes to ensure that all threats identified have a suitable countermeasure, if not the test should fail.

This approach is very useful to perform security regression testing that fits nicely with your CICD practices. At every deployment the automated test will check if the countermeasures are still being correctly applied or whether they have regressed.

Conclusion

In conclusion, testing is definitely a good thing and AppSec professional should be encouraged to test their applications whether they follow the BDD approach or not. However, knowing what to test for makes your testing strategy significantly more accurate and easier to develop and maintain.