Penetration Testing – But Why?

This article has also been published on Emmas blog.


I am a penetration tester – a legal, ethical hacker. But I am more comfortable with calling myself a security tester or a security analyst, or a SecDevOps professional.

The most common distinction between vulnerability assessment and penetration testing is that the former is automated and the latter manual. However, that’s an over-simplification. Reading this excellent research paper (“Does penetration testing need standardisation?”, Knowles, Baron, McGarr, 2015), the delivery of penetration testing services are of varying type and quality. Specifically communicating and fixing the findings often fall short. And truly  – isn’t fixing the issues the whole point?

I once again tried my sketchnoting skills. In blue are findings from the paper, red are my own remarks.

The certificate I’m studying, Offensive Security Certified Professional, teaches how to perform exploitations end to end – from reconaissance to remote shell. This is a very useful skillset, because it requires you to apply the Hacker Mindset and truly understand the OWASP TOP-10.

However, the bulk of security work in the field is not fancy pwnage and haxx0r exploit development. In most people’s minds, however, this is what security professionals do all day long, and so they think that deep knowledge in that particular skillset is essential.

According to the paper, many companies don’t want actual exploit development in their penetration test. An intrusive penetration test can cause disturbances on the system. And a reasonable management is likely to accept a vulnerability, if the pen tester can present a plausible scenario in words, also without working exploit code.

What really upsets me in this study is the poor quality of the reports:

“Often basically a Nessus output in PDF format” (Knowles, Baron, McGarr, 2015, page 14)

Translated into human readable: any student in my higher vocational studies could do that in the first semester, but without the prize tag of specialist consultants.

The customer expects and pays for a human’s skill: prioritization, attack scenarios, root cause analyses. Such a delivery is downright unethical. The “ethical” in ethical hacking is not only about having permission, but also to deliver what you’re paid for.


Penetration Test Checklist:

  • Do you know what your assets and business values are?
  • Do you already have someone in your organization that has and takes responsibility for security? (CISO, a security minded sysadmin or developer)
  • Do you have a process for prioritizing bugs in general?
  • Are you working towards CI and DevOps?
  • Do you have a nice, prestigeless team?
  • Are you updating your software?
  • Are you running vulnerability scanners?
  • Do you have a vulnerability disclosure program?

If all those boxes are ticked: Congratulations! Performing penetration tests is a cost-effective and smart thing to do. If not, you should probably start somewhere higher up in the list.

2 thoughts on “Penetration Testing – But Why?

    1. My view of Certified Ethical Hacker (CEH) is that it’s not good bang for the buck and the money is better spent elsewhere.

      One of the modules in my education used this curriculum and this book by Matt Walker.
      I highly recommend this book, because it’s as hilarious as a course book will ever get. You will also learn a lot from it. But taking the certificate for almost 1000 dollars is in my opinion not money well spent. Read the book, skip the exam :slightly_smiling_face:

      Otherwise, how do I learn? Being at the Chaos Communications Congress every year I always learn a ton. Their lectures are all published online. I go to local meetups. I try to continuously learn all the time through various sources that I find organically.

Leave a Reply

Your email address will not be published. Required fields are marked *