Recently, I tried to hire an AppSec contractor to add some capacity to our team. The request went out to some contracting firms and we received six resumes just a few days after posting. Now, the problem is that my team and I don’t have the capacity to interview that many candidates or even a subset of candidates. So I decided to send all of the candidates a few relatively straight forward web app vulnerability challenges, or at least I thought they should have been straight forward or relatively simple to solve. The experiment showed us that the consultants, all of which claim to have 5+ years AppSec experience, couldn’t identify major flaws in some stripped down CTF challenges. Their responses left me completely baffled and disheartened. I fear for our industry.
I do not consider myself an expert, nor will I ever claim to be one. What I’ve always found to be the case is that the more you learn, them more you realize you have to learn. Be humble. And to quote an actual expert in the infosec industry:
If you don’t think you are a newb, you’re not trying hard enough. - HD Moore
The responses below also show me a clear lack of professionalism on a number of fronts. First and foremost, if you don’t know the answer please state so. In my experience, honesty and transparency has been key to building a great relationship with highly skilled development teams at a Fortune 100. Technologies in use in development are changing so rapidly it’s impossible to know everything, and that’s OK. IMHO, the correct way to approach a question, problem or you don’t know is to simply state to your client that you’re not familiar with that specific technology or configuration, but will research it and provide a high-quality analysis shortly. Do not make up some technobable or simply guess, have some humility.
The Challenges
Each of the “experts” below were given the same three challenges. These challenges were pulled from our past internal CTFs we hold to teach and reinforce skill sets with our internal security and development teams.
- Wide open S3 buckets - A static website was hosted on a poorly configured S3 bucket.
- Server-Side Request Forgery - While not your typical “OWASP Top Ten” vulnerability, it should be in every senior app tester’s skill set to detect and exploit this vulnerability. This challenge was hosted an an AWS EC2 and the goal was to have the tester pull the AMI ID from the metadata service. It was the hardest of the 3 and I figured few if any would identify it and I would happily accept simply the identification of the SSRF vulnerability.
- NoSQL Injection - this was a 101. As one of my colleagues put it, the old-school equivalent of
' or 1=1--.
Expert Responses
Below are the responses we received from the candidates. Of the four respondents to the challenges, there was only one correct answer.
| Years of Experience | Challenge 1 | Challenge 2 | Challenge 3 | Correct Answers | |
|---|---|---|---|---|---|
| Expert 1 | 5 | The value of the flag can be identified by first identifying the IP address of the particular URL. I found the IP address as well as information about the free ports in both tcp/udp by using nmap command. After identifying the IP address, i used ncap commands to connect into the server (nc -u/-t ip address port) and opened new terminal in order to listen to the server(nc -lnvc ip address). By using this method we would get hashed values and after obtaining the hashed values, we have to copy & paste in online hash converting tools! By this way we would obtain the flag information. | For this particular challenge I used burp suit proxy to obtain the AMI ID and server information. When we turn on the interceptor in burp suit, all the request to the DNS server goes through burp suit where we can even modify the header’s information and send request to DNS server. | Finally for this third challenge I used burp suit and OWASP ZAP to get the information about the URL. By usiing the interceptor i got all the information about the server as well as the other details regarding the free ports and request and response from the server. | None |
| Expert 2 | 6 | 01E18CABD9192987 | ami-327f7372 | D38B05454554EAE8 | None |
| Expert 3 | 8 | This link is not secured there is no SSL config it is on HTTP | 3925709 | No Response | None |
| Expert 4 | 8 | CTF{l3aky_buck3t} | 2216684 | url of the challenge | Challenge 1 |
Wrap-up
I’m not posting these responses to shame the testers. I’m disappointed that folks in our industry can be actively working for 5+ years and not have a grasp of concepts outside the OWASP Top Ten. How are non-technical hiring managers supposed to weed trough resumes to identify qualified candidates when they’re not capable of conducting technical tests like this? Organizations are entrusting individuals like this to protect some of their most valued assets and the skill level I’m seeing from “senior” resources is sub-par and unequipped to handle today’s modern applications. I’ll be sending all candidates who took the time to attempt the challenges a detailed writeup of the challenges so they can further their skill set.
If you agree or disagree with any of this, hit me up on twitter: ztgrace.