Pentester Diaries Ep5: Understanding Severity Ratings

Welcome back to Pentester Diaries, a podcast series that aims to take off the hacker hoodie and have a real conversation about this growing profession.

In this episode, Jon Helmus talks with Joan Bono, a long-time Cobalt Core pentester. They will take a look at understanding pentest severity ratings.

Pentester Diaries is multi-formatted, with audio, video, and written versions of the episode below.

Watch the video podcast here:

Prefer to read the episode? Below you can find a transcription of the podcast:

Jon Helmus: [00:15]

Welcome to Pentester Diaries, a podcast series provided by Cobalt.io. I'm your host Jon Helmus and this week, we're interviewing Joan Bono. Joan, welcome to the show.

Joan Bono: [00:26]

Hi, thank you for inviting me.

Jon Helmus: [00:28]

You're most certainly welcome. Joan has expertise in various types of methodologies within pentesting. He's one of the top members in the Cobalt Core community. This week, we're going to be talking about how we can actually understand pentest severity findings and the ratings that go with them. Before we kick off into the content of this episode, Joan, it would be really great for you to talk a little bit about yourself, your background, and any insights you want the listeners to know about you.

Joan Bono: [01:00]

My name is Joan. I'm from Spain. I've been working in security for six years now. I live in Prague because I like the cold climate we have here.

Jon Helmus: [01:21]

I, too, like the cold climate. I live in the Pacific Northwest, where it's typically cold for the most part. To start setting the stage for the conversation, we know what a CVSS Score (Common Vulnerability Scoring System) is, what makes up that CVSS Score because we use that as a matrix or a framework, to assign severities to these findings that we find within targeted organizations we are pentesting. However, we do want to make sure that we translate it to the audience. Could you elaborate on what a CVSS Score is and what makes up the score itself?

Joan Bono: [02:04]

CVSS is an open framework we use during our test to categorize the vulnerabilities. Basically, it's what we use to assign a severity level. When you see a vulnerability is rated high, medium, or low, this number you see after the severity, sometimes you see WebLogic issue 9.8 CVSS Scores. That's the score we use.

Jon Helmus: [02:34]

Like you were saying, you can go and actually pull the CVSS Score from a designated area. You can actually go on NIST and find the calculator. Based on that, we find a CVSS Score associated with a vulnerability that we were able to exploit, sometimes it can vary a little bit in severity ratings. But before we dive into that specific topic, can you walk through an example of how you would compare different scenarios for the CVSS that you find. Meaning, if you find an RCE (Remote Code Execution) and SQL Injection, how do you rate those and how do you contrast them differently based on a production or test system?

Joan Bono: [03:26]

It's more based on personal experience. For example, when you see an SQL Injection or Remote Code Execution, you automatically think about it as a high severity, but it depends on the environment. From a business perspective, imagine you have an SQL injection and you can get information from a database, which just contains the name of a product on a website, and maybe you consider that as a high because it's a SQL Injection and it's serious. But from the client perspective, they could say this is not a high severity. And they could say – we don't need to fix that as soon as, for example, reflective Cross-Site Scripting, which affects the final use of the customers of this application. Yes, it's an injection, but you only can get the names of the products by navigating on the website.

Jon Helmus: [04:26]

Most definitely. I know as a pentester myself, it's also a matter of asking, is this a system that is generating revenue for the client or the organization, whatever the target is? What is the value of that asset? What is the value of that asset towards the organization? You have to kind of mimic the severity based on that along with the CVSS Score.

You mentioned it with SQL Injection. Yes, SQL injection is a high but what kind of information are you able to pull from it? A junior pentester might say, ‘Oh, I found SQLI’ and immediately slap a critical to it and after actually evaluating it you'll say, ‘Well, you were only able to pull an arbitrary table that didn't have any proprietary information in it so we're going to need to bring that down a little bit to a high, because you still have that injection point there which is not good. As a pentester that has worked with software developers, when there's a development issue where it shows they do not understand how to properly secure the website or how to write the code. It still needs to be high because it needs to be evaluated as such.

However, while we're on the topic, you had mentioned the severity of the systems, the impact that those systems have. In your own experience, I'd love to know your thoughts on the availability? As a penterter myself, I found instances where there were 20 instances of Cross-Site Scripting (XSS) on an organization's website. To say, ‘okay, I'm going to slap a critical on this,’ that's going to give you little to no time to fix it. How would you associate that, where you found a large amount of a certain critical finding, such as XSS or SQL injection, and the bandwidth or the amount of people to fix that, are not there? How would you, as a professional pentester, go about evaluating that?

Joan Bono: [06:42]

Depending on the information extracted from the databases. For example, if they have just one developer and the issue could affect the customers, we cannot decrease the severity of that just because they don't have the capacity to cover all these issues. In this case, I've seen companies just shutting down an entire platform just because of that. I'm talking about a platform that companies are using just to put how many hours they’re employees are working, where every employee logs in this page and then they develop a system in which the employee enters how many hours they’ve worked on each project. I've seen a company like that compromise their own platform to the point where no employee was able to write how many hours they worked for that week just because of that kind of injection. They knew that they were not able to get that fixed and that page was exposed to the internet.

Using that injection, it was possible not to write to the database, but you were able to see the values for each user, how many hours, names of the project, and that sensitive information. The company needs to evaluate, especially if a company works with medical data, addresses, client contacts, or company names. Of course you don't want that injection to affect you because otherwise if you say ‘wait, yes, this affects our business, then our employees are not able to record how many hours they are working in each project.’ We don't know exactly how to change that, but our clients don't want to be exposed that way because we are providing services to the organization and then you have a vulnerability and you're exposing their data. That's something internally, depending on the business impact. They will have to ask themselves: ‘what do we prefer? To work more hours next week? To ask our employees to write down how many hours they work on each project? Or to lose our customers because we have an injection because we have exposed their names. How much we pay them? How much money do we get from them, which is the main point of contact of that company for us.’ These are all questions they will have to evaluate.

Based on the capacity to remediate an issue, we cannot lower an issue. What we can do, in this case, when we are talking about a serious vulnerability when you're doing a pentest, imagine you have ten low severity issues, and just one medium. In the low severity issues, you have SSL (Secure Sockets Layer), DLS and these configurations or some headers missing, or a JavaScript library that is outdated, JQuery outdated, for example. In this case, you can just increase the severity for the one you considered low but it's the highest one in the lowest. So you may say ‘I can increase this little bit because this is more important than the other one, and this is an easy fix so this could be fixed quickly. They just have one medium so let's put two mediums to make sure they will take a look at this one and they will fix this one.’ Because there may be clients that do not dedicate as much attention to low severity issues, as much as they focus on the high and medium. Due to this, they may forget completely about the low severity ones. Of course, the highest severity is the most important to fix, you have to put more effort into fixing the highest severities than the medium or the low, but you have severities that may be low at that moment that could become high severity or medium issues in the future.

Jon Helmus: [11:24]

With all that being said, I think to funnel that all into one thing is that you have to make sure that you, as a pentester, translate the severity of each finding. And yes, it may take a little extra work, but as you mentioned earlier, you might have to bring the severity down. Sometimes I think us, as pentesters, are not always conscious about that. When someone shuts down their website to fix it, they're losing money. We may think about it, but we're not directly impacted by it. The client is impacted by that. At the same time, like you said, you're having to communicate that risk to the client and that is critical.

Essentially that's what we do. We communicate and we're professional risk assessors. It's crucial that we help our clients understand the risk. Aside from helping them understand the risk, is there anything you want to add to it besides everything that you just said about how you communicate the risk portion to them? Even the remediation portion is there. As a pentester, how do you help out a client with understanding the risk while also implementing the remediation?

Joan Bono: [12:45]

For me, it has the same importance. If the client needs to know what's the vulnerability, yes, but also, they need to know how the remediation process works. Again, going to the same point about the low severity issues, sometimes they may say ‘they are low, we don't care about these ones.’ But they are really easy to fix and they are improving the security a lot. For example, if you have the XSS HTTP header missing. That can be fixed by just adding one line to the web server configuration. This can prevent the Cross-Site Scripting issues. For example, this Cross-Site Scripting can be used to steal user information, to investigate the user, or cause a Denial-Of-Service attack but just by adding one line, can try to prevent a high in the future, or at least it will not be as easy to trigger. That's the main thing. The calculator gives you another view about the security of your application or your company. It’s scored as high, medium, and low, and not implied as, ‘we have to take care of this and we don't need to take care of this.’

Jon Helmus: [14:05]

It's crucial to us as pentesters that we help them along in the entire way. In the past, I've worked with consulting agencies, and I think any pentesters that are listening in that have worked with consulting agencies, understand that there's places where we have worked where it's like ‘nope, you assign it a severity. You send it to them and then it's out of your hands and it's all in their hands. You don't need to worry about it anymore.’ I think it's important to us as pentesters that we really help them through the remediation process, as well.

Another question to start coming to the tail end of the episode is that we already talked about how does a high go to a low? You talked about SQL Injection. SQL Injection is not great to have. If you look at it from a systematic issue level, it's going to be okay. We found this table, but it doesn't have anything in it. It doesn't have anything that's proprietary, sensitive, anything that could really impact any part of the business. It's random data. You can turn that high into a medium or low. However, I would love to know your take on how you take a low severity finding and turn it into a high? I think that's something that the listeners would really be interested in understanding, because we find lows all the time, but how do you turn that low into a high?

Joan Bono: [15:35]

I've seen this. Every company has their own guidelines, their security guidelines or some basis they have to cover. Let's say, the company security handbook says, for example, that TLS or SSL configuration issues are automatically low because to exploit a vulnerability of this type, you need to perform a Man In The Middle attack and then get the traffic and then decrypt the traffic. It's pretty much impossible to do that. In perfect conditions, everything works and so on but in real life, it's really complicated. For example, TLS 1.2 was deprecated from March 2020. In April or May, during an assessment, there was a company, who had this protocol deprecated. The same way we are deprecating Telnet, we are deprecating FTP, and TLS 1.2. This severity issue, which was a low, they asked to increase it as a high because it was a protocol, which was deprecated. Sorry, TLS 1.1.

Jon Helmus: [16:54]

I was going to say that was TLS 1.1

Joan Bono: [16:57]

TLS 1.1 is deprecated and they ask it to increase it to a high.

Jon Helmus: [17:05]

TLS 1.1 or TLS 1.2, aside from the semantics of the version, is that the takeaway from that is the complexity of exploiting that is difficult. You have to have a lot of different variables, however, as you just said, it indicates another larger issue that's outside of the technical realm. It's saying, ‘Okay, you're not abiding by your policy. You're overstepping the policy. You're not understanding the policy so there's that systematic issue there.’ Because of that, this was deprecated over a year ago and you still have it, why do you still have it? You need to rank it as high and I love that. I think everybody's listening in is going to be like, okay well ‘no, I didn't realize that so now we're going to start seeing a lot more high findings definitely all over the place.’ I'd love to know your comments on it. I think as we start out as junior pentesters, we really focus on the technical part and as we mature, we mature into the field where we understand that it's not all about the technical portions. We have to understand that there's three big pillars that make up the infrastructure. It's the processes, the operations, and the technical procedures. We have to understand that when we're assessing something, whether it's an asset or even a pentester looking at a policy, we have to take all three of those things into account. What do you think about that?

Joan Bono: [18:53]

That's what happens. When we start, we’re just focusing on getting remote code execution on time. That's the only thing that matters. You just want a metasploitable session open. It's the only thing you are looking for. But when you are just getting more experience, you know you also have to focus on the low and the medium things because it's not always about getting a reverse share. Another company is paying you to look for this. They want to be secure and it's not only about having a remote code execution, because maybe they don't have anything like that, and maybe they don't have a database running there. But maybe they have a sensitive information exposure.

For example, they have a file there inside a config folder that is allowing listing and a directory listing is a low severity issue. Then, you have a configuration file with all the internal variables. You have the AWS access keys. You have API keys. You have usernames and passwords and this low severity stuff is what really matters for the company, because it contains special information, and that's also important because it can cost a lot of money to the company. They don't have a remote code execution yet, but they can get a fine if this personal information is exposed.

In Europe, we have this data privacy law. If you're a small company and you have, let's say 100 customers, and you expose their data, you will probably have to close your company because you are not going to be able to pay the fine. That is something we must keep in mind. As pentesters, we are not only looking for the high severity things. We are also looking for the recommendations there because maybe we are not able to spread something, but the techniques are evolving every day, every minute. You try a payload which is not valid anymore because Cloudflare is blocking it. But if you look on Twitter one hour later and you have a bypass for that. That's something we learn. Every pentester we learn something but the more people we work with, the more things we learn. With the more companies you work with, the more things you know that they are not only worried about the high severity things. That's something important, I think.

Jon Helmus: [21:30]

It's a learning experience. The only way to understand it is to do it. You don’t learn how policies work by going on to Hack The Box. You learn how to do the technical portion, but you might also not necessarily understand like, ‘Oh, I'm not able to hit this target because Cloudflare is in front of it so I need to be able to access it internally or learn the bypass to get past Cloudflare.’ Because as a junior pentester, if you see any Cloudflare or any kind of web application firewall, you're going to try to hit the target that you're assigned and you're going to see port 80 and 443 open, you're going to be, ‘Oh, that's it?’ And then you might go to the next thing. That's what I think sometimes as a junior pentester, we would see and have someone who gains experience, you understand no this is a WAF. We need to understand that we need to either bypass this or if we can't bypass this, we need to be given a bypass to assess the internal portion because we're still not able to assess that scoped target or target environment.

It comes with experience and that's why experience plays such a key part in being a pentester. I think that's why it's crucial for junior pentesters to understand that if you're having trouble finding that gig that you want as a pentester or the next level. Technical expertise doesn't play the only part of you being a pentester. It's also the knowledge of understanding environments, understanding business culture, and understanding the way operations within an enterprise essentially work.

In my mind, pentesting has turned into less of a hack-of-all-the-things to more of a mature business process. It's going to be interesting to see how this field has quickly turned into a business process that helps out organizations mature. And at the end of the day, I think something that has resonated throughout our entire conversation is that pentesting isn't all about getting that shell. It's not all about getting that SQLI’s. As a pentester, the thought in your mind during the entire engagement and interaction with the client should be: how can I make this client better than they were before.

We're wrapping up on time now. Joan, thanks again for being on the Pentester Diaries, loved the conversation. As final points from the conversation, what would you advise all the listeners based on what we talked about today? What would be the key takeaways from this conversation?

Joan Bono: [24:21]

I would say, especially on these last minutes, that a live pentest is not a CTF. Pentest is not OSCP. Pentest is not Hack The Box. Pentest is understanding the client requirements and the client needs.

Jon Helmus: [24:43]

It's a business process, everyone. While you got to be a hacker, you also got to be a businessperson while you do it. Thanks again, Joan. Thanks for coming on the show. For everyone who has been listening in, the guest today was Joan Bono. I'm your host, Jon Helmus and we will catch you on the next episode.