Web Security Testing with OWASP ZAP and Selenium

Have you ever wondered how we can actually find security vulnerabilities in Web Applications? There are guidelines from global security organizations that can be followed from Security Experts on how to efficiently perform penetration and security tests in your application. To review the top 10 vulnerabilities refer to OWASP Top 10 Risks

Introduction to Security Testing

There are multiple scanners in the software community commercial or open-source that gives the ability to penetration testers and security engineers to scan their application for known vulnerabilities.

Now most of the scanners are having CI/CD support and works well side by side with Selenium which is the tool that simulates user actions in our browsers.

OWASP Zed Attack Proxy (ZAP)

The OWASP Zed Attack Proxy (ZAP) is one of the world’s most popular free security tools and is actively maintained by hundreds of international volunteers. It can help you automatically find security vulnerabilities in your web applications while you are developing and testing your
applications. Its also a great tool for experienced pen testers to use for manual security testing.

Objective

To use OWASP ZAP, to detect web application vulnerabilities in a CI/CD pipeline

Problem

Web applications have Basic Authentication, User Logins and Form Validation which stops Scanner in its tracks

Solution

Use Selenium test scripts to drive ZAP. A project may include already selenium scripts for functional testing. Active scans actively modify the recorded requests and responses to determine further vulnerabilities

CI/CD Setup

Let's create a CI pipeline that will start ZAP in headless mode, run our functional tests that will perform two types of scan (active/passive), store results of scanning alerts in HTML Reports and tear down the server.

CI/CD Steps:

• Start ZAP
• Run Selenium Scripts (Passive Scan)
• Wait for Passive scan to complete
• Start Active Scan
• Wait for Active scan to complete
• Retrieve alerts and report

OWASP ZAP Installation

OWASP ZAP can be installed with multiple ways but we prefer to use Docker which is the simplest way to bring up the server.

 stage('OWASP ZAP setup'){
    sh "docker pull owasp/zap2docker-stable"
    sh "docker run --rm -d -u zap --name zap -p 4449:4449 -i 
        owasp/zap2docker-stable 
        zap.sh # A start up script provided by ZAP
        -daemon # Start in a headless configuration
        -host 0.0.0.0 # The ZAP host
        -port 4449 # The ZAP port 
        -config api.addrs.addr.name=.*
        -config api.addrs.addr.regex=true # Allow any source IP to connect 
        -config api.key=testypon" # Api key to be used 
    }

Now that we brought up ZAP in headless mode navigating to localhost in port 4449 you will be able to see:

WebDriver Integration

Lets see how we can integrate ZAP with our WebDriver instance that will drive the user interaction with our application:

DesiredCapabilities caps = new DesiredCapabilities();
Proxy proxy = new Proxy();
proxy.setProxyType(Proxy.ProxyType.PAC);
StringBuilder strBuilder = new StringBuilder();
strBuilder
.append("http://localhost:4449")
.append("/proxy.pac?apikey=")
.append(zapApiKey);
proxy.setProxyAutoconfigUrl(strBuilder.toString());
caps.setCapability(CapabilityType.PROXY, proxy);

Now pass the capabilities in your WebDriver Object as usual to intercept traffic from ZAP.

If the target web application has security response headers in place, specifically Strict-Transport-Security the WebDriver should be configured as follows:

caps.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
caps.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);

Passive Scan

Passive scans record the requests and responses sent to the web application and creates alerts for detected vulnerabilities. Also they are triggered whenever we access the application from WebDriver.

Active scan

Active scans actively modify the recorded requests and responses to determine further vulnerabilities for the application.

ZAP API in Selenium

For the reasons of simplicity we created a Driver(Wrapper to the actual ZAP API) called ZAPDriver to map to all needed API calls from our selenium scripts and will drive the execution of the scans. Lets see the most needed fuction for the scans:

First add your dependencies in the test project:

<dependency>
 <groupId>org.zaproxy</groupId>
 <artifactId>zap-clientapi</artifactId>
 <version>${zapapi.version}</version>
</dependency>

Now lets create our ZAPDriver to start interacting with the API functions:

@Service
@Profile("Security")
public class ZapDriver implements Spider, ScanningProxy, ContextModifier, Authentication {

    @Value("${zap.enabled:false}")
    private boolean zap;

    @Value("${zap.host:}")
    private String zapHost;

    @Value("${zap.base.url}")
    private String zapBaseUrl;

    @Value("${zap.port:0000}")
    private int zapPort;

    @Value("${zap.api.key}")
    String zapApiKey;

    private static final String MINIMUM_ZAP_VERSION = "2.6"; // Weekly builds are also allowed.

    private ClientApi clientApi;

    @PostConstruct
    public void initializeScanner() {
        this.clientApi = new ClientApi(zapHost, zapPort, zapApiKey);
        this.secData = new SecurityData();
        validateMinimumRequiredZapVersion();
        setAttackMode();
    }

In the PostContruct phase of the Object we initialize an instance of the ClientAPI to perform the need HTTP calls to ZAP from within our tests

Step 1: Enable Scanner

Now if you enable Passive or Active scanners you can do that with different ways. You can find the scanner policy name from the API and use the id to enable specific scanners for example (SQL Injection or Cross Site Scripting) but for the sake of simplicity we can enable all scanners and get a unified report with different categories of alerts:

     @Override
    public void enableAllScanners() throws ProxyException {
        try {
            clientApi.pscan.setEnabled("true");
            ApiResponse response = clientApi.ascan.enableAllScanners(null);
            log.trace(String.format("ZAP OK response for api call %s!!!", response.getName()));
        } catch (ClientApiException e) {
            throw new ProxyException(e);
        }
    }

The above will enable all active scanners and you have also another call if you want to enable all passive scanners as well

Step 2: Spidering

Spider is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).

  @Override
    public void spider(String url) {
        try {
            ApiResponse response = clientApi.spider.scan(url, null, null, null, null);
            log.trace(String.format("ZAP OK response for api call %s!!!", response.getName()));
        } catch (ClientApiException e) {
            log.error("Exception trying to spider " + e.getDetail());
        }
    }
 @Override
    public void excludeFromSpider(String regex) {
        try {
            ApiResponse response = clientApi.spider.excludeFromScan(regex);
            log.trace(String.format("ZAP OK response for api call %s!!!", response.getName()));
        } catch (ClientApiException e) {
            throw new ProxyException(e);
        }
    }

You can exclude also URLs from third party providers that you do not want to spider in order to speed up the crawling process and produce valid alerts for your application.

Step 3: Scanning

After spidering your application you need to scan for gathering our alerts based on the enabled policy of the scanners previously

 @Override
    public void scan(String url) throws ProxyException {
        try {
            ApiResponse response = clientApi.ascan.scan(url, "true", "false", null, null, null);
            log.trace(String.format("ZAP OK response for api call %s!!!", response.getName()));
        } catch (ClientApiException e) {
            throw new ProxyException(e);
        }
    }

Great!!!

Tip: Be careful that the times of the scan are different per application. You need to adjust the scan/spider timeout per your need

Step 4: Reporting

After scan is complete we need to create a report for the alerts and store it on our CI server.

byte[] htmlReport = scanner.getHtmlReport()
Path pathToFile = Paths.get(path)
Files.createDirectories(pathToFile.getParent())
Files.write(pathToFile, htmlReport)
allureService.html(pathToFile.toFile(),"OWASP ZAP Report")

 @Override
    public byte[] getHtmlReport() throws ProxyException {
        try {
            return clientApi.core.htmlreport();
        } catch (ClientApiException e) {
            throw new ProxyException(e);
        }
    }

Now the report for example will summarize all alerts with description and possible solutions provided from OWASP ZAP Organization:

Conclusion

SecDevOps is the philosophy of integrating security practices within the DevOps process. SecDevOps involves creating a 'Security as Code' culture with ongoing, flexible collaboration between release engineers and security teams. There are plenty tools and platforms that can be used in the release cycles of your application and we reviewed how to use the most known open-source tool.