//
you're reading...
WebDriver

BrowserMobProxy meets JBrowserDriver

While I was casually skimming through the internet, I stumbled into this new flavour of webdriver which is being recommended as an alternate for “ghostdriver“. Its called “jBrowserDriver“.

The github page should pretty much help you get started with using it.

In this post we will just look at how do we couple a proxy server such as “BrowserMobProxy” with “jBrowserDriver“.

Without further adieu, here’s how we do it.

We would first need to do some preparatory steps related to certificates, else you will encounter certificate related errors such as the one below :

[2016-09-03T15:35:49.476][Instance 1][Port 52167] java.io.IOException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target: https://www.google.com.ar/?gws_rd=ssl
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at com.machinepublishers.jbrowserdriver.StreamConnection.exec(StreamConnection.java:360)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at com.machinepublishers.jbrowserdriver.StreamConnection.getResponseCode(StreamConnection.java:440)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at com.sun.webkit.network.URLLoader.receiveResponse(URLLoader.java:414)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at com.sun.webkit.network.URLLoader.doRun(URLLoader.java:163)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at com.sun.webkit.network.URLLoader.lambda$run$91(URLLoader.java:128)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at java.security.AccessController.doPrivileged(Native Method)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at com.sun.webkit.network.URLLoader.run(URLLoader.java:127)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[2016-09-03T15:35:49.476][Instance 1][Port 52167] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[2016-09-03T15:35:49.477][Instance 1][Port 52167] 	at java.lang.Thread.run(Thread.java:745)

  • First follow the instructions detailed here to download the firefox’s default certificates to your local file system.
    Certificate is available at :
https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
  • Now follow the instructions detailed here to download the BMP certificate to your local file system.

Certificate is available at :

https://raw.githubusercontent.com/lightbody/browsermob-proxy/master/browsermob-core/src/main/resources/sslSupport/ca-certificate-rsa.cer

Once you have both the certificates, use any text editor of your choice to append the certificate contents that you got for BMP, to the certificate that is associated with jBrowserDriver.

In the coming example, lets assume its going to be available in the location

src/main/resources/local-certs.crt

Now lets look at how a simple example looks like :

import com.machinepublishers.jbrowserdriver.JBrowserDriver;
import com.machinepublishers.jbrowserdriver.ProxyConfig;
import com.machinepublishers.jbrowserdriver.Settings;
import net.lightbody.bmp.BrowserMobProxyServer;
import net.lightbody.bmp.client.ClientUtil;
import org.openqa.selenium.WebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

import java.io.IOException;

public class JBrowserDriverWithBmp {
    public WebDriver driver;
    public BrowserMobProxyServer server;
    private static final String url = "https://the-internet.herokuapp.com/";

    @BeforeMethod
    public void setUp() {
        server = new BrowserMobProxyServer();
        server.start();
        ProxyConfig proxyConfig = new ProxyConfig(ProxyConfig.Type.HTTP, ClientUtil.getConnectableAddress()
            .getCanonicalHostName(), server.getPort());
        Settings.Builder builder = Settings.builder();
        builder = builder.proxy(proxyConfig).ssl("src/main/resources/local-certs.crt");
        driver = new JBrowserDriver(builder.build());
        server.newHar();
    }

    @Test
    public void testExchange() throws InterruptedException {
        driver.get(url);
        Assert.assertEquals(driver.getTitle(), "The Internet");
        String actualUrl = server.getHar().getLog().getEntries().get(0).getRequest().getUrl();
        Assert.assertEquals(actualUrl, url);
    }

    @AfterMethod
    public void closeBrowser() throws IOException {
        if (driver != null) {
            driver.quit();
        }
        if (server != null) {
            server.stop();
        }
    }
}

PS : The instructions in terms of setting up the CLASSPATH via Maven dependencies for both BrowserMobProxy and jBrowserDriver is available for reference in the respective Github repositories.

For the sake of convenience a ready to use merged certificate can be downloaded from here

After browsing through the codebase, I realised that there are a lot more easier ways of getting this done.

Here are some interesting values that can be passed to ssl() method of the Builder.

Settings.builder().ssl("compatible");

This causes jBrowserDriver to automatically work with a downloaded copy of

https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
Settings.builder().ssl("trustanything");

This causes jBrowserDriver to “literally” disable any certificate validations. If your test environment uses self signed certificates (or) if you don’t want to deal with any certificate related issues, then you would be using this. Please be very careful when using this because it can expose yourself to any “man in the middle attacks”.

Lets say you are operating in a Grid environment and your node and your BMP server are going to be running on different machines. You don’t want to use “trustanything” and disable cert validation but want to use a valid cert procedure as described in this blog, you can have the “merged certificate file” [ which I spoke about] and have it hosted on a web-server and pass on the location of it using :

String ssl = "https://gist.githubusercontent.com/krmahadevan/59b45361fd39e6df125b16b2443a5ce7/raw/213c0fccba5585fc4e3f5caea24ad1b0d1aadbe2/local-certs.crt";
Settings.builder().ssl(url);

I haven’t been able to get this mode to work properly though.

For more information take a look at the javadocs here.

Advertisements

Discussion

2 thoughts on “BrowserMobProxy meets JBrowserDriver

  1. The URL to the cert authorities did not work because of a bug in jBrowserDriver. Cert authorities specified with a URL remain cached for 2 days and changes to settings do not force the cache to update (it should). See issue https://github.com/MachinePublishers/jBrowserDriver/issues/210

    Cheers!

    Posted by DH | October 18, 2016, 8:16 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: