Selenium is a very popular automation tool, Selenium 4 has introduced many advantages which are helpful for testers. Chrome DevTools Protocol (CDP) is one of the major features which selenium has introduced recently. As the Chrome DevTools Protocol is now available with selenium 4 we can control the network on Chrome browsers. In this tutorial let’s discuss some of the features of Network Interception
- Capture Request with Selenium 4
- Capture Response with Selenium 4
- Block Network URLs with Selenium 4
- Load Insecure website in Selenium 4
- Mock Network using Selenium 4
- Control Network Speed/Emulation with Selenium4
Capture Network Request in Selenium 4
Selenium 4 has introduced CDP, using CDP you get access to browser Network features. You can get access to the Network HTTP requests when you make the call to the particular domain. For example, if you type URL https://www.rahulshettyacademy.com internally it makes requests to a lot of assets such as CSS, images, javascript, it may even call the API to load the data on the browser.
To capture Network Request in Selenium 4, you need to follow below steps.
The CDP features are made available in the selenium using DevTool package/class so we need to create the instance.
ChromeDriver driver = new ChromeDriver();
DevTools devTool = driver.getDevTools();
devTool.createSession();
In the above code, we care about creating driver instance first, using driver instance we are getting dev tools and then we are creating session.
The DevTool provides the command send() which allows us to send the command to CDP, CDP has documented list of Chrome DevTools Protocol command, by referring their documentation, to capture network request we need to
- Enable the Network
- Add the Listener to requestWillBeSent event
So the code looks like below
devTool.send(Network.enable(Optional.empty(), Optional.empty(),
Optional.empty()));
devTool.addListener(Network.requestWillBeSent(), requestSent -> {
System.out.println("Request URL => " + requestSent.getRequest().getUrl());
System.out.println("Request Method => " + requestSent.getRequest().getMethod());
System.out.println("Request Headers => " + requestSent.getRequest().getHeaders().toString());
System.out.println("-------------------------------------------------");
});
Once we are done with Network capturing, we need to navigate to the URL
driver.get("https://rahulshettyacademy.com/#/index");
Putting Everything Together
Example: Capture Network Request Using CDP in Selenium 4
import java.util.Optional;
import org.junit.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v94.network.Network;
public class CaptureRequestSelenium {
@Test
public void captureRequestSelenium() {
System.setProperty("webdriver.chrome.driver", "path to chromedriver ");
ChromeDriver driver = new ChromeDriver();
DevTools devTool = driver.getDevTools();
devTool.createSession();
devTool.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
devTool.addListener(Network.requestWillBeSent(), requestSent -> {
System.out.println("Request URL => " + requestSent.getRequest().getUrl());
System.out.println("Request Method => " + requestSent.getRequest().getMethod());
System.out.println("Request Headers => " + requestSent.getRequest().getHeaders().toString());
System.out.println("------------------------------------------------------");
});
driver.get("https://rahulshettyacademy.com/#/index");
}
}
Once you Execute the above Code the Output looks like below
Output : Network Request CDP Selenium 4
Capturing Network Response with CDP in Selenium 4
Above we have captured network requests similarly we can capture network response, For example, if you are looking for status for API, we can get it that as well.
To Capture Network response we need to do
- Enable Network
- Add Listener to Network.responseReceived event
Code Looks like below
Example: Capture Network Response with Chrome DevTools Protocol in Selenium 4
import java.util.Optional;
import org.junit.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v94.network.Network;
public class CaptureResponseSelenium {
@Test
public void captureRequstSelenium() {
System.setProperty("webdriver.chrome.driver", "path to chromedriver");
ChromeDriver driver = new ChromeDriver();
DevTools devTool = driver.getDevTools();
devTool.createSession();
devTool.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
devTool.addListener(Network.responseReceived(), responseReceieved -> {
System.out.println("Response Url => " + responseReceieved.getResponse().getUrl());
System.out.println("Response Status => " + responseReceieved.getResponse().getStatus());
System.out.println("Response Headers => " + responseReceieved.getResponse().getHeaders().toString());
System.out.println("Response MIME Type => " + responseReceieved.getResponse().getMimeType().toString());
System.out.println("------------------------------------------------------");
});
driver.get("http://google.com");
}
}
Output: Network Response CDP Selenium
Blocking URL using Chrome DevTools Protocol in Selenium
Consider example, if you want to validate the website blocking all the .css files (that is website without styles), you need to block all the urls/files which are having extensions with .css. Similarly, you might have a scenario to test blocking certain API calls. These scenarios can be tested using Chrome DevToos Protocol(CDP) in selenium 4.
The Network.setBlockedURLs () provides option to block the List of URLs
If you want to block all the .css you can simply use
devTool.send(Network.setBlockedURLs(List.of("*.css"))); // Blocks all css
If you want to block a particular API or URL you can use like below
devTool.send(Network.setBlockedURLs(List.of(
"https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png"
)));
Example: Blocking API/URL/CSS in Selenium 4
Before Blocking CSS website looks like below
import org.junit.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v94.network.Network;
public class BlockURL {
@Test
public void blockUrl() {
System.setProperty("webdriver.chrome.driver", "path to chromedriver");
ChromeDriver driver = new ChromeDriver();
DevTools devTool = driver.getDevTools();
devTool.createSession();
devTool.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
devTool.send(Network.setBlockedURLs(List.of("*.css"))); // Blocks all css files
/*
* Block request by URL below code
devTool.send(Network.setBlockedURLs( List.of(
"https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png"
)));
*/
devTool.addListener(Network.loadingFailed(), loadingFailed -> {
System.out.println("Blocking reason: " + loadingFailed.getBlockedReason().get());
});
driver.get("https://rahulshettyacademy.com/#/index");
}
}
After Executing the above code it blocks all the styles and the Website will be loaded like below.
Output : Blocking URL in Selenium with CDP
Loading Insecure Website using CDP in Selenium 4
The browser usually blocks, we are trying to load the HTTPS website if the certificate has some errors. Many times, we will not be able to proceed with such website automation. CDP provides the option to ignore those certificate errors and we can load that website on the browser and test it.
The setIgnoreCertificateErrors command in CDP can be used with Selenium 4 and we can proceed with testing.
Security.setIgnoreCertificateErrors(true)
Example : Load Insecure webpage with CDP in Selenium 4
import org.junit.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v94.security.Security;
public class InsecureWebsite {
@Test
public void ignoreCertificate() {
System.setProperty("webdriver.chrome.driver", "path to chromedriver");
ChromeDriver driver = new ChromeDriver();
DevTools devTool = driver.getDevTools();
devTool.createSession();
devTool.send(Security.setIgnoreCertificateErrors(true));
driver.get("https://untrusted-root.badssl.com/");
}
}
The above code loads the website though it's not secure and has an HTTPS certificate error.
Network Emulation and Controlling Network Speed using CDP in Selenium 4
Consider we have a scenario, we need to test our website how it loads under slow internet connection, like 2G, 3G, 4G, etc. If our website is tested under slow bandwidth conditions and if it is optimized to work in slow bandwidth it works normally. The Selenium 4 with Chrome DevTools Protocol provides option to control the network bandwidth it is called network emulation.
Network emulation can be done using the command Network.emulateNetworkConditions, The selenium DevTools wrapper takes argument such as offline, latency, downloadThroughput, uploadThroughput, connectionType. We need to call this method to emulate network conditions.
import java.util.Optional;
import org.junit.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v94.network.Network;
import org.openqa.selenium.devtools.v94.network.model.ConnectionType;
public class NetworkSpeed {
@Test
public void networkSpeed() {
System.setProperty("webdriver.chrome.driver", "path to chromedriver");
ChromeDriver driver = new ChromeDriver();
DevTools devTool = driver.getDevTools();
devTool.createSession();
devTool.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
devTool.send(Network.emulateNetworkConditions(false, 20, 20, 50, Optional.of(ConnectionType.CELLULAR3G)));
driver.get("https://rahulshettyacademy.com/#/index");
}
}
Once we execute the above code, it emulates the network as per said parameters and loads the website. This helps to test website performance under slow network conditions.
Mock API using Chrome DevTools Protocol in Selenium 4
Consider the example, we have a website which internally calls some API, now what happens if we change the API parameters, what happens if we change the request information. Mocking can be done in selenium using CDP commands
Let’s consider the example,
Navigate to https://rahulshettyacademy.com/angularAppdemo/
Click on Virtual Library
It loads the table with book details.
If you open the Chrome Network Tab, internally it is making call to /GetBook.php?AuthorName=Shetty
Now, let’s change the network URL parameters programmatically,
We have /GetBook.php?AuthorName=Shetty API call for that lets replace Shetty with Unkown and lets see what happens
Example : Mock API Requests in Selenium 4 Using CDP
import java.util.Optional;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v94.fetch.Fetch;
public class MockNetwork {
@Test
public void mockAPISelenium() {
System.setProperty("webdriver.chrome.driver", "path to chromedriver");
ChromeDriver driver = new ChromeDriver();
DevTools devtools = driver.getDevTools();
devtools.createSession();
devtools.send(Fetch.enable(Optional.empty(), Optional.empty()));
devtools.addListener(Fetch.requestPaused(), req -> {
if (req.getRequest().getUrl().contains("=Shetty")) {
String mock = req.getRequest().getUrl().replace("=shetty", "=Unknown");
devtools.send(Fetch.continueRequest(req.getRequestId(), Optional.of(mock), Optional.empty(),
Optional.empty(), Optional.empty(), Optional.empty()));
} else {
devtools.send(Fetch.continueRequest(req.getRequestId(), Optional.of(req.getRequest().getUrl()),
Optional.empty(), Optional.empty(), Optional.empty(), Optional.empty()));
}
});
driver.get("https://www.rahulshettyacademy.com/angularAppdemo/");
driver.findElement(By.xpath("//button[contains(text(),'Virtual Library')] ")).click();
}
}
Let’s understand the Code,
devtools.send(Fetch.enable(Optional.empty(), Optional.empty())); : We are enabling Featch
devtools.addListener(Fetch.requestPaused(), req -> { : In this line we are adding listener to the CDP evernt Fetch.requestPaused, this helps to get the each request that is made from the browser.
if (req.getRequest().getUrl().contains("=Shetty")) { : Our scenario is monitor only request having parameter “Shetty”, and we are going to replace that with Unkown, so we are verifuing the request is having parameter “Shetty”
String mock = req.getRequest().getUrl().replace("=shetty", "=Unknown"); : If the URL is having Shetty then we need replace original url with Unknown that is what we are doing here, this is mock URL we are building from orginal URL
devtools.send(Fetch.continueRequest(req.getRequestId(), Optional.of(mock), Optional.empty(),Optional.empty(), Optional.empty(), Optional.empty())); : Now, we need to continue with Mock Request, we have replaced original request with mock request in this line and continuing with mock request. That means browser sends the mocked request in place of original request.
else {devtools.send(Fetch.continueRequest(req.getRequestId(), Optional.of(req.getRequest().getUrl()),Optional.empty(), Optional.empty(), Optional.empty(), Optional.empty())); }
We are not bothered about the request if the request doesn’t contain the “Shetty” parameter so else part we are just continuing with the original URL
Let’s execute the above code see what happens
Output : Mock API Requests in Selenium CDP
In this tutorial we have learned a lot about CDP with Selenium 4, Chrome DevTools Protocol has a lot of features, we tried to help you to understand some basics which help you to perform testing seamlessly in your organization.
The Rahul Shetty Academy continues to bring articles, courses that help you to upskill your knowledge. Rahul Shetty Academy has designed many courses on trending automation tools. Every topic is covered in depth and detail, whenever Rahul Shetty Academy design courses it also considers Beginner, Intermediate and Advanced level and cost-effective.
Author: Ganesh Hegde | LinkedIn
Ganesh Hegde is a Professional SDET/QA he has expertise in many automation tools such as Selenium, Cypress, TestCafe, Playwright, WebdriverIO. He is passionate about Blog Writing and Technical Content Writing.