Automation Testing Interview Questions and Answers

Answer

You do not say no, you say what the return is. The honest answer is that automating everything is usually a negative return investment, and the way to prove it is arithmetic a delivery manager can follow. Take a case that takes 4 minutes to run manually and is executed once per release across 24 releases a year: that is 96 minutes of manual effort a year.

Automating it costs roughly 2 to 4 hours to write and, from experience, 20 to 30 percent of that build cost again every year in maintenance. That case never pays back. Now take the login plus search plus add to cart plus UPI payment journey that gets run in every one of 6 environments before every release: that is hundreds of executions a year, and it pays back inside two sprints.

So the rule I give is: automate what is stable, repeated, data heavy, deterministic and high business risk. Never automate one off cases, exploratory testing, anything whose pass criteria is human judgement like visual polish or copy tone, features still being redesigned every sprint, and flows where the oracle sits outside your control such as a live third party OTP gateway. Then I would rank the 1,200 cases by risk times execution frequency, take the top 150 to 200 that cover the revenue critical journeys, and commit to those in the quarter with a measurable target: regression cycle time down from 3 days to 4 hours. That is a number he can take to his own review.

Answer

The pyramid says most of your automated checks should be fast unit tests at the base, a smaller layer of integration and API tests in the middle, and a thin layer of end to end UI tests at the top. The reason is cost per test, not purity. A unit test runs in milliseconds, fails for exactly one reason, and points at one function.

A UI test runs in 20 to 90 seconds, touches the browser, the network, the backend, the database and three third party services, and when it goes red you have to investigate all of them. An inverted pyramid with 900 UI tests and 40 unit tests fails in four predictable ways. First, runtime: at 45 seconds average and even 10 parallel threads that suite is over an hour, so it stops running on pull requests and moves to nightly, which means developers get feedback the next morning instead of in 10 minutes.

Second, flakiness compounds: if each test is 99 percent reliable, 900 of them give you roughly a 1 in 10,000 chance of an all green run, so red becomes normal and people stop reading the report. Third, maintenance: a single navbar redesign breaks 300 tests, and the team spends its sprint fixing tests instead of finding bugs. Fourth, diagnosis cost: a failing UI test tells you something is broken, not what.

The fix is not to delete UI tests, it is to push coverage down. Keep 40 to 80 UI journeys that prove the critical paths work end to end, move the field level validation and business rules to API and unit level, and let the pyramid rebuild itself over two or three quarters.

Answer

Linear is record and playback, one long script per case with hardcoded data. It is fast to produce and impossible to maintain, and it is why so many recorded suites get abandoned. Modular breaks the script into reusable functions, login, search, checkout, so a change lands in one place.

Data driven separates the data from the logic, so the same script runs from Excel, CSV, a JSON file or a TestNG DataProvider, which is what you want for 40 GST rate combinations. Keyword driven pushes it further: the test is a table of keywords like openBrowser, enterText, clickElement, and a runtime engine maps keywords to code, which sounds like it lets manual testers write automation but in practice creates a second programming language nobody debugs well. Hybrid is the honest combination, modular page classes plus external data plus reusable utilities.

BDD wraps the whole thing in Gherkin Given When Then via Cucumber or SpecFlow so that non technical stakeholders can read the scenarios. In 2026 I would build hybrid: Page Object or component objects for structure, external data through a data provider or fixture, a config layer per environment, a reporting layer, and a driver or fixture factory. I would add BDD only if product owners or business analysts genuinely read and edit the feature files, because Cucumber adds a glue code layer and a regex or Cucumber Expression maintenance burden that pays for itself only when there is a real non technical reader. Indian services projects often mandate Cucumber contractually, so know it, but be able to say when it is theatre.

// Data driven with TestNG DataProvider
public class GstTest {

  @DataProvider(name = "gstSlabs")
  public Object[][] gstSlabs() {
    return new Object[][] {
      { "1000", "5",  "1050.00" },
      { "1000", "12", "1120.00" },
      { "1000", "18", "1180.00" },
      { "1000", "28", "1280.00" }
    };
  }

  @Test(dataProvider = "gstSlabs")
  public void invoiceTotalIncludesGst(String base, String rate, String expected) {
    InvoicePage page = new InvoicePage(driver);
    page.enterBaseAmount(base).selectGstRate(rate);
    Assert.assertEquals(page.getTotal(), expected);
  }
}

Answer

Page Object Model puts every locator and every page level action for one screen inside one class, so tests talk in business language, loginPage.loginAs(user, password), and never see a By locator. When the login page markup changes, one class changes and every test that uses it keeps working. Page Factory is Selenium's older helper on top of that: you annotate fields with @FindBy and call PageFactory.initElements(driver, this), which creates lazy proxies for the WebElements.

It looks tidy but it has real problems, the proxy re resolves the element on each call which hides where the lookup happens, it interacts badly with pages that re render because you get StaleElementReferenceException at odd points, and it gives you no place to put a wait. Most teams in 2026 write plain POM with explicit By fields and a helper that waits before acting, and skip Page Factory entirely. POM turns into an anti pattern in three situations.

First, the god object: a HomePage class with 120 methods because someone kept adding to it instead of splitting into header, filter panel and results grid component objects. Second, assertions inside the page object, which couples the page to one test's expectations and makes it useless for the next test. Third, page objects that return void everywhere, so you cannot chain and cannot express navigation, a good page method returns the page object you land on. Modern Playwright and Cypress suites often use component objects or plain fixtures instead, because the locator API is already expressive enough that a thin wrapper adds little.

// Plain POM, no Page Factory
public class LoginPage {
  private final WebDriver driver;
  private final WebDriverWait wait;

  private final By mobile = By.cssSelector("[data-testid='login-mobile']");
  private final By otp    = By.cssSelector("[data-testid='login-otp']");
  private final By submit = By.cssSelector("[data-testid='login-submit']");

  public LoginPage(WebDriver driver) {
    this.driver = driver;
    this.wait = new WebDriverWait(driver, Duration.ofSeconds(10));
  }

  public LoginPage enterMobile(String value) {
    wait.until(ExpectedConditions.visibilityOfElementLocated(mobile)).sendKeys(value);
    return this;
  }

  // returns the page you land on, so callers can chain
  public DashboardPage submitOtp(String code) {
    driver.findElement(otp).sendKeys(code);
    driver.findElement(submit).click();
    return new DashboardPage(driver);
  }
}

Answer

An implicit wait is a global setting on the driver, driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10)), that makes every findElement poll for up to that long before throwing NoSuchElementException. It is a blunt instrument: it only waits for presence in the DOM, not for visibility, not for clickability, and it slows down every negative assertion because checking that an element is absent now takes the full 10 seconds. An explicit wait is per condition, new WebDriverWait(driver, Duration.ofSeconds(10)).until(ExpectedConditions.elementToBeClickable(locator)), and it waits for the state you actually care about.

A FluentWait is the general form of the explicit wait where you set the total timeout, the polling interval and the exception types to ignore, which is what you use for a slowly polling dashboard where you want to ignore StaleElementReferenceException while the widget re renders. The mixing problem is the classic Selenium trap and it is worth stating precisely: the documented behaviour is that mixing them can cause unpredictable wait times, because the implicit wait lives inside the driver's element lookup while the explicit wait wraps that lookup in its own polling loop. Each poll of the explicit wait now blocks for the implicit timeout before returning, so a wait you declared as 10 seconds can effectively become 10 times 10 or longer, and in some driver and version combinations it fails much earlier than you expect instead.

The fix is one line of policy: set the implicit wait to zero, or never set it at all, and use explicit waits everywhere. Never use Thread.sleep, it is either too short and flaky or too long and slow.

// Do not do this
driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10));
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("total")));
// each poll blocks for the implicit timeout, real wait time is unpredictable

// Do this instead: no implicit wait at all, explicit everywhere
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(15));
WebElement payNow = wait.until(
    ExpectedConditions.elementToBeClickable(By.cssSelector("[data-testid='pay-now']")));
payNow.click();

// FluentWait for a widget that re renders while polling
Wait<WebDriver> fluent = new FluentWait<>(driver)
    .withTimeout(Duration.ofSeconds(30))
    .pollingEvery(Duration.ofMillis(500))
    .ignoring(NoSuchElementException.class)
    .ignoring(StaleElementReferenceException.class);

fluent.until(d -> d.findElement(By.cssSelector("[data-testid='order-status']"))
                   .getText().equals("CONFIRMED"));

Answer

The ranking I use is: a dedicated test attribute first, then id if it is genuinely stable and not framework generated, then a semantic CSS selector, then a relative XPath, and absolute XPath never. Index based locators like (//div[@class='card'])[3] or /html/body/div[2]/div[4]/span break because they encode position, not identity. A developer adds a promo banner, an ad slot loads, a sponsored card gets injected, or the list reorders by relevance, and your third card is now someone else's, so the test does not fail loudly, it asserts against the wrong row and either passes wrongly or fails with a confusing message.

Class based selectors are almost as fragile once the app uses Tailwind or CSS modules, because the class is either a utility soup that changes with a design tweak or a hashed name like css-1x7f9k that changes on every build. That is why the durable answer is a contract with the developers: agree that every element a test needs carries a data-testid attribute, that the value is treated as an API and is not renamed without telling QA, and that removing one is a code review comment. In practice you get this by adding the attributes yourself in a pull request for the top journeys and showing the flake rate drop.

CSS versus XPath: CSS is faster in most browsers, more readable, and enough for 90 percent of cases; XPath earns its place when you need to traverse upward with ancestor or parent, or select by visible text with normalize-space, which CSS cannot do. In Playwright, prefer getByRole and getByLabel, they map to the accessibility tree and survive markup churn.

// Fragile
driver.findElement(By.xpath("(//div[@class='job-card'])[3]//button"));
driver.findElement(By.xpath("/html/body/div[2]/div[4]/span"));
driver.findElement(By.className("css-1x7f9k"));

// Durable
driver.findElement(By.cssSelector("[data-testid='job-card-apply']"));
driver.findElement(By.xpath("//button[normalize-space()='Apply Now']"));
driver.findElement(By.xpath("//span[text()='Razorpay']/ancestor::div[@data-testid='job-card']"));

// Playwright, accessibility first
await page.getByRole("button", { name: "Apply Now" }).click();
await page.getByLabel("Mobile number").fill("9876543210");
await page.getByTestId("job-card-apply").first().click();

Answer

Four things matter in practice. First, Selenium Manager, shipped from Selenium 4.6, which auto discovers or downloads the right driver binary for the browser installed on the machine. It removed the entire class of build failures where the Chrome on the Jenkins agent auto updated overnight and every job died with SessionNotCreatedException about a chromedriver version mismatch.

It also made WebDriverManager, the third party library nearly every Indian project used, optional. In Selenium 4.11 and later you simply write new ChromeDriver() with no setup line. Second, the W3C WebDriver protocol became the only wire protocol, so the old JSON Wire Protocol and the DesiredCapabilities class are gone, replaced by ChromeOptions, FirefoxOptions and EdgeOptions.

Suites written in 2018 fail to compile until this is fixed. Third, relative locators, RelativeLocator.with(By.tagName("input")).below(By.id("password")), which let you locate an element by its visual position relative to another one. They are genuinely useful for forms with no usable attributes, but they depend on rendered geometry, so they can behave differently across viewport sizes and are not something to build a whole suite on.

Fourth, real Chrome DevTools Protocol access through devTools.send(...), which gives you network interception, geolocation and console log capture, and the newer WebDriver BiDi standard which is the cross browser successor. Also worth naming: new window and tab handling via driver.switchTo().newWindow(WindowType.TAB), and full page screenshots on Firefox. Interviewers use this question to tell whether you have actually upgraded a project or only read the release notes.

// Selenium 4.11 and later, no WebDriverManager, no system property
WebDriver driver = new ChromeDriver();

// Options replaced DesiredCapabilities
ChromeOptions options = new ChromeOptions();
options.addArguments("headless=new");
options.setAcceptInsecureCerts(true);
WebDriver headless = new ChromeDriver(options);

// Relative locators
import static org.openqa.selenium.support.locators.RelativeLocator.with;

WebElement otpField = driver.findElement(
    with(By.tagName("input")).below(By.id("mobile")).above(By.id("submit")));

// New tab without JavaScript hacks
driver.switchTo().newWindow(WindowType.TAB);
driver.get("https://goodspace.ai/jobs");

Answer

I pick on four axes: what the team can already write, what the application is, what the CI budget is, and what the existing suite costs to keep. Selenium is the widest, it drives every browser, has bindings for Java, Python, C#, JavaScript and Ruby, works with real mobile browsers through Appium, and is still what most Indian services job descriptions ask for because those projects are Java shops with a decade of investment. Its weakness is that synchronisation is your problem, so every team reinvents waits and every suite is flaky until someone disciplines it.

Playwright is the strongest default for a new web project in 2026: auto waiting built into every action, web first assertions that retry, a single API across Chromium, Firefox and WebKit, browser contexts that give you a clean isolated session in milliseconds instead of a fresh browser launch, tracing with a time travel viewer, and built in parallel workers. It has Java and Python bindings too, though the TypeScript one gets features first. Cypress has the best developer experience for frontend engineers, runs in the browser with time travel debugging, and is popular where the same team writes app code and tests; its constraints are that it is JavaScript only, historically weaker on multi tab and multi origin flows, and parallelisation across machines pushes you toward its paid dashboard.

Practically: greenfield web product, Playwright with TypeScript. Existing large Java and TestNG suite with trained people, stay on Selenium 4 and fix the waits. Frontend team owning its own tests in a React codebase, Cypress is defensible. Say the trade off out loud, do not just declare a favourite.

Answer

The execution order is BeforeSuite, then BeforeTest, then BeforeClass, then for each test method BeforeMethod, the @Test itself, AfterMethod, then AfterClass, AfterTest, AfterSuite. In a real framework BeforeSuite reads config and starts the report, BeforeMethod creates the driver and BeforeClass or BeforeTest handles login state, so that each method gets a clean browser. priority is a simple integer ordering hint within a class, lower runs first, default is 0, and methods with the same priority run in alphabetical order. It does not create a dependency: a priority 1 method still runs even if the priority 0 method failed, so using priority to express ordering of a multi step journey is a mistake that produces cascading confusing failures. dependsOnMethods creates a real dependency: if the depended on method fails, the dependent one is reported as SKIP, not FAIL, which is exactly what you want when the login step failed and running checkout is pointless.

By default it is a hard dependency; add alwaysRun = true to run anyway. groups tag methods into named buckets like smoke, regression, payment or sanity, and you select them from testng.xml or the command line, which is how you run a 12 minute smoke set on every pull request and the full regression nightly. The trap interviewers probe is overusing dependsOnMethods to build long chains, because it makes tests order dependent and impossible to run individually. Prefer independent tests that set up their own state through API calls, and reserve dependencies for genuine preconditions.

public class CheckoutTest extends BaseTest {

  @Test(groups = { "smoke" }, priority = 0)
  public void userCanLogin() {
    Assert.assertTrue(new LoginPage(driver).loginAs("9876543210").isDashboardVisible());
  }

  // SKIPPED, not FAILED, if userCanLogin fails
  @Test(groups = { "regression" }, dependsOnMethods = { "userCanLogin" })
  public void userCanPayWithUpi() {
    Assert.assertEquals(new CheckoutPage(driver).payWithUpi("test@upi"), "SUCCESS");
  }

  @AfterMethod(alwaysRun = true)
  public void captureOnFailure(ITestResult result) {
    if (result.getStatus() == ITestResult.FAILURE) {
      Screenshots.capture(driver, result.getName());
    }
  }
}

Answer

A DataProvider is a method annotated with @DataProvider that returns Object[][] or an Iterator<Object[]>, where each inner array is one set of arguments for one execution of the test. TestNG runs the test once per row and reports each row as a separate result, so a failure tells you exactly which data combination broke. The parameter tag in testng.xml is different in both mechanism and intent: it injects a single static string value from XML into a method annotated with @Parameters, and it is meant for environment level configuration, browser name, base URL, environment tag, not for test data sets.

The practical differences are that a DataProvider is code so it can read from Excel via Apache POI, a CSV, a JSON file, a database or an API and can compute values at runtime, whereas parameters are fixed strings in the suite file. A DataProvider can also be marked parallel = true so its rows execute concurrently, and it can accept a Method or ITestContext argument so the same provider returns different data depending on which test asked for it, which is how you keep one provider serving several tests. The mistakes I look for when reviewing: reading a 4,000 row Excel file inside a DataProvider so collection alone takes a minute, and returning production like personal data in a file committed to git. Keep the provider small, generate data where you can, and pull large or sensitive sets from a seeded API instead.

// DataProvider reading from a JSON fixture, one result row per case
@DataProvider(name = "loginData", parallel = true)
public Object[][] loginData(Method m) {
  return JsonData.rows("testdata/login_" + m.getName() + ".json");
}

@Test(dataProvider = "loginData")
public void loginValidation(String mobile, String otp, String expectedMessage) {
  Assert.assertEquals(new LoginPage(driver).attempt(mobile, otp), expectedMessage);
}

// testng.xml parameter: environment config, not test data
// <suite name="regression">
//   <parameter name="browser" value="chrome"/>
//   <parameter name="baseUrl" value="https://staging.goodspace.ai"/>
//   <test name="checkout">
//     <classes><class name="tests.CheckoutTest"/></classes>
//   </test>
// </suite>

@Parameters({ "browser", "baseUrl" })
@BeforeMethod
public void setUp(String browser, String baseUrl) {
  DriverFactory.init(browser);
  DriverFactory.get().get(baseUrl);
}

Answer

The rule is that the pull request gate must be fast enough that a developer waits for it, which in practice means under 10 to 15 minutes total. So the PR lane gets unit tests, API tests, static analysis, a build, and a smoke set of maybe 25 to 60 UI journeys covering login, search, the primary conversion flow and payment. Anything longer goes elsewhere.

The nightly or scheduled lane runs the full regression across browsers, the long data heavy suites, and the cross device matrix, and it is allowed to take an hour or two because nobody is blocked on it. A third lane, triggered on merge to the release branch or on a deploy to staging, runs the critical path suite against the freshly deployed environment so you know the artifact you are about to promote actually works. The reason not to run everything on every PR is not just time, it is signal: a two hour suite with a 3 percent flake rate will go red on unrelated pull requests, developers will learn to click rerun without reading, and the gate stops meaning anything.

On the mechanics side, in Jenkins this is a declarative pipeline with stages for build, test and publish, running on a labelled agent, archiving the Allure or Extent results and publishing the HTML report. In GitHub Actions it is a workflow with pull_request and schedule triggers, a job matrix for browsers or shards, and artifact upload for reports, videos and traces. The important detail either way is that the pipeline must fail the build on test failure, and must upload evidence on failure, otherwise nobody can debug a red run.

# .github/workflows/e2e.yml
name: e2e
on:
  pull_request:
    branches: [ development ]
  schedule:
    - cron: "30 18 * * *"   # nightly, 00:00 IST

jobs:
  smoke:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: "21"
          cache: maven
      - name: Run smoke suite
        run: mvn test -Dgroups=smoke -Dbrowser=chrome
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: allure-results
          path: target/allure-results

  regression:
    if: github.event_name == 'schedule'
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - run: mvn test -Dgroups=regression -Dshard=${{ matrix.shard }} -DtotalShards=4

Answer

A flaky test is one that produces different results on the same code and the same environment, passing sometimes and failing sometimes without anything changing. It is worse than a failing test, because a failing test gets fixed and a flaky test gets rerun until it goes green, which trains the whole team to ignore red. My first three checks, in order.

One, is it a synchronisation problem: did the test assert on an element before the network call resolved, before the animation finished, or before a React re render replaced the node. This is 60 to 70 percent of flakes in practice, and the tell is that the failure is a NoSuchElementException, a StaleElementReferenceException or an assertion on empty text. Two, is it a test data or state problem: does the test assume a user, a coupon, an order or a job posting exists, and does another test or another engineer's manual run mutate it.

The tell is that it fails in the full suite but passes alone, or fails only when run second. Three, is it environment or timing: a slower CI agent, a different viewport, a container without fonts so the layout shifts, a cron job that runs at midnight, a time zone difference between the agent in UTC and assertions written for IST, or a rate limit on a shared staging API. Beyond that I would look at ordering dependence and parallel interference. The important discipline is that I do not add a retry until I know which of these it is, because a retry on a genuine race condition hides a real product bug that will hit a user on a slow 4G connection.

Answer

For a native HTML select element, Selenium gives you the Select class with selectByVisibleText, selectByValue and selectByIndex, plus getOptions, getFirstSelectedOption and, for multi selects, deselectAll. Prefer selectByValue where you can, because visible text changes with copy edits and localisation and index changes when an option is inserted. Checkboxes and radio buttons are plain clicks, but always read isSelected first rather than blindly clicking, otherwise a test that runs twice toggles it off; the safe pattern is if (!box.isSelected()) box.click().

Browser level dialogs, the JavaScript alert, confirm and prompt, are not in the DOM, so you cannot find them with a locator; you switch to them with driver.switchTo().alert() and then accept, dismiss, getText or sendKeys. Wrap it with ExpectedConditions.alertIsPresent because the dialog appears asynchronously. The part that catches people out is that almost nothing in a modern application is a native select.

React, Angular and Vue component libraries render a button plus a floating div of list items, often in a portal attached to the end of body rather than inside the form. The Select class throws UnexpectedTagNameException there. You handle it as two interactions: click the trigger, wait for the option list to be visible, then click the option by its text or test id, and if the list is virtualised you may have to type into a search input to bring the option into the DOM at all. On Playwright the equivalent is selectOption for native selects, and getByRole with listbox and option roles for the custom case.

// Native select
Select state = new Select(driver.findElement(By.id("state")));
state.selectByValue("KA");
Assert.assertEquals(state.getFirstSelectedOption().getText(), "Karnataka");

// Checkbox, idempotent
WebElement terms = driver.findElement(By.cssSelector("[data-testid='accept-terms']"));
if (!terms.isSelected()) { terms.click(); }

// Browser dialog
new WebDriverWait(driver, Duration.ofSeconds(10))
    .until(ExpectedConditions.alertIsPresent());
Alert alert = driver.switchTo().alert();
Assert.assertTrue(alert.getText().contains("Delete this resume"));
alert.accept();

// Custom React dropdown rendered in a portal
driver.findElement(By.cssSelector("[data-testid='city-trigger']")).click();
new WebDriverWait(driver, Duration.ofSeconds(10)).until(
    ExpectedConditions.visibilityOfElementLocated(By.cssSelector("[role='listbox']")));
driver.findElement(By.xpath("//div[@role='option'][normalize-space()='Bengaluru']")).click();

Answer

The driver should be created and destroyed by the framework, never by a test. In a TestNG framework that means a BaseTest with a @BeforeMethod that asks a DriverFactory for a driver and an @AfterMethod with alwaysRun = true that quits it, so a test that throws still releases the browser. Creating per method gives you a clean browser profile, no leaked cookies and no session bleed between tests, at the cost of a few seconds of launch time per test; creating per class is faster but couples tests together, and you will eventually get a failure caused by leftover state.

The factory itself reads the browser from a system property or config so the same suite runs on Chrome locally and headless Chrome in CI, applies the Options object, sets window size explicitly because a headless agent defaults to a small viewport and your responsive layout will render the mobile navigation, and sets page load and script timeouts. Two mistakes come up constantly. First, calling driver.close() instead of driver.quit(): close shuts one window, quit ends the session and kills the driver process, so a suite using close leaks chromedriver processes until the agent runs out of memory.

Second, holding the driver in a plain static field, which works fine single threaded and breaks immediately under parallel execution because every thread overwrites the same reference. The correct shape is a ThreadLocal in the factory. Also make failure capture part of the lifecycle: take the screenshot and dump the page source in the AfterMethod before you quit, because after quit there is nothing left to capture.

public class BaseTest {

  @BeforeMethod(alwaysRun = true)
  public void startDriver() {
    DriverFactory.create(System.getProperty("browser", "chrome"));
    DriverFactory.get().manage().window().setSize(new Dimension(1440, 900));
  }

  @AfterMethod(alwaysRun = true)
  public void stopDriver(ITestResult result) {
    if (result.getStatus() == ITestResult.FAILURE) {
      Screenshots.capture(DriverFactory.get(), result.getName());
      Artifacts.savePageSource(DriverFactory.get(), result.getName());
    }
    DriverFactory.quit();   // quit, not close
  }
}

Answer

REST Assured gives you a fluent given, when, then syntax over HTTP: given the headers, body and auth, when you post to the path, then assert status code, response time, headers and body fields with Hamcrest matchers or by deserialising into a POJO. A login test asserts a 200, a token present and non empty, the correct user id echoed back, and often a schema match using matchesJsonSchemaInClasspath so a field silently changing type gets caught. The reason to test it at the API layer is cost and precision.

The UI login test takes 15 to 40 seconds, needs a browser, breaks when the design team moves the button, and when it fails you still do not know whether the fault was the frontend, the network or the auth service. The API test takes 200 milliseconds, needs no browser, runs on every pull request, and when it fails it points directly at the auth service. So you keep exactly one UI login test to prove the screen wires up correctly, and you push the 20 validation cases, wrong OTP, expired OTP, blocked user, rate limit exceeded, malformed mobile number, to the API layer where each is a fast independent test. The second reason is that API calls become your setup mechanism: the UI checkout test logs in through the API, grabs the token, injects it into local storage or as a cookie, and starts the browser already authenticated, which removes 30 seconds and one whole class of flakiness from every UI test in the suite.

import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;

public class LoginApiTest {

  @Test
  public void validOtpReturnsToken() {
    given()
        .baseUri(Config.get("apiBaseUrl"))
        .contentType("application/json")
        .body("{\"mobile\":\"9876543210\",\"otp\":\"000000\"}")
    .when()
        .post("/api/v1/auth/verify-otp")
    .then()
        .statusCode(200)
        .time(lessThan(1500L))
        .body("token", not(emptyOrNullString()))
        .body("user.mobile", equalTo("9876543210"));
  }

  @Test
  public void expiredOtpIsRejected() {
    given().baseUri(Config.get("apiBaseUrl")).contentType("application/json")
        .body("{\"mobile\":\"9876543210\",\"otp\":\"111111\"}")
    .when().post("/api/v1/auth/verify-otp")
    .then().statusCode(401).body("code", equalTo("OTP_EXPIRED"));
  }
}

Answer

Both produce HTML reports. ExtentReports is a library you call from listeners, you build the report yourself, log steps, attach screenshots, and it is very common in Indian services projects because it is easy to bolt onto an existing TestNG suite. Allure is a reporting framework: your runner writes JSON result files, the Allure command line renders them into a site, and it gives you history and trends across runs, categories of defects, severity labels, and links to the test management or bug tracker via @Issue and @TmsLink annotations.

If you need trend lines showing flake rate and pass rate over the last 30 builds, Allure is the better answer. What a delivery manager reads is not what an engineer reads. The engineer wants the stack trace, the screenshot at failure, the page source, the browser console log and, on Playwright, the trace file that lets them replay the run.

The manager reads four things: did the build pass, how long did it take, which business areas failed, and is it getting better or worse. So the top of the report should carry pass and fail counts, total duration, a breakdown by feature or module rather than by class name, and the trend. Two practices make reports actually useful: attach the screenshot and the failure evidence automatically from a TestNG listener or an AfterMethod so nobody has to remember, and tag every test with the module and severity so the summary groups by something the business recognises, Payments, Onboarding, Search, not com.company.tests.regression.Batch7.

// Allure annotations that make the report readable to non engineers
@Epic("Payments")
@Feature("UPI checkout")
@Severity(SeverityLevel.CRITICAL)
@Issue("PAY-2481")
public class UpiCheckoutTest extends BaseTest {

  @Test
  @Story("User pays a subscription with UPI")
  public void upiPaymentSucceeds() {
    Allure.step("Open checkout", () -> new CartPage(driver).proceedToCheckout());
    Allure.step("Pay with UPI", () -> new CheckoutPage(driver).payWithUpi("test@upi"));
    Assert.assertEquals(new OrderPage(driver).status(), "SUCCESS");
  }

  @Attachment(value = "screenshot", type = "image/png")
  public byte[] screenshot() {
    return ((TakesScreenshot) DriverFactory.get()).getScreenshotAs(OutputType.BYTES);
  }
}

Answer

Test count is an activity metric, not an outcome metric. You can double it in a month by parametrising a form validation test and change nothing about product quality, and it actively rewards the wrong behaviour because a bigger suite is slower and more expensive to maintain. The metrics I would report are these.

Defect escape rate: how many bugs reached production that the suite should have caught, tracked per release. This is the metric that answers the only question the business asks, is automation preventing incidents. Suite runtime, both the pull request lane and the full regression, because if the gate creeps past 15 minutes developers start bypassing it.

Flake rate, measured as the percentage of test executions that changed result without a code change, with a target under 1 percent, because that number determines whether anyone trusts the report. Coverage of critical user journeys, expressed as a checklist of business flows rather than a code coverage percentage: signup, login, search, apply, payment, refund, either automated end to end or not. Mean time to diagnose a failure, since a suite that takes 40 minutes to triage is not saving anyone time.

And automation maintenance effort as a share of the QA sprint, because when that crosses 30 percent the framework is telling you something. Code coverage percentage is worth tracking at unit level but is a poor automation metric, an end to end suite can execute 80 percent of lines and assert almost nothing. When a manager asks for a number, give the escape rate and the regression cycle time, those two move budget.

Answer

Smoke is the build acceptance check: a shallow, broad set that proves the deployment is not fundamentally broken, the app loads, login works, the main pages render, the primary transaction completes. If smoke fails, you reject the build and nobody wastes time testing further. It should be small and fast, 20 to 60 tests, under 10 minutes, and it should be the most stable code in your repository because a flaky smoke suite blocks everyone.

Sanity is narrow and deep: after a specific fix or a small change, you verify that one area works, plus its immediate neighbours. It is often not fully automated because its scope changes with the change under test. Regression is broad and deep: everything you have, run to confirm that new work did not break old behaviour, and it is the suite that grows every sprint and needs the most pruning discipline.

Deciding what goes where: smoke gets the flows where failure means the product is unusable or unable to earn revenue, which for a jobseeker product like Goodspace means signup, login, job search, apply and the payment checkout. Regression gets everything else including edge cases, validation rules, permission matrices and browser specific behaviour. In the pipeline, smoke runs after every deploy to any environment and on every pull request, regression runs nightly or on release branches. Tag the tests with TestNG groups or Playwright grep tags rather than maintaining separate folders, because separate folders drift and you end up with duplicate copies of the same test.

Answer

First I make the failure reproducible instead of anecdotal. I run the single test on the agent in a loop, 30 or 50 times, using TestNG invocationCount or a shell loop, and record how often it fails and with what exception. A flake you cannot reproduce on demand cannot be fixed, only guessed at.

Second I collect evidence at the moment of failure, which means the AfterMethod already captures a screenshot, the page source, the browser console log via the logging preferences or CDP, and on Playwright the trace file. Nine times out of ten the screenshot alone tells the story: a spinner still visible, an empty results grid, a cookie banner covering the button, a mobile layout because the headless viewport defaulted to 800 by 600. Third I ask what is different about the agent, and I check it rather than assuming: CPU and memory contention because eight jobs share the node, so the app renders slower and a 5 second wait is not enough; a headless browser where animations and lazy loaded images behave differently; the agent clock in UTC while the assertion expects IST, which breaks anything comparing a displayed date; missing fonts in the container changing text width and therefore element positions; and network latency to a staging API that is fast from the office.

Fourth I check isolation: does it fail only when the full suite runs, which points at shared test data or parallel interference rather than the test itself. Only after I know the category do I fix it, usually by replacing a positional or timing assumption with an explicit wait on the real condition, or by making the test create its own data. Retry is the last resort and it goes on with a ticket and an expiry, never quietly.

// Step 1: make it reproducible on the agent
@Test(invocationCount = 30, threadPoolSize = 1)
public void searchReturnsResults() { ... }

// Step 2: capture real evidence, not just the stack trace
@AfterMethod(alwaysRun = true)
public void onFailure(ITestResult result) {
  if (result.getStatus() != ITestResult.FAILURE) return;
  WebDriver d = DriverFactory.get();
  Artifacts.png(((TakesScreenshot) d).getScreenshotAs(OutputType.BYTES), result.getName());
  Artifacts.text(d.getPageSource(), result.getName() + ".html");
  for (LogEntry e : d.manage().logs().get(LogType.BROWSER)) {
    Artifacts.append(result.getName() + ".console.log", e.getLevel() + " " + e.getMessage());
  }
}

// Playwright equivalent: trace on first retry, then open the trace viewer
// playwright.config.ts
// use: { trace: "on-first-retry", video: "retain-on-failure", screenshot: "only-on-failure" }

Answer

TestNG runs parallel tests on separate threads inside one JVM. A static WebDriver field is one memory slot shared by every thread, so when thread B starts and assigns its own ChromeDriver to that field, thread A's reference is silently replaced. Thread A then drives thread B's browser, and you see the classic symptoms: a test navigating to a page it never requested, assertions failing on data belonging to another test, NoSuchSessionException when a thread quits the driver another thread is still using, and failures that move around between runs so nobody can reproduce them.

ThreadLocal fixes it by giving each thread its own value behind the same API: ThreadLocal<WebDriver> holds a separate driver per thread, get() returns the calling thread's instance, and remove() clears it. The factory pattern is create, get, quit, where quit calls driver.quit() and then tl.remove(), and the remove is not optional. If you leave the entry in place, the thread pool reuses the thread for the next test and hands it a dead session, plus you leak memory across a long run.

The same discipline applies to anything else per test: the ExtentTest node, the current user or token, and the test data context all need ThreadLocal or a per test context object rather than statics. Two follow ups interviewers ask. What thread count is safe: it is bound by agent CPU and memory, roughly one browser per available core with 2 GB per Chrome instance, so a 4 core, 8 GB agent handles 3 to 4 comfortably and more just makes everything slower and flakier. And are your tests actually independent, because ThreadLocal fixes driver sharing but not two tests both mutating the same seeded user.

public final class DriverFactory {

  private static final ThreadLocal<WebDriver> TL = new ThreadLocal<>();

  public static void create(String browser) {
    WebDriver driver;
    if ("firefox".equalsIgnoreCase(browser)) {
      driver = new FirefoxDriver();
    } else {
      ChromeOptions o = new ChromeOptions();
      if (Boolean.getBoolean("headless")) o.addArguments("headless=new");
      o.addArguments("window-size=1440,900");
      driver = new ChromeDriver(o);
    }
    TL.set(driver);
  }

  public static WebDriver get() {
    WebDriver d = TL.get();
    if (d == null) throw new IllegalStateException("Driver not initialised on " + Thread.currentThread().getName());
    return d;
  }

  public static void quit() {
    WebDriver d = TL.get();
    if (d != null) { d.quit(); TL.remove(); }   // remove() is mandatory
  }
}

Answer

The parallel attribute on the suite tag tells TestNG what unit to distribute across threads, and thread-count caps how many run at once. parallel = methods runs every @Test method on its own thread, which gives maximum concurrency and demands complete test independence, including no shared instance fields on the test class since methods of the same class then run concurrently on the same instance. parallel = classes runs each class on one thread, so methods inside a class stay sequential and can safely share instance state, which is the setting most Java suites should use because it maps naturally to a BaseTest with a per method driver. parallel = tests runs each test tag concurrently, which is how you parallelise by browser or by module, one test tag per browser with a parameter. parallel = instances is for factory created instances of the same class. There is also a separate thread pool control on the @Test annotation itself, threadPoolSize with invocationCount, and parallel = true on a DataProvider which parallelises the rows of one provider and uses the data provider thread count rather than the suite thread count, a distinction that trips people up when they set thread-count to 8 and nothing speeds up. Practical guidance: start with parallel = classes and thread-count 3 or 4, measure, and only push higher if the agent has the cores. Before switching anything to parallel, prove independence by running the suite in a shuffled order, because parallel execution mostly just exposes ordering dependencies you already had.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="regression" parallel="classes" thread-count="4"
       data-provider-thread-count="4" verbose="1">

  <listeners>
    <listener class-name="listeners.RetryListener"/>
    <listener class-name="io.qameta.allure.testng.AllureTestNg"/>
  </listeners>

  <test name="chrome">
    <parameter name="browser" value="chrome"/>
    <groups><run><include name="regression"/></run></groups>
    <packages><package name="tests.*"/></packages>
  </test>

  <test name="firefox">
    <parameter name="browser" value="firefox"/>
    <groups><run><include name="smoke"/></run></groups>
    <packages><package name="tests.*"/></packages>
  </test>
</suite>

Answer

Playwright performs actionability checks before every action. Before a click it waits for the element to be attached to the DOM, visible, stable meaning it has stopped moving between animation frames, able to receive events meaning nothing is covering it at the click point, and enabled. It retries these checks until they all pass or the timeout expires, so page.getByRole('button', { name: 'Apply' }).click() already contains the wait that a Selenium test has to write by hand with WebDriverWait and elementToBeClickable.

The stability check is the one that solves a whole category of Selenium flakes, the element that exists and is visible but is still sliding in from a CSS transition so the click lands on empty space. Web first assertions are the assertion side of the same idea: expect(locator).toBeVisible() or expect(locator).toHaveText('SUCCESS') poll and retry until the condition holds or the timeout is hit, whereas a plain expect(await locator.textContent()).toBe('SUCCESS') reads once and fails immediately if the value has not arrived. That distinction is the single most common review comment on a Playwright suite, because the non retrying form looks correct and is a race condition.

Two things auto waiting does not solve, and interviewers like to hear you say so. It cannot know about your application's own idea of readiness, so if a spinner disappears before data renders you still need an explicit wait on the data itself. And it does not wait for network by default, so for a page that loads results asynchronously you use page.waitForResponse on the specific API call or assert on the rendered result, not a blanket networkidle wait, which is discouraged because it is flaky on pages with polling or analytics beacons.

// Selenium: the wait is your job
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(15));
wait.until(ExpectedConditions.elementToBeClickable(By.cssSelector("[data-testid='apply']"))).click();
wait.until(ExpectedConditions.textToBe(By.cssSelector("[data-testid='status']"), "APPLIED"));

// Playwright: actionability is built in, assertions retry
import { test, expect } from "@playwright/test";

test("jobseeker can apply to a job", async ({ page }) => {
  await page.goto("/jobs/senior-qa-bengaluru");
  await page.getByRole("button", { name: "Apply Now" }).click();

  // retries until it holds or times out
  await expect(page.getByTestId("application-status")).toHaveText("APPLIED");

  // wait on the specific call, not networkidle
  const res = await page.waitForResponse(r =>
    r.url().includes("/api/v1/applications") && r.status() === 201);
  expect((await res.json()).jobId).toBe("senior-qa-bengaluru");
});

// Wrong: reads once, races the network
// expect(await page.getByTestId("application-status").textContent()).toBe("APPLIED");

Answer

A shadow DOM is an encapsulated subtree attached to a host element, used by web components and by design systems built on Lit or Stencil, and increasingly by browser rendered controls. A normal CSS selector or XPath cannot cross the boundary, so driver.findElement on an element inside the shadow tree throws NoSuchElementException even though you can see it in DevTools. In Selenium 4 the supported route is getShadowRoot() on the host element, which returns a SearchContext you can call findElement on with a CSS selector, and you chain that for nested shadow roots.

Two limits matter: XPath does not work inside a shadow root, only CSS, and getShadowRoot only reaches open shadow roots. A closed shadow root is not accessible through WebDriver at all, and the only options then are a JavaScript hook the developers expose, or asking them to open it, which is a legitimate answer in an interview. Before Selenium 4 people used a JavascriptExecutor returning shadowRoot, and you still see that in older suites.

Playwright pierces open shadow DOM automatically for CSS and for its role and text based locators, so getByRole and a plain CSS selector just work and you rarely think about it, which is one of the concrete reasons teams with component heavy design systems migrate. Cypress needs includeShadowDom set in config or per command. In all cases, iframes are a separate concern and shadow piercing does not cross an iframe boundary, you still have to switch frames first.

// Selenium 4: getShadowRoot, CSS only, open roots only
WebElement host = driver.findElement(By.cssSelector("gs-date-picker"));
SearchContext shadow = host.getShadowRoot();
WebElement input = shadow.findElement(By.cssSelector("input.date-input"));
input.sendKeys("17/08/2026");

// Nested shadow roots: chain
SearchContext inner = shadow.findElement(By.cssSelector("gs-calendar")).getShadowRoot();
inner.findElement(By.cssSelector("[data-day='17']")).click();

// Legacy fallback still seen in old suites
SearchContext legacy = (SearchContext) ((JavascriptExecutor) driver)
    .executeScript("return arguments[0].shadowRoot", host);

// Playwright: pierces open shadow DOM by default
await page.locator("gs-date-picker input.date-input").fill("17/08/2026");
await page.getByRole("button", { name: "17" }).click();

Answer

These are three separate context switches and mixing them up is the usual bug. An iframe is a nested browsing context inside the same page: you must call driver.switchTo().frame(...) by index, by name or id, or by WebElement before any locator inside it will resolve, and you must call switchTo().defaultContent() to get back, or switchTo().parentFrame() to go up one level. A very common Indian scenario is exactly this, the Razorpay or PayU checkout renders inside an iframe, and candidates fail the flow because they search for the card field on the parent document.

Prefer switching by WebElement wrapped in an explicit wait using frameToBeAvailableAndSwitchToIt, since iframe ids are often generated per session. Tabs and windows are separate handles: driver.getWindowHandles() returns a set, you diff it against the handle you started with to find the new one, switch to it, act, close it, then switch back explicitly, because after closing a window the driver has no current window and the next command throws NoSuchWindowException. Wait for the new handle to appear rather than assuming it is instant, ExpectedConditions.numberOfWindowsToBe(2) is the clean way.

Native dialogs, the alert, confirm, prompt and the basic auth or print dialog, are outside the DOM entirely, handled through switchTo().alert(). In Playwright all three are much simpler: frameLocator for iframes, a page event on the browser context for a popup so you await the new page object, and a dialog event handler you register before the action that triggers it, since Playwright auto dismisses dialogs unless you attach a listener.

// iframe: wait and switch by element, never by hardcoded index
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(20));
wait.until(ExpectedConditions.frameToBeAvailableAndSwitchToIt(
    By.cssSelector("iframe.razorpay-checkout-frame")));
driver.findElement(By.cssSelector("[name='card_number']")).sendKeys("4111111111111111");
driver.switchTo().defaultContent();

// new tab: diff the handles
String parent = driver.getWindowHandle();
driver.findElement(By.cssSelector("[data-testid='pay-via-netbanking']")).click();
wait.until(ExpectedConditions.numberOfWindowsToBe(2));
for (String h : driver.getWindowHandles()) {
  if (!h.equals(parent)) { driver.switchTo().window(h); break; }
}
driver.findElement(By.id("bank-submit")).click();
driver.close();
driver.switchTo().window(parent);   // mandatory after close

// Playwright
const frame = page.frameLocator("iframe.razorpay-checkout-frame");
await frame.getByPlaceholder("Card number").fill("4111111111111111");

const [bankTab] = await Promise.all([
  page.context().waitForEvent("page"),
  page.getByTestId("pay-via-netbanking").click(),
]);
await bankTab.getByRole("button", { name: "Submit" }).click();

page.on("dialog", d => d.accept());   // register before triggering

Answer

The rule is that every test creates the state it needs and cleans up after itself, and it never depends on a record someone seeded six months ago. The reason is that shared staging environments are shared: another team truncates a table, a manual tester spends the credits on your test account, a nightly job expires the coupon, and your test fails for reasons that have nothing to do with the code under test. Creating state through the UI is the wrong way to do it, because a checkout test that clicks through signup, profile completion, resume upload and plan selection just to reach the payment screen spends 3 minutes and inherits the flakiness of four unrelated pages, and when signup breaks, your payment test fails and reports the wrong thing.

So setup goes through the API: call the signup endpoint, call the profile endpoint, upload the resume via the file endpoint, get a token, inject it into local storage or as a cookie, then open the browser directly on the checkout URL. That takes 2 seconds and fails only for reasons the test cares about. Where the API cannot express the state, seed the database directly with a SQL fixture, but keep it in the framework behind a data service so tests do not embed SQL.

Two more disciplines. Unique data per run: suffix emails and mobile numbers with a run id or timestamp so parallel runs and reruns never collide, and prefer generated data over a fixed pool. And PII: Indian test data routinely ends up carrying real names, real mobile numbers, PAN and Aadhaar shaped identifiers pulled from a production dump, which is both a DPDP Act problem and a security review finding. Use masked or synthetic data, keep the generators in code, and never commit a customer extract into the test repository.

// Setup via API, then start the browser already authenticated
public class TestDataService {

  public static Session seededJobseeker() {
    String suffix = System.currentTimeMillis() + "-" + Thread.currentThread().getId();
    String mobile = "98" + suffix.substring(suffix.length() - 8);

    Response r = given().baseUri(Config.apiBase()).contentType("application/json")
        .body(Map.of("mobile", mobile, "name", "QA User " + suffix))
        .post("/api/v1/test/seed-jobseeker")
        .then().statusCode(201).extract().response();

    return new Session(r.path("token"), r.path("userId"), mobile);
  }
}

// Inject the token so the UI test skips login entirely
Session s = TestDataService.seededJobseeker();
driver.get(Config.baseUrl() + "/blank");
((JavascriptExecutor) driver).executeScript(
    "window.localStorage.setItem('token', arguments[0]);", s.token());
driver.get(Config.baseUrl() + "/premium/cart");

Answer

Configuration must come from outside the code, resolved in a documented precedence order, and secrets must never sit in the repository. The layered approach I use: a properties or YAML file per environment holding non secret values, base URL, API base, timeouts, feature flags, browser default; a system property or environment variable that overrides any of those, so CI can pass the environment name and the browser without editing files; and secrets injected only as environment variables from the CI secret store, Jenkins credentials binding, GitHub Actions secrets, AWS Secrets Manager or Vault. The config loader reads env, then system property, then file, then a default, and it fails fast with a clear message if a required key is missing rather than sending requests to null.

Practical rules that come up in review. Never commit a .env or a config with a real token, and add a pre commit or secret scanning check because it will happen at least once. Never point an automated write test at production; production gets a read only smoke suite with a clearly marked test account and no destructive actions, and destructive tests are guarded by an assertion on the environment name.

Keep timeouts configurable per environment because a shared staging box is genuinely slower than local and hardcoding 5 seconds guarantees CI flakes. And put the resolved environment name in the report header, because half the failed run investigations in a services project end with someone realising the suite ran against the wrong environment.

public final class Config {
  private static final Properties FILE = load(
      "config/" + resolveEnv() + ".properties");

  private static String resolveEnv() {
    return firstNonBlank(System.getenv("TEST_ENV"),
                         System.getProperty("env"), "staging");
  }

  public static String get(String key) {
    String v = firstNonBlank(System.getenv(key.toUpperCase().replace('.', '_')),
                             System.getProperty(key),
                             FILE.getProperty(key));
    if (v == null) throw new IllegalStateException("Missing config key: " + key);
    return v;
  }

  public static void assertNotProduction() {
    if ("production".equals(resolveEnv()))
      throw new IllegalStateException("Destructive test blocked on production");
  }
}

// config/staging.properties  (no secrets here)
// baseUrl=https://staging.goodspace.ai
// apiBaseUrl=https://staging-api.goodspace.ai
// defaultTimeoutSeconds=20

// secrets arrive as env vars from the CI store
// RAZORPAY_TEST_KEY, DB_PASSWORD, OTP_BYPASS_TOKEN

Answer

Selenium Grid 4 gives you a hub and node topology, or the newer distributed mode with router, distributor, session map, queue and nodes, and you can run it standalone for simple cases. Self hosting is cheapest per hour and gives you full control and access to internal environments that are not exposed to the internet, which matters a lot for a staging system behind a VPN. The cost is that someone has to own it: nodes go stale, browser versions drift, a hung session eats a slot, and you cannot test Safari on macOS or real iOS devices without buying Apple hardware.

Docker changes the operational story, the selenium docker images plus docker compose or a Kubernetes deployment give you disposable, versioned nodes, a video recorder container, and a clean node per session, which removes most of the staleness problem. That is what I would run for Chrome and Firefox at scale on an internal environment. Cloud grids buy you the two things you cannot easily self host: the real browser and OS matrix including Safari, Edge on Windows and older versions, and real Indian mobile devices for Appium.

They also give you video, logs and a dashboard your manager can open. The honest Indian market picture is that services projects on client budgets very often have BrowserStack or LambdaTest already, because LambdaTest is India based and priced competitively and BrowserStack is an Indian company with strong enterprise presence, and per parallel session pricing means teams buy 5 or 10 parallels and queue everything through them. Product startups more often run Playwright in GitHub Actions containers, where the browsers ship with the tool and a grid is not needed at all, and buy a cloud device farm only for the mobile app.

Answer

TestNG was built for exactly this use case, so it ships things an end to end suite needs out of the box: XML suite files that define what runs without touching code, groups for smoke and regression selection, dependsOnMethods for genuine preconditions, DataProvider including parallel rows, built in parallel execution at suite, test, class and method level, IRetryAnalyzer for controlled retries, and a rich listener model, ITestListener, IInvokedMethodListener, ISuiteListener, that reporting layers like Allure and ExtentReports hook into. JUnit 5 is the stronger unit testing framework and has caught up considerably: the Jupiter model with extensions instead of runners, nested tests, ParameterizedTest with argument sources, tags for filtering, dynamic tests, assumptions, and parallel execution configured through junit-platform.properties. It integrates more naturally with Spring Boot testing, and if your organisation already uses it for unit tests, using it for integration tests keeps one toolchain.

The practical decision in India is close to settled: Selenium plus TestNG plus Maven is the stack in the overwhelming majority of services job descriptions, so a candidate who only knows JUnit will fail keyword screening. For a new suite I would still pick TestNG for UI end to end work because suite XML, groups and the retry and listener model are exactly what a regression pipeline needs, and JUnit 5 for unit and Spring integration tests, and I would not consider it a religious question. What interviewers actually check is whether you know that TestNG SKIPs dependent methods, that its parallelism is configured in XML, and that retry logic in TestNG is IRetryAnalyzer while JUnit 5 has no built in retry at all without an extension.

Answer

I agree to a narrow, visible, expiring retry and I refuse a blanket one. A blanket retry across the whole suite does one thing reliably: it converts genuine intermittent product bugs into green builds. If a payment confirmation renders correctly 4 times out of 5 because of a race between the webhook and the UI poll, that is not a test problem, that is a user in Indore seeing a failed payment screen for a payment that actually succeeded, and a retry deletes the only signal you had.

So the deal I offer is this. Retries are opt in per test, not global, applied through IRetryAnalyzer attached only to tests on a known flake list. Every retried test carries a ticket id and a date, and the list is reviewed every sprint, so a retry is a loan not a gift.

Every retry is recorded and reported: the report must show that a test passed on attempt 2, and the dashboard must show flake rate as a first class metric, because a suite where 40 tests pass on retry is not a healthy suite even though the build is green. Second, I would put a quarantine lane in place instead of retries wherever possible: the flaky test is moved out of the blocking suite into a non blocking job that still runs and still reports, so the gate stays trustworthy while the test gets fixed, and there is a rule that a quarantined test is either fixed or deleted within two sprints. Deleting a test that nobody will fix is better than keeping a red one everyone ignores. Finally, I would report the flake list upward, because a persistent flake almost always maps to a real product race condition.

public class RetryAnalyzer implements IRetryAnalyzer {
  private int attempt = 0;
  private static final int MAX = 1;   // one extra attempt, not three

  @Override
  public boolean retry(ITestResult result) {
    if (attempt < MAX) {
      attempt++;
      Reporter.log("RETRY " + result.getName() + " attempt " + attempt, true);
      Flake.record(result.getName());   // feeds the flake rate dashboard
      return true;
    }
    return false;
  }
}

// Opt in per test, with a ticket and an expiry, never suite wide
@Test(retryAnalyzer = RetryAnalyzer.class)
@FlakeWaiver(ticket = "QA-4412", expires = "2026-09-30")
public void walletBalanceUpdatesAfterRefund() { ... }

// Quarantine lane: still runs, does not block the merge
// <test name="quarantine"><groups><run><include name="quarantined"/></run></groups></test>

Answer

Upload: if the page uses a real input with type file, do not click it and do not try to drive the operating system dialog, because WebDriver cannot see native dialogs. Send the absolute path directly to the input element with sendKeys, and Selenium hands the file to the browser. The path must be absolute and must exist on the machine running the browser, which is why this breaks on a remote grid; there you attach a LocalFileDetector to the RemoteWebDriver so the file is uploaded to the node first.

If the input is visually hidden behind a styled label, which most modern designs do, sendKeys still works on the hidden input, you just have to locate the input rather than the label. If the page uses a drag and drop zone with no input at all, you either dispatch a synthetic DataTransfer drop event through JavascriptExecutor or, better, ask the developers to keep a real input in the DOM. Playwright makes this cleaner with setInputFiles, and it also has a filechooser event for the no input case.

Download: configure the browser rather than fighting it, set download.default_directory in Chrome preferences to a per test temp folder, disable the PDF viewer with plugins.always_open_pdf_externally so PDFs actually download, then poll that directory for the file to appear and for its size to stop changing, because a partially written file is a classic flake. Then validate the content, not just its existence: for a resume PDF, open it with PDFBox and assert on the extracted text, page count and that it is not zero bytes. On a remote grid you cannot see the node's filesystem, so either use the Selenium 4 downloads endpoint, or bypass the browser and fetch the download URL with an HTTP client carrying the session cookies.

// Upload: sendKeys on the input, absolute path
WebElement input = driver.findElement(By.cssSelector("input[type='file']"));
input.sendKeys(Paths.get("src/test/resources/files/resume.pdf").toAbsolutePath().toString());

// Remote grid: file must reach the node
((RemoteWebDriver) driver).setFileDetector(new LocalFileDetector());

// Download: configure Chrome, then poll for a settled file
Map<String, Object> prefs = new HashMap<>();
prefs.put("download.default_directory", downloadDir.toAbsolutePath().toString());
prefs.put("plugins.always_open_pdf_externally", true);
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", prefs);

// Wait for the file to exist AND stop growing
Path pdf = Files.wait(downloadDir, "resume-*.pdf", Duration.ofSeconds(30));

// Validate content, not just existence
try (PDDocument doc = PDDocument.load(pdf.toFile())) {
  String text = new PDFTextStripper().getText(doc);
  Assert.assertEquals(doc.getNumberOfPages(), 2);
  Assert.assertTrue(text.contains("Senior QA Automation Engineer"));
}

// Playwright
await page.setInputFiles("input[type=file]", "tests/files/resume.pdf");
const [download] = await Promise.all([
  page.waitForEvent("download"),
  page.getByRole("button", { name: "Download PDF" }).click(),
]);
await download.saveAs("artifacts/resume.pdf");

Answer

You do not automate the third party, you automate around it, and you get agreement from engineering to make the environment testable. For OTP, which gates almost every Indian login and payment, the options in order of preference are: a fixed bypass OTP for numbers in a designated test range on non production environments, so 000000 always works for 98xxxxxx numbers, which is what most Indian product teams actually implement; an internal API or admin endpoint that returns the last OTP issued for a mobile number, which the test calls and then types; reading it from the database or a Redis key through a test data service; or, for a real device test, a mail or SMS catching service such as a Twilio test number or a Mailosaur inbox for email OTP. For 2FA with a TOTP authenticator, this one is genuinely automatable: store the shared secret for the test account and generate the six digit code in the test with a TOTP library, since that is exactly what the authenticator app does.

For captcha, the correct engineering answer is to disable it in test environments by configuration or by using the reCAPTCHA test site key that always passes, and to never use a captcha solving service, which is against the provider's terms and will not survive a security review. For UPI and payment gateways, use the gateway's sandbox with its documented test credentials, Razorpay test mode with the standard test card and test UPI handle, and never touch a live gateway. The framing that wins the interview: say clearly that testability is a product requirement you negotiate before the sprint, not a hack you invent at the end.

// Preferred: environment level bypass agreed with engineering
String otp = Config.get("otp.bypass");          // "000000" on staging

// Fallback: internal test endpoint returns the last issued OTP
String issued = given().baseUri(Config.apiBase())
    .header("X-Test-Token", System.getenv("OTP_BYPASS_TOKEN"))
    .get("/api/v1/test/last-otp/{mobile}", mobile)
    .then().statusCode(200).extract().path("otp");

// TOTP 2FA is fully automatable from the shared secret
GoogleAuthenticator gauth = new GoogleAuthenticator();
int code = gauth.getTotpPassword(System.getenv("TOTP_SECRET"));
loginPage.enterAuthCode(String.valueOf(code));

// Captcha: use the provider's always-pass test key in non production
// RECAPTCHA_SITE_KEY=6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI  (test key)

// Payments: gateway sandbox only
// Razorpay test mode: card 4111 1111 1111 1111, UPI success@razorpay

Answer

It is worth far less than it was in 2015 and more than zero. Chromium now underpins Chrome, Edge, Opera, Brave and Samsung Internet, so a huge share of what used to be separate targets is one engine, and modern CSS and JavaScript differences between evergreen browsers are small. What still genuinely differs is WebKit, meaning Safari on macOS and every browser on iOS since iOS browsers are all WebKit under the hood, and that is where real bugs appear: date input rendering, flexbox and gap edge cases, video and audio autoplay, scroll behaviour, IndexedDB quirks, and payment sheet behaviour.

So the matrix should be driven by your own analytics, not by a template. For a typical Indian consumer product the traffic is heavily Android Chrome, then desktop Chrome, then iOS Safari at a much smaller share of sessions but often a disproportionate share of paying users in metros, then Edge on Windows from corporate desktops. That gives you a defensible matrix: run the full regression on Chromium only, run the smoke and the critical journeys on WebKit and on Android Chrome viewports, and add Edge if the analytics justify it.

Do not run every test on every browser, that multiplies runtime and maintenance for almost no extra defect discovery, and it is the single most common way a suite becomes too slow to gate anything. Playwright makes the narrow matrix cheap because the same test runs on chromium, firefox and webkit projects with a config change. Also separate cross browser from responsive: most of what teams call cross browser bugs are actually viewport and layout bugs, and those are cheaper to catch with a viewport matrix or visual regression than with three browser engines.

Answer

The question I ask for each check is: what is this test's oracle. If the answer is a business rule, a status code, a calculation, a permission or a data transformation, it belongs at the API layer, because those rules live in the backend and testing them through the browser adds 30 seconds and four failure modes without adding information. If the answer is that the user can complete a journey on screen, that the right thing rendered, that the form wires to the right endpoint, or that a client side state machine works, it belongs at the UI layer.

Concretely, for a job application flow: the rules about who can apply, duplicate application rejection, quota limits and the resulting notification payload are API tests; that a jobseeker can search, open a job, click Apply and see the confirmation is one UI test. That split is what turns a 900 test UI suite into a 90 test UI suite plus 400 fast API tests with better coverage. Playwright's APIRequestContext makes the hybrid case easy, because it can share a browser context's cookies and storage state.

So a test can log in once through the API, reuse the same storage state for both the UI and the API assertions, do the UI action, then assert the backend actually recorded it, which catches the case where the screen shows success but the write failed. It also works the other way: seed a record over the API, then open the UI and assert it renders. Use request as a standalone fixture for pure API tests so they do not launch a browser at all.

import { test, expect, request } from "@playwright/test";

// Pure API test, no browser launched
test("duplicate application is rejected", async ({ request }) => {
  const first = await request.post("/api/v1/applications", {
    data: { jobId: "qa-lead-blr", userId: seededUser.id },
  });
  expect(first.status()).toBe(201);

  const second = await request.post("/api/v1/applications", {
    data: { jobId: "qa-lead-blr", userId: seededUser.id },
  });
  expect(second.status()).toBe(409);
  expect((await second.json()).code).toBe("ALREADY_APPLIED");
});

// Hybrid: act in the UI, verify in the backend
test("apply from UI persists on the server", async ({ page, request }) => {
  await page.goto("/jobs/qa-lead-blr");
  await page.getByRole("button", { name: "Apply Now" }).click();
  await expect(page.getByTestId("application-status")).toHaveText("APPLIED");

  const res = await request.get("/api/v1/applications/me");
  const ids = (await res.json()).map((a) => a.jobId);
  expect(ids).toContain("qa-lead-blr");
});

Answer

Appium is a server that speaks the W3C WebDriver protocol and translates your commands into platform automation calls through a driver. On Android that driver is UiAutomator2, which builds on Google's UiAutomator and Instrumentation and locates elements by resource-id, accessibility id, class name, UiSelector or XPath. On iOS it is XCUITest, which drives Apple's own XCUITest framework and uses accessibility id, predicate strings and class chains, and which requires macOS with Xcode, a provisioning profile and code signing for anything on a real device.

From Appium 2 the drivers are installed separately with the appium driver command rather than shipping in the box, and capabilities are vendor prefixed under appium colon, which is the migration that breaks older suites. The practical differences: accessibility id is the only locator that works identically across both, so ask developers to set contentDescription on Android and accessibilityIdentifier on iOS and you can share a large part of your page object layer; XPath works on both and is slow on both, badly so on iOS where the accessibility tree is expensive to walk; and gestures differ, with W3C actions being the portable route and platform specific mobile commands existing for scroll and swipe. Emulators and simulators are right for the bulk of functional regression in CI: they are free, fast to reset to a clean state, scriptable and parallelisable.

Real devices are necessary for anything the emulator cannot honestly reproduce, camera and biometrics, push notifications, real network conditions on Indian 4G, battery and thermal behaviour, performance and jank, UPI intent handoff to an actual PhonePe or Google Pay app, and OEM skin quirks on Xiaomi, Samsung, Vivo and Oppo builds, which matter a great deal in the Indian market. Most teams run emulator based regression and a small real device set on a cloud device farm.

// Appium 2 capabilities, Android
UiAutomator2Options options = new UiAutomator2Options();
options.setPlatformName("Android")
       .setAutomationName("UiAutomator2")
       .setDeviceName("Pixel_7_API_34")
       .setApp("/builds/goodspace-staging.apk")
       .setAppPackage("ai.goodspace.app")
       .setAppActivity(".MainActivity")
       .setNoReset(false)
       .setNewCommandTimeout(Duration.ofSeconds(120));

AndroidDriver driver = new AndroidDriver(new URL("http://127.0.0.1:4723"), options);

// accessibility id is the one locator that ports across platforms
driver.findElement(AppiumBy.accessibilityId("login-mobile")).sendKeys("9876543210");
driver.findElement(AppiumBy.androidUIAutomator(
    "new UiSelector().resourceId(\"ai.goodspace.app:id/submit\")")).click();

// iOS equivalent
// XCUITestOptions ios = new XCUITestOptions()
//     .setPlatformName("iOS").setAutomationName("XCUITest")
//     .setDeviceName("iPhone 15").setPlatformVersion("17.4")
//     .setApp("/builds/Goodspace.app");
// driver.findElement(AppiumBy.iOSNsPredicateString("name == 'login-submit'")).click();

Answer

I would say plainly that the suite is paying the cost of BDD without collecting the benefit, and I would propose one of two directions rather than pretending it is fine. BDD's value is a shared language: business analysts, product owners and developers read and edit Given When Then scenarios, and the conversation happens before the code is written. The cost is a real layer of machinery, feature files, step definition glue with Cucumber Expressions or regex, a runner configuration, a hooks class, and the very specific failure modes that come with it, ambiguous step definitions when two glue methods match one sentence, undefined steps that silently pass in some configurations, scenario outlines that generate more cases than anyone reviews, and step reuse producing sentences like Given I click the third button which is not business language at all, just Selenium in English.

If the business genuinely will not engage, option one is to revive it properly: run a three amigos session per story, write scenarios collaboratively before development, and keep steps at the behaviour level, and measure whether product owners actually edit files within a quarter. If that fails, option two is to migrate off it: keep the feature files as living documentation for the top 40 business flows, and rewrite the rest as plain TestNG or Playwright tests with descriptive names, which removes the glue layer and makes debugging one hop shorter. What I would not do is quietly keep writing feature files because that is what the framework does, since every new scenario adds glue maintenance for an audience of zero. In Indian services projects Cucumber is frequently a contractual deliverable, in which case say so and scope the layer to what the contract requires.

Answer

I would attack it in five passes, measured at every step. First, measure: get per test duration from the TestNG or Playwright report, sort descending, and you will usually find that 20 percent of the tests consume 60 to 70 percent of the runtime, and that a handful contain literal sleeps. Removing Thread.sleep calls and replacing them with explicit waits alone often takes 15 to 20 percent off.

Second, cut setup: every test that logs in through the UI, completes onboarding through the UI, or uploads a file through the UI to reach its actual assertion is spending minutes on someone else's feature. Move that to API seeding plus token injection and start the browser on the target page. This is usually the biggest single win.

Third, parallelise properly: parallel classes with a thread count matched to agent cores, ThreadLocal drivers, and independent data per test. Going from 1 to 6 effective threads is close to a 5x reduction if the tests are genuinely independent, and it is the step that exposes every ordering dependency you had. Fourth, shard across machines: split the suite into N shards by historical duration rather than alphabetically, run them as a CI matrix, and merge the reports.

Four shards times six threads gets a 4 hour suite into the 15 to 25 minute range. Fifth, delete and demote: find tests that have never failed in 200 runs and duplicate coverage that already exists at the API level, and move them down the pyramid or remove them. A suite is a codebase and it needs pruning. Then protect the number by failing the build if the smoke lane crosses its budget, otherwise it silently creeps back.

# Shard by historical duration across a CI matrix, then merge reports
jobs:
  e2e:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - name: Run shard
        env:
          SHARD_INDEX: ${{ matrix.shard }}
          SHARD_TOTAL: 4
        run: npx playwright test
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: blob-report-${{ matrix.shard }}
          path: blob-report

  merge:
    needs: e2e
    if: always()
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with: { pattern: blob-report-*, merge-multiple: true, path: all-blob-reports }
      - run: npx playwright merge-reports ./all-blob-reports

Answer

First I check whether the tool is actually the problem. If the suite is flaky because of hardcoded sleeps, shared test data and a static driver, Playwright will inherit every one of those problems in a new syntax, and I will have spent six months to arrive at the same flake rate. So the honest first step is to measure: flake rate, average test duration, maintenance hours per sprint, and how much of the failure volume is synchronisation.

If the majority is synchronisation and stale elements, Playwright's auto waiting and web first assertions genuinely remove that class, and the case is real. Then the plan is strangler pattern, never big bang. New tests are written in Playwright from day one.

The two suites run side by side in CI as separate jobs with merged reporting, so coverage never dips. Migration proceeds by business area in priority order, starting with the flakiest and most valuable journeys, and each migrated area is deleted from the Selenium suite in the same pull request so the two never drift. Shared assets move first: test data seeding through the API is tool independent, so it is reused rather than rewritten.

I would expect roughly 12 to 18 months for 1,400 tests alongside normal delivery, and I would say that number out loud rather than promising a quarter. The manager pitch is not technical, it is three numbers: PR feedback time drops from X to Y because the suite parallelises natively and runs faster, flake rate drops from X percent to under 1 percent so the gate becomes trustworthy again, and maintenance hours per sprint drop, which is capacity returned to testing. If those numbers are not compelling on your suite, the honest answer is do not migrate, fix the waits and the data instead.

Answer

Self healing works by recording, at authoring or at last successful run, a fingerprint of each element: its attributes, text, position, neighbours and a slice of the DOM around it. When the primary locator misses, the engine scores candidate elements against that fingerprint and picks the closest, then optionally rewrites the locator. Testim, mabl, Functionize, Katalon and Tricentis sell versions of this, and Selenium and Playwright ecosystems have open source and AI assisted equivalents.

It genuinely reduces the churn from cosmetic refactors, a renamed class or a moved div. The limits are the part leadership does not hear. First, a healed locator can be wrong in a way a broken one cannot: if the developer renamed the Save button and added a Delete button in the same place, the healer may bind to Delete and your test now passes while doing something dangerous, so healing without a review step converts hard failures into silent incorrectness.

Second, healing masks a real signal, because a locator breaking is often the earliest indication that the UI contract changed, and if the tool quietly patches it nobody reviews the change. Third, it does not help with the actual majority of maintenance, which is behaviour changes, new required fields, new consent screens and changed business rules, not locator drift. Fourth, AI generated tests today produce plausible shallow assertions, they check that a page loaded rather than that the invoice total is 1,180, and someone still has to know what to assert. My position is: enable healing in report only mode so it tells you what it would have healed and opens a pull request, keep a data-testid contract with developers as the primary defence, and treat AI generation as a first draft accelerator reviewed by a human, not as coverage.

Answer

Visual regression captures a screenshot of a page or component, compares it against an approved baseline, and surfaces the diff for a human to approve or reject. Percy and Chromatic sit close to the component and pull request workflow, Applitools uses what it calls visual AI to compare at a perceptual level rather than pixel by pixel, and Playwright ships toHaveScreenshot for a self hosted version with a configurable pixel threshold. It is the right tool for a class of defect that functional assertions cannot catch: a CSS change that overlaps the price with the button, a font failing to load, a broken responsive breakpoint, a component library upgrade shifting spacing across 40 screens.

The failure mode is noise, and it is severe enough that most abandoned visual suites died of it. The noise sources are predictable: dynamic content such as timestamps, names, job counts and rotating banners; animations and carousels captured mid transition; font rendering and antialiasing differences between your Mac and the Linux CI container; scrollbars; ad or third party iframes; and randomised A/B variants. The controls are equally predictable.

Always capture in the same containerised environment, never compare a local screenshot to a CI baseline. Freeze the clock and seed randomness so dynamic values are deterministic. Mask or ignore known dynamic regions, both Percy and Playwright support masking selectors.

Disable animations globally with a CSS injection. Prefer component level snapshots over full page ones, because a component diff is reviewable and a full page diff is a wall of red. And set an explicit approval workflow with a named owner, because a visual suite where nobody approves baselines degrades to everyone clicking approve all within a month, which is worse than not having it.

// Playwright self hosted visual check with the noise controls applied
import { test, expect } from "@playwright/test";

test.beforeEach(async ({ page }) => {
  // freeze animations and transitions
  await page.addStyleTag({ content: "*,*::before,*::after{animation:none!important;transition:none!important;}" });
  // freeze the clock so relative timestamps are deterministic
  await page.clock.setFixedTime(new Date("2026-08-17T10:00:00+05:30"));
});

test("job card renders correctly", async ({ page }) => {
  await page.goto("/jobs/qa-lead-blr");
  await expect(page.getByTestId("job-card")).toHaveScreenshot("job-card.png", {
    maxDiffPixelRatio: 0.01,
    mask: [page.getByTestId("posted-ago"), page.getByTestId("applicant-count")],
  });
});

// playwright.config.ts: always compare like for like
// snapshotPathTemplate: "{testDir}/__screenshots__/{projectName}/{arg}{ext}"
// run baselines and comparisons in the same container image

Answer

Contract testing verifies the agreement between a consumer and a provider without running both together. In Pact's consumer driven model, the consumer's test declares the request it will make and the response it expects, and runs against a Pact mock provider, which produces a pact file describing that interaction. The provider then replays those interactions against its real implementation in its own build, using provider states to set up the data each interaction assumes, and it fails if it no longer satisfies the contract.

The Pact Broker, or PactFlow, stores the pacts, versions them against git commits and application versions, and answers the question that makes this useful in a pipeline: can I deploy this version of the consumer to this environment given what is currently deployed there. That check is the can-i-deploy step, and it is the actual product. It replaces a whole category of expensive end to end integration tests.

In a microservice estate where a jobseeker app talks to eight services, spinning all eight up in one environment to test integration is slow, flaky and always slightly out of date. Contract tests give each pair fast, isolated feedback in each team's own build and catch the real failure, which is a provider removing or retyping a field a consumer relies on. What contract testing does not do, and interviewers check this: it does not verify business behaviour, performance, or that the whole journey works, so you still keep a thin end to end layer for the critical revenue paths.

It also needs organisational buy in, because the provider team has to run the verification in their pipeline and treat a broken contract as a build failure, and without that agreement the pact files just accumulate unverified. Bi directional contract testing with OpenAPI specifications is the lighter variant when the provider will not adopt consumer driven pacts.

// Consumer side (Pact JVM, JUnit 5)
@ExtendWith(PactConsumerTestExt.class)
@PactTestFor(providerName = "jobs-service")
class ApplicationsConsumerPactTest {

  @Pact(consumer = "jobseeker-web")
  public RequestResponsePact jobDetail(PactDslWithProvider builder) {
    return builder
        .given("job qa-lead-blr exists and is open")
        .uponReceiving("a request for job detail")
          .path("/api/v1/jobs/qa-lead-blr").method("GET")
        .willRespondWith()
          .status(200)
          .body(new PactDslJsonBody()
              .stringType("id", "qa-lead-blr")
              .stringType("title", "QA Lead")
              .booleanType("open", true)
              .numberType("salaryMaxLpa", 22))
        .toPact();
  }

  @Test
  @PactTestFor(pactMethod = "jobDetail")
  void clientParsesJobDetail(MockServer mock) {
    Job job = new JobsClient(mock.getUrl()).fetch("qa-lead-blr");
    assertEquals("QA Lead", job.title());
  }
}

// Pipeline gate against the broker before promoting a build
// pact-broker can-i-deploy \  (single line in CI)
//   pacticipant=jobseeker-web version=$GIT_SHA to-environment=production

Answer

An automation engineer is expected to own performance awareness even when a separate performance team owns the big load tests. That means knowing the vocabulary and being able to run a baseline. The vocabulary interviewers check: throughput in requests per second, concurrency or virtual users, latency reported as percentiles not averages because p95 and p99 are what users feel, error rate under load, ramp up and soak duration, and the difference between a load test at expected traffic, a stress test past the breaking point, a spike test simulating a flash sale or an IPL advertisement, and a soak test that reveals memory leaks over hours.

JMeter is the incumbent in Indian enterprises: a Java tool with a GUI for building plans, a huge plugin ecosystem, protocol coverage beyond HTTP including JDBC, JMS and FTP, and distributed execution across load generators. Its plans are XML, which review and merge badly, and the GUI should be used for authoring only, never for the actual run. k6 is the modern alternative: tests are JavaScript, so they live in git and get reviewed like code, it is written in Go so a single machine drives far more virtual users, thresholds are declared inside the test so the run itself passes or fails, and it integrates cleanly into CI. Gatling occupies similar ground with Scala or Java DSLs.

Where they fit in a pipeline: a full load test does not belong on a pull request, it is too slow and needs a production like environment. What does belong on every build is a small smoke level performance check with thresholds, for example the login and search endpoints must hold p95 under 800 milliseconds at 20 virtual users, which catches the N plus 1 query someone introduced. The full load and soak runs are scheduled, weekly or before a release, on a dedicated environment with a comparable dataset, because load testing against a staging database with 500 rows tells you nothing about a production table with 40 million.

// k6 script lives in git, thresholds make the run self grading
import http from "k6/http";
import { check, sleep } from "k6";

export const options = {
  stages: [
    { duration: "2m", target: 50 },   // ramp up
    { duration: "5m", target: 50 },   // steady
    { duration: "1m", target: 0 },    // ramp down
  ],
  thresholds: {
    "http_req_duration{endpoint:search}": ["p(95)<800", "p(99)<1500"],
    http_req_failed: ["rate<0.01"],
  },
};

export default function () {
  const res = http.get("https://staging-api.goodspace.ai/api/v1/jobs?q=qa&city=bengaluru",
    { tags: { endpoint: "search" } });
  check(res, {
    "status is 200": (r) => r.status === 200,
    "has results": (r) => r.json("results").length > 0,
  });
  sleep(1);
}

Answer

Classic WebDriver is request and response: you ask the browser to do something and it answers. That model cannot tell you about things that happen on their own, a console error, a failed network request, a JavaScript exception, so suites historically polled or scraped logs. Selenium 4 exposed Chrome DevTools Protocol access, which opens a bidirectional channel to Chromium browsers and unlocks a set of things that are genuinely useful for testing: intercepting and stubbing network responses so you can simulate a 500 from the payments API without touching the backend, throttling the network to emulate an Indian 3G connection and prove your loading states work, capturing all console errors and failing the test if any appear, overriding geolocation to test city based job results, injecting basic auth headers, capturing performance metrics, and blocking third party analytics that slow the run.

The catch is that CDP is Chromium only and version specific, which is why a Selenium version supports a small window of CDP versions and you see warnings about a mismatch when Chrome updates ahead of your Selenium dependency. WebDriver BiDi is the standards track answer to exactly that: a W3C bidirectional protocol implemented across Chromium and Firefox, with Safari following, that provides the same class of capability, log events, network interception, script evaluation, in a cross browser way. In Selenium 4 recent versions and Selenium 5 the BiDi APIs are the ones to build on, and CDP is the legacy path being wound down.

Playwright and Puppeteer have had this capability from the start through their own protocol handling, which is precisely why network stubbing and console assertions feel native there and bolted on in Selenium. Mentioning that BiDi replaces CDP is a strong senior signal.

// Fail the test if the app logs any console error, via BiDi
try (LogInspector logs = new LogInspector(driver)) {
  List<String> errors = new CopyOnWriteArrayList<>();
  logs.onConsoleEntry(entry -> {
    if ("error".equalsIgnoreCase(entry.getLevel().toString())) errors.add(entry.getText());
  });

  driver.get(Config.baseUrl() + "/premium/cart");
  new CartPage(driver).proceedToCheckout();
  Assert.assertTrue(errors.isEmpty(), "Console errors: " + errors);
}

// CDP: stub a failing payments API and prove the error state renders
DevTools devTools = ((HasDevTools) driver).getDevTools();
devTools.createSession();
devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
devTools.send(Network.setBlockedURLs(List.of("*/api/v1/payments/initiate")));

// CDP: emulate an Indian 3G connection
devTools.send(Network.emulateNetworkConditions(
    false, 300, 400_000, 400_000, Optional.of(ConnectionType.CELLULAR3G),
    Optional.empty(), Optional.empty(), Optional.empty()));

Answer

That symptom narrows the space immediately: the test is fine, the interaction between tests is not. I would work through four hypotheses in order. One, shared mutable test data.

Two tests use the same user, the same coupon code, the same job posting or the same wallet, and one mutates what the other asserts. The proof is to run just those two together and watch it fail, and the fix is unique seeded data per test with a run scoped suffix. Two, shared framework state.

A static field somewhere: a static WebDriver, a static ExtentTest, a static config map being mutated, a static current user in a utility class, or a singleton API client whose auth token gets overwritten by whichever thread logged in last. The tell is exceptions that mention a session or an object the test never created. Three, server side contention.

A rate limiter that trips at 100 requests a minute per IP and your six threads share the CI egress IP, a database deadlock on the same row, a background job that expires sessions, or a feature flag another test toggles globally, which is the nastiest because the toggle affects unrelated tests for a random window. Four, infrastructure. Six Chrome instances on a 4 core, 8 GB agent means CPU starvation, the app renders slowly, and waits that were adequate become marginal, which shows up as timing failures scattered randomly rather than in one test.

The diagnostic tooling: run with a fixed seed and a recorded order, then bisect by running halves of the suite to find the interacting pair; log a run id and thread name into every request header so you can correlate a server log line to a specific test; and check whether the failure moves when you change the thread count, because a failure that disappears at thread count 1 and worsens at 12 is contention, not logic. The permanent fix is independence, not sequencing, because forcing sequential execution just hides it and doubles your runtime.

Answer

The core decision is to split the framework from the tests. The framework, driver or fixture management, waits and interaction helpers, config resolution, API clients, test data services, reporting and listeners, becomes a versioned library published to an internal Maven or npm registry, owned by a small core group. Each team keeps its own test repository, or its own directory in a monorepo, depending on the shared library at a pinned version, and owns its own tests.

That gives you consistent plumbing without a single repository where five teams' pull requests collide daily. Versioning matters: teams upgrade the library on their own schedule, breaking changes get a major bump and a migration note, and the core group does not get to break everyone on a Tuesday. Second decision is contracts, not conventions, for the things that must be uniform: the locator strategy with data-testid, the config precedence, the reporting tags for module and severity, and the rule that no test may depend on data it did not create.

Enforce these in code review and, where you can, in the framework itself, for example by making the base test refuse to start if the environment is not resolved. Third is CI shape: one reusable workflow or shared Jenkins library that every team calls, so sharding, artifact upload, report merging and the flake dashboard work identically without five copies. Fourth is ownership of quality: each team owns its own suite's green status and flake rate, published on a shared dashboard, because a central QA team owning everyone's tests becomes a bottleneck and the teams stop caring.

The core group owns the library, the CI templates, the dashboard and the standards. Finally, plan for the cross cutting suite: the two or three journeys that span all five teams' services need one owner, otherwise they belong to nobody and rot.

Answer

A gate is only worth having if the team respects it, and a team respects it only when it is fast, deterministic and proportionate. So I would define blocking and non blocking tiers explicitly. Blocking on a pull request: compilation and lint, unit tests, API contract or component tests, and the smoke UI suite, all together under about 15 minutes, plus a hard rule that a new failing test is not merged with a skip annotation.

Blocking on promotion to production: the critical journey suite green against the actual release artifact in staging, the contract check that this consumer version is compatible with what is deployed, a security scan with no new high severity findings, and a performance threshold check on the key endpoints. Non blocking but visible: full cross browser regression, visual diffs pending approval, accessibility scan findings, code coverage movement, and the quarantine lane. Those get reported to the team and tracked, but they do not stop a release, because gating on a two hour flaky suite means people will find a way around it and you lose the gate entirely.

Two things make this work in practice. First, an explicit override path with an audit trail: a named person can force a deploy with a recorded reason, because a gate with no legitimate override becomes a gate people disable. Second, flake budget enforcement: if the blocking suite's flake rate crosses one percent, fixing it becomes the team's priority ahead of new tests, since a flaky blocking gate is worse than no gate. I would also gate on evidence, not just status: the pipeline must publish the report, the trace and the screenshots for every failure, because a red gate nobody can diagnose becomes a rubber stamped override within two weeks.

Answer

Days 1 to 15, measure and choose. I would sit through one full regression cycle to see where the two weeks actually go, count how many of the cases are genuinely re run every release, and find the top five defects that escaped to production in the last quarter, because those tell me what coverage is actually missing. I would also establish the baseline numbers I will be judged on: regression cycle duration, escape count, and the number of releases per quarter.

Tool choice follows the team, if they are Java people, Selenium 4 with TestNG and Maven; if they are a JavaScript frontend team, Playwright with TypeScript. Days 16 to 45, build thin and vertical. I would automate the five to eight highest value journeys end to end, and while doing it build only the framework each one needs: driver or fixture factory, config layer, page or component objects, API based test data seeding, and reporting.

Wire it into CI from the very first test, running on every pull request, because a suite that only runs on someone's laptop dies. The visible deliverable at day 45 is a green pipeline that runs the critical path in under 10 minutes on every merge. Days 46 to 90, widen and prove value.

Push the API layer hard, because that is where coverage gets cheap: 100 API tests in the time 20 UI tests would take. Add the regression areas by risk order, introduce the smoke versus regression split, and start reporting flake rate from day one so it never becomes normal. Train two manual testers to write tests against the framework, because a one person automation effort dies when that person leaves. The 90 day claim I would make is concrete: regression from two weeks to two days with the critical path verified on every merge, not full automation, which would be a promise I could not keep.