API Testing Interview Questions and Answers

Answer

An idempotent method produces the same server state whether you call it once or twenty times. GET, PUT and DELETE are defined as idempotent, HEAD and OPTIONS are too, and POST and PATCH are not. The word is about state, not about the response body: a repeated DELETE legitimately returns 204 the first time and 404 afterwards, and that is still idempotent because the resource is gone either way.

POST means create a new subordinate resource, so every call is intended to make another one. That is exactly why a payment endpoint modelled as POST /payments is dangerous. If the mobile network drops the response after the server has already debited the customer, the client sees a timeout, retries, and the gateway creates a second payment.

On a UPI collect flow that is a real second debit from a real bank account, and the customer sees two entries in the passbook. PUT does not have this problem because it targets a specific URI: PUT /users/8812/address written twice leaves exactly one address, the second write simply overwrites the first with identical content. The industry fix for POST is an idempotency key, a client generated unique header that the server stores with the first response and replays for any repeat, which is what Razorpay and Stripe both do. As a tester your job is to prove the guarantee rather than assume it: fire the same POST twice with the same key, assert exactly one record in the payments table, and assert the second response is the stored one and not a fresh 201.

# Retry without an idempotency key: two payments, two debits
curl -i -X POST https://api.example.in/v1/payments \
  -H "Content-Type: application/json" \
  -d '{"amount": 49900, "currency": "INR", "vpa": "aarav@okhdfcbank"}'
# 201 Created  {"payment_id": "pay_A1", "status": "created"}
# (client times out, retries the exact same call)
# 201 Created  {"payment_id": "pay_A2", "status": "created"}   <- duplicate debit

# Same call, now idempotent
curl -i -X POST https://api.example.in/v1/payments \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: 7f2c1b90-3d2e-4a11-9c55-0b1a2c3d4e5f" \
  -d '{"amount": 49900, "currency": "INR", "vpa": "aarav@okhdfcbank"}'
# 201 Created  {"payment_id": "pay_A1"}
# retry with the SAME key -> 200 OK, same pay_A1, no second debit

# PUT is naturally idempotent
curl -X PUT https://api.example.in/v1/users/8812/address \
  -H "Content-Type: application/json" \
  -d '{"line1": "12 MG Road", "city": "Bengaluru", "pin": "560001"}'
# run it 10 times, still exactly one address row

Answer

400 Bad Request means the server could not understand or parse the request: malformed JSON, a missing required field, a string where an integer belongs, a broken Content-Type. 422 Unprocessable Content means the syntax is fine and the server understood you perfectly, but the content violates a semantic rule. An email that is well formed but already registered is the textbook 422 case, as is a date of birth that parses correctly but makes the user eleven years old, or a transfer amount that is a valid number but exceeds the daily UPI limit. The practical rule I use in bug reports: if a client could fix the request by correcting its serialisation, it is 400; if the client would have to change the meaning of the data or the state of the world, it is 422.

Where this gets contested is that plenty of teams, including large Indian fintechs, return 400 for everything and put the detail in an error code field. That is a defensible convention if it is consistent and documented, and a bad bug report is one that says the status is wrong without asking whether the API has a documented error contract. So I raise it as a consistency defect: this endpoint returns 422 for a duplicate email while the sibling endpoint returns 400 for a duplicate phone number, which forces every client to special case them.

The other half of the assertion is the body. Whichever code the team picks, the response should carry a machine readable code such as EMAIL_ALREADY_REGISTERED and a field pointer, because clients must never string match on human readable messages that a copywriter will change next sprint.

// 400: the server could not parse it
// POST /v1/users   {"email": "aarav@", "age": "twenty"}
{
  "error": {
    "code": "MALFORMED_REQUEST",
    "message": "age must be an integer",
    "field": "age"
  }
}

// 422: parsed fine, business rule says no
// POST /v1/users   {"email": "aarav@example.in", "age": 24}
{
  "error": {
    "code": "EMAIL_ALREADY_REGISTERED",
    "message": "An account with this email already exists",
    "field": "email"
  }
}

// The assertion that actually protects clients
pm.test("duplicate email is a stable machine readable error", function () {
  pm.expect(pm.response.code).to.be.oneOf([400, 422]);
  pm.expect(pm.response.json().error.code).to.eql("EMAIL_ALREADY_REGISTERED");
});

Answer

401 Unauthorized is really unauthenticated: the server does not know who you are. No Authorization header, an expired JWT, a revoked API key, a signature that does not verify. A correct 401 must include a WWW-Authenticate header telling the client what scheme to use, which almost no API actually does, and it is a fair thing to raise. 403 Forbidden means the server knows exactly who you are and is refusing anyway: your token is valid but you lack the scope, your role is viewer and you tried to delete, your account is suspended, or your IP is not allowlisted.

The tester friendly summary is that 401 is fixable by logging in again and 403 is not. The 404 twist is about information leakage. If GET /v1/orders/90211 returns 403 when the order belongs to another customer, the API has just confirmed that order 90211 exists.

An attacker can enumerate the ID space and learn how many orders you process per day, which is competitive intelligence, and for a loan or health record it can be far worse. Returning 404 for resources you are not allowed to see makes not found and not yours indistinguishable, which is why most security reviews recommend it for object level authorization on sensitive resources. As a tester I write both cases explicitly: with a valid token for user A, request user B's order and assert 404 with an empty body, and separately assert that a genuinely nonexistent ID returns the identical status and body shape, because a difference in response time or body length reintroduces the leak.

# No token at all
curl -i https://api.example.in/v1/orders/90211
# 401 Unauthorized
# WWW-Authenticate: Bearer realm="api"

# Valid token, but the read:orders scope is missing
curl -i https://api.example.in/v1/orders/90211 -H "Authorization: Bearer $VIEWER_TOKEN"
# 403 Forbidden   {"error": {"code": "INSUFFICIENT_SCOPE"}}

# Valid token for user A, order belongs to user B
curl -i https://api.example.in/v1/orders/90211 -H "Authorization: Bearer $USER_A_TOKEN"
# 404 Not Found   (identical to a genuinely missing order)

# The test that catches the leak
pm.test("foreign order is indistinguishable from a missing one", function () {
  pm.response.to.have.status(404);
  pm.expect(pm.response.text()).to.eql(pm.collectionVariables.get("missingOrderBody"));
});

Answer

This is the question senior interviewers love, because there is a defensible answer on both sides and they want to hear you reason rather than recite. The purist position: HTTP status describes the outcome of the HTTP request, and a body carrying a failure means the request itself succeeded, the transport worked, the payload parsed, the server processed it and is reporting a business result. By that reading, a payment that the bank declined is a perfectly successful API call about an unsuccessful payment, and 200 is correct.

The pragmatic objection is what actually bites teams: every generic HTTP client, retry wrapper, load balancer, monitoring dashboard and alerting rule treats 2xx as fine. A gateway that reports declines as 200 will show a green success rate dashboard through an outage where every single transaction is failing, and a naive client written by a partner will happily mark the order as paid. The compromise most well designed Indian payment APIs land on is that the HTTP layer reflects the API call and a required status field carries the money outcome, with the contract documented loudly and with declines also surfaced as a distinct error code.

What matters in the interview is that your test never trusts the status code alone. I assert 200 for the transport, then assert the business status explicitly, and I add a negative test that a declined payment does not create a fulfilled order, because the real defect is almost never the status code, it is the downstream system that treated 200 as paid.

pm.test("transport succeeded", function () {
  pm.response.to.have.status(200);
});

pm.test("declined payment is reported as failed, not silently swallowed", function () {
  var body = pm.response.json();
  pm.expect(body).to.have.property("status");
  pm.expect(body.status).to.eql("failed");
  pm.expect(body.error_code).to.eql("BANK_DECLINED");
  pm.expect(body).to.have.property("payment_id");
});

pm.test("a failed payment must NOT mark the order as paid", function () {
  pm.sendRequest({
    url: pm.environment.get("baseUrl") + "/v1/orders/" + pm.collectionVariables.get("orderId"),
    method: "GET",
    header: { Authorization: "Bearer " + pm.environment.get("token") }
  }, function (err, res) {
    pm.expect(res.json().payment_status).to.eql("pending");
  });
});

Answer

Path parameters identify the resource and are part of the URI, so they are mandatory by construction: GET /v1/merchants/{merchantId}/settlements/{settlementId}. Testing focuses on type mismatch (a string where a numeric ID is expected should be 400 or 404, never a 500 stack trace), non-existent IDs, IDs belonging to another tenant, and URL encoding when the identifier contains a slash or a plus. Query parameters filter, sort and paginate, and they are optional far more often, so the tests are about defaults and combinations: what happens with no parameters at all, with limit=0, limit=100000, a negative offset, an unknown sort field, two conflicting filters, and a repeated parameter such as status=paid&status=failed which some frameworks read as a list and others as last wins.

Query strings are also logged by nginx and every proxy in the path, which is why a token or an Aadhaar number in a query parameter is a defect you should raise on sight. Headers carry cross cutting concerns: Authorization, Content-Type, Accept, Idempotency-Key, X-Request-Id, and API version headers. The interesting tests are the missing and wrong cases, for example sending a JSON body with Content-Type: text/plain should be 415 Unsupported Media Type, and an unsupported Accept value should be 406. Body parameters carry the payload for POST, PUT and PATCH, and this is where the volume of testing sits: required versus optional fields, type validation, boundaries, nested objects, empty arrays, nulls versus omitted keys (which mean different things in PATCH), and extra unknown fields, which should be rejected or ignored but never quietly assigned to a privileged column.

# path + query + header + body on one call
curl -i -X POST \
  "https://api.example.in/v1/merchants/MRC_88/settlements?dry_run=true&currency=INR" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: 3a91-..." \
  -H "X-Request-Id: qa-run-4412" \
  -d '{"amount": 250000, "account_id": "acc_77", "notes": {"batch": "aug-payout"}}'

# path: wrong type must not 500
curl -o /dev/null -w "%{http_code}\n" https://api.example.in/v1/merchants/'; DROP TABLE'/settlements

# query: boundary and repeat behaviour
curl -s "https://api.example.in/v1/settlements?limit=0"
curl -s "https://api.example.in/v1/settlements?limit=100000"
curl -s "https://api.example.in/v1/settlements?status=paid&status=failed"

# header: JSON body with the wrong content type -> expect 415
curl -i -X POST https://api.example.in/v1/settlements \
  -H "Content-Type: text/plain" -d '{"amount": 100}'

Answer

A test that only checks status 200 passes on almost any regression, including an endpoint that returns an empty object. A serious API test asserts on six layers. First, the status code and the reason it is that code.

Second, the schema, meaning the response validates against a JSON Schema or the OpenAPI response model, which catches a renamed field, a removed field or a type change from number to string the moment it lands. Third, field types and formats, because schema validation only helps if the schema is specific: amount is an integer in paise, not a float, created_at is an RFC 3339 timestamp, vpa matches the UPI handle pattern. Fourth, business values, the part most candidates skip, which means recomputing the expected result rather than echoing the response back at itself.

If you POST an order for two items at 249 rupees with 18 percent GST, assert the total is 58764 paise, do not assert that total equals response.total. Fifth, headers and non functional properties: Content-Type, cache directives on a personalised endpoint, an X-Request-Id you can grep in logs, and a response time budget appropriate for the endpoint. Sixth, side effects, which is what separates API testing from response shape checking.

Did the row actually land in MySQL, did the Kafka event get published, did the ledger balance change by exactly the amount, did the webhook fire, and just as importantly did nothing else change. I also assert the absence of things, because a response leaking pan or aadhaar_number or password_hash is a defect even when every positive assertion passes.

pm.test("status and timing", function () {
  pm.response.to.have.status(201);
  pm.expect(pm.response.responseTime).to.be.below(800);
  pm.expect(pm.response.headers.get("Content-Type")).to.include("application/json");
  pm.expect(pm.response.headers.get("X-Request-Id")).to.be.a("string");
});

pm.test("business value is recomputed, not echoed", function () {
  var body = pm.response.json();
  var expected = 2 * 24900;
  var withGst = Math.round(expected * 1.18);
  pm.expect(body.subtotal_paise).to.eql(expected);
  pm.expect(body.total_paise).to.eql(withGst);
  pm.expect(body.total_paise).to.be.a("number");
});

pm.test("no PII leakage", function () {
  var raw = pm.response.text();
  ["aadhaar", "pan", "password_hash", "otp"].forEach(function (k) {
    pm.expect(raw.toLowerCase()).to.not.include(k);
  });
});

Answer

SOAP is a protocol, REST is an architectural style, and that difference drives everything. A SOAP service exposes one endpoint over POST, carries an XML envelope with a Header and a Body, and is described by a WSDL that defines every operation, message type and fault. REST exposes many resource URIs, uses the HTTP verbs to express intent, and is usually described by OpenAPI with JSON payloads.

For SOAP the WSDL is a gift: it is machine readable and strongly typed, so SoapUI or ReadyAPI can generate a request skeleton for every operation, and XSD validation gives you schema checking for free. Your assertions use XPath rather than JSONPath, and errors arrive as a SOAP Fault inside a 200 or 500 response with faultcode and faultstring rather than as an HTTP status, so a test that only checks for 200 will pass on every failure. That is the classic SOAP trap.

Namespaces matter too: an XPath that ignores the namespace prefix silently matches nothing and your assertion passes vacuously against an empty node set. Practically this still matters in India because core banking, insurance and many government integrations expose SOAP, so services majors like TCS, Infosys and Cognizant continue to staff SOAP testers for BFSI accounts. In a mixed estate you often test a REST facade in Postman or REST Assured and the SOAP backend in SoapUI, and the highest value tests are the translation ones: does the REST wrapper map a SOAP Fault to a sensible HTTP status, and does it preserve decimal precision on amounts when converting XML to JSON.

# SOAP request: one endpoint, POST, XML envelope
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns:ban="http://bank.example.in/accounts">
  <soapenv:Header/>
  <soapenv:Body>
    <ban:GetBalance>
      <ban:accountNumber>50100234567890</ban:accountNumber>
    </ban:GetBalance>
  </soapenv:Body>
</soapenv:Envelope>

# Failure arrives as a Fault, NOT as a 4xx
<soapenv:Body>
  <soapenv:Fault>
    <faultcode>soapenv:Client</faultcode>
    <faultstring>ACCOUNT_NOT_FOUND</faultstring>
  </soapenv:Fault>
</soapenv:Body>

// REST Assured asserting on XML, namespace aware
given().contentType("text/xml").body(envelope)
.when().post("/services/AccountService")
.then().statusCode(200)
  .body("Envelope.Body.Fault.faultstring", equalTo("ACCOUNT_NOT_FOUND"));

Answer

GraphQL usually exposes a single POST /graphql endpoint, so status codes stop being your signal. A GraphQL server commonly returns 200 with an errors array alongside a partial data object, which means a test asserting only on 200 will pass while half the response is null. The first rule is therefore to assert that errors is absent or empty, then assert on the data shape.

The test surface also changes: instead of fixed responses you test query shapes, so you check that the client can ask for exactly the fields it needs, that a nested query does not explode into N+1 database calls, that unknown fields are rejected by the schema, and that query depth or complexity limits exist, because an unlimited nested query is a denial of service vector. Introspection should usually be disabled in production, and I test for that explicitly. Authorization is field level in GraphQL, so a viewer might be allowed to fetch user.name but not user.phone in the same query, which needs its own test matrix. gRPC is different again: it is binary Protocol Buffers over HTTP/2 with a .proto contract, so you cannot curl it casually.

You use grpcurl, Postman's gRPC support, or generated stubs in Java or Go. Statuses are gRPC codes such as OK, INVALID_ARGUMENT, NOT_FOUND, DEADLINE_EXCEEDED and UNAVAILABLE rather than HTTP codes, deadlines are first class so timeout behaviour is testable, and streaming (server, client and bidirectional) needs tests that assert on the sequence of messages. The compensation is that the .proto file is a real contract, so backward compatibility testing (never reuse a field number, never change a type) becomes a concrete checklist.

// GraphQL: 200 with errors is the trap
// POST /graphql
{
  "query": "query($id: ID!) { order(id: $id) { id total items { sku qty } } }",
  "variables": { "id": "90211" }
}

// Response: HTTP 200, but half of it failed
{
  "data": { "order": { "id": "90211", "total": null, "items": [] } },
  "errors": [{ "message": "Not authorised to read total", "path": ["order", "total"] }]
}

pm.test("graphql call has no partial errors", function () {
  pm.response.to.have.status(200);
  var body = pm.response.json();
  pm.expect(body.errors, JSON.stringify(body.errors)).to.be.undefined;
  pm.expect(body.data.order.total).to.be.a("number");
});

# gRPC with grpcurl
grpcurl -plaintext -d '{"order_id": "90211"}' \
  localhost:50051 payments.OrderService/GetOrder
# expect code = OK; error path expects code = NOT_FOUND, not HTTP 404

Answer

Postman has five variable scopes and resolves them from narrowest to broadest: local (data and script variables that live only for the current request or run), data (values from the CSV or JSON file fed to the collection runner), environment, collection, and finally global. When the same name exists in several scopes the narrowest one wins, so a data file column called baseUrl overrides the environment baseUrl, and an environment value overrides a collection value. Globals are the widest and the most dangerous, because they persist across every collection in your workspace and are the classic reason a test passes on your machine.

The failure story goes like this: three months ago you set a global token while debugging. Your environment does not define token, so locally the global fills the gap and every request authenticates. In CI, Newman only loads the environment file you pass with -e, globals are not exported unless you explicitly pass -g, so token resolves to an empty string and every request gets 401.

The fix is discipline: define everything a collection needs at collection or environment scope, never rely on globals, and add a pre-request script at collection level that fails loudly if a required variable is missing rather than sending a request with the literal placeholder. Also know the difference between initial value, which is what gets exported and shared, and current value, which stays on your machine. Committing an environment file where the initial value of an API key is filled in is one of the most common ways Indian teams leak staging credentials into a public repo.

// Precedence, narrowest wins:
// local  >  data (CSV/JSON)  >  environment  >  collection  >  global

pm.globals.set("baseUrl", "https://global.example.in");
pm.collectionVariables.set("baseUrl", "https://collection.example.in");
pm.environment.set("baseUrl", "https://staging.example.in");
console.log(pm.variables.get("baseUrl"));  // staging wins

// Collection level pre-request script: fail fast on missing config
var required = ["baseUrl", "apiKey", "merchantId"];
required.forEach(function (name) {
  var v = pm.variables.get(name);
  if (!v || String(v).indexOf("{{") === 0) {
    throw new Error("Missing variable: " + name + ". Did you select an environment?");
  }
});

# Newman does NOT carry your globals unless you pass them
newman run payments.postman_collection.json -e staging.postman_environment.json -g globals.json

Answer

pm is Postman's scripting API, available in both the pre-request and the test script tabs. pm.test(name, fn) registers a named assertion block, and every block runs independently so one failure does not hide the others, which is why you should write several small pm.test blocks rather than one giant one. Inside, pm.expect is a bundled Chai expect, so you get the full BDD chain: to.eql for deep equality, to.equal for strict equality on primitives, to.have.property, to.be.an('array'), to.include, to.match with a regular expression, to.be.oneOf, and to.have.lengthOf. pm.response gives you code, status, responseTime, responseSize, headers.get(name), text() and json(). pm.response.to.have.status(201) and pm.response.to.be.ok are sugar over the same thing. pm.request lets you read what you sent, which is useful when asserting that a request id you generated came back. For sharing state you have pm.environment, pm.collectionVariables and pm.globals with get, set and unset, and pm.sendRequest for firing an extra call from inside a script.

Two practical points interviewers listen for. First, name your tests as sentences that read in the runner output, because a failing test called Test1 tells CI nothing. Second, pass a message as the second argument to pm.expect so the failure output includes the actual value, for example pm.expect(body.status, JSON.stringify(body)).to.eql('applied'), otherwise you get expected undefined to equal applied and have to rerun manually to see the payload.

pm.test("application created", function () {
  pm.response.to.have.status(201);
});

pm.test("response shape is right", function () {
  var body = pm.response.json();
  pm.expect(body, JSON.stringify(body)).to.have.property("application_id");
  pm.expect(body.status).to.eql("applied");
  pm.expect(body.job).to.be.an("object");
  pm.expect(body.job.job_id).to.eql(pm.collectionVariables.get("jobId"));
  pm.expect(body.applied_at).to.match(/^\d{4}-\d{2}-\d{2}T/);
});

pm.test("resume url is a signed s3 link that expires", function () {
  var url = pm.response.json().resume_url;
  pm.expect(url).to.include("X-Amz-Expires");
  pm.expect(url).to.not.include("AWS_SECRET");
});

pm.test("applying twice to the same job is rejected", function () {
  pm.sendRequest({
    url: pm.environment.get("baseUrl") + "/v1/applications",
    method: "POST",
    header: { "Content-Type": "application/json", Authorization: "Bearer " + pm.environment.get("token") },
    body: { mode: "raw", raw: JSON.stringify({ job_id: pm.collectionVariables.get("jobId") }) }
  }, function (err, res) {
    pm.expect(res.code).to.be.oneOf([409, 422]);
  });
});

Answer

Chaining means extracting a value from one response and storing it in a variable that later requests reference with double curly braces. The mechanics are simple: in the test script of the login request, read the response, then call pm.environment.set or pm.collectionVariables.set, then use {{token}} in the Authorization header of subsequent requests. The judgement is in which scope you write to.

Use collection variables for values created during the run such as orderId or paymentId, because they are scoped to the collection and do not pollute the shared environment file. Use environment variables for things that legitimately belong to the environment such as the token for that host. Never write run specific ids into globals.

The things that break: first, ordering. Chaining assumes requests run in collection order, which is true in the runner and in Newman, but a colleague running a single request from the middle of the collection gets a stale or empty variable and reports a phantom bug. Guard against it by having each request check for its prerequisite and skip with a clear message.

Second, staleness. If the chained id persists in the environment file, a rerun may pass because it is asserting against yesterday's order which still exists. Clear run scoped variables in the collection level pre-request or at the end of the run.

Third, asynchronicity. If the create call returns 202 and the resource is not yet queryable, the next request 404s intermittently, which needs a polling step rather than a longer sleep.

// Request 1: POST /v1/auth/login   (Tests tab)
var body = pm.response.json();
pm.test("login returns a token", function () {
  pm.expect(body.access_token).to.be.a("string").and.not.empty;
});
pm.environment.set("token", body.access_token);
pm.environment.set("tokenExpiresAt", Date.now() + body.expires_in * 1000);

// Request 2: POST /v1/orders
// Header:  Authorization: Bearer {{token}}
var order = pm.response.json();
pm.collectionVariables.set("orderId", order.order_id);

// Request 3: GET /v1/orders/{{orderId}}   (Pre-request tab, guard the chain)
if (!pm.collectionVariables.get("orderId")) {
  throw new Error("orderId missing. Run the whole collection, not this request alone.");
}

// End of run: clean up so a rerun cannot pass on stale data
pm.collectionVariables.unset("orderId");

Answer

A pre-request script runs immediately before the request is sent, and it can be attached at collection, folder or request level, running outermost first. The obvious use is computing a dynamic value, but the interesting uses are the ones that make a collection portable. Generating a fresh idempotency key or a unique email per run so reruns do not collide with yesterday's data.

Computing a signature: many Indian payment and logistics APIs require an HMAC SHA256 of the payload plus a timestamp, and CryptoJS is bundled into the Postman sandbox so you can compute it inline. Refreshing an expired token by checking a stored expiry and calling the auth endpoint with pm.sendRequest only when needed, rather than adding a login request before every call. Skipping a request conditionally with postman.setNextRequest to branch a flow.

Seeding test data by calling an internal setup endpoint. Validating configuration and failing fast when a required variable is missing. The two constraints to remember are that the sandbox is JavaScript with a limited library set (CryptoJS, Lodash, moment, uuid, ajv, chai, atob and btoa are available, arbitrary npm packages are not), and that pm.sendRequest is asynchronous with a callback, so any variable you set inside it must be set inside the callback or the main request will fire before the value exists. That async mistake is the single most common bug in real Postman collections.

// Collection level pre-request: fresh idempotency key + HMAC signature
var uuid = require("uuid");
pm.collectionVariables.set("idemKey", uuid.v4());

var ts = Math.floor(Date.now() / 1000);
var payload = pm.request.body ? pm.request.body.raw : "";
var toSign = ts + "." + payload;
var secret = pm.environment.get("webhookSecret");
var sig = CryptoJS.HmacSHA256(toSign, secret).toString(CryptoJS.enc.Hex);

pm.request.headers.upsert({ key: "X-Timestamp", value: String(ts) });
pm.request.headers.upsert({ key: "X-Signature", value: sig });

// Unique data per run so reruns do not collide
pm.collectionVariables.set("testEmail", "qa+" + Date.now() + "@goodspace.ai");

// Async trap: set inside the callback, never after it
pm.sendRequest({ url: pm.environment.get("baseUrl") + "/v1/config", method: "GET" }, function (err, res) {
  pm.collectionVariables.set("gstRate", res.json().gst_rate);   // correct
});
// pm.collectionVariables.set("gstRate", ???)  <- runs BEFORE the callback

Answer

The collection runner accepts a CSV or JSON data file and runs the whole collection once per row, exposing each column as a data scope variable. In the CSV the first row is the header and each header name becomes a variable you reference with double curly braces in the URL, body, headers or scripts, or read in code with pm.iterationData.get('vpa'). Because data scope sits just below local in precedence, a column named baseUrl will override the environment, which is occasionally what you want and more often an accident.

The pattern that makes this valuable is putting the expected result in the CSV alongside the input, so one request plus one test script covers thirty validation cases: a column of UPI VPAs with a column of expected status codes and expected error codes, then a test that asserts the response matches the row. Practical rules from real use: quote any field containing a comma, save the file as UTF-8 without a BOM because a BOM corrupts the first header name and produces a maddening undefined on only the first column, remember every value arrives as a string so 200 from the CSV is not equal to the numeric response code without a parseInt or a loose comparison, and keep the file small enough that a failure is diagnosable. For CI, the same file is passed to Newman with -d, so the exact iteration that failed appears in the JUnit report. The classic mistake is embedding real customer phone numbers or PAN values in the CSV and committing it, which turns a convenience file into a data protection incident.

# upi_cases.csv
# vpa,amount,expectedStatus,expectedCode
# aarav@okhdfcbank,49900,201,
# aarav@,49900,422,INVALID_VPA
# aarav@okhdfcbank,0,422,AMOUNT_TOO_LOW
# aarav@okhdfcbank,-100,400,MALFORMED_REQUEST

// Request body uses the columns directly
// { "vpa": "{{vpa}}", "amount": {{amount}} }

// Tests tab
var expectedStatus = parseInt(pm.iterationData.get("expectedStatus"), 10);
var expectedCode = pm.iterationData.get("expectedCode");

pm.test("row " + pm.info.iteration + " [" + pm.iterationData.get("vpa") + "] returns " + expectedStatus, function () {
  pm.response.to.have.status(expectedStatus);
  if (expectedCode) {
    pm.expect(pm.response.json().error.code).to.eql(expectedCode);
  }
});

# Same file in CI
newman run payments.postman_collection.json -e staging.json -d upi_cases.csv

Answer

Newman is Postman's command line runner, installed with npm install -g newman, and it runs the exported collection JSON with an environment file, an optional data file and one or more reporters. The plumbing is easy, so what interviewers are really checking is whether you have made the pipeline trustworthy. Four things matter.

First, exit codes: Newman exits non zero when any assertion fails, which is what fails the Jenkins or GitHub Actions job, so never pipe the output through something that swallows the code. Second, reporting: the cli reporter is for humans, the junit reporter produces XML your CI can render as a test report with per assertion detail, and newman-reporter-htmlextra gives a shareable HTML artefact that non technical stakeholders will actually open. Third, secrets: the environment file committed to the repo must contain placeholders only, with real tokens injected at runtime from the CI secret store using the environment variable substitution feature, because a committed staging key is the most common leak in Indian repos.

Fourth, where it runs in the pipeline: API smoke tests belong immediately after deploy to staging, gated so a failure blocks promotion, and the full regression can run nightly. Two operational details worth naming: use -n to run multiple iterations and the delay-request flag to avoid tripping rate limits on a shared staging gateway, and keep a separate fast smoke collection of ten requests, because a forty minute suite in the deploy gate will be disabled by the third person who is blocked by it.

# install
npm install -g newman newman-reporter-htmlextra

# secrets injected at runtime, the committed template holds placeholders only
envsubst < envs/staging.template.json > /tmp/staging.json

# smoke run in the deploy gate (reports land in ./newman/)
newman run collections/payments.smoke.json \
  -e /tmp/staging.json \
  -r cli,junit,htmlextra

# data driven nightly regression, 3 passes
newman run collections/payments.regression.json \
  -e /tmp/staging.json -d data/upi_cases.csv -n 3 \
  -r cli,junit

# exit code drives the pipeline
# 0 = all assertions passed, 1 = assertion failure, other = runtime error
echo "newman exit: $?"

Answer

A JSON Schema is a declarative description of a payload: which fields exist, their types, formats, enums, numeric ranges, array constraints and which are required. You validate a response against it with Ajv, which is bundled into the Postman sandbox, or with the JSON Schema Validator library in Java, or with the matchesJsonSchemaInClasspath matcher in REST Assured. What it catches that hand written assertions miss is the whole class of contract drift you did not think to check.

A backend that renames total to total_amount, changes amount from an integer in paise to a float in rupees, starts sending null for a field that used to be a string, or drops a field entirely will sail past a test that only checks status and a couple of properties, and will break the Android client in production. Two settings do most of the work. required lists the fields a client can depend on, and additionalProperties set to false makes the schema strict so a new unexpected field fails the test. That strictness is a deliberate choice: for a consumer you own, strict is right because you want to know when the producer changes anything, but for a third party API you consume it will fail on every harmless addition, so allow additions there and be strict about your own services. Keep the schemas in the repo next to the tests rather than pasted into a Postman script, generate the first version from a real response, then tighten it by hand, because generated schemas are permissive by default and a permissive schema is a test that always passes.

// schemas/payment.schema.json
{
  "type": "object",
  "required": ["payment_id", "amount_paise", "currency", "status", "created_at"],
  "additionalProperties": false,
  "properties": {
    "payment_id": { "type": "string", "pattern": "^pay_[A-Za-z0-9]{10,}$" },
    "amount_paise": { "type": "integer", "minimum": 100 },
    "currency": { "type": "string", "enum": ["INR"] },
    "status": { "type": "string", "enum": ["created", "authorized", "captured", "failed", "refunded"] },
    "vpa": { "type": ["string", "null"] },
    "created_at": { "type": "string", "format": "date-time" }
  }
}

// Postman: Ajv is bundled
var Ajv = require("ajv");
var ajv = new Ajv({ allErrors: true });
var schema = JSON.parse(pm.collectionVariables.get("paymentSchema"));
var validate = ajv.compile(schema);

pm.test("payment response matches the contract", function () {
  var ok = validate(pm.response.json());
  pm.expect(ok, JSON.stringify(validate.errors)).to.be.true;
});

Answer

Content-Type is the first, because an API that returns JSON with text/html or a missing charset will break strict clients and often signals that an error page from nginx has replaced the real response. Asserting it turns a confusing parse failure into a clear diagnosis. Cache-Control matters most on personalised endpoints: if GET /v1/me comes back with a public cacheable directive, a CDN or a corporate proxy can serve one user's profile to another, which is a genuine security defect and one I have seen reach staging more than once.

Assert no-store on anything user specific. X-Request-Id or a correlation id header is the thread that ties a failing test to a server log line, so I assert that it exists and, better, that it echoes the value I sent, because an API that generates its own id and ignores mine makes distributed debugging much harder. Location must be present on a 201 and must point at the created resource, and a surprising number of APIs omit it.

Retry-After should accompany 429 and 503, otherwise clients guess and hammer the service. Security headers deserve a check: Strict-Transport-Security, X-Content-Type-Options nosniff, and the absence of Server or X-Powered-By headers that advertise your exact framework version to anyone scanning. Set-Cookie flags on session APIs need HttpOnly, Secure and an appropriate SameSite. Finally the CORS headers on a browser facing API, since a wildcard Access-Control-Allow-Origin combined with credentials is a real misconfiguration.

pm.test("content type is json", function () {
  pm.expect(pm.response.headers.get("Content-Type")).to.include("application/json");
});

pm.test("personalised response is never cacheable", function () {
  var cc = pm.response.headers.get("Cache-Control") || "";
  pm.expect(cc.toLowerCase(), "Cache-Control was: " + cc).to.include("no-store");
});

pm.test("correlation id echoes what we sent", function () {
  pm.expect(pm.response.headers.get("X-Request-Id")).to.eql(pm.request.headers.get("X-Request-Id"));
});

pm.test("201 carries a Location pointing at the new resource", function () {
  pm.response.to.have.status(201);
  pm.expect(pm.response.headers.get("Location")).to.include("/v1/payments/" + pm.response.json().payment_id);
});

pm.test("no framework fingerprinting headers", function () {
  pm.expect(pm.response.headers.has("X-Powered-By")).to.be.false;
  pm.expect(pm.response.headers.get("X-Content-Type-Options")).to.eql("nosniff");
});

Answer

Because UI setup is slow, brittle and tests nothing you care about. To test a refund you need a captured payment, and clicking through signup, KYC, product selection, checkout and a payment simulator takes ninety seconds and breaks whenever a designer moves a button. Four API calls do the same in two seconds and fail with a clear status code when something is wrong.

Setting up through the API also means you can create precise states that the UI cannot reach at all: an order stuck in a pending settlement state, a user whose token expires in thirty seconds, a wallet with exactly one rupee, a partially refunded payment. It parallelises safely because each test can create its own tenant or user rather than fighting over one shared login, which is the biggest single cause of flaky suites in Indian QA teams that share a staging account. It also makes cleanup possible, since anything you created via the API you can delete via the API.

Where it fails: the API path may not exercise the same validation the UI does, so you can create data the product would never allow and then write a test for an impossible state, which produces a bug report the developer rightly rejects. Second, if setup calls the same endpoint that the test is verifying, a bug in that endpoint hides itself. Third, if the setup API is internal and unversioned it will change without warning and break every test at once. The rule I follow is that setup uses stable public or dedicated seed endpoints, and the endpoint under test is never used to build its own precondition.

# Setup for one refund test, four calls, roughly two seconds
curl -s -X POST $API/v1/test/users -H "X-Seed-Key: $SEED_KEY" \
  -d '{"email":"qa+refund-4412@goodspace.ai","kyc":"verified"}'
# -> {"user_id":"usr_4412","token":"eyJhbGci..."}

curl -s -X POST $API/v1/orders -H "Authorization: Bearer $TOKEN" \
  -d '{"items":[{"sku":"PLAN_PRO","qty":1}]}'
# -> {"order_id":"ord_991","amount_paise":99900}

curl -s -X POST $API/v1/payments -H "Authorization: Bearer $TOKEN" \
  -H "Idempotency-Key: seed-4412" \
  -d '{"order_id":"ord_991","method":"upi","vpa":"success@razorpay"}'
# -> {"payment_id":"pay_A1","status":"captured"}

# NOW the actual test
curl -i -X POST $API/v1/payments/pay_A1/refund -H "Authorization: Bearer $TOKEN" \
  -d '{"amount_paise": 49950, "reason": "partial"}'

# teardown
curl -s -X DELETE $API/v1/test/users/usr_4412 -H "X-Seed-Key: $SEED_KEY"

Answer

First I reproduce the exact request outside the framework, because half of API failures are the harness rather than the API. Newman and REST Assured can both print the resolved request, and I turn that into a curl command with the real URL, headers and body so I can rerun it by hand. If curl succeeds where the test failed, the bug is in variable resolution, ordering or a stale token, not in the product.

Second, I read the whole response, not the assertion message. Status, body, and headers together usually name the problem: a 401 with WWW-Authenticate means the token expired mid run, a 403 means the seeded user lacks a role, an HTML body with a 502 means nginx never reached the service, a 404 on a chained id means the create step failed earlier and the report only shows the downstream failure. Third, I grab the correlation id.

If I sent X-Request-Id I already have it, otherwise I take the one the response returned, and I search the service logs and the tracing tool for it, which turns guesswork into the actual stack trace or the downstream timeout. Fourth, I separate environment from defect: was staging redeployed, did a config value change, is the bank sandbox down, did someone else's run delete my seed data. Fifth, I check whether it fails consistently by running just that request five times, because an intermittent failure points to timing, ordering or an async flow rather than a logic bug. Only then do I raise a defect, and it includes the curl reproduction, the full response, the correlation id and the environment, because a bug report without those gets returned within the hour.

# 1. Reproduce outside the framework, verbosely
curl -sv -X POST "$BASE/v1/payments/pay_A1/refund" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "X-Request-Id: qa-debug-$(date +%s)" \
  -d '{"amount_paise": 49950}' 2>&1 | tee /tmp/refund.log

# 2. Timing breakdown separates "slow API" from "slow DNS"
curl -o /dev/null -s -w \
  "dns:%{time_namelookup} connect:%{time_connect} tls:%{time_appconnect} ttfb:%{time_starttransfer} total:%{time_total}\n" \
  "$BASE/v1/payments/pay_A1"

# 3. Is the token actually valid, or expired mid run?
echo $TOKEN | cut -d. -f2 | base64 -d 2>/dev/null | python3 -m json.tool

# 4. Consistency check
for i in 1 2 3 4 5; do
  curl -o /dev/null -s -w "%{http_code} " "$BASE/v1/payments/pay_A1"
done; echo

Answer

A JWT is three base64url segments separated by dots: header, payload, signature. The header names the algorithm and often a key id, the payload carries claims, and the signature is computed over the first two segments with either a shared secret (HS256) or a private key (RS256). The critical thing to say out loud is that base64 is encoding, not encryption.

Anyone holding the token can read the payload, so a JWT that contains a PAN, an Aadhaar number, a phone number or an internal role description is leaking data to every browser, every log aggregator and every proxy in the path. As a tester I decode the payload and assert on the registered claims. exp must exist and must actually be enforced, which I prove by taking an expired token and asserting 401 rather than trusting the code. iat and nbf should be sane relative to server time, and a clock skew of a few minutes is normal but ten minutes is a defect waiting to happen. iss must match the expected issuer and aud must match this API, because an API that accepts a valid token minted for a different service is a real vulnerability in a microservice estate. sub identifies the user, and I assert that the token for user A cannot read user B's data. Then the attack cases: tamper one character in the payload and assert 401, change alg to none and assert 401, sign with the wrong secret and assert 401, and reuse a token after logout to check whether revocation exists at all, which for stateless JWTs it usually does not.

// Decoded payload of a real-shaped JWT
{
  "sub": "usr_4412",
  "iss": "https://auth.example.in",
  "aud": "payments-api",
  "iat": 1786000000,
  "exp": 1786003600,
  "scope": "read:payments write:refunds",
  "role": "merchant_admin"
}

// Postman: decode and assert without any library
var token = pm.environment.get("token");
var claims = JSON.parse(atob(token.split(".")[1].replace(/-/g, "+").replace(/_/g, "/")));

pm.test("token claims are correct and short lived", function () {
  pm.expect(claims.iss).to.eql("https://auth.example.in");
  pm.expect(claims.aud).to.eql("payments-api");
  pm.expect(claims.exp - claims.iat).to.be.at.most(3600);
  pm.expect(claims.exp * 1000).to.be.above(Date.now());
});

pm.test("no PII inside the token", function () {
  var raw = JSON.stringify(claims).toLowerCase();
  ["aadhaar", "pan", "mobile", "dob"].forEach(function (k) {
    pm.expect(raw).to.not.include(k);
  });
});

# tampered token must be rejected
curl -o /dev/null -s -w "%{http_code}\n" $BASE/v1/payments \
  -H "Authorization: Bearer ${TOKEN%?}X"   # expect 401

Answer

Client credentials is the machine to machine grant. Your test harness posts client_id, client_secret and grant_type=client_credentials to the token endpoint and gets back an access token with a scope and an expiry. It is fully automatable, needs no browser, and is what you should ask for when the team says the API is OAuth protected but the tests are server side.

Authorization code flow involves a human: the user is redirected to the identity provider, logs in, consents, is redirected back to the client with a short lived code, and the client exchanges that code plus a PKCE verifier for tokens. You cannot automate the consent screen reliably, and trying to drive it with Selenium is how suites become flaky, especially once the provider adds MFA or a captcha. The practical strategies, in order of preference: ask the identity team for a client credentials client scoped to the test environment; use the resource owner password grant if the provider still supports it for test users only, since it is deprecated but very convenient in staging; obtain a long lived refresh token once by hand, store it in the CI secret store, and have the suite exchange it for access tokens at the start of every run; or, as a last resort, drive the browser once per run and cache the token.

Whichever you use, the token must be fetched once and shared, not once per request, or you will rate limit your own identity provider. And the tests for the flow itself, meaning wrong client secret, expired code, reused code, missing scope, are separate negative tests and belong in the suite too.

# Client credentials: fully automatable
curl -s -X POST https://auth.example.in/oauth2/token \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=client_credentials" \
  -d "client_id=$CI_CLIENT_ID" \
  -d "client_secret=$CI_CLIENT_SECRET" \
  -d "scope=read:payments write:refunds"
# {"access_token":"eyJ...","token_type":"Bearer","expires_in":3600,"scope":"read:payments write:refunds"}

# Authorization code: step 2 needs a human, so we cache a refresh token instead
curl -s -X POST https://auth.example.in/oauth2/token \
  -d "grant_type=refresh_token" \
  -d "refresh_token=$CI_REFRESH_TOKEN" \
  -d "client_id=$CI_CLIENT_ID"

// Negative tests that belong in the suite
pm.test("reusing an authorization code fails", function () {
  pm.response.to.have.status(400);
  pm.expect(pm.response.json().error).to.eql("invalid_grant");
});

pm.test("token without write scope cannot refund", function () {
  pm.response.to.have.status(403);
  pm.expect(pm.response.json().error.code).to.eql("INSUFFICIENT_SCOPE");
});

Answer

The naive fixes are both wrong: logging in before every request triples your request count and can trip the identity provider's rate limit, and asking the backend team for a token with a twenty four hour expiry weakens production security for a test convenience. The correct pattern is a lazy refresh guarded by a stored expiry. Store the token and its absolute expiry timestamp when you first authenticate.

Before each request, compare the expiry with the current time minus a safety margin of thirty to sixty seconds, and only when it is within that margin do you call the token endpoint again. In Postman this lives in a collection level pre-request script, which runs before every request in the collection, and the refresh call uses pm.sendRequest with the new values set inside the callback. In REST Assured or any Java suite the same logic belongs in a token provider class, guarded so that parallel threads do not all refresh at once, typically with a synchronized block or an AtomicReference holding a cached token.

Two details interviewers probe. First, the safety margin, because a token that expires in two seconds will fail the request you are about to send even though it was technically still valid when you checked. Second, the retry on 401: even with a margin, a clock skew or a server side revocation can produce a 401, so a robust suite catches a single 401, refreshes once, retries once, and fails properly on the second. Do not retry indefinitely, because an actually revoked credential then produces an infinite loop instead of a clear failure.

// Collection level pre-request script in Postman
var expiresAt = Number(pm.environment.get("tokenExpiresAt") || 0);
var marginMs = 60 * 1000;

if (!pm.environment.get("token") || Date.now() > expiresAt - marginMs) {
  pm.sendRequest({
    url: pm.environment.get("authUrl") + "/oauth2/token",
    method: "POST",
    header: { "Content-Type": "application/x-www-form-urlencoded" },
    body: {
      mode: "urlencoded",
      urlencoded: [
        { key: "grant_type", value: "client_credentials" },
        { key: "client_id", value: pm.environment.get("clientId") },
        { key: "client_secret", value: pm.environment.get("clientSecret") }
      ]
    }
  }, function (err, res) {
    if (err || res.code !== 200) { throw new Error("token refresh failed: " + (err || res.code)); }
    var b = res.json();
    pm.environment.set("token", b.access_token);
    pm.environment.set("tokenExpiresAt", Date.now() + b.expires_in * 1000);
  });
}

// Java equivalent, thread safe for parallel REST Assured runs
public final class TokenProvider {
    private static final AtomicReference<Token> CACHE = new AtomicReference<>();
    public static synchronized String get() {
        Token t = CACHE.get();
        if (t == null || t.expiresAt().isBefore(Instant.now().plusSeconds(60))) {
            t = fetchFromIdp();
            CACHE.set(t);
        }
        return t.value();
    }
}

Answer

A webhook is an unauthenticated POST from the internet to your server, so the only thing standing between you and a forged payment confirmation is the signature. Razorpay computes an HMAC SHA256 of the exact raw request body using your webhook secret and sends it in the X-Razorpay-Signature header. Your server must recompute it and compare.

The tests split into three groups. Positive: send a correctly signed payload and assert the order moves to paid and the event is recorded. Negative, which is where the real bugs are: send the same payload with a wrong signature, with the header missing entirely, with an empty header, with a signature computed using a different secret, and with a valid signature over a different body, and assert every one is rejected with 400 and, crucially, that no state changed.

I have seen production code that logged a signature mismatch and then processed the payload anyway, so the state assertion matters more than the status code. Replay is the third group: capture a genuinely valid signed request and send it again, then assert the payment is not credited twice, which requires the handler to be idempotent on the event id. Two implementation details worth naming in an interview.

The comparison must be constant time, using hmac.compare_digest in Python or MessageDigest.isEqual in Java, because a naive string equality leaks timing information. And the signature is computed over the raw bytes, so any middleware that parses and re-serialises JSON before the verifier sees it will change key order or whitespace and break verification in a way that looks random.

# Generate a valid signature for a test payload
BODY='{"event":"payment.captured","payload":{"payment":{"entity":{"id":"pay_A1","amount":49900,"order_id":"ord_991"}}}}'
SIG=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$WEBHOOK_SECRET" -r | cut -d' ' -f1)

# 1. Valid signature -> accepted
curl -i -X POST $BASE/webhooks/razorpay \
  -H "Content-Type: application/json" \
  -H "X-Razorpay-Signature: $SIG" -d "$BODY"      # expect 200, order becomes paid

# 2. Tampered amount, original signature -> must be rejected
curl -i -X POST $BASE/webhooks/razorpay \
  -H "X-Razorpay-Signature: $SIG" \
  -d '{"event":"payment.captured","payload":{"payment":{"entity":{"id":"pay_A1","amount":9999900}}}}'
# expect 400 AND no ledger movement

# 3. Missing header -> 400
curl -o /dev/null -s -w "%{http_code}\n" -X POST $BASE/webhooks/razorpay -d "$BODY"

# 4. Replay the valid one -> 200 but exactly one credit
curl -s -X POST $BASE/webhooks/razorpay -H "X-Razorpay-Signature: $SIG" -d "$BODY"
# then assert: SELECT count(*) FROM ledger WHERE payment_id='pay_A1' -> 1

Answer

202 means the request was accepted for processing and nothing has happened yet, so asserting on the 202 body proves almost nothing. This is the shape of refunds, bulk uploads, KYC verification, payout batches and anything that hands off to a queue. There are three legitimate ways to observe the outcome.

Polling is the most common and works whenever the API exposes a status resource: the 202 response should return a job id or a Location header, and you poll it on an interval with a hard timeout and, ideally, exponential backoff. The rules that keep this sane are a bounded number of attempts, an explicit failure message naming the last observed state, and never using a bare sleep, because a fixed sleep is either too short (flaky) or too long (a suite that takes an hour). Callback receivers are the second approach: you stand up a small HTTP endpoint that the system under test calls when finished, and the test waits on it.

Locally that can be a lightweight server in the test process, and for cloud environments teams use a public receiver or a tunnel. This is closest to production behaviour and lets you assert on the callback payload and its signature. Queue or database inspection is the third: read the Kafka topic, the SQS queue or the database row directly.

It is fast and precise but couples the test to internals, so I use it for diagnosis and for asserting side effects, not as the primary contract check. Whichever you pick, also test the failure path, meaning what the status resource says when processing fails, and test that a terminal state is genuinely terminal.

// 202 response you must not assert success on
// POST /v1/refunds -> 202 Accepted
// Location: /v1/refunds/rfnd_77/status
{ "refund_id": "rfnd_77", "status": "queued" }

// Postman polling with backoff and a hard cap
var attempt = Number(pm.collectionVariables.get("pollAttempt") || 0);
var body = pm.response.json();

if (body.status === "processed") {
  pm.test("refund settled with the right amount", function () {
    pm.expect(body.amount_paise).to.eql(49950);
    pm.expect(body.utr).to.be.a("string");
  });
  pm.collectionVariables.unset("pollAttempt");
} else if (body.status === "failed") {
  pm.test("refund must not fail", function () { pm.expect.fail("terminal failure: " + body.failure_reason); });
} else if (attempt < 12) {
  pm.collectionVariables.set("pollAttempt", attempt + 1);
  setTimeout(function () {}, Math.min(500 * Math.pow(2, attempt), 8000));
  postman.setNextRequest("GET refund status");
} else {
  pm.test("refund reached a terminal state", function () {
    pm.expect.fail("still " + body.status + " after 12 polls");
  });
}

Answer

Start by pinning down the contract, because implementations differ: is the key scoped per merchant or globally, how long is it retained, does a repeat return the stored response or a fresh one, and what happens when the same key arrives with a different body. Then the cases. Same key and same body sent twice must produce exactly one payment, and the second response should be the stored one, ideally with a header marking it as a replay.

Same key with a different body must be rejected, usually 409 or 422, because that means the client has a bug and silently returning the original payment would hide a real mismatch. Different keys with identical bodies must create two payments, since that is a legitimate case, a customer buying the same thing twice. Concurrency is the case that finds real bugs: fire the same key twice simultaneously and assert one succeeds and the other either waits and returns the same result or returns a clear in progress conflict, never two payments.

A poorly implemented check-then-insert without a unique constraint will fail this every time. Then the failure interaction: if the first attempt failed with a 500 or the bank declined it, does the key lock in that failure forever, which would prevent a legitimate retry. Most gateways only store terminal successful responses, so verify the documented behaviour. Also test key retention, meaning the same key after the retention window, missing key behaviour, a key that is too long or contains control characters, and finally the database level assertion, because the only thing that really proves it worked is exactly one row and exactly one debit.

# 1. Same key, same body, twice
KEY=$(uuidgen)
for i in 1 2; do
  curl -s -o /tmp/r$i.json -w "%{http_code}\n" -X POST $BASE/v1/payments \
    -H "Idempotency-Key: $KEY" -H "Content-Type: application/json" \
    -d '{"order_id":"ord_991","amount_paise":49900,"vpa":"aarav@okhdfcbank"}'
done
diff /tmp/r1.json /tmp/r2.json && echo "same payment returned"
# assert in DB: SELECT count(*) FROM payments WHERE order_id='ord_991' -> 1

# 2. Same key, DIFFERENT body -> must conflict
curl -i -X POST $BASE/v1/payments -H "Idempotency-Key: $KEY" \
  -d '{"order_id":"ord_991","amount_paise":100,"vpa":"aarav@okhdfcbank"}'
# expect 409 / 422, error code IDEMPOTENCY_KEY_REUSED

# 3. Concurrency: two identical calls at the same instant
for i in 1 2; do
  curl -s -o /tmp/c$i.json -X POST $BASE/v1/payments \
    -H "Idempotency-Key: $KEY-race" \
    -d '{"order_id":"ord_992","amount_paise":49900}' &
done; wait
# assert exactly ONE payment row for ord_992

# 4. Different keys, same body -> two payments is CORRECT
curl -s -X POST $BASE/v1/payments -H "Idempotency-Key: $(uuidgen)" -d '{"order_id":"ord_993","amount_paise":49900}'

Answer

First establish the policy, because you cannot test a limit you cannot state: how many requests, over what window, keyed by what (API key, user, IP, or endpoint), is the window fixed or sliding, and is there a burst allowance. Then the cases. Just under the limit must all succeed.

Crossing the limit must return 429 Too Many Requests, and a correct 429 carries a Retry-After header with either seconds or an HTTP date, plus ideally X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset so a well behaved client can pace itself instead of guessing. Waiting the advertised Retry-After must actually restore service, and this is where implementations frequently lie: I assert that a request sent one second before the advertised reset still fails and one sent just after succeeds. Isolation matters too, so verify that hitting the limit on one API key does not throttle a different key, otherwise one noisy merchant can take down everyone.

Check the counter is not reset by a new connection or a different node behind the load balancer, which happens when the limit is held in per instance memory rather than Redis, and you can detect it by sending the burst and watching the allowed count exceed the documented limit. Two things I always add: assert that authentication endpoints are rate limited more aggressively than reads, because that is your brute force defence, and confirm that your own regression suite does not trip the limit, which is the most common cause of a nightly run going red for reasons that have nothing to do with the product.

# Burst past the documented limit of 60 per minute
for i in $(seq 1 70); do
  curl -s -o /dev/null -D - -w "%{http_code} " $BASE/v1/payments -H "Authorization: Bearer $TOKEN" \
    | grep -i -E 'x-ratelimit|retry-after' | tr '\n' ' '
  echo
done

# Expected shape of the 429
# HTTP/1.1 429 Too Many Requests
# Retry-After: 23
# X-RateLimit-Limit: 60
# X-RateLimit-Remaining: 0
# X-RateLimit-Reset: 1786003623
# {"error": {"code": "RATE_LIMIT_EXCEEDED", "message": "Try again in 23 seconds"}}

pm.test("429 tells the client how to behave", function () {
  pm.response.to.have.status(429);
  var retry = pm.response.headers.get("Retry-After");
  pm.expect(retry, "Retry-After missing").to.be.a("string");
  pm.expect(Number(retry)).to.be.above(0).and.below(3600);
  pm.expect(pm.response.headers.get("X-RateLimit-Remaining")).to.eql("0");
});

# Isolation: a different key must NOT be throttled
curl -o /dev/null -s -w "%{http_code}\n" $BASE/v1/payments -H "Authorization: Bearer $OTHER_KEY"

Answer

Offset pagination uses page and limit or offset and limit, and it is easy to reason about until the underlying data changes. If a new payment is inserted at the top of a list sorted by created_at descending while you are walking pages, everything shifts down by one, so the last item of page one reappears as the first item of page two and you see a duplicate. If a row is deleted instead, one record shifts up and is skipped entirely, which is worse because a reconciliation job silently misses a transaction.

Offset also degrades badly at depth, since offset 100000 makes the database count and discard a hundred thousand rows on every request. Cursor pagination hands back an opaque token pointing at the last item under a stable sort, so the next page continues from that position regardless of insertions above it. It fixes duplicates and skips, but brings its own test cases: an expired or malformed cursor must return a clean 400 rather than a 500, a cursor from a different filter or sort order must be rejected rather than silently returning wrong data, and cursors must not be guessable if they encode internal ids.

For both styles the core cases are the same: default limit when none is supplied, limit boundaries at zero, one, maximum and beyond maximum, the last page, an empty result set, and a total count if the API claims one. The test that actually finds the bug is the concurrent one: start walking pages, insert and delete rows in the middle of the walk, collect every id, and assert the union has no duplicates and no gaps against a known set.

# Offset pagination, unstable under writes
curl -s "$BASE/v1/payments?limit=20&offset=0"    # ids 100..81
# another user creates a payment (id 101) right now
curl -s "$BASE/v1/payments?limit=20&offset=20"   # ids 81..62  <- id 81 seen TWICE

# Cursor pagination, stable
curl -s "$BASE/v1/payments?limit=20"
# { "data": [...], "next_cursor": "eyJjcmVhdGVkX2F0IjoiMjAyNi0wOC0xN1QwOTowMCJ9" }
curl -s "$BASE/v1/payments?limit=20&cursor=eyJjcmVhdGVkX2F0IjoiMjAyNi0wOC0xN1QwOTowMCJ9"

// The assertion that catches duplicates and skips
var seen = JSON.parse(pm.collectionVariables.get("seenIds") || "[]");
var page = pm.response.json().data.map(function (p) { return p.payment_id; });

pm.test("no duplicate ids across pages", function () {
  var dupes = page.filter(function (id) { return seen.indexOf(id) !== -1; });
  pm.expect(dupes, "duplicated across pages: " + dupes).to.be.empty;
});
pm.collectionVariables.set("seenIds", JSON.stringify(seen.concat(page)));

# Malformed cursor must be 400, never 500
curl -o /dev/null -s -w "%{http_code}\n" "$BASE/v1/payments?cursor=not-a-cursor"

Answer

REST Assured expresses an HTTP call as a Gherkin style chain. given() owns everything about the request: base URI and base path, headers, content type, authentication, query and path parameters, cookies, the body, and any filters or specifications. when() owns the action, meaning the verb and the path, and each verb method actually fires the request. then() owns the assertions: statusCode, header, cookie, time, and body with a JsonPath expression plus a Hamcrest matcher such as equalTo, hasItem, hasSize, containsString, greaterThan or notNullValue. The chain returns a ValidatableResponse, and calling extract() at the end gives you the raw response so you can pull a value out for the next call or deserialise it into a POJO. Points that separate a real answer from a memorised one.

The body matcher syntax is Groovy GPath, not strict JSONPath, so you write body("data.items[0].sku", equalTo("PLAN_PRO")) without a dollar prefix, and you get Groovy collection operations like findAll and collect, which are extremely powerful for asserting over lists. Assertions in then() are chained and, by default, the first failure aborts, so use a soft assertion approach or split the test if you need all failures reported. log().ifValidationFails() attached to then() prints the request and response only when something breaks, which keeps CI output readable but still debuggable. And REST Assured is a client library, not a framework, so the test lifecycle, data providers, parallelism and reporting come from TestNG or JUnit around it.

import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;

@Test
public void capturedPaymentHasCorrectShapeAndValues() {
    given()
        .baseUri("https://api.example.in")
        .basePath("/v1")
        .header("Authorization", "Bearer " + TokenProvider.get())
        .header("X-Request-Id", "qa-" + UUID.randomUUID())
        .contentType(ContentType.JSON)
        .pathParam("paymentId", "pay_A1")
        .queryParam("expand", "order")
    .when()
        .get("/payments/{paymentId}")
    .then()
        .log().ifValidationFails()
        .statusCode(200)
        .time(lessThan(800L))
        .header("Content-Type", containsString("application/json"))
        .body("payment_id", equalTo("pay_A1"))
        .body("amount_paise", equalTo(49900))
        .body("currency", equalTo("INR"))
        .body("status", is(oneOf("captured", "authorized")))
        .body("order.order_id", notNullValue())
        .body("$", not(hasKey("pan")))
        .body("refunds.findAll { it.status == 'processed' }.size()", equalTo(0));
}

// extract() when you need the value downstream
String refundId =
    given().header("Authorization", "Bearer " + TokenProvider.get())
        .contentType(ContentType.JSON)
        .body("{\"amount_paise\": 49950}")
    .when().post("/v1/payments/pay_A1/refund")
    .then().statusCode(202)
    .extract().path("refund_id");

Answer

In a suite of two hundred tests, every one repeats the base URI, the auth header, the content type and a handful of common assertions. RequestSpecification captures the request half once and ResponseSpecification captures the shared assertions, and both are built with RequestSpecBuilder and ResponseSpecBuilder, then applied with spec(). The payoff is not just fewer lines: when the base path moves from /v1 to /v2 or the auth scheme changes, you edit one builder rather than two hundred tests, and when you add a global assertion such as every response must carry an X-Request-Id and must complete under two seconds, every existing test inherits it immediately.

That last point is the one interviewers like, because it turns a cross cutting requirement into a one line change. In practice I keep a base spec factory that reads the environment from a system property so the same suite runs against local, staging and preprod, and I build the token lazily inside the spec so tests do not each authenticate. There are two gotchas.

First, a spec is not immutable in the way people assume: reusing the same builder instance across parallel threads while mutating it produces cross talk, so build the spec fresh per test or make it genuinely immutable and shared read only. Second, over specifying hurts, because if the shared ResponseSpecification asserts statusCode 200 then no negative test can use it, so keep a permissive common spec for headers and timing and put status codes in the individual tests. RestAssured.requestSpecification and RestAssured.responseSpecification set them globally, which is convenient but hides behaviour, so I prefer explicit spec() calls.

public final class Specs {

    public static RequestSpecification base() {
        return new RequestSpecBuilder()
            .setBaseUri(System.getProperty("env.baseUri", "https://staging-api.example.in"))
            .setBasePath("/v1")
            .setContentType(ContentType.JSON)
            .addHeader("Authorization", "Bearer " + TokenProvider.get())
            .addHeader("X-Request-Id", "qa-" + UUID.randomUUID())
            .addFilter(new AllureRestAssured())
            .log(LogDetail.URI)
            .build();
    }

    // Permissive on purpose: no statusCode here, so negative tests can reuse it
    public static ResponseSpecification common() {
        return new ResponseSpecBuilder()
            .expectHeader("Content-Type", containsString("application/json"))
            .expectResponseTime(lessThan(2L), TimeUnit.SECONDS)
            .build();
    }
}

@Test
public void refundIsAccepted() {
    given().spec(Specs.base())
        .body(new RefundRequest(49950, "customer_request"))
    .when()
        .post("/payments/pay_A1/refund")
    .then()
        .spec(Specs.common())
        .statusCode(202)
        .body("status", equalTo("queued"));
}

Answer

REST Assured exposes extraction through extract().path() for a single value, extract().jsonPath() for a reusable JsonPath object, and extract().response() for everything. The expression language is Groovy GPath rather than the Jayway JSONPath most people know from Postman, and that difference is the source of most confusion. There is no dollar sign root, you index arrays with square brackets, and you get Groovy collection methods, so you can write data.findAll { it.status == 'captured' }.sum { it.amount_paise } directly in an assertion and compute a total without writing a loop.

Negative indices work, so data[-1] is the last element, and data.size() gives the count. The surprises are worth naming. Type coercion is strict: a JSON number extracted with path() comes back as Integer if it fits and Float or BigDecimal otherwise, so equalTo(49900) passes but equalTo(49900L) fails with a confusing message, and money fields are best extracted as String or BigDecimal to avoid float comparison entirely.

A path that does not exist returns null rather than throwing, so an assertion against a misspelled field silently passes when you compare with nullValue and silently fails in a confusing way otherwise. Extracting a list of a single element still gives you a List, not the element. For XML, XmlPath is the equivalent and namespaces must be declared with using(NamespaceContext) or your expressions match nothing at all, which produces vacuous passes. Finally, calling extract() after then() consumes the response, so keep one Response object if you need multiple reads rather than firing the request again.

Response res =
    given().spec(Specs.base())
    .when().get("/payments?limit=100")
    .then().statusCode(200)
    .extract().response();

JsonPath jp = res.jsonPath();

// simple extraction
String firstId = jp.getString("data[0].payment_id");
int count      = jp.getInt("data.size()");
String lastVpa = jp.getString("data[-1].vpa");

// Groovy collection power: no loops needed
List<String> failed = jp.getList("data.findAll { it.status == 'failed' }.payment_id");
int capturedTotal   = jp.getInt("data.findAll { it.status == 'captured' }.sum { it.amount_paise }");

Assert.assertEquals(capturedTotal, 149700, "captured total mismatch");
Assert.assertTrue(failed.isEmpty(), "unexpected failures: " + failed);

// Money as String to dodge float comparison
String amount = jp.getString("data[0].amount_display");
Assert.assertEquals(new BigDecimal(amount), new BigDecimal("499.00"));

// XML with namespaces, or your XPath matches nothing
XmlPath xp = new XmlPath(res.asString())
    .using(new XmlPathConfig().declaredNamespaces(Map.of("ban", "http://bank.example.in/accounts")));
String balance = xp.getString("Envelope.Body.GetBalanceResponse.balance");

Answer

Raw JSON strings in Java are painful and unsafe. Every quote needs escaping, the compiler cannot help you, a renamed field only fails at runtime, and building a variant payload means string concatenation. If Jackson or Gson is on the classpath, REST Assured serialises any object you pass to body() into JSON automatically based on the content type, and deserialises the response with extract().as(SomeClass.class).

Now the payload is a typed object: your IDE autocompletes fields, a rename is a compile error across every test, and you can build variants with a builder or by cloning and mutating one field, which makes boundary testing far cleaner. Deserialising the response gives you real objects to assert on, so you can compare an entire expected object with equals rather than writing fifteen body() matchers, and you can run business logic on the result, for example recomputing GST from line items and comparing it to the returned total. The trade offs interviewers expect you to know.

First, a POJO with strict Jackson configuration will fail on unknown properties, which is either a feature (it catches contract drift, same benefit as strict JSON Schema) or a nuisance for third party APIs, controlled with FAIL_ON_UNKNOWN_PROPERTIES. Second, POJOs hide malformed payloads: you can no longer easily send a field with the wrong type or an extra unexpected key, which is exactly what negative tests need, so keep raw string or Map based payloads for the malformed cases. Third, use BigDecimal for money and never double, and prefer integer paise, because a float total in a payments test will eventually produce a rounding mismatch that costs you an afternoon.

public class RefundRequest {
    @JsonProperty("amount_paise") private long amountPaise;
    @JsonProperty("reason")       private String reason;
    @JsonProperty("speed")        private String speed = "normal";
    // constructor, getters, builder omitted
}

@JsonIgnoreProperties(ignoreUnknown = false)   // strict: catches contract drift
public class RefundResponse {
    @JsonProperty("refund_id")    public String refundId;
    @JsonProperty("status")       public String status;
    @JsonProperty("amount_paise") public long amountPaise;
    @JsonProperty("created_at")   public OffsetDateTime createdAt;
}

@Test
public void partialRefundReturnsTypedResponse() {
    RefundResponse refund =
        given().spec(Specs.base())
            .body(new RefundRequest(49950, "customer_request"))
        .when().post("/payments/pay_A1/refund")
        .then().statusCode(202)
        .extract().as(RefundResponse.class);

    Assert.assertEquals(refund.amountPaise, 49950);
    Assert.assertEquals(refund.status, "queued");
    Assert.assertTrue(refund.refundId.startsWith("rfnd_"));
}

// Malformed payloads still need raw form
@Test
public void amountAsStringIsRejected() {
    given().spec(Specs.base())
        .body("{\"amount_paise\": \"49950\"}")
    .when().post("/payments/pay_A1/refund")
    .then().statusCode(400);
}

Answer

A filter implements the Filter interface and sits in the request pipeline, so it can inspect or modify the request before it goes out and the response before assertions run. The common uses are logging, injecting a correlation id or auth token on every call, capturing request and response into an Allure or ExtentReports attachment, measuring latency, and retrying a specific class of transient failure. RestAssured ships RequestLoggingFilter and ResponseLoggingFilter, and ResponseLoggingFilter.logResponseIfStatusCodeIs(greaterThan(399)) is the sensible default because it prints only when something went wrong.

The PII problem is real in Indian projects. KYC and onboarding APIs carry Aadhaar numbers, PAN, bank account numbers and OTPs in request bodies, and a suite that logs full payloads writes them into the Jenkins console, the CI artefact store and any log shipper attached to it. That is a data protection incident created by the QA team.

The fix is a custom filter that redacts before writing: parse the body, replace known sensitive keys with a masked value, mask the Authorization header, and only then log. Masking should keep the last four digits at most, and it should be key based rather than regex based on the value, since a regex for a twelve digit number will also mangle amounts and timestamps. I also strip secrets from the curl reproduction we print for failures, and I make the redaction list a shared constant so a new sensitive field is added in one place. If the team uses Allure, the same filter attaches the redacted request and response to the report, which gives you the debugging value without the exposure.

public class RedactingLogFilter implements Filter {

    private static final Set<String> SENSITIVE =
        Set.of("aadhaar", "aadhaar_number", "pan", "account_number", "otp", "cvv", "password");

    @Override
    public Response filter(FilterableRequestSpecification req,
                           FilterableResponseSpecification res,
                           FilterContext ctx) {

        Response response = ctx.next(req, res);

        if (response.statusCode() >= 400) {
            System.out.println("REQ  " + req.getMethod() + " " + req.getURI());
            System.out.println("AUTH Bearer ****redacted****");
            System.out.println("BODY " + redact(req.getBody()));
            System.out.println("RESP " + response.statusCode() + " " + redact(response.asString()));
        }
        return response;
    }

    private String redact(Object body) {
        if (body == null) return "";
        try {
            ObjectNode node = (ObjectNode) new ObjectMapper().readTree(body.toString());
            SENSITIVE.forEach(k -> { if (node.has(k)) node.put(k, "****"); });
            return node.toString();
        } catch (Exception e) {
            return "<unparseable body suppressed>";
        }
    }
}

// wire it once in the shared spec
// .addFilter(new RedactingLogFilter())
// .addFilter(ResponseLoggingFilter.logResponseIfStatusCodeIs(greaterThan(399)))

Answer

Maven gives you dependency management and the surefire plugin that runs the tests; TestNG gives you grouping, parallelism, data providers, listeners and retry. A structure that survives contact with a real team looks like this: src/test/java split into clients (thin wrappers per API resource that own the endpoint paths and return typed responses), models (the POJOs), tests (assertions only), and a base or util package holding the specs, the token provider and the filters. Test classes never contain URLs, because when a path changes you want to edit one client class.

Configuration comes from a properties file per environment plus system property overrides, so mvn test -Denv=staging picks the right base URI and credentials, and CI passes secrets as system properties or environment variables rather than committing them. testng.xml defines suites by group: smoke for the deploy gate, regression for nightly, and a separate suite for the long running or downstream dependent tests you do not want blocking a release. TestNG groups let one class contribute to several suites. Parallelism is set in the suite file with parallel=methods and a thread count, which is where most suites break, because shared static state such as one mutable RequestSpecification or a shared test user causes cross talk.

Make tests independent, create their own data, and clean it up in an @AfterMethod. Add an IRetryAnalyzer that retries once for genuinely transient network failures and reports the retry, but never let it mask a real flaky test, and wire an ITestListener that attaches the redacted request and response to the report on failure.

# pom.xml
<dependencies>
  <dependency><groupId>io.rest-assured</groupId><artifactId>rest-assured</artifactId><version>5.5.0</version><scope>test</scope></dependency>
  <dependency><groupId>org.testng</groupId><artifactId>testng</artifactId><version>7.10.2</version><scope>test</scope></dependency>
  <dependency><groupId>com.fasterxml.jackson.core</groupId><artifactId>jackson-databind</artifactId><version>2.17.2</version><scope>test</scope></dependency>
</dependencies>

<build><plugins>
  <plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.2.5</version>
    <configuration>
      <suiteXmlFiles><suiteXmlFile>${suite}</suiteXmlFile></suiteXmlFiles>
      <systemPropertyVariables>
        <env.baseUri>${env.baseUri}</env.baseUri>
      </systemPropertyVariables>
    </configuration>
  </plugin>
</plugins></build>

# testng-smoke.xml
<suite name="smoke" parallel="methods" thread-count="6">
  <listeners><listener class-name="qa.listeners.AllureAttachListener"/></listeners>
  <test name="payments-smoke">
    <groups><run><include name="smoke"/></run></groups>
    <packages><package name="qa.tests.*"/></packages>
  </test>
</suite>

# CI
mvn -B clean test -Dsuite=testng-smoke.xml -Denv.baseUri=https://staging-api.example.in \
  -Dclient.secret=$STAGING_CLIENT_SECRET

Answer

A Postman mock server takes saved example responses from a collection and serves them at a generated URL, matching an incoming request to the closest saved example. Its real value is unblocking work. When the backend for a new checkout flow is two sprints away but the OpenAPI spec is agreed, the frontend team and the QA team can both build against the mock immediately: the app developer wires the integration, and I write and debug my assertions, my schema files and my chaining logic so that on the day the real service appears the suite is ready.

It is also useful for reproducing responses that are painful to trigger for real, such as a specific bank decline code or a 500 from a partner, and for demoing a flow when staging is down. Where it is the wrong tool: a mock proves nothing about the real service. It has no database, no business logic, no concurrency, no latency profile and no authorization, so a suite that passes against a mock and is never rerun against the real API creates a dangerous illusion of coverage.

Mocks also drift, because someone updates the API and forgets the saved examples, at which point your tests are validating a fiction. The discipline is to treat mocks as scaffolding: keep the examples generated from or validated against the OpenAPI spec, run the same collection against staging as soon as it exists, and never let the mock run in the release gate. If you need conditional or stateful behaviour, richer request matching, fault injection or delay simulation, WireMock or Mockoon is the better choice, and for provider verification you want Pact rather than a mock at all.

# A saved example in the collection becomes the mock response
# Request:  GET /v1/payments/pay_A1
# Header matched:  x-mock-response-name: captured-payment

curl -s https://a1b2c3.mock.pstmn.io/v1/payments/pay_A1 \
  -H "x-api-key: $POSTMAN_MOCK_KEY" \
  -H "x-mock-response-name: bank-declined"
# {"payment_id":"pay_A1","status":"failed","error_code":"BANK_DECLINED"}

// Same collection, two environments: mock now, staging later
// envs/mock.json      baseUrl = https://a1b2c3.mock.pstmn.io
// envs/staging.json   baseUrl = https://staging-api.example.in

# Build the assertions against the mock today
newman run checkout.postman_collection.json -e envs/mock.json

# Re-point at the real service the day it lands, no test changes
newman run checkout.postman_collection.json -e envs/staging.json

Answer

They answer different questions. A CI run answers is this build correct, and it runs on a code event, against staging, before anything reaches users, and its job is to block a bad deploy. A Postman monitor answers is production working right now, and it runs on a schedule from Postman's cloud regions, hitting the live environment continuously and alerting when something breaks between deploys.

That distinction matters because plenty of production incidents have nothing to do with a deploy: an expired TLS certificate, a rotated third party key, a DNS change, a partner gateway going down, a Redis eviction that suddenly makes a cached endpoint slow. CI would never catch those because CI is not running. Practically I keep a small production monitor of read only, side effect free checks: health endpoints, a public catalogue read, a login with a dedicated synthetic account, and latency assertions.

Never run a monitor that creates payments or sends notifications, because it will pollute production data and, on a payments product, actually move money. Monitors can run from multiple regions, which is how you notice that the API is fine from Mumbai but slow from Singapore, and they can post failures to Slack or a webhook. Their limits are worth stating: they run in Postman's cloud so they cannot reach a private VPC endpoint without a Postman agent, they have execution time limits, and the alerting is basic compared with a real observability stack. For serious production monitoring most Indian product teams eventually move to their own synthetic checks in Datadog, Grafana or a scheduled Newman job on their own infrastructure, and keep Postman monitors for quick coverage and for APIs that are genuinely public.

Answer

Downstream sandboxes in India are genuinely unreliable: Aadhaar and bank sandboxes have maintenance windows, rate caps and unannounced outages, and a suite that goes red every time one of them hiccups gets ignored, which is worse than having no suite. WireMock lets you stand up a real HTTP server that stubs the dependency. You define stub mappings matching on method, URL, headers and body patterns, and return canned status codes and bodies, and you point the service under test at the WireMock URL through configuration rather than changing code.

What makes WireMock better than a static mock is what it can simulate: fixed and random delays so you can test your own timeout handling, fault injection such as an empty response, a connection reset or malformed data, stateful scenarios so the first call returns pending and the second returns verified, and request verification so you can assert your service actually called the dependency with the right payload and the right number of times. That last capability is the one people forget, and it is how you prove a retry policy works. Record and playback mode is the fastest way to build realistic stubs: point WireMock at the real sandbox while it is up, capture the traffic, then replay it forever.

The discipline is to be explicit about what stubbing buys you. Stubbed tests verify your integration logic, your error handling and your retries, and they must be fast and always green. They do not verify that the real sandbox still behaves this way, so you keep a small separate contract suite that runs against the real dependency on a schedule and is allowed to be flagged as an environment issue rather than blocking the pipeline.

// WireMock stub: happy path
stubFor(post(urlEqualTo("/uidai/v2/verify"))
    .withHeader("Content-Type", equalTo("application/json"))
    .withRequestBody(matchingJsonPath("$.reference_id"))
    .willReturn(aResponse()
        .withStatus(200)
        .withHeader("Content-Type", "application/json")
        .withBody("{\"status\":\"verified\",\"name_match\":true,\"ref\":\"UID-8812\"}")
        .withFixedDelay(400)));

// Timeout behaviour: does OUR service degrade gracefully?
stubFor(post(urlEqualTo("/uidai/v2/verify"))
    .inScenario("outage").whenScenarioStateIs("slow")
    .willReturn(aResponse().withFixedDelay(30000)));

// Fault injection
stubFor(post(urlEqualTo("/uidai/v2/verify"))
    .inScenario("outage").whenScenarioStateIs("reset")
    .willReturn(aResponse().withFault(Fault.CONNECTION_RESET_BY_PEER)));

// Stateful: pending first, verified second
stubFor(get(urlPathMatching("/uidai/v2/status/.*"))
    .inScenario("kyc").whenScenarioStateIs(Scenario.STARTED)
    .willReturn(okJson("{\"status\":\"pending\"}"))
    .willSetStateTo("done"));

// Prove the retry policy actually retried
verify(3, postRequestedFor(urlEqualTo("/uidai/v2/verify")));

Answer

If the OpenAPI document is genuinely maintained, it is the cheapest testing asset you have. Four things you can do with it. Generate request skeletons and a Postman collection so you are not hand typing forty endpoints, which is what the import feature is for.

Validate live responses against the response schemas defined in the spec, using a validator that takes the OpenAPI document directly, so you do not maintain a second set of JSON Schema files that drift from it. Generate boundary cases automatically from the constraints already declared, since minLength, maximum, enum and pattern tell you exactly what the invalid values are. And run a spec linter such as Spectral in CI so the document itself stays healthy, catching missing examples, undocumented error responses and inconsistent naming.

Drift detection is the important part, because in most Indian teams the spec is written once and then quietly diverges from the code. There are two directions of drift. The spec says something the API no longer does, which you catch by validating real staging responses against the spec on every nightly run and failing when a response does not match.

And the API does something the spec never mentioned, which you catch by comparing the set of live endpoints and fields against the document, or by making the validation strict on additional properties. The strongest version is to generate the spec from the code annotations so it cannot drift, and then diff the generated spec against the committed one in CI, failing the build on an unreviewed change. That turns every contract change into a visible pull request diff, which is exactly where a breaking change should be argued about rather than discovered by a mobile client in production.

# Lint the spec itself in CI
npx @stoplight/spectral-cli lint openapi.yaml -r .spectral.yaml

# Validate live staging responses against the spec (schemathesis)
schemathesis run https://staging-api.example.in/openapi.json \
  -H "Authorization: Bearer $TOKEN" -c all

# Fail the build when the generated spec differs from the committed one
mvn -q springdoc:generate -Dspec.out=build/openapi.json
diff <(jq -S . openapi.json) <(jq -S . build/openapi.json) \
  || { echo "API contract changed without updating openapi.json"; exit 1; }

// REST Assured: validate against the spec, not a hand written schema
import static io.restassured.module.jsv.JsonSchemaValidator.matchesJsonSchemaInClasspath;

given().spec(Specs.base())
.when().get("/payments/pay_A1")
.then().statusCode(200)
  .body(matchesJsonSchemaInClasspath("schemas/payment.schema.json"));

Answer

In a microservice estate the expensive failure is a provider changing a field that some consumer depended on. Full integration testing catches it but requires every service deployed together in one environment, which is slow, shared, and the first thing to break when one team deploys mid run. Pact inverts the problem.

Each consumer writes a test against a local Pact mock describing exactly the requests it makes and the responses it needs, and running that test generates a pact file, a machine readable contract. The pact is published to a broker. The provider then runs verification: it replays every interaction from every consumer's pact against its real implementation and fails if any expectation is broken.

The key property is that the contract describes only what consumers actually use, so a provider is free to add fields or change anything nobody depends on, and gets a hard failure the moment it breaks something real. Two mechanisms make this work in practice. Provider states let a pact say given a captured payment exists, and the provider sets up that state before replaying, which is how you avoid needing shared data.

Can-i-deploy queries the broker in the deployment pipeline and blocks a release when the version you are shipping has not been verified against the consumers currently in production. The limits are worth stating honestly: Pact verifies shape and semantics of the interaction, not business correctness or performance, it is a poor fit for public APIs with unknown consumers where OpenAPI plus schema validation is better, and it needs both teams bought in, because a pact nobody verifies is just a file. Used well it removes the need for most cross service integration tests and makes independent deploys safe.

// Consumer side (Node): describe only what you actually use
const provider = new PactV3({ consumer: "checkout-web", provider: "payments-api" });

provider
  .given("a captured payment pay_A1 exists")
  .uponReceiving("a request for payment status")
  .withRequest({ method: "GET", path: "/v1/payments/pay_A1",
                 headers: { Authorization: like("Bearer token") } })
  .willRespondWith({
    status: 200,
    headers: { "Content-Type": "application/json" },
    body: {
      payment_id: like("pay_A1"),
      status: term({ matcher: "captured|failed|refunded", generate: "captured" }),
      amount_paise: integer(49900)
    }
  });

# publish (PACT_BROKER_BASE_URL and PACT_BROKER_TOKEN come from CI secrets)
pact-broker publish ./pacts -a $GIT_SHA

# provider verification in the payments-api pipeline
mvn test -Dpact.verifier.publishResults=true -Dpact.provider.version=$GIT_SHA

# deployment gate: fails if a consumer currently in production would break
pact-broker can-i-deploy -a payments-api -e $GIT_SHA

Answer

BOLA, also called IDOR, is number one on the OWASP API Security Top 10 because it is trivially exploitable, extremely common, and completely invisible to functional testing. The pattern: an endpoint takes a resource id, checks that the caller is authenticated, and forgets to check that the caller owns that resource. On a lending product that means user A can read user B's loan application, complete with income proof, PAN and bank statements, by changing a number in the URL.

Functional tests never catch it because every test uses its own token against its own data. The suite has to be built deliberately. Create at least two users in different tenants plus one privileged role, and store both tokens.

For every endpoint that accepts an identifier, generate the cross access case: A's token against B's resource, on every verb, because teams often protect GET and forget PATCH and DELETE. Expect 404, and assert the response is indistinguishable from a genuinely missing resource. Then the variations that catch partial fixes: the id in the body rather than the path, a nested id such as /loans/{id}/documents/{docId} where the parent is checked but the child is not, an id inside a bulk or batch request where only the first element is validated, a filter parameter such as user_id that overrides the token identity, and an export or report endpoint that ignores tenant scoping entirely.

Also test predictable identifiers, since sequential ids make enumeration trivial and UUIDs make it impractical. The right way to run this is generated rather than hand written: drive it from the OpenAPI spec so every new endpoint automatically gets a cross tenant case, otherwise the coverage decays the moment someone ships a new route.

# Two real users, two real tokens
A_TOKEN=$(login qa+a@goodspace.ai); A_LOAN=loan_5001
B_TOKEN=$(login qa+b@goodspace.ai); B_LOAN=loan_5002

# Cross access on every verb, not just GET
for VERB in GET PATCH DELETE; do
  CODE=$(curl -o /dev/null -s -w "%{http_code}" -X $VERB \
    "$BASE/v1/loans/$B_LOAN" -H "Authorization: Bearer $A_TOKEN")
  echo "$VERB $B_LOAN as user A -> $CODE"   # must be 404 every time
done

# Nested child: parent checked, child forgotten
curl -o /dev/null -s -w "%{http_code}\n" \
  "$BASE/v1/loans/$A_LOAN/documents/doc_of_user_b" -H "Authorization: Bearer $A_TOKEN"

# Id in the body, not the path
curl -i -X POST $BASE/v1/loans/repay -H "Authorization: Bearer $A_TOKEN" \
  -d '{"loan_id":"loan_5002","amount_paise":100000}'

# Filter parameter overriding token identity
curl -s "$BASE/v1/loans?user_id=usr_b" -H "Authorization: Bearer $A_TOKEN" | jq '.data | length'

# Batch where only element[0] is validated
curl -i -X POST $BASE/v1/loans/bulk-status -H "Authorization: Bearer $A_TOKEN" \
  -d '{"loan_ids":["loan_5001","loan_5002"]}'

Answer

These are two sides of trusting the object mapper. Mass assignment happens when a framework binds every field in the request body straight onto a model, so a client can set attributes the UI never exposes. The test is to take a legitimate update request and add privileged fields: role admin, kyc_status verified, credit_limit, is_internal, wallet_balance, merchant_id, verified_at.

Then the crucial second step, which candidates routinely miss: a 200 response is not the answer, you must re read the resource or query the database, because a well behaved framework ignores unknown fields silently while a vulnerable one accepts them silently, and both return 200. If the field stuck, you have found a serious defect. On a KYC API, setting kyc_status to verified means bypassing identity verification entirely.

Excessive data exposure is the reverse: the API returns the whole model and expects the client to display a subset. A GET on a profile that returns aadhaar_number, pan, full bank account number, internal risk_score and password_hash is a breach regardless of what the app renders, because anyone can read the raw response. The test asserts on absence, with a shared deny list of forbidden keys applied to every response in the suite, plus a check that anything genuinely needed is masked to last four digits.

Nested and expanded objects are where this hides: a loan response that embeds the full applicant object, or an ?expand=user parameter that bypasses the serialiser. List endpoints are worse than single reads because they leak in bulk. I run the deny list check as a global assertion rather than per endpoint, so a new field added anywhere trips it.

# Mass assignment attempt: privileged fields smuggled into a normal update
curl -s -X PATCH $BASE/v1/users/me -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"display_name":"Aarav","kyc_status":"verified","role":"admin","credit_limit_paise":50000000}'
# 200 OK means NOTHING. Re-read:
curl -s $BASE/v1/users/me -H "Authorization: Bearer $TOKEN" | jq '{kyc_status, role, credit_limit_paise}'
# expect kyc_status "pending", role "user", credit_limit unchanged

// Global deny list, applied to every response in the suite
var FORBIDDEN = ["aadhaar_number", "pan", "password_hash", "otp", "risk_score",
                 "internal_notes", "account_number", "ifsc_secret"];

pm.test("no excessive data exposure", function () {
  var raw = pm.response.text();
  FORBIDDEN.forEach(function (key) {
    pm.expect(raw, "leaked field: " + key).to.not.include("\"" + key + "\"");
  });
});

pm.test("sensitive identifiers are masked, not returned in full", function () {
  var body = pm.response.json();
  if (body.pan_masked) {
    pm.expect(body.pan_masked).to.match(/^[X*]{6}\d{4}$/);
  }
});

# Expansion parameters bypass serialisers, test them explicitly
curl -s "$BASE/v1/loans/loan_5001?expand=applicant" -H "Authorization: Bearer $TOKEN" | jq 'keys'

Answer

The false sense of security comes from ORMs. Teams assume parameterised queries everywhere, but injection survives in the gaps: raw SQL for reporting and reconciliation queries, dynamic ORDER BY and column names which cannot be parameterised, LIKE patterns built by concatenation, search endpoints that pass user input to Elasticsearch or a NoSQL query builder, and anything shelling out to a command such as a PDF generator or an image resizer. So the test surface is every parameter, not just the obvious ones, including headers and JSON values, not only query strings.

The signals I look for are not the classic error page. A 500 with a database exception is the easy case. The realistic ones are behavioural: a sort parameter that accepts an arbitrary column name and changes the result order, a filter that returns more rows when you append an always true condition, a timing difference when you inject a sleep, and NoSQL operator injection where sending an object such as a not equal operator in place of a string bypasses an authentication check entirely.

On the templating side, server side template injection in a notification or invoice endpoint is worth probing because Indian products often render SMS and email templates from user supplied fields. Command injection shows up wherever a filename or a path from the request reaches a shell. The discipline is to combine automated scanning with targeted manual work: run OWASP ZAP or Burp in the pipeline for the broad sweep, and hand write the cases for the endpoints where you know the implementation uses dynamic SQL. And every finding must be confirmed by effect, not by an error message, since a 500 can equally mean a null pointer.

# Classic, usually blocked, still worth one pass
curl -s "$BASE/v1/payments?q=aarav'%20OR%20'1'%3D'1" -H "Authorization: Bearer $TOKEN" | jq '.data | length'

# The real gap: dynamic ORDER BY cannot be parameterised
curl -s "$BASE/v1/payments?sort=amount_paise" | jq '.data[0].payment_id'
curl -s "$BASE/v1/payments?sort=(CASE WHEN 1=1 THEN amount_paise ELSE id END)" | jq '.data[0].payment_id'
# a different ordering here means user input reaches the SQL text

# NoSQL operator injection: object where a string is expected
curl -i -X POST $BASE/v1/auth/login -H "Content-Type: application/json" \
  -d '{"email":"admin@example.in","password":{"$ne":null}}'
# a 200 here is a total authentication bypass

# Time based confirmation, effect not error message
curl -o /dev/null -s -w "%{time_total}\n" "$BASE/v1/reports?from=2026-08-01"
curl -o /dev/null -s -w "%{time_total}\n" "$BASE/v1/reports?from=2026-08-01'%20AND%20SLEEP(5)%23"

# Command injection where a filename reaches a shell
curl -X POST $BASE/v1/invoices/render -d '{"filename":"inv.pdf; id"}'

# Broad sweep in CI
docker run -t ghcr.io/zaproxy/zaproxy zap-api-scan.py -t $BASE/openapi.json -f openapi -r zap.html

Answer

Load testing runs the expected production traffic to confirm the system meets its targets at normal volume, and it is the baseline everything else is compared against. Stress testing deliberately pushes past capacity to find the breaking point and, more importantly, the failure mode: does the service shed load gracefully with 429s and 503s, or does it queue until threads exhaust and take the database with it. Spike testing applies a sudden vertical jump, which is the realistic Indian scenario, a flash sale on Flipkart or Meesho, an IPL match ending, a salary day at 10am, and it tests autoscaling reaction time and connection pool behaviour rather than steady throughput.

Soak testing holds a moderate load for hours or days to expose the slow failures: memory leaks, connection pool exhaustion, unbounded caches, log disks filling, and the classic one where a token cached at startup expires eight hours in and everything fails at once. Averages are useless because latency distributions are heavily right skewed. If ninety five requests take 100ms and five take 5 seconds, the average is 345ms and looks acceptable while one in twenty users waits five seconds. p95 and p99 describe the tail that users actually complain about, and on a payments API the tail is where retries, double submissions and support tickets originate.

Two refinements that mark an experienced answer. Percentiles must never be averaged across intervals or across load generators, because the average of percentiles is not a percentile, which is a real defect in naively built dashboards. And coordinated omission matters: a load generator that waits for a slow response before sending the next request under reports the tail badly, which is why open model tools and correctly configured JMeter throughput timers matter.

// k6: four profiles, one script
export const options = {
  scenarios: {
    load:   { executor: "constant-arrival-rate", rate: 400, timeUnit: "1s",
              duration: "15m", preAllocatedVUs: 200 },
    spike:  { executor: "ramping-arrival-rate", startRate: 100, timeUnit: "1s",
              preAllocatedVUs: 500, startTime: "20m",
              stages: [{ target: 100, duration: "1m" },
                       { target: 3000, duration: "30s" },
                       { target: 100, duration: "3m" }] },
    soak:   { executor: "constant-arrival-rate", rate: 150, timeUnit: "1s",
              duration: "8h", preAllocatedVUs: 150, startTime: "30m" }
  },
  thresholds: {
    "http_req_duration{endpoint:collect}": ["p(95)<800", "p(99)<2000"],
    "http_req_failed": ["rate<0.005"],
    "checks": ["rate>0.995"]
  }
};

// Why the average lies
// 95 requests @ 100ms, 5 requests @ 5000ms
// mean = 345ms  (looks fine)
// p95  = 100ms, p99 = 5000ms  (one in twenty users waits 5 seconds)

# arrival rate executors avoid coordinated omission
k6 run -o experimental-prometheus-rw upi_collect.js

Answer

The plan mirrors the real transaction, not a single endpoint. A UPI collect involves creating an order, initiating a collect request, the payer approving it out of band, and a webhook landing asynchronously, so the JMeter plan has a thread group per stage with realistic think time between them rather than one hot loop hammering a single URL. Structure: a setup thread group that authenticates once and stores the token in a property, a main thread group driving the collect creation, an HTTP Request Defaults element for the host, a CSV Data Set Config supplying distinct VPAs and amounts so every virtual user works on unique data (reusing one VPA turns the test into a lock contention benchmark), a JSON Extractor to carry the payment id forward, and assertions kept deliberately light because heavy assertions consume the generator's own CPU.

For arrival control use the Concurrency Thread Group with a Throughput Shaping Timer so you drive requests per second rather than threads, since thread count is not load. At 2000 rps a single machine will not do it: JMeter is heavy per thread, so you run distributed mode with several load generators or you use k6 or Gatling which handle this volume far more cheaply, and you always run the generators in the same region as the service or you are measuring the internet. Always run in non GUI mode. What I watch is more important than the numbers JMeter prints: p95 and p99 per stage rather than aggregate, error rate broken down by status code, and the server side signals, meaning CPU, database connection pool saturation, GC pauses, queue depth for the webhook consumer, and the downstream NPCI or bank sandbox response time, because in almost every Indian payment load test the bottleneck turns out to be a connection pool or the downstream, not application code.

Test Plan: upi-collect-2000rps
  User Defined Variables: BASE_URL, MERCHANT_ID
  HTTP Request Defaults: https://perf-api.example.in : 443, keep-alive on
  HTTP Cookie Manager, HTTP Cache Manager (disabled for API tests)

  setUp Thread Group (1 thread, 1 loop)
    HTTP Request  POST /oauth2/token
    JSON Extractor -> access_token
    JSR223 PostProcessor: props.put("token", vars.get("access_token"))

  Thread Group: collect  (Concurrency Thread Group)
    Throughput Shaping Timer:
      0    -> 200 rps  over 120s   (warm up)
      200  -> 2000 rps over 300s   (ramp)
      2000 -> 2000 rps for 900s    (steady state, this is the measurement window)
    CSV Data Set Config: vpas.csv  (vpa, amount) recycle=false, sharing=all threads
    HTTP Header Manager: Authorization: Bearer ${__P(token)}
    HTTP Request  POST /v1/payments/collect
      body {"vpa":"${vpa}","amount_paise":${amount},"merchant_id":"${MERCHANT_ID}"}
    JSON Extractor -> payment_id
    Response Assertion: status 201 only  (keep assertions cheap)
    Constant Timer: 800ms  (think time before status poll)
    HTTP Request  GET /v1/payments/${payment_id}

  Backend Listener -> InfluxDB -> Grafana   (no GUI listeners)

# non GUI, distributed
jmeter -n -t upi-collect.jmx -R 10.0.1.11,10.0.1.12,10.0.1.13 -l results.jtl -e -o report/

Answer

CORS is a browser policy, not a server security control, which is exactly why curl succeeds. The browser enforces the same origin policy and will not let JavaScript on one origin read a response from another unless the server explicitly permits it with Access-Control-Allow-Origin. For simple requests, meaning GET, HEAD or POST with a basic content type and no custom headers, the browser sends the request and then blocks the response if the header is missing, so the server actually processed it.

That distinction matters: a failed CORS POST may have already created the record. For anything else, and that includes any request with an Authorization header or Content-Type application/json, the browser first sends an OPTIONS preflight carrying Access-Control-Request-Method and Access-Control-Request-Headers, and only proceeds if the response allows them. Most CORS bugs are a preflight problem: the OPTIONS route is not implemented and returns 404 or 405, or the gateway requires authentication on OPTIONS, which is a guaranteed failure because browsers never send credentials on a preflight.

Testing it means sending the OPTIONS request explicitly with curl and asserting on the response headers, then checking the actual call. The security cases matter as much as the functional one. Access-Control-Allow-Origin set to a wildcard together with Access-Control-Allow-Credentials true is invalid and browsers reject it, but a server that reflects whatever Origin the client sent while allowing credentials is effectively wide open, so I always send a hostile Origin and assert it is not echoed back. I also check Access-Control-Max-Age, since a missing value makes the browser preflight on every single call and doubles the request count on a mobile connection.

# The preflight the browser sends, reproduced by hand
curl -i -X OPTIONS https://api.example.in/v1/payments \
  -H "Origin: https://app.goodspace.ai" \
  -H "Access-Control-Request-Method: POST" \
  -H "Access-Control-Request-Headers: authorization,content-type"

# Expected
# HTTP/1.1 204 No Content
# Access-Control-Allow-Origin: https://app.goodspace.ai
# Access-Control-Allow-Methods: GET, POST, PATCH, DELETE, OPTIONS
# Access-Control-Allow-Headers: authorization, content-type, x-request-id
# Access-Control-Allow-Credentials: true
# Access-Control-Max-Age: 600

# Common failure: gateway demands auth on OPTIONS -> browsers never send it
# HTTP/1.1 401 Unauthorized      <- CORS error in the browser, curl still works

# Security case: does the server reflect ANY origin?
curl -si -X OPTIONS https://api.example.in/v1/payments \
  -H "Origin: https://attacker.example.com" \
  -H "Access-Control-Request-Method: POST" | grep -i access-control-allow-origin
# echoing attacker.example.com back while allowing credentials is a real vulnerability

# Wildcard plus credentials is invalid and will be rejected by the browser
# Access-Control-Allow-Origin: *
# Access-Control-Allow-Credentials: true

Answer

First I get data rather than opinions: for two weeks I tag every failure as a real defect, an environment problem, or a test problem, and I record which step failed. In almost every suite I have seen, the majority of failures cluster in setup steps such as login, navigation and data creation, not in the assertion the test was written for. That immediately tells you what to move.

The rewrite is in three moves. Move all setup to the API, so a test that needs a captured payment seeds it with four HTTP calls in two seconds rather than clicking through checkout, which removes most of the flakiness and most of the runtime at once. Move all business rule verification to API tests, since GST calculation, refund eligibility, credit limits, interest computation and validation rules do not need a browser and are far more precisely assertable through JSON, including negative and boundary cases that are painful in a UI.

Then keep in the UI only what genuinely requires a browser: that the element renders, that the flow is navigable, that a payment redirect returns correctly, and a small number of critical end to end journeys, typically five to fifteen for a product, not four hundred. What is left is a UI suite of maybe thirty minutes and an API suite of a few minutes that runs on every commit. Two things make the change stick.

Report the coverage you moved rather than the tests you deleted, because deleting tests reads as reducing coverage to a manager unless you show the replacement. And add a quarantine mechanism, where a test that fails intermittently is moved out of the gate with an owner and a deadline, so the suite stays trustworthy while the flake is actually fixed rather than being re run until it passes.

Answer

The pattern of only at night is itself evidence, and the first job is to find what is different about that time and that context rather than rerunning until it goes green. I work through five hypotheses in order. Timing: nightly runs often collide with scheduled jobs, backups, a settlement batch or a cache warm, so I check the deploy and cron calendar for the window.

Data: overnight cleanup jobs, a database refresh from production, or a colleague's suite deleting shared users will break tests that depend on data created earlier, and the fix is per run data rather than shared fixtures. Concurrency: the nightly run may be the only time the suite runs fully parallel, so tests that share a user, an idempotency key or a sequence collide only there. Configuration: staging is redeployed at night with a different feature flag, a rotated secret, or a config value that only CI passes, and comparing the resolved configuration between the two runs settles it quickly.

Downstream: bank and Aadhaar sandboxes have maintenance windows overnight, which is a very common cause in Indian fintech and looks exactly like a product bug. The tooling that makes this fast is observability rather than cleverness. Every request from the suite carries an X-Request-Id containing the run id and the test name, so a failure in the report links directly to the server logs and traces for that exact call.

Then I compare the failing nightly trace with a passing manual one side by side, and the divergence, a different downstream latency, a different feature flag, a missing header, is usually visible immediately. I do not close it as environmental without naming the specific mechanism, because environmental is where real defects go to hide.

// Make every request traceable back to the exact test and run
String requestId = String.format("qa-%s-%s-%s",
    System.getenv().getOrDefault("CI_RUN_ID", "local"),
    testContext.getName(),
    UUID.randomUUID().toString().substring(0, 8));

given().spec(Specs.base()).header("X-Request-Id", requestId)
.when().post("/v1/payments")
.then().statusCode(201);

# On failure, pull the exact server-side story
grep "qa-4412-refundIsAccepted" /var/log/payments/app.log | jq -r '.msg'

# Diff the two environments' resolved config
diff <(curl -s $BASE/internal/config -H "X-Debug-Key: $K" | jq -S 'del(.secrets)') \
     <(cat config/expected-staging.json | jq -S .)

# Was the downstream sandbox actually up during the run window?
curl -s "$METRICS/api/v1/query_range?query=upstream_latency_p99{svc='uidai'}&start=$T1&end=$T2&step=60" \
  | jq '.data.result[0].values[-5:]'

# Is it order dependent, or genuinely intermittent?
mvn test -Dtest=RefundTest -DrerunFailingTestsCount=0 -Dsurefire.runOrder=random

Answer

The goal is a pipeline people believe. A suite that is red half the time teaches everyone to ignore red, which is more dangerous than having no suite, so the strategy is to separate what you control from what you do not. Split the suite into three tiers.

Tier one is your own logic against stubbed dependencies with WireMock or Mockoon: these must be fast, deterministic and always green, and they gate the deploy. They cover your request construction, your response handling, your retry and timeout policy, your signature verification and your state machine, all of which are yours to get right. Tier two is contract verification against the real sandbox, run on a schedule rather than on every commit, whose job is to answer whether the real dependency still behaves like your stubs.

When it fails it raises a dependency alert, not a build failure, and it must report clearly whether the sandbox was reachable at all, because down is a different signal from changed. Tier three is a small production or preprod smoke against real integrations, monitored rather than gating. Alongside the tiers, three practices.

Health check before running: probe the downstream first and, if it is down, mark the dependent tests as skipped with an explicit reason instead of failing them, so the report reads accurately. Distinguish failure types in the reporting: a connection reset or a 503 from the bank is an infrastructure event, an assertion mismatch is a defect, and they must not appear in the same bucket. And use the outage productively, because a flaky downstream is a genuine production condition, so test your own behaviour under it: does your service time out cleanly, does it retry with backoff, does it leave the payment in a recoverable state, does the customer see a sane message rather than a spinner.

# Tier 1: gate the deploy, stubs only, always green
mvn test -Dsuite=testng-stubbed.xml -Dwiremock.port=8089

# Tier 2: scheduled contract check against the REAL sandbox
# health probe first, skip rather than fail when it is down
if ! curl -sf -m 5 "$UIDAI_SANDBOX/health" > /dev/null; then
  echo "::warning::UIDAI sandbox unreachable, marking dependency suite SKIPPED"
  exit 0
fi
mvn test -Dsuite=testng-realdeps.xml

// TestNG: skip with a reason instead of failing on infrastructure
@BeforeMethod(alwaysRun = true)
public void requireDownstream() {
    if (!Downstream.isHealthy("npci")) {
        throw new SkipException("NPCI sandbox unhealthy, test not executed");
    }
}

// Turn the outage into a test: prove OUR behaviour is correct
stubFor(post("/npci/collect").willReturn(aResponse().withFixedDelay(31000)));

@Test(groups = "stubbed")
public void npciTimeoutLeavesPaymentRecoverable() {
    given().spec(Specs.base()).body(new CollectRequest("aarav@okhdfcbank", 49900))
    .when().post("/v1/payments/collect")
    .then().statusCode(202);

    // must be pending and retryable, never silently failed or double debited
    given().spec(Specs.base()).when().get("/v1/payments/" + id)
    .then().body("status", equalTo("pending"))
           .body("is_retryable", equalTo(true));
    verify(moreThanOrExactly(2), postRequestedFor(urlEqualTo("/npci/collect")));
}