AWS Interview Questions and Answers

Answer

An IAM user is a long-lived identity with a permanent credential attached, either a console password or an access key pair. An IAM role is an identity with no permanent credentials at all; principals assume it through STS and receive temporary credentials that expire, typically after one hour but configurable up to twelve via MaxSessionDuration. A policy is neither an identity nor a credential, it is a JSON document listing Effect, Action, Resource and optional Condition blocks, attached to a user, group, role or resource.

Every role carries two distinct policies that candidates routinely conflate: the trust policy, which answers who is allowed to call sts:AssumeRole on this role, and the permission policy, which answers what the assumed session can then do. In 2026 the correct default for humans is no IAM users at all. Federate through IAM Identity Center so engineers get short-lived credentials from their SSO provider, and reserve long-lived access keys for the rare legacy integration that cannot assume a role.

For compute, attach a role: an instance profile on EC2, a task role on ECS, an execution role on Lambda, IRSA or EKS Pod Identity on Kubernetes. Interviewers probe two specific things. First, role chaining, where one assumed role assumes another, caps the session at one hour and ignores the longer MaxSessionDuration. Second, when a third party assumes a role in your account, the trust policy must require an sts:ExternalId condition, otherwise you have created the confused deputy vulnerability that AWS documentation warns about.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowVendorToAssume",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::111122223333:root" },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": { "sts:ExternalId": "gs-prod-7f2c9" }
      }
    }
  ]
}

// Assume it and inspect the session
// aws sts assume-role --role-arn arn:aws:iam::444455556666:role/VendorAudit \
//   --role-session-name audit --external-id gs-prod-7f2c9
// aws sts get-caller-identity

Answer

S3 Standard is the default: millisecond access, no minimum storage duration, no retrieval fee. Standard-IA and One Zone-IA cost less per GB but add a per-GB retrieval charge, a 30 day minimum billable duration and a 128 KB minimum billable object size. Glacier Instant Retrieval keeps millisecond access at archive pricing with a 90 day minimum.

Glacier Flexible Retrieval restores in minutes to hours, also 90 days. Glacier Deep Archive is the cheapest tier at a 180 day minimum and roughly twelve hour standard restores. Intelligent-Tiering moves objects between access tiers automatically for a small per-object monitoring fee and is the safe choice when you genuinely cannot predict access patterns.

For logs the standard shape is Standard for 30 days while people are still debugging, Standard-IA or Glacier Instant Retrieval to 90 days for incident forensics, Deep Archive to the retention limit your compliance team mandates, then expiry. Two gotchas decide this question. First, transitions cost money per thousand objects, so moving fifty million tiny JSON log files to Glacier can cost more in transition requests than the storage you save; aggregate them into larger objects first, or set an ObjectSizeGreaterThan filter. Second, always add an AbortIncompleteMultipartUpload rule, because failed multipart uploads leave invisible parts that you keep paying for and that never show up in the console object list.

{
  "Rules": [
    {
      "ID": "app-logs-tiering",
      "Status": "Enabled",
      "Filter": {
        "And": {
          "Prefix": "logs/",
          "ObjectSizeGreaterThan": 131072
        }
      },
      "Transitions": [
        { "Days": 30, "StorageClass": "STANDARD_IA" },
        { "Days": 90, "StorageClass": "DEEP_ARCHIVE" }
      ],
      "Expiration": { "Days": 2555 }
    },
    {
      "ID": "kill-orphan-multipart",
      "Status": "Enabled",
      "Filter": {},
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}

// aws s3api put-bucket-lifecycle-configuration \
//   --bucket gs-app-logs --lifecycle-configuration file://lifecycle.json

Answer

On-Demand is the per-second baseline with no commitment. Spot sells spare capacity at a large discount but AWS can reclaim the instance with a two minute interruption notice delivered through the instance metadata service, plus an earlier rebalance recommendation event on EventBridge. Reserved Instances commit you to a specific instance family in a region for one or three years and can include a capacity reservation.

Savings Plans commit you to a rupee-per-hour spend instead of a specific instance shape: Compute Savings Plans are the flexible variant that follows you across families, regions, Fargate and Lambda, while EC2 Instance Savings Plans discount harder but lock the family and region. The practical pattern is Savings Plans for your steady baseline, On-Demand for the variable head, and Spot for anything stateless and interruptible such as CI runners, batch jobs and Karpenter-managed worker pools. The T-series question catches people out.

T3, T3a, T4g and similar burstable instances earn CPU credits at a fixed rate per hour and spend them when CPU exceeds the baseline, which is as low as 5 to 20 percent depending on size. When credits run out, a standard-mode instance throttles hard and looks like a mysterious application slowdown, while unlimited mode instead keeps performance and silently bills surplus credits, which is how a small t3.micro fleet can produce a startling line item. In India, teams also underuse Graviton; moving to m7g or r8g typically buys a meaningful price-performance gain on the same workload.

# Spot with a price cap, no shell quoting needed with shorthand syntax
aws ec2 run-instances \
  --image-id ami-0abcd1234efgh5678 \
  --instance-type m7g.large \
  --instance-market-options MarketType=spot,SpotOptions={MaxPrice=0.032,SpotInstanceType=one-time} \
  --count 1

# Poll the interruption notice from inside the instance (IMDSv2)
TOKEN=$(curl -sX PUT http://169.254.169.254/latest/api/token \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/spot/instance-action

# Watch burstable credits before assuming the app is slow
aws cloudwatch get-metric-statistics --namespace AWS/EC2 \
  --metric-name CPUCreditBalance --statistics Minimum --period 300 \
  --dimensions Name=InstanceId,Value=i-0123456789abcdef0 \
  --start-time 2026-08-10T00:00:00Z --end-time 2026-08-11T00:00:00Z

Answer

A subnet is public if its route table has a 0.0.0.0/0 route pointing at an Internet Gateway, and private if it does not. That is the entire distinction; there is no public flag on a subnet beyond the auto-assign public IP setting. Instances in a private subnet still need outbound internet for package installs and third-party API calls, so you place a NAT Gateway in a public subnet and route 0.0.0.0/0 from the private route table to it.

The NAT performs source address translation using an Elastic IP, letting outbound connections work while blocking unsolicited inbound. For production you deploy one NAT Gateway per Availability Zone and point each AZ private route table at its local NAT, because a single shared NAT is both a single point of failure and a source of cross-AZ data transfer charges on every byte. Two facts interviewers like.

AWS reserves five IP addresses in every subnet, the network address, VPC router, DNS, a future-use address and broadcast, so a /28 gives you eleven usable addresses, not sixteen. And NAT Gateway is priced both per hour and per GB processed, which makes it one of the top three surprise line items on Indian startup bills. The fix is VPC endpoints: a free gateway endpoint for S3 and DynamoDB routes that traffic off the NAT entirely, while interface endpoints powered by PrivateLink handle services such as Secrets Manager, ECR and SQS for a smaller hourly plus per-GB fee.

# /16 for the VPC, /20 per subnet leaves room to grow
aws ec2 create-vpc --cidr-block 10.20.0.0/16
aws ec2 create-subnet --vpc-id vpc-0a1 --cidr-block 10.20.0.0/20  --availability-zone ap-south-1a
aws ec2 create-subnet --vpc-id vpc-0a1 --cidr-block 10.20.16.0/20 --availability-zone ap-south-1a

# NAT lives in the public subnet, private route table points at it
aws ec2 create-nat-gateway --subnet-id subnet-pub-1a --allocation-id eipalloc-01
aws ec2 create-route --route-table-id rtb-priv-1a \
  --destination-cidr-block 0.0.0.0/0 --nat-gateway-id nat-01

# Take S3 and DynamoDB off the NAT completely (free, gateway type)
aws ec2 create-vpc-endpoint --vpc-id vpc-0a1 \
  --service-name com.amazonaws.ap-south-1.s3 \
  --route-table-ids rtb-priv-1a rtb-priv-1b

Answer

A security group is stateful and attaches to an ENI. If you allow inbound TCP 443, the response traffic is automatically permitted regardless of your outbound rules, because the connection is tracked. Security groups support allow rules only, there is no deny, and evaluation is a union across every group attached to the interface.

Their best feature is that the source of a rule can be another security group ID rather than a CIDR, so you can say the app tier accepts 8080 only from the load balancer security group and never has to know its IP addresses. A network ACL is stateless and attaches to a subnet. It supports both allow and deny, rules are evaluated in ascending rule-number order and the first match wins, and because it is stateless you must write the return path yourself.

This is exactly where people get burned: they allow inbound 443 on a NACL, forget an outbound rule for the ephemeral port range 1024 to 65535, and every TLS handshake hangs while the security group looks perfectly correct. The right mental model is that security groups are your primary access control and NACLs are a coarse subnet-level backstop, useful mainly for blocking a specific abusive CIDR, since a security group cannot express deny. Watch the quotas too: 60 inbound plus 60 outbound rules per security group and 5 security groups per ENI by default, both adjustable, and a NACL evaluates a maximum of 20 rules per direction before you need a quota increase.

# App tier accepts 8080 only from the ALB security group, never a CIDR
aws ec2 authorize-security-group-ingress \
  --group-id sg-app \
  --protocol tcp --port 8080 --source-group sg-alb

# NACLs are stateless: inbound rule alone is not enough
aws ec2 create-network-acl-entry --network-acl-id acl-01 --rule-number 100 \
  --protocol tcp --port-range From=443,To=443 --cidr-block 0.0.0.0/0 \
  --rule-action allow --ingress

# ... the return path must be opened explicitly
aws ec2 create-network-acl-entry --network-acl-id acl-01 --rule-number 100 \
  --protocol tcp --port-range From=1024,To=65535 --cidr-block 0.0.0.0/0 \
  --rule-action allow --egress

Answer

A Lambda invocation runs in an execution environment, essentially a Firecracker microVM. When no warm environment is available, Lambda creates one and runs the INIT phase: it downloads your deployment package or container image, starts any extensions, bootstraps the language runtime, and executes everything at module scope in your file before the handler is ever called. INIT has a ten second guard rail, after which Lambda restarts initialization.

Then comes INVOKE, which runs your handler function, and eventually SHUTDOWN when the environment is reclaimed, usually after a period of idleness that AWS does not contractually guarantee. Cold start duration is driven by package size, runtime choice, whether you use a container image, and configured memory, since more memory means proportionally more vCPU during INIT too. Practical numbers in 2026: a small Node.js or Python function initializes in tens of milliseconds, a fat Java or .NET application can take seconds without SnapStart.

On billing, the historical rule was that on-demand invocations were billed for INVOKE duration while INIT was free, and provisioned concurrency was billed differently; AWS has been tightening this, so quote the pricing page rather than folklore in an interview. The design consequence is unchanged either way. Create SDK clients, database pools and parsed configuration at module scope so they are reused across warm invocations, and keep the handler body doing only per-request work. Creating a DynamoDBClient inside the handler is the single most common performance bug in production Lambda code.

// index.mjs on nodejs22.x
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, GetCommand } from '@aws-sdk/lib-dynamodb';

// INIT phase: runs once per execution environment, reused when warm
const ddb = DynamoDBDocumentClient.from(new DynamoDBClient({}));
const TABLE = process.env.TABLE_NAME;
let warm = false;

export const handler = async (event) => {
  const coldStart = !warm;
  warm = true;

  const res = await ddb.send(new GetCommand({
    TableName: TABLE,
    Key: { pk: `USER#${event.userId}`, sk: 'PROFILE' },
  }));

  return {
    statusCode: res.Item ? 200 : 404,
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ item: res.Item ?? null, coldStart }),
  };
};

Answer

Since December 2020, S3 provides strong read-after-write consistency for all object operations in all regions at no extra cost and with no performance penalty. A successful PUT of a new object, an overwrite of an existing object, and a DELETE are all immediately visible to any subsequent GET, HEAD or LIST from any client. That killed a whole category of workarounds from the older era, such as writing a marker object then polling, or maintaining a DynamoDB index just to know whether a file had landed.

Many candidates still recite the old eventual-consistency answer, and interviewers notice. What is not strongly consistent is everything above the object layer. Bucket configuration changes propagate eventually: bucket policies, IAM policy updates, lifecycle rules, CORS configuration, replication configuration and Block Public Access settings can take a short while to apply globally, which is why a freshly tightened policy sometimes appears to still allow a request.

Cross-Region and Same-Region Replication are asynchronous by design; use S3 Replication Time Control if you need a fifteen minute service level objective on replication, and never assume the replica has the object simply because the source write returned 200. Read-after-write also says nothing about durability of an in-flight multipart upload, which becomes an object only after CompleteMultipartUpload. Finally, if versioning is enabled, a DELETE without a version ID inserts a delete marker rather than removing data, so the object is gone from a plain GET but is still billed and still recoverable.

Answer

gp3 is the correct default for almost everything. It ships a flat baseline of 3000 IOPS and 125 MB/s regardless of volume size, and you provision extra IOPS and throughput independently of capacity, up to 16000 IOPS and 1000 MB/s. gp2 is the older generation where performance is tied to size at 3 IOPS per GB with a burst bucket, meaning a 20 GB gp2 root volume silently throttles once its burst credits drain, a classic cause of mystery latency on small instances. gp3 is also cheaper per GB than gp2, so migrating existing gp2 volumes is one of the easiest cost wins available. io2 and io2 Block Express are for databases that need sustained sub-millisecond latency, up to 256000 IOPS, higher durability, and Multi-Attach across instances for clustered filesystems. st1 and sc1 are throughput-optimised HDDs suited to sequential workloads such as log processing and big data scans; they behave terribly under random IO. Instance store NVMe is the fastest option by far but is physically attached and lost on stop or terminate, so it is a cache tier, never a source of truth.

Resizing is online but has two steps that people forget. Modifying the volume grows the block device, and then you must grow the partition and the filesystem inside the guest with growpart plus resize2fs or xfs_growfs. You also cannot shrink an EBS volume at all; the only path is to create a smaller volume and copy the data across.

# Modify the volume (online, no detach needed)
aws ec2 modify-volume --volume-id vol-0123456789abcdef0 \
  --volume-type gp3 --size 200 --iops 6000 --throughput 250

aws ec2 describe-volumes-modifications --volume-id vol-0123456789abcdef0 \
  --query "VolumesModifications[].[ModificationState,Progress]" --output table

# Inside the instance: grow the partition, then the filesystem
lsblk
sudo growpart /dev/nvme0n1 1
sudo xfs_growfs -d /            # XFS, e.g. Amazon Linux 2023
# sudo resize2fs /dev/nvme0n1p1 # ext4

Answer

A Region is an isolated geographic deployment of AWS with its own control plane; ap-south-1 is Mumbai and ap-south-2 is Hyderabad. An Availability Zone is one or more discrete data centres inside a Region with independent power, cooling and networking, connected to sibling AZs over low latency private links. Local Zones extend a parent Region closer to a metro for latency-sensitive workloads, and AWS operates several in India including Delhi, Kolkata and Chennai.

Edge locations are the CloudFront and Route 53 points of presence, far more numerous, used for content caching, TLS termination and Lambda@Edge or CloudFront Functions, not for running your database. For an Indian product, ap-south-1 is the default because it has the deepest service coverage and the lowest latency to most of the user base, and you spread across at least three AZs. ap-south-2 matters when you need in-country disaster recovery that survives losing Mumbai entirely, which regulated fintech and health platforms increasingly do, and it also helps with DPDP Act comfort since data never leaves India. Two operational details show seniority.

Cross-AZ data transfer is billed in both directions, so a chatty microservice mesh that ignores topology can spend more on inter-AZ traffic than on compute; keep hot paths zone-local where you can. And AZ names such as ap-south-1a are per-account aliases mapped to different physical zones for different accounts, so when you coordinate placement across accounts you must compare AZ IDs like aps1-az1, not the friendly names.

# AZ names are per-account aliases; AZ IDs are physical
aws ec2 describe-availability-zones --region ap-south-1 \
  --query "AvailabilityZones[].[ZoneName,ZoneId,ZoneType]" --output table

# What exists in Hyderabad
aws ec2 describe-availability-zones --region ap-south-2 --output table

# Include Local Zones and Wavelength in the listing
aws ec2 describe-availability-zones --region ap-south-1 \
  --all-availability-zones \
  --filters Name=zone-type,Values=local-zone --output table

Answer

CloudWatch metrics are time-series numbers stored under a namespace with dimensions. AWS services publish them for free at five minute granularity under basic monitoring; detailed monitoring gives one minute for a fee, and custom metrics can go to one second resolution. Logs are a separate product: log groups contain log streams, you pay for ingestion, storage and any Logs Insights queries you run, and retention defaults to Never Expire, which is how log storage quietly becomes a five figure monthly line.

Alarms sit on metrics, not logs, and evaluate a statistic over a period against a threshold, moving between OK, ALARM and INSUFFICIENT_DATA. A useful alarm has four properties. It uses an M out of N evaluation so a single noisy datapoint does not page anyone.

It sets treatMissingData deliberately, because the default of missing can leave an alarm stuck in INSUFFICIENT_DATA forever when a Lambda simply stops being invoked. It alarms on user-visible symptoms such as p99 latency, 5xx rate or queue age rather than on CPU, which is a cause and not a symptom. And it routes somewhere a human actually reads.

Two facts interviewers like to hear. EC2 does not publish memory or disk usage by default, you need the CloudWatch agent, and candidates who say otherwise reveal they have not run EC2 in production. And metric filters or Embedded Metric Format let you emit metrics from log lines without a synchronous PutMetricData call on the request path, which is both cheaper and faster than instrumenting each function with the API.

# Page on sustained 5xx, not on a single blip
aws cloudwatch put-metric-alarm \
  --alarm-name alb-5xx-sustained \
  --namespace AWS/ApplicationELB \
  --metric-name HTTPCode_Target_5XX_Count \
  --dimensions Name=LoadBalancer,Value=app/gs-prod/50dc6c495c0c9188 \
  --statistic Sum --period 60 --threshold 25 \
  --comparison-operator GreaterThanThreshold \
  --evaluation-periods 5 --datapoints-to-alarm 3 \
  --treat-missing-data notBreaching \
  --alarm-actions arn:aws:sns:ap-south-1:111122223333:oncall

# Stop paying to store logs forever
aws logs put-retention-policy --log-group-name /aws/lambda/checkout --retention-in-days 30

Answer

A presigned URL embeds a SigV4 signature in the query string so an anonymous client can perform one specific S3 operation, usually GetObject or PutObject, for a limited time. It lets a browser or mobile app upload directly to S3 instead of streaming bytes through your API server, which removes a bandwidth bottleneck, avoids Lambda payload limits, and cuts NAT and compute cost. The signature is computed with the credentials of whoever generated it, and here is the failure mode candidates miss: the URL cannot outlive the credentials that signed it.

The SigV4 maximum expiry is seven days, but if you sign from a Lambda execution role or any assumed role, the temporary session may expire in an hour, and the URL dies with it even though you asked for seven days. Signing with a long-lived IAM user key gives the full window but reintroduces a permanent credential you now have to protect. The second failure mode is trusting the client.

A plain presigned PUT lets the caller upload a file of any size and any content type to that key. Use a presigned POST policy instead when you need to enforce a content-length-range and an allowed content type, or attach an S3 event that validates and quarantines the object after upload. Also remember that a presigned URL is a bearer token: anyone who obtains it can use it until expiry, so keep expiry short, log the generation, and never put one in a URL that ends up in a shared browser history or referrer header.

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

const s3 = new S3Client({ region: 'ap-south-1' });

export async function uploadUrl(userId, filename) {
  const cmd = new PutObjectCommand({
    Bucket: 'gs-user-uploads',
    Key: `resumes/${userId}/${filename}`,
    ContentType: 'application/pdf',
    ServerSideEncryption: 'aws:kms',
    SSEKMSKeyId: process.env.UPLOAD_KEY_ARN,
  });

  // 300s: long enough for a slow 4G upload, short enough to limit leak damage
  return getSignedUrl(s3, cmd, { expiresIn: 300 });
}

Answer

Application Load Balancer works at layer 7. It routes on host, path, HTTP header, query string, source IP and method, supports weighted target groups for canaries, terminates TLS, speaks HTTP/2 and gRPC, integrates with AWS WAF and Cognito or OIDC authentication, and can target Lambda functions directly. Network Load Balancer works at layer 4.

It gives you a static IP per AZ (and optionally a Bring Your Own IP), handles millions of connections per second at very low latency, can pass TLS straight through to the backend, and preserves the client source IP when targets are registered by instance ID. Gateway Load Balancer is a different animal, used to insert third-party network appliances such as firewalls into the traffic path over the GENEVE protocol on port 6081. Choose ALB for normal HTTP services, NLB when you need static IPs for a partner allowlist, non-HTTP protocols, or extreme throughput, and GWLB only for inline security appliances.

The defaults that bite: ALB idle timeout is 60 seconds, so a long-polling or slow report endpoint returns 504 unless you raise it and set your backend keep-alive higher than the ALB value, otherwise the target closes a connection the ALB still believes is usable and you see sporadic 502s. Cross-zone load balancing is on by default and free for ALB but off by default for NLB and billed when enabled. Deregistration delay defaults to 300 seconds, which makes deployments feel slow until you tune it against your longest request.

Answer

An Auto Scaling Group is defined by a launch template plus minimum, maximum and desired capacity, spread across subnets in multiple AZs. Scaling policies change desired capacity. Target tracking is the one you should reach for by default: you name a metric such as average CPU utilisation or ALBRequestCountPerTarget and a target value, and AWS manages the underlying alarms and step sizes for you.

Step scaling adds a specific number of instances per alarm breach band and is useful when you know your capacity curve. Simple scaling is the legacy option with a cooldown and should be avoided. Scheduled actions handle predictable patterns, which in India often means scaling up before the 9 pm traffic peak on a consumer app.

Predictive scaling uses machine learning on historical load to pre-warm capacity ahead of a known daily shape. The health check grace period is the number of seconds after an instance launches during which ASG ignores health check results. If your application takes 120 seconds to boot and the grace period is 60, the ELB health check fails, ASG terminates the instance, launches another, and you get an infinite replacement loop that looks like a scaling storm.

Set the grace period above your realistic worst-case boot time. Two more things senior candidates mention: turn on ELB health checks in addition to EC2 status checks, because an instance can be running perfectly while the application inside is dead, and use warm pools or Instance Refresh with a MinHealthyPercentage when boot time is genuinely long or the fleet is large.

# Target tracking on requests per target beats CPU for web tiers
aws autoscaling put-scaling-policy \
  --auto-scaling-group-name gs-api-asg \
  --policy-name rps-target \
  --policy-type TargetTrackingScaling \
  --target-tracking-configuration file://tt.json

# tt.json
{
  "TargetValue": 800.0,
  "PredefinedMetricSpecification": {
    "PredefinedMetricType": "ALBRequestCountPerTarget",
    "ResourceLabel": "app/gs-prod/50dc6c49/targetgroup/gs-api/9e7c1f2a"
  },
  "DisableScaleIn": false
}

# Give slow-booting apps room before health checks count
aws autoscaling update-auto-scaling-group \
  --auto-scaling-group-name gs-api-asg \
  --health-check-type ELB --health-check-grace-period 300

Answer

RDS is managed PostgreSQL, MySQL, MariaDB, Oracle or SQL Server on EBS. You pick it when you want a familiar engine, full SQL, standard extensions, and predictable pricing, and when your workload fits comfortably on one writer. Aurora is AWS reimplementing the storage layer under MySQL and PostgreSQL: the database writes log records to a distributed storage fleet that keeps six copies across three AZs, which makes replicas cheap to add (up to fifteen), failover fast, and restores near-instant through cloning.

Aurora Serverless v2 scales capacity in fine-grained Aurora Capacity Units and can now scale to zero for dev environments, which makes it attractive for spiky or unpredictable workloads. DynamoDB is a different contract altogether: a managed key-value and document store with single-digit millisecond latency at any scale, but you must design around access patterns, not entities, and you give up joins, ad hoc queries and strong secondary index consistency on GSIs. The honest interview answer is that DynamoDB wins when you know your access patterns, need predictable latency under heavy scale, and want on-demand capacity with no instance to size.

Relational wins when queries are exploratory, reporting matters, or the team is small and SQL is the shared language. Most Indian startups over-reach for DynamoDB early, then discover that their product manager wants a report nobody modelled and they are now writing an export pipeline into Athena or Redshift. Choosing Postgres on Aurora and adding DynamoDB later for the two genuinely hot tables is usually the safer sequence.

Answer

Modern S3 defaults are much safer than the ones most tutorials describe. Since April 2023 new buckets have S3 Block Public Access enabled and Object Ownership set to Bucket owner enforced, which disables ACLs entirely so every object is owned by the bucket owner and access is governed only by policies. Since January 2023 all new objects are encrypted with SSE-S3 by default at no cost.

So the correct answer to how you make a bucket private is largely that you leave the defaults alone and resist any tutorial telling you to turn Block Public Access off. Access is then granted three ways: an identity policy on the caller, a resource policy on the bucket, or a role assumed cross-account. A good bucket policy is explicit about principal and prefix, and adds a Deny for any request not using TLS, since aws:SecureTransport false is still allowed unless you block it.

For content that genuinely must be public, do not open the bucket; put CloudFront in front with Origin Access Control, which keeps the bucket private and gives you caching, WAF and logging as a bonus. To verify rather than hope, run IAM Access Analyzer for S3, which reports buckets reachable from outside your account or organization, and enable S3 Storage Lens or Macie if you handle personal data under the DPDP Act. The most common real breach path in Indian startups is not a public bucket at all, it is an over-broad s3:GetObject on Resource star inside an application role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyPlaintextTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::gs-user-uploads",
        "arn:aws:s3:::gs-user-uploads/*"
      ],
      "Condition": { "Bool": { "aws:SecureTransport": "false" } }
    },
    {
      "Sid": "AllowCloudFrontOACOnly",
      "Effect": "Allow",
      "Principal": { "Service": "cloudfront.amazonaws.com" },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::gs-user-uploads/public/*",
      "Condition": {
        "StringEquals": {
          "AWS:SourceArn": "arn:aws:cloudfront::111122223333:distribution/E1ABCDEF2GHIJ"
        }
      }
    }
  ]
}

Answer

Route 53 offers simple, weighted, latency-based, failover, geolocation, geoproximity, multivalue answer and IP-based routing. For active-passive across regions you use failover routing with a primary and a secondary record, each associated with a Route 53 health check; when the primary health check fails, Route 53 stops returning it and clients resolve to the secondary. For active-active you use latency-based routing so users in India resolve to ap-south-1 while users in Europe resolve to eu-west-1, with health checks attached so an unhealthy region drops out automatically.

Weighted routing is what you use for a gradual migration or a canary, for example shifting five percent of traffic to a new stack. Geolocation routing answers based on the user country and is the tool for data residency or language routing rather than performance. Multivalue answer returns up to eight healthy records and is a cheap client-side load spread, not a substitute for a load balancer.

An alias record is a Route 53 specific extension that points at an AWS resource such as an ALB, CloudFront distribution, API Gateway domain, S3 website endpoint or another record in the same hosted zone. It resolves at the DNS layer to the current addresses of that resource, so it survives the underlying IPs changing, it can be created at the zone apex where CNAME is illegal, and alias queries to AWS targets are not billed. The failure mode to mention is DNS TTL: browsers, operating systems and corporate resolvers cache answers, so failover is never instant and a 300 second TTL means minutes of stale routing during an incident.

Answer

The SDKs and CLI walk a fixed credential provider chain and stop at the first source that yields credentials. The order is roughly: explicit parameters in code, environment variables such as AWS_ACCESS_KEY_ID and AWS_SESSION_TOKEN, the web identity token file used by IRSA on EKS, the shared credentials and config files under ~/.aws with the selected profile, container credentials from the ECS or EKS agent endpoint, and finally the EC2 instance metadata service. Understanding this order matters because most credential bugs are precedence bugs: a stale AWS_PROFILE or a leftover AWS_ACCESS_KEY_ID in your shell will quietly override the profile you thought you were using, and the resulting error is a confusing AccessDenied against the wrong account.

The clean 2026 setup is IAM Identity Center. You run aws configure sso once, define a profile per account and permission set, and the CLI opens a browser to fetch short-lived credentials that refresh automatically, so there is nothing long-lived on the laptop at all. For cross-account work you define a profile with role_arn and source_profile, and the CLI performs the AssumeRole for you.

Always confirm which identity you are actually operating as before running anything destructive; aws sts get-caller-identity is the two second check that prevents applying a Terraform plan to production instead of staging. In organisations with many accounts, add a shell prompt segment showing the active profile and region, because the single most expensive human error in cloud operations is running the right command in the wrong account.

# ~/.aws/config
[sso-session gs]
sso_start_url = https://goodspace.awsapps.com/start
sso_region = ap-south-1
sso_registration_scopes = sso:account:access

[profile prod]
sso_session = gs
sso_account_id = 111122223333
sso_role_name = PowerUser
region = ap-south-1

[profile prod-deploy]
role_arn = arn:aws:iam::111122223333:role/DeployRole
source_profile = prod
region = ap-south-1

# Usage
aws sso login --sso-session gs
AWS_PROFILE=prod-deploy aws sts get-caller-identity

Answer

Start in Cost Explorer with a daily granularity view grouped by service to find when the step change happened and which service moved, then re-group by usage type to see the specific meter, because Amazon EC2 as a service line hides very different things: instance hours, EBS storage, snapshot storage, data transfer and NAT processing all report under related lines. Once you know the meter, group by linked account, then by tag or resource. If the account has the Cost and Usage Report enabled into S3, query it with Athena for resource-level attribution, which Cost Explorer alone does not always give you.

In practice the same handful of causes come up repeatedly. NAT Gateway data processing rises because a workload started pulling container images or S3 objects through the NAT instead of an endpoint. Cross-AZ data transfer rises after a deployment spread pods across zones.

CloudWatch Logs ingestion rises because someone left debug logging on, and log groups with no retention policy keep compounding storage. S3 costs rise from request charges on a chatty client or from versioning keeping every overwrite. And an EBS snapshot schedule with no deletion policy grows forever.

The organisational fix is prevention: enforce a tagging policy through AWS Organizations, activate those tags as cost allocation tags, set AWS Budgets with anomaly detection so a spike alerts within a day rather than at month end, and review the Cost Optimization Hub recommendations. For Indian entities, remember invoices carry GST and the console shows USD, so finance reconciliation needs the tax invoice from the billing console, not the Cost Explorer figure.

# Where did the step change happen, and in which meter?
aws ce get-cost-and-usage \
  --time-period Start=2026-07-01,End=2026-08-01 \
  --granularity DAILY --metrics UnblendedCost \
  --group-by Type=DIMENSION,Key=USAGE_TYPE \
  --filter file://ec2-filter.json

# Anomaly detection so the next spike alerts in a day
aws ce create-anomaly-monitor --anomaly-monitor \
  MonitorName=svc-monitor,MonitorType=DIMENSIONAL,MonitorDimension=SERVICE

# Log groups quietly retaining forever
aws logs describe-log-groups \
  --query "logGroups[?retentionInDays==null].logGroupName" --output text

Answer

One execution environment handles exactly one invocation at a time. There is no request-level parallelism inside a single environment, so concurrency equals the number of simultaneous invocations, which for a steady workload is roughly requests per second multiplied by average duration in seconds. A function serving 200 requests per second at 250 ms therefore needs about 50 concurrent environments.

The account has a regional concurrent execution quota, 1000 by default and raisable on request, shared by every function in that region. Since late 2023 each function can also scale independently by up to 1000 additional concurrent executions every 10 seconds, so one bursting function no longer starves its neighbours the way it did under the old shared burst pool. Reserved concurrency carves a slice of the account quota and pins it to one function.

It does two things at once, which trips people up: it guarantees that much capacity for that function, and it caps the function at exactly that number, throttling anything beyond it. Setting reserved concurrency to zero is the standard emergency kill switch for a misbehaving function. Provisioned concurrency is unrelated to limits; it pre-initializes a number of environments and keeps them warm so INIT never runs on the request path, and it is billed hourly whether you use it or not. What interviewers listen for is the throttle behaviour: a synchronous caller gets a 429 with TooManyRequestsException, an asynchronous invoke is retried internally with backoff for up to six hours before going to the destination or dead letter queue, and a stream source such as Kinesis or DynamoDB Streams simply blocks the shard, so lag grows instead of errors appearing.

# Protect a shared RDS instance from a Lambda stampede
aws lambda put-function-concurrency \
  --function-name order-writer --reserved-concurrent-executions 40

# Emergency stop: throttle everything without deleting the function
aws lambda put-function-concurrency \
  --function-name runaway-worker --reserved-concurrent-executions 0

# Keep 20 environments warm on a published version (not on $LATEST)
aws lambda put-provisioned-concurrency-config \
  --function-name checkout --qualifier live \
  --provisioned-concurrent-executions 20

# Are we actually being throttled?
aws cloudwatch get-metric-statistics --namespace AWS/Lambda \
  --metric-name Throttles --statistics Sum --period 60 \
  --dimensions Name=FunctionName,Value=order-writer \
  --start-time 2026-08-11T00:00:00Z --end-time 2026-08-11T06:00:00Z

Answer

Memory is the only performance dial Lambda exposes. You set it between 128 MB and 10240 MB, and CPU, network bandwidth and disk throughput all scale linearly with that number. The reference point worth memorising is 1769 MB, which corresponds to one full vCPU.

Below that you get a fraction of a core, above it you get more than one, and around 3538 MB you cross into two vCPUs, which only helps if your code is actually multi-threaded or your runtime parallelises work such as JVM garbage collection or Node.js crypto in the libuv thread pool. Because billing is GB-seconds, doubling memory doubles the per-millisecond price but often more than halves the duration on CPU-bound work, so the total cost falls while latency improves. This is the counterintuitive result interviewers want you to state: 128 MB is frequently the most expensive setting you can choose.

The tuning method is empirical, not theoretical. Run the open source AWS Lambda Power Tuning state machine, which invokes your function across a range of memory settings with real payloads and plots cost against duration so you can pick the knee of the curve. Do not extrapolate from a colleague's function; an IO-bound function waiting on DynamoDB gains almost nothing above 512 MB, while a function parsing a large PDF or resizing images keeps improving to several gigabytes. Two related settings often get missed: ephemeral storage at /tmp defaults to 512 MB and is configurable to 10240 MB, and it is billed separately above the free 512 MB.

# Give a CPU-bound image resizer real cores and disk
aws lambda update-function-configuration \
  --function-name thumbnailer \
  --memory-size 3008 \
  --ephemeral-storage Size=2048 \
  --timeout 60

# Deploy the power tuning state machine, then run it
aws serverlessrepo create-cloud-formation-change-set \
  --application-id arn:aws:serverlessrepo:us-east-1:451282441545:applications/aws-lambda-power-tuning \
  --stack-name lambda-power-tuning --capabilities CAPABILITY_IAM

# Input for the state machine execution
{
  "lambdaARN": "arn:aws:lambda:ap-south-1:111122223333:function:thumbnailer",
  "powerValues": [512, 1024, 1769, 3008, 5120],
  "num": 30,
  "payload": { "key": "uploads/sample.jpg" },
  "strategy": "balanced"
}

Answer

SnapStart attacks cold starts from a different angle. Instead of keeping environments warm, Lambda initializes your function once when you publish a version, takes an encrypted Firecracker microVM snapshot of the fully initialized memory and disk state, and then resumes copies of that snapshot on demand. Because the expensive part, classloading in Java or importing heavy Python modules, has already happened, resume times are dramatically shorter than a real cold start.

It launched for Java on Corretto and has since been extended to Python and .NET runtimes. The economics differ sharply from provisioned concurrency: SnapStart has no hourly charge for idle capacity, though you do pay for snapshot caching and restore, so it suits spiky and unpredictable traffic where paying to keep environments warm around the clock would be wasteful. Provisioned concurrency still wins when you need a hard guarantee of zero initialization on a latency-critical path, or when your runtime is not supported.

The correctness trap is uniqueness. Everything captured in the snapshot is duplicated into every resumed environment, so anything that must be unique per environment breaks: pseudorandom number generator state, cached credentials or connection handles that expire, timestamps computed at init, unique identifiers generated once at startup. Java offers CRaC beforeCheckpoint and afterRestore hooks, and the other runtimes expose equivalent restore hooks, so you re-seed randomness and reopen connections after restore. SnapStart also requires published versions; it does not apply to $LATEST, which surprises teams that deploy by overwriting the function code directly.

// Java: re-establish per-environment uniqueness after a snapshot resume
import org.crac.Context;
import org.crac.Core;
import org.crac.Resource;

public class Handler implements RequestHandler<Req, Res>, Resource {
  private HikariDataSource pool;

  public Handler() {
    Core.getGlobalContext().register(this);
    this.pool = buildPool();
  }

  @Override
  public void beforeCheckpoint(Context<? extends Resource> c) {
    pool.close();               // never snapshot a live socket
  }

  @Override
  public void afterRestore(Context<? extends Resource> c) {
    this.pool = buildPool();    // fresh connections per resumed environment
  }
}

// aws lambda update-function-configuration --function-name pricing \
//   --snap-start ApplyOn=PublishedVersions
// aws lambda publish-version --function-name pricing

Answer

DynamoDB hashes the partition key to place an item on a physical partition. Each partition serves up to 3000 read capacity units and 1000 write capacity units per second and holds up to 10 GB. If your key distribution concentrates traffic, one partition hits those ceilings while the table as a whole looks under-provisioned, and you get ProvisionedThroughputExceededException even though the console shows spare capacity.

That is a hot partition. Adaptive capacity absorbs mild imbalance by borrowing unused throughput from other partitions and can isolate a persistently hot key onto its own partition, but it is a safety net, not a licence to design badly, and it does not fix a key with only three distinct values. The classic mistake is using something low cardinality as the partition key: a status field, a tenant identifier when one tenant is ten times bigger than the rest, or a date when all of today's writes land on one value.

Fixes are write sharding, where you append a calculated suffix such as ORDER#2026-08-11#7 across N shards and fan out reads across them, or restructuring so the partition key is naturally high cardinality like a user or order identifier and the sort key carries the time dimension. On indexes: a local secondary index shares the table partition key, must be created with the table, supports strongly consistent reads, and constrains that item collection to 10 GB. A global secondary index has its own partition and sort key, its own capacity, is always eventually consistent, and a write that violates its projection simply does not appear, which is how sparse indexes are built deliberately.

# High-cardinality PK, time in the SK, plus a sparse GSI on open orders only
aws dynamodb create-table --table-name Orders \
  --attribute-definitions \
    AttributeName=pk,AttributeType=S \
    AttributeName=sk,AttributeType=S \
    AttributeName=openStatus,AttributeType=S \
  --key-schema AttributeName=pk,KeyType=HASH AttributeName=sk,KeyType=RANGE \
  --billing-mode PAY_PER_REQUEST \
  --global-secondary-indexes file://gsi.json

# gsi.json : items without openStatus never enter the index
[
  {
    "IndexName": "gsi-open-orders",
    "KeySchema": [
      { "AttributeName": "openStatus", "KeyType": "HASH" },
      { "AttributeName": "sk", "KeyType": "RANGE" }
    ],
    "Projection": { "ProjectionType": "KEYS_ONLY" }
  }
]

# Find the offending key when throttles start
# CloudWatch Contributor Insights for DynamoDB ranks partition keys by traffic
aws dynamodb update-contributor-insights --table-name Orders \
  --contributor-insights-action ENABLE

Answer

Single-table design stores multiple entity types in one table using generic attribute names, typically pk and sk, with type prefixes encoded into the key values. A user profile might be pk USER#42 and sk PROFILE, that user's orders pk USER#42 and sk ORDER#2026-08-11#a91, and the order's line items under the same partition. Because everything for one access pattern lives in one partition, a single Query with a begins_with condition on the sort key returns the profile and the last twenty orders in one request, which is the join you gave up when leaving SQL.

GSI overloading extends the idea: you add generic gsi1pk and gsi1sk attributes and populate them differently per entity type, so one index serves several inverted access patterns rather than creating one index per query. The discipline that makes this work is designing from access patterns backwards. You write down every query the application will make, then define keys that satisfy them, rather than modelling entities first.

Two honest caveats belong in your answer. Single-table design makes analytics and ad hoc reporting genuinely painful, so most teams stream the table to S3 through DynamoDB Streams or an export and query it in Athena. And it makes onboarding harder, because the table is unreadable without the access pattern document. Never use Scan in a hot path; it reads every item and consumes capacity proportional to table size, so a Scan with a FilterExpression still bills you for everything it read before filtering, which is a favourite interview trap.

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, QueryCommand } from '@aws-sdk/lib-dynamodb';

const ddb = DynamoDBDocumentClient.from(new DynamoDBClient({}));

// Profile plus the 20 most recent orders in ONE request, one partition
export async function userDashboard(userId) {
  const { Items } = await ddb.send(new QueryCommand({
    TableName: 'AppTable',
    KeyConditionExpression: 'pk = :pk AND (sk = :profile OR begins_with(sk, :order))',
    ExpressionAttributeValues: {
      ':pk': `USER#${userId}`,
      ':profile': 'PROFILE',
      ':order': 'ORDER#',
    },
    ScanIndexForward: false,   // newest sort keys first
    Limit: 21,
  }));

  return {
    profile: Items.find((i) => i.sk === 'PROFILE'),
    orders: Items.filter((i) => i.sk.startsWith('ORDER#')),
  };
}

Answer

A standard queue gives at-least-once delivery, best-effort ordering and effectively unlimited throughput. A FIFO queue gives ordering within a message group and deduplication inside a five minute window, at a lower throughput ceiling: 300 transactions per second per API action without batching and 3000 with batching, raised substantially in high throughput mode. MessageGroupId is the key design decision on FIFO, because ordering is per group and messages in different groups process in parallel; using one group for the whole queue serialises everything and destroys throughput.

Visibility timeout is the mechanism behind most duplicate bugs. When a consumer receives a message, SQS hides it for the visibility timeout, thirty seconds by default and up to twelve hours. If the consumer finishes and calls DeleteMessage, the message is gone.

If the consumer crashes, or simply takes longer than the timeout, the message reappears and another consumer picks it up while the first is still working, so the same payment gets processed twice. The fix is to make visibility timeout comfortably longer than your worst-case processing time, extend it mid-flight with ChangeMessageVisibility for long jobs, and make the consumer idempotent regardless, because at-least-once means duplicates are a contract, not a bug. Add a redrive policy with maxReceiveCount so poison messages land in a dead letter queue instead of looping forever, and enable long polling with ReceiveMessageWaitTimeSeconds set to 20, which both reduces empty receive charges and improves latency. Default retention is four days and the maximum is fourteen.

# Visibility comfortably above worst-case processing, long polling on
aws sqs set-queue-attributes --queue-url $Q --attributes file://attrs.json

# attrs.json
{
  "VisibilityTimeout": "300",
  "ReceiveMessageWaitTimeSeconds": "20",
  "MessageRetentionPeriod": "1209600",
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:111122223333:orders-dlq\",\"maxReceiveCount\":\"5\"}"
}

# Extend the lease on a job that is running long instead of losing it
aws sqs change-message-visibility --queue-url $Q \
  --receipt-handle $RECEIPT --visibility-timeout 600

# How stale is the backlog right now?
aws sqs get-queue-attributes --queue-url $Q \
  --attribute-names ApproximateAgeOfOldestMessage ApproximateNumberOfMessages

Answer

The event source mapping is a Lambda-managed poller, not something running in your function. It long-polls the queue, groups messages into batches up to BatchSize, waits up to MaximumBatchingWindowInSeconds to fill a batch, and invokes your function. It then deletes the whole batch if the function returns successfully.

That last detail is the one that catches teams out: with the default configuration, if one message in a batch of ten throws, all ten become visible again and the nine that already succeeded get reprocessed, so a single poison message can multiply your side effects. The fix is to set FunctionResponseTypes to ReportBatchItemFailures and return a batchItemFailures array containing the message IDs that failed. Lambda then deletes the successes and only redelivers the named failures.

Be careful with the contract: returning a malformed response or throwing after partially processing means the whole batch is retried, and if you sort by sequence for stream sources, the semantics differ again. The hard operational rule is that queue visibility timeout must be at least six times your function timeout, which is what AWS enforces in the console, otherwise messages reappear while the poller still owns them. Other useful knobs: ScalingConfig MaximumConcurrency caps how many concurrent invocations the poller creates, which protects a downstream database far better than reserved concurrency because throttled invocations do not count as failures against your redrive policy; and FilterCriteria lets you discard irrelevant messages inside the mapping so you are not billed for invocations that do nothing.

// Return only the message IDs that genuinely failed
export const handler = async (event) => {
  const batchItemFailures = [];

  for (const record of event.Records) {
    try {
      await processOrder(JSON.parse(record.body));
    } catch (err) {
      console.error({ msg: 'record failed', id: record.messageId, err: err.message });
      batchItemFailures.push({ itemIdentifier: record.messageId });
    }
  }

  return { batchItemFailures };
};

// aws lambda create-event-source-mapping \
//   --function-name order-worker \
//   --event-source-arn arn:aws:sqs:ap-south-1:111122223333:orders \
//   --batch-size 10 --maximum-batching-window-in-seconds 5 \
//   --function-response-types ReportBatchItemFailures \
//   --scaling-config MaximumConcurrency=25

Answer

S3 can publish object events directly to SNS, SQS or Lambda, or it can publish everything to EventBridge with a single bucket setting. Direct notifications are the lowest latency and cheapest path, but they are limited: filtering is only by prefix and suffix, and historically overlapping configurations on the same prefix were rejected, so two teams both wanting notifications on uploads/ end up fighting over one configuration. EventBridge removes those limits.

You get content-based filtering on any field of the event, up to five targets per rule, many rules over the same events, an archive and replay facility that is invaluable after an incident, schema discovery, and cross-account routing through an event bus. The trade-offs are slightly higher latency and per-event cost. For an upload pipeline, the shape most teams settle on is S3 to EventBridge to SQS to Lambda rather than S3 straight to Lambda.

The queue gives you a buffer against burst uploads, a natural concurrency cap through the event source mapping, a dead letter queue for poison objects, and the ability to redrive after a bad deploy, none of which you get invoking Lambda directly. Two delivery caveats to raise unprompted: S3 event delivery is at-least-once, so your processor must be idempotent on object key plus version ID, and events are not ordered, so a rapid create then delete on the same key can arrive in either order. If ordering genuinely matters, enable bucket versioning and treat the version ID as the unit of work rather than the key.

# Turn on EventBridge for the bucket once
aws s3api put-bucket-notification-configuration --bucket gs-user-uploads \
  --notification-configuration file://notif.json

# notif.json
{ "EventBridgeConfiguration": {} }

# Rule: only PDFs above 1 MB under resumes/, straight to a buffering queue
{
  "source": ["aws.s3"],
  "detail-type": ["Object Created"],
  "detail": {
    "bucket": { "name": ["gs-user-uploads"] },
    "object": {
      "key": [{ "prefix": "resumes/" }, { "suffix": ".pdf" }],
      "size": [{ "numeric": [">", 1048576] }]
    }
  }
}

# aws events put-rule --name resume-uploads --event-pattern file://pattern.json
# aws events put-targets --rule resume-uploads \
#   --targets Id=1,Arn=arn:aws:sqs:ap-south-1:111122223333:resume-work

Answer

Attaching a Lambda function to a VPC used to be a performance disaster because Lambda created and attached an elastic network interface per execution environment, adding many seconds to every cold start. That was fixed in 2019 with Hyperplane. Lambda now creates a small number of shared ENIs per unique combination of subnet and security group, at function create or update time rather than at invoke time, and every execution environment for that function multiplexes over them.

So the honest answer to whether VPC attachment slows cold starts today is no, it does not meaningfully, and saying otherwise dates you. What does still bite is connectivity and cost. A VPC-attached function has no route to the public internet unless its subnets are private with a NAT Gateway route, and it can never have a public IP of its own.

Teams attach a function to a VPC to reach RDS, then discover it can no longer call a payment gateway or even the S3 API. The correct fixes are a gateway VPC endpoint for S3 and DynamoDB, which is free, and interface endpoints for Secrets Manager, KMS, SQS and anything else on the hot path, falling back to a NAT Gateway only for genuinely external calls. Also watch subnet IP capacity, since Hyperplane ENIs plus your other workloads share the address space, and always place the function in at least two AZs so a single zone impairment does not take the function down. Finally, if a function only talks to public AWS APIs, do not put it in a VPC at all.

# Attach across two AZs, never one
aws lambda update-function-configuration --function-name reporting \
  --vpc-config SubnetIds=subnet-priv-1a,subnet-priv-1b,SecurityGroupIds=sg-lambda

# Free gateway endpoint, keeps S3 traffic off the NAT
aws ec2 create-vpc-endpoint --vpc-id vpc-0a1 \
  --vpc-endpoint-type Gateway \
  --service-name com.amazonaws.ap-south-1.s3 \
  --route-table-ids rtb-priv-1a rtb-priv-1b

# Interface endpoint for Secrets Manager so init does not need NAT
aws ec2 create-vpc-endpoint --vpc-id vpc-0a1 \
  --vpc-endpoint-type Interface \
  --service-name com.amazonaws.ap-south-1.secretsmanager \
  --subnet-ids subnet-priv-1a subnet-priv-1b \
  --security-group-ids sg-endpoints --private-dns-enabled

Answer

IAM evaluates every request against the full set of applicable policies, and the algorithm is deterministic. Start from an implicit deny. Any explicit Deny anywhere in the evaluation ends it immediately, no matter what else allows.

Then organization policies apply: a service control policy that does not allow the action blocks it even for the account root, and resource control policies apply the same idea to resource-side access. Then permissions boundaries cap what an identity policy can grant, then session policies passed at AssumeRole time cap it further. Finally, for same-account access an allow in either the identity policy or the resource policy is sufficient, while for cross-account access you need an allow on both sides, which is the single most common cause of a puzzling AccessDenied.

Debugging is a checklist, not a guess. Read the CloudTrail event, because the errorMessage often names the exact policy type that denied you, including the SCP. Run the IAM policy simulator against the specific principal, action and resource ARN.

Confirm you are the principal you think you are with sts get-caller-identity, since a stale profile explains a surprising share of these. Check the resource ARN format carefully: an s3:ListBucket needs the bucket ARN while s3:GetObject needs the bucket-slash-star ARN, and a policy with only the latter denies listing. Check conditions that are invisible in the console summary, such as aws:SourceIp, aws:SourceVpce or an aws:PrincipalTag mismatch in an ABAC setup. Finally check KMS: a request to read an encrypted object also needs kms:Decrypt on the key, and the key policy must permit it independently.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListNeedsBucketArn",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::gs-reports",
      "Condition": { "StringLike": { "s3:prefix": ["team/${aws:PrincipalTag/team}/*"] } }
    },
    {
      "Sid": "GetNeedsObjectArn",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "kms:Decrypt"],
      "Resource": [
        "arn:aws:s3:::gs-reports/team/${aws:PrincipalTag/team}/*",
        "arn:aws:kms:ap-south-1:111122223333:key/2f9a-..."
      ]
    }
  ]
}

# Prove it before shipping
# aws iam simulate-principal-policy \
#   --policy-source-arn arn:aws:iam::111122223333:role/AnalystRole \
#   --action-names s3:GetObject \
#   --resource-arns arn:aws:s3:::gs-reports/team/growth/q2.csv

Answer

A KMS key, formerly called a customer master key, lives inside a FIPS validated hardware security module and its key material never leaves. That is the security property you are buying, and it is also the constraint: every Encrypt or Decrypt call is a network round trip to KMS, and the direct Encrypt API accepts at most 4 KB of plaintext. So you cannot encrypt a 200 MB video by calling kms:Encrypt on it.

Envelope encryption is the standard answer. You call GenerateDataKey, which returns two things: a plaintext data key and the same key encrypted under your KMS key. You encrypt the payload locally with the plaintext key using AES-GCM, immediately discard the plaintext key from memory, and store the encrypted data key alongside the ciphertext.

To read the data back you send the encrypted data key to kms:Decrypt, get the plaintext key, and decrypt locally. This is exactly what SSE-KMS on S3, encrypted EBS volumes and RDS encryption do under the hood. Points that separate a strong answer: use an encryption context, a set of key-value pairs that is bound into the ciphertext and logged in CloudTrail, so a data key issued for one tenant cannot decrypt another tenant's blob.

Cache data keys with the AWS Encryption SDK when encrypting many small objects, because KMS is billed per ten thousand requests and throttles at a per-region rate. And remember that a KMS key policy is mandatory and authoritative; unlike most services, an IAM policy alone does not grant access unless the key policy delegates to IAM in the account.

import { KMSClient, GenerateDataKeyCommand, DecryptCommand } from '@aws-sdk/client-kms';
import { createCipheriv, createDecipheriv, randomBytes } from 'node:crypto';

const kms = new KMSClient({ region: 'ap-south-1' });
const KEY_ID = process.env.KMS_KEY_ARN;

export async function seal(plaintext, tenantId) {
  const { Plaintext, CiphertextBlob } = await kms.send(new GenerateDataKeyCommand({
    KeyId: KEY_ID,
    KeySpec: 'AES_256',
    EncryptionContext: { tenant: tenantId },
  }));

  const iv = randomBytes(12);
  const cipher = createCipheriv('aes-256-gcm', Plaintext, iv);
  const body = Buffer.concat([cipher.update(plaintext, 'utf8'), cipher.final()]);

  Plaintext.fill(0); // scrub the data key from memory

  return {
    wrappedKey: Buffer.from(CiphertextBlob).toString('base64'),
    iv: iv.toString('base64'),
    tag: cipher.getAuthTag().toString('base64'),
    body: body.toString('base64'),
  };
}

Answer

ECS with the Fargate launch type removes the host entirely: you declare CPU and memory in the task definition and AWS runs the container, billing per vCPU-second and GB-second. It is the right default for most teams because there are no AMIs to patch and no cluster capacity to babysit. ECS on EC2 gives you the host back, which matters when you need GPUs, privileged containers, custom kernels, per-host daemons, or simply cheaper steady-state compute through Spot and Savings Plans on instances you keep busy.

EKS is for teams that want the Kubernetes ecosystem, Helm charts, operators, CRDs and portability across clouds, and it costs a control plane fee per cluster plus the operational overhead of running Kubernetes properly. A useful framing for interviews: pick ECS Fargate unless you can name the specific Kubernetes feature you need, because EKS bought without that reason turns into a full-time platform team. The role question is the one that separates people who have actually deployed.

A task execution role is used by the ECS agent, not your code: it pulls the image from ECR, fetches secrets referenced in the task definition, and writes container logs to CloudWatch. A task role is assumed by the application running inside the container and grants the permissions your code uses, such as reading from S3 or writing to DynamoDB. Mixing them produces two classic symptoms: a task that fails to start with a CannotPullContainerError because the execution role lacks ECR permissions, or a task that starts fine and then throws AccessDenied at runtime because the task role was never attached.

{
  "family": "gs-api",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "1024",
  "memory": "2048",
  "runtimePlatform": { "cpuArchitecture": "ARM64", "operatingSystemFamily": "LINUX" },
  "executionRoleArn": "arn:aws:iam::111122223333:role/ecsTaskExecutionRole",
  "taskRoleArn": "arn:aws:iam::111122223333:role/gsApiTaskRole",
  "containerDefinitions": [
    {
      "name": "api",
      "image": "111122223333.dkr.ecr.ap-south-1.amazonaws.com/gs-api:2026.08.11",
      "portMappings": [{ "containerPort": 8080, "protocol": "tcp" }],
      "secrets": [
        { "name": "DB_PASSWORD", "valueFrom": "arn:aws:secretsmanager:ap-south-1:111122223333:secret:prod/db-AbCdEf" }
      ],
      "healthCheck": {
        "command": ["CMD-SHELL", "curl -f http://localhost:8080/healthz || exit 1"],
        "interval": 15, "timeout": 5, "retries": 3, "startPeriod": 60
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/gs-api",
          "awslogs-region": "ap-south-1",
          "awslogs-stream-prefix": "api"
        }
      }
    }
  ]
}

Answer

CloudFormation is the native engine. It holds state for you, supports change sets so you can preview a diff before applying, detects drift when someone edits a resource by hand, and StackSets deploy the same template across many accounts and regions in an Organization. Its weaknesses are template verbosity, slow rollbacks and a per-stack resource ceiling that pushes large systems into nested stacks.

CDK is not a competitor to CloudFormation; it is a code-first way to synthesize CloudFormation templates in TypeScript, Python, Java or Go, with L1 constructs mapping one to one to resources, L2 constructs adding sensible defaults and IAM wiring, and L3 patterns assembling whole architectures. Terraform is the multi-cloud option with its own state file, an explicit plan phase, and a far larger provider ecosystem, at the cost of owning state storage and locking yourself, usually an S3 backend with locking. The recovery question is a real operations test.

When an update fails and the rollback also fails, the stack lands in UPDATE_ROLLBACK_FAILED and refuses further updates. You fix the underlying resource by hand, for example reattaching a deleted security group, then call ContinueUpdateRollback, optionally passing ResourcesToSkip for resources that can never roll back. DELETE_FAILED usually means a dependency CloudFormation cannot remove, most often a non-empty S3 bucket, a log group recreated outside the stack, or an ENI still attached by a Lambda function. Mention retention too: a DeletionPolicy of Retain on stateful resources such as databases and buckets is what stops a bad delete from destroying production data.

# Never apply blind: read the change set first
aws cloudformation create-change-set --stack-name gs-prod-api \
  --template-body file://template.yaml --change-set-name pr-4821 \
  --capabilities CAPABILITY_NAMED_IAM
aws cloudformation describe-change-set --stack-name gs-prod-api \
  --change-set-name pr-4821 \
  --query "Changes[].ResourceChange.[Action,LogicalResourceId,Replacement]" --output table

# Has anyone edited resources by hand?
aws cloudformation detect-stack-drift --stack-name gs-prod-api

# Stuck rollback: repair the resource, then continue, skipping the hopeless one
aws cloudformation continue-update-rollback --stack-name gs-prod-api \
  --resources-to-skip LegacyMigrationCustomResource

# Template guard rails
Resources:
  OrdersTable:
    Type: AWS::DynamoDB::Table
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain

Answer

HTTP API is the modern default. It costs meaningfully less per million requests than REST API, has lower latency, supports built-in JWT authorizers and Lambda authorizers, and uses the simpler payload format 2.0. REST API is the older, heavier product, and you choose it when you need something it uniquely offers: request and response transformation with VTL mapping templates, API keys with usage plans for partner rate limiting, response caching, private APIs reachable only through a VPC endpoint, WAF association, or fine-grained request validation.

An ALB with a Lambda target is the third option and it becomes attractive at very high request volume where per-request API Gateway pricing dominates, but you give up authorizers, usage plans, throttling and request validation, and you inherit the ALB payload size cap of 1 MB. The migration trap is payload format. Under format 1.0 you read event.httpMethod, event.path and event.queryStringParameters.

Under 2.0 those move to event.requestContext.http.method, event.rawPath and event.rawQueryString, cookies arrive as an array rather than a header, and the response shape is lenient: returning a plain object gets serialised to JSON with a 200 automatically, which quietly hides bugs when you meant to return a status code. Teams that copy a REST API handler into an HTTP API and see undefined method values are hitting exactly this. Also remember the integration timeout: the long-standing hard limit was 29 seconds, and although that maximum can now be raised through a quota increase on regional APIs, the right design for anything slower is to return 202 with a job identifier and poll, not to stretch the timeout.

// Handler that survives both payload formats during a migration
export const handler = async (event) => {
  const isV2 = event.version === '2.0';
  const method = isV2 ? event.requestContext.http.method : event.httpMethod;
  const path = isV2 ? event.rawPath : event.path;
  const query = isV2
    ? Object.fromEntries(new URLSearchParams(event.rawQueryString))
    : event.queryStringParameters ?? {};

  if (method !== 'GET') {
    return { statusCode: 405, body: JSON.stringify({ error: 'method_not_allowed' }) };
  }

  // Always return an explicit statusCode: format 2.0 will otherwise assume 200
  return {
    statusCode: 200,
    headers: { 'content-type': 'application/json', 'cache-control': 'no-store' },
    body: JSON.stringify({ path, page: Number(query.page ?? 1) }),
  };
};

Answer

Standard RDS runs a normal MySQL or PostgreSQL engine on top of an EBS volume, with a Multi-AZ standby kept in sync by block-level replication. Aurora replaces the storage engine entirely. The database writes redo log records, not full data pages, to a purpose-built distributed storage fleet that keeps six copies of every 10 GB segment spread across three Availability Zones, using a quorum for writes and reads so it tolerates losing an entire AZ plus one more copy without data loss.

Three consequences matter in interviews. Replicas attach to the same shared storage instead of replaying a binlog, so replica lag is typically milliseconds and adding a reader is fast and cheap, up to fifteen of them. Failover promotes an existing reader and usually completes in well under a minute.

And backups and clones are storage-level operations, so restoring or cloning a large database does not mean copying it. Aurora Serverless v2 scales an instance vertically in fine-grained Aurora Capacity Units, roughly 2 GiB of memory plus proportional CPU per unit, adjusting in seconds without dropping connections, and it can scale down to zero for idle environments. It is the right call for spiky, unpredictable or low-duty-cycle workloads, and for development and staging where a provisioned instance sits idle overnight.

It is the wrong call for a steady, well-understood production load, where a reserved provisioned instance is simply cheaper. Two further levers worth naming: Blue/Green Deployments for near-zero-downtime major version upgrades, and Aurora I/O-Optimized pricing when IO charges exceed roughly a quarter of your Aurora bill.

# Serverless v2 cluster that can idle to zero on a dev environment
aws rds create-db-cluster --db-cluster-identifier gs-dev \
  --engine aurora-postgresql --engine-version 16.4 \
  --serverless-v2-scaling-configuration MinCapacity=0,MaxCapacity=8 \
  --master-username gsadmin --manage-master-user-password

aws rds create-db-instance --db-instance-identifier gs-dev-writer \
  --db-cluster-identifier gs-dev --db-instance-class db.serverless \
  --engine aurora-postgresql

# Major version upgrade with a rehearsed switchover instead of downtime
aws rds create-blue-green-deployment \
  --blue-green-deployment-name pg16-upgrade \
  --source arn:aws:rds:ap-south-1:111122223333:cluster:gs-prod \
  --target-engine-version 16.4
aws rds switchover-blue-green-deployment \
  --blue-green-deployment-identifier bgd-abc123 --switchover-timeout 300

Answer

A Multi-AZ instance deployment keeps a synchronous standby in another AZ. The standby serves no traffic at all; it exists purely for availability and durability. On failure, RDS flips the CNAME behind the endpoint to the standby, which typically takes a minute or two.

A read replica is asynchronous, can serve read traffic, may lag under write pressure, and can be promoted to a standalone writer or created cross-region for disaster recovery. Note the two do different jobs: read replicas are for scale, Multi-AZ is for availability, and candidates who offer read replicas as a high availability answer get marked down. The Multi-AZ DB cluster is the newer option for MySQL and PostgreSQL, with two readable standbys using semi-synchronous replication and failover typically faster than the instance deployment, so it gives you availability and a bit of read scale together.

RDS Proxy sits between the application and the database and solves two specific problems. It pools and multiplexes connections, which matters enormously for Lambda, where a thousand concurrent invocations otherwise open a thousand Postgres connections and exhaust max_connections. And it shortens perceived failover, because the proxy holds the client connection open and reconnects to the new writer underneath.

The catch to mention is pinning: if a session sets a variable, opens a transaction spanning calls, uses temporary tables or prepared statements in ways the proxy cannot reason about, the proxy pins that backend connection to the client and you lose multiplexing entirely. Also remember DNS caching after failover, since a JVM with an infinite DNS TTL will keep dialling the old writer.

# Stop Lambda from exhausting max_connections
aws rds create-db-proxy --db-proxy-name gs-prod-proxy \
  --engine-family POSTGRESQL \
  --auth AuthScheme=SECRETS,SecretArn=arn:aws:secretsmanager:ap-south-1:111122223333:secret:prod/db-AbCdEf,IAMAuth=REQUIRED \
  --role-arn arn:aws:iam::111122223333:role/rds-proxy-role \
  --vpc-subnet-ids subnet-priv-1a subnet-priv-1b \
  --require-tls --idle-client-timeout 900

# Watch for pinning: if this is high, multiplexing is not happening
aws cloudwatch get-metric-statistics --namespace AWS/RDS \
  --metric-name DatabaseConnectionsCurrentlySessionPinned \
  --dimensions Name=ProxyName,Value=gs-prod-proxy \
  --statistics Maximum --period 300 \
  --start-time 2026-08-11T00:00:00Z --end-time 2026-08-11T06:00:00Z

# JVM clients: never cache DNS forever across a failover
# -Dnetworkaddress.cache.ttl=5

Answer

Use three layers and be explicit about what each one proves. Layer one is unit tests where the SDK is mocked in process: aws-sdk-client-mock for JavaScript v3, moto for Python, or the built-in interfaces and fakes in Go. These are fast, run on every commit, and prove your branching logic, retry handling and error mapping.

They prove nothing about IAM or service semantics. Layer two is a local emulator, usually LocalStack, often started through Testcontainers so the test suite owns its lifecycle. This is good for exercising real client code paths against S3, SQS, DynamoDB and Lambda, and DynamoDB Local specifically is faithful enough to catch key schema and index mistakes.

The important caveat, and the thing that shows experience, is that emulators do not enforce IAM the way AWS does and their behaviour diverges on edge cases, so a green LocalStack suite can still fail on the first real deploy with AccessDenied. Layer three is an ephemeral real-cloud stack: deploy the CDK or SAM application into a sandbox account under a per-branch stack name, run integration tests against it, tear it down. That is the only layer that validates IAM policies, resource policies, KMS grants and service quotas.

For serverless specifically, sam local invoke and sam local start-api give you handler-level debugging with a real event payload, and you should keep a folder of recorded production events to replay. Finally, keep at least one post-deploy smoke test that hits the real endpoint, because the failure you most want to catch is a permission you only granted in staging.

import { mockClient } from 'aws-sdk-client-mock';
import { SQSClient, SendMessageCommand } from '@aws-sdk/client-sqs';
import { DynamoDBDocumentClient, PutCommand } from '@aws-sdk/lib-dynamodb';
import { describe, it, expect, beforeEach } from 'vitest';
import { placeOrder } from '../src/orders.js';

const sqsMock = mockClient(SQSClient);
const ddbMock = mockClient(DynamoDBDocumentClient);

beforeEach(() => { sqsMock.reset(); ddbMock.reset(); });

describe('placeOrder', () => {
  it('does not enqueue when the conditional write loses', async () => {
    ddbMock.on(PutCommand).rejects(
      Object.assign(new Error('conditional'), { name: 'ConditionalCheckFailedException' }),
    );

    await expect(placeOrder({ id: 'o-1' })).resolves.toEqual({ duplicate: true });
    expect(sqsMock.commandCalls(SendMessageCommand)).toHaveLength(0);
  });
});

Answer

S3 scales to at least 3500 PUT, COPY, POST or DELETE requests per second and 5500 GET or HEAD requests per second per partitioned prefix, and there is no limit on the number of prefixes. S3 repartitions automatically as traffic grows, so the old advice about prepending a random hash to keys is obsolete; what still helps is spreading heavy parallel work across distinct prefixes so you are not hammering a single one while it repartitions. When you exceed the current capacity you get HTTP 503 SlowDown, and the correct response is exponential backoff with jitter, which the SDKs do for you if you leave retries enabled.

For a 20 GB object, multipart upload is not optional: single PutObject is capped at 5 GB. Multipart splits the object into parts, minimum 5 MB each except the last, maximum 10000 parts, uploads them in parallel, and assembles them with CompleteMultipartUpload. The high-level helpers do this for you: the CLI switches to multipart above a configurable threshold, and the JavaScript Upload class from @aws-sdk/lib-storage handles part sizing, concurrency and retries.

Failed parts persist and keep costing money until an AbortIncompleteMultipartUpload lifecycle rule removes them. For downloads, byte-range fetches let you parallelise and resume. Transfer Acceleration routes uploads through a CloudFront edge and is worth measuring when clients are far from the bucket region, for example a Mumbai bucket serving uploads from North America. And when you need single-digit millisecond latency at very high request rates, S3 Express One Zone directory buckets trade multi-AZ durability for speed.

import { S3Client } from '@aws-sdk/client-s3';
import { Upload } from '@aws-sdk/lib-storage';
import { createReadStream } from 'node:fs';

const s3 = new S3Client({ region: 'ap-south-1', maxAttempts: 5 });

const upload = new Upload({
  client: s3,
  params: {
    Bucket: 'gs-media',
    Key: 'raw/interview-2026-08-11.mp4',
    Body: createReadStream('/data/interview.mp4'),
    ContentType: 'video/mp4',
  },
  queueSize: 6,              // parallel parts in flight
  partSize: 64 * 1024 * 1024, // 64 MB parts keeps us well under 10000
  leavePartsOnError: false,   // abort on failure instead of leaking parts
});

upload.on('httpUploadProgress', (p) => console.log(p.loaded, '/', p.total));
await upload.done();

// CLI equivalent tuning
// aws configure set default.s3.multipart_threshold 128MB
// aws configure set default.s3.max_concurrent_requests 20

Answer

Start from blast radius and quota isolation, not from org charts. Accounts are free, service quotas are per account, and an IAM mistake cannot cross an account boundary, so the unit of isolation should be the environment. A workable layout is a management account that does nothing except billing and Organizations, a log archive account holding the organization CloudTrail trail and Config history with S3 Object Lock, a security tooling account as delegated administrator for GuardDuty, Security Hub, Detective and IAM Access Analyzer, a shared services account for the CI runners, container registry and shared networking, and then one account per workload per environment: payments-prod, payments-staging, jobs-prod, and so on.

Organize them into OUs so policies apply by function. Control Tower can set this up as a landing zone if you would rather not hand-build it. Service control policies are the guard rails, and the subtlety worth stating is that an SCP never grants anything; it only sets the maximum permissions an account can have, so it caps even the account root user.

Good SCPs deny things nobody should ever do: using regions outside ap-south-1 and ap-south-2, disabling CloudTrail or GuardDuty, deleting the log archive bucket, creating IAM users when you have Identity Center. Resource control policies extend the same organization-wide guard rail to the resource side, for example denying any S3 access from principals outside your organization. Humans get in exclusively through IAM Identity Center permission sets, and pipelines assume a deployment role per account with an ExternalId or an OIDC trust from your CI provider.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "IndiaRegionsOnly",
      "Effect": "Deny",
      "NotAction": ["iam:*", "sts:*", "organizations:*", "cloudfront:*", "route53:*", "support:*"],
      "Resource": "*",
      "Condition": {
        "StringNotEquals": {
          "aws:RequestedRegion": ["ap-south-1", "ap-south-2"]
        }
      }
    },
    {
      "Sid": "ProtectAuditTrail",
      "Effect": "Deny",
      "Action": [
        "cloudtrail:StopLogging",
        "cloudtrail:DeleteTrail",
        "guardduty:DeleteDetector",
        "config:DeleteConfigurationRecorder"
      ],
      "Resource": "*"
    },
    {
      "Sid": "NoLongLivedIamUsers",
      "Effect": "Deny",
      "Action": ["iam:CreateUser", "iam:CreateAccessKey"],
      "Resource": "*"
    }
  ]
}

Answer

Work in order of rupees saved per engineering hour, and measure before touching anything. First, visibility: enable the Cost and Usage Report into S3, query it in Athena, and enforce a tagging policy through Organizations so every resource carries owner, environment and service. Untagged spend is the reason most reduction programmes stall.

Second, commitment coverage. Look at Savings Plans and Reserved Instance coverage across EC2, Fargate, Lambda, RDS, ElastiCache and OpenSearch; moving a steady baseline from On-Demand to a one year no-upfront Compute Savings Plan is a large saving with no code change and low risk. Third, architecture-free wins: migrate gp2 volumes to gp3, delete unattached volumes, old snapshots and idle Elastic IPs, set retention on every CloudWatch log group and move verbose groups to the Infrequent Access log class, and apply S3 lifecycle rules plus Intelligent-Tiering.

Fourth, the silent network bill: put gateway endpoints in front of S3 and DynamoDB so NAT stops processing that traffic, add interface endpoints for ECR and Secrets Manager, and make sure chatty services are zone-aware so you stop paying for cross-AZ hops on every internal call. Fifth, compute shape: move to Graviton where your runtime supports it, right-size using Compute Optimizer rather than intuition, and shift stateless batch and CI onto Spot. Sixth, the harder engineering: DynamoDB provisioned with autoscaling instead of on-demand once traffic is predictable, and aggressive caching in CloudFront or ElastiCache to cut origin work. For Indian entities also confirm that credits, the correct billing region and GST invoicing are set up, because reconciliation errors can look like overspend.

-- Athena over the Cost and Usage Report: what actually grew month over month
SELECT
  line_item_product_code            AS service,
  line_item_usage_type              AS usage_type,
  resource_tags_user_service        AS owner_service,
  round(sum(CASE WHEN month = 7 THEN line_item_unblended_cost END), 2) AS jul,
  round(sum(CASE WHEN month = 8 THEN line_item_unblended_cost END), 2) AS aug
FROM cur.gs_billing
WHERE year = '2026' AND month IN (7, 8)
  AND line_item_line_item_type = 'Usage'
GROUP BY 1, 2, 3
HAVING sum(CASE WHEN month = 8 THEN line_item_unblended_cost END) > 50000
ORDER BY aug - jul DESC
LIMIT 25;

-- Orphaned volumes nobody remembers creating
-- aws ec2 describe-volumes --filters Name=status,Values=available \
--   --query 'Volumes[].[VolumeId,Size,VolumeType,CreateTime]' --output table

Answer

Accept that exactly-once delivery does not exist and aim for exactly-once effect instead. Every layer in the path can duplicate: SQS standard guarantees at-least-once, Lambda retries asynchronous invocations, an event source mapping redelivers a whole batch when one record fails, and a network timeout on your own payment call may have succeeded server side. So idempotency has to live in your code, keyed on something the producer controls.

Derive an idempotency key from the business event, an order identifier plus an operation name, never from the SQS message ID, because a redrive or a republish produces a new message ID for the same logical event. Store that key in DynamoDB with a conditional write using attribute_not_exists, which is an atomic compare-and-set at the item level. If the write succeeds you are the first processor and may proceed; if it fails with ConditionalCheckFailedException you are a duplicate and should return the stored result rather than reprocessing.

Handle concurrency by writing an IN_PROGRESS record with a short expiry first, then updating it to COMPLETE with the response payload, so two simultaneous copies do not both slip through the check. Add a TTL attribute so records self-expire after your replay window rather than growing forever. Beyond the consumer, use the transactional outbox pattern on the producer side: write the domain change and the outbox row in one database transaction, then relay the outbox to SQS, or let DynamoDB Streams do the relaying, which removes the dual-write problem where the database commits and the publish fails. Powertools for AWS Lambda ships this idempotency logic if you would rather not maintain it.

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, PutCommand, UpdateCommand, GetCommand } from '@aws-sdk/lib-dynamodb';

const ddb = DynamoDBDocumentClient.from(new DynamoDBClient({}));
const TTL_HOURS = 48;

export async function chargeOnce(orderId, amountPaise, charge) {
  const key = `IDEMP#charge#${orderId}`;
  const now = Math.floor(Date.now() / 1000);

  try {
    await ddb.send(new PutCommand({
      TableName: 'AppTable',
      Item: { pk: key, sk: 'LOCK', state: 'IN_PROGRESS', ttl: now + TTL_HOURS * 3600 },
      ConditionExpression: 'attribute_not_exists(pk)',
    }));
  } catch (err) {
    if (err.name !== 'ConditionalCheckFailedException') throw err;
    const { Item } = await ddb.send(new GetCommand({ TableName: 'AppTable', Key: { pk: key, sk: 'LOCK' } }));
    // Still in flight elsewhere: throw so SQS redelivers after the visibility timeout
    if (!Item || Item.state !== 'COMPLETE') throw new Error('in_progress_retry_later');
    return { duplicate: true, result: Item.result };
  }

  const result = await charge(orderId, amountPaise);

  await ddb.send(new UpdateCommand({
    TableName: 'AppTable',
    Key: { pk: key, sk: 'LOCK' },
    UpdateExpression: 'SET #s = :done, #r = :result',
    ExpressionAttributeNames: { '#s': 'state', '#r': 'result' },
    ExpressionAttributeValues: { ':done': 'COMPLETE', ':result': result },
  }));

  return { duplicate: false, result };
}

Answer

The four standard tiers are backup and restore, pilot light, warm standby and multi-site active-active, in ascending order of cost and descending order of recovery time. A fifteen minute recovery time objective with near-zero data loss rules out backup and restore, which is measured in hours, and rules out pilot light unless you have rehearsed the scale-up. Warm standby is the honest fit: a scaled-down but fully functional copy of the stack running continuously in the second region, with data replicating live, and a scaling event plus a DNS switch on failover.

For an Indian payments platform the second region is usually ap-south-2 Hyderabad, which keeps data inside the country for regulatory comfort. The data layer decides your recovery point objective. Aurora Global Database replicates cross-region with typically sub-second lag and supports managed planned failover.

DynamoDB global tables give multi-region active-active with last-writer-wins conflict resolution. S3 Cross-Region Replication with Replication Time Control gives a fifteen minute service level objective, and EBS snapshots and AMIs must be copied by Data Lifecycle Manager, not by hand. Route 53 health checks plus failover records handle the switch, and Route 53 Application Recovery Controller gives you routing controls you can flip deliberately rather than relying on health checks to notice.

What actually fails in real failovers is rarely the database. It is unraised service quotas in the standby region, because your Lambda concurrency and EC2 vCPU limits there are still at defaults. It is secrets and KMS keys that were never replicated, so the application starts and cannot decrypt anything.

It is DNS TTLs longer than the RTO. And it is the failover never having been tested, which is why a quarterly game day belongs in the answer.

Answer

Separate the three signals and pick the cheapest correct mechanism for each. For metrics, do not call PutMetricData on the request path: it is a synchronous API call that adds latency and is billed per metric. Use Embedded Metric Format instead, where you write a specially structured JSON log line and CloudWatch extracts metrics from it asynchronously at no additional metric publishing cost.

Powertools for AWS Lambda emits EMF for you. Watch cardinality ruthlessly, because every unique combination of dimension values is a separate billed custom metric, and adding userId as a dimension is how a monitoring bill reaches six figures. For traces, X-Ray is the managed option with sampling rules so you trace one request per second plus a percentage of the rest rather than everything, and ADOT, the AWS Distro for OpenTelemetry, is the vendor-neutral path if you want to send the same spans to Grafana Tempo, Honeycomb or a self-hosted collector.

OpenTelemetry is the right default in 2026 because it keeps you portable. For logs, structure everything as JSON with a correlation identifier propagated from the entry point so a single query can reconstruct a request across functions, set retention on every log group, and move verbose but rarely queried groups to the Infrequent Access log class. Sample debug logs rather than emitting them for every invocation, and never log full request payloads containing personal data, which matters under the DPDP Act. Finally, tie it to outcomes: define SLOs on user-visible latency and error rate, alarm on burn rate rather than raw thresholds, and delete every dashboard nobody has opened in a quarter.

// One structured line: CloudWatch turns this into metrics, no API call
console.log(JSON.stringify({
  _aws: {
    Timestamp: Date.now(),
    CloudWatchMetrics: [{
      Namespace: "GoodSpace/Checkout",
      Dimensions: [["Service", "Stage"]],
      Metrics: [
        { Name: "PaymentLatency", Unit: "Milliseconds" },
        { Name: "PaymentFailure", Unit: "Count" }
      ]
    }]
  },
  Service: "checkout",
  Stage: "prod",
  PaymentLatency: 412,
  PaymentFailure: 0,
  correlationId: "c-8f21ab",
  orderId: "o-99213"       // a field, NOT a dimension: keeps cardinality flat
}));

# Logs Insights: p99 by route for the last hour
fields @timestamp, route, duration
| filter ispresent(duration) and level = "info"
| stats count() as n, pct(duration, 99) as p99 by route
| sort p99 desc
| limit 20

Answer

First identify which limit you hit, because the remedies differ. ProvisionedThroughputExceededException from DynamoDB is a capacity or key distribution problem. 503 SlowDown from S3 means the prefix is repartitioning. ThrottlingException or a bare Rate exceeded from control plane APIs such as EC2 DescribeInstances, CloudFormation or IAM means you exceeded a request rate quota that is often not adjustable and not documented.

TooManyRequestsException from Lambda means concurrency. KMS throttles per region on cryptographic operations, which is why unbatched envelope encryption in a tight loop falls over. Then look at your own behaviour, because the usual root cause is a client that fans out with no coordination: a thousand Lambda invocations all calling DescribeStacks at once will throttle no matter how you tune retries.

The SDK side matters. AWS SDKs default to standard retry mode with a small number of attempts and a token bucket retry quota that deliberately stops a retry storm from amplifying an outage. Adaptive mode adds client-side rate limiting that measures throttles and slows the client down automatically, which helps a batch job but is a poor fit for a latency-sensitive request path because it will delay everyone.

Always use exponential backoff with full jitter rather than fixed backoff, since synchronised retries recreate the same spike. Beyond retries, fix the shape of the load: batch calls where the API supports it, cache describe-style responses that change rarely, use pagination with a sensible page size, spread work over time with SQS and a capped event source mapping concurrency, and request a quota increase through Service Quotas for the limits that are adjustable. Finally, only retry idempotent operations, and log the retry count so throttling is visible before it becomes an incident.

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { NodeHttpHandler } from '@smithy/node-http-handler';

// Batch worker: let the SDK throttle itself rather than hammering the service
const ddb = new DynamoDBClient({
  region: 'ap-south-1',
  retryMode: 'adaptive',   // client-side rate limiting; use 'standard' on request paths
  maxAttempts: 6,
  requestHandler: new NodeHttpHandler({
    connectionTimeout: 1000,
    requestTimeout: 3000,   // fail fast instead of holding a Lambda open
  }),
});

// Full jitter, the only backoff that does not resynchronise the herd
export async function withBackoff(fn, attempts = 5) {
  for (let i = 0; i < attempts; i++) {
    try {
      return await fn();
    } catch (err) {
      const retryable = ['ThrottlingException', 'ProvisionedThroughputExceededException', 'SlowDown']
        .includes(err.name);
      if (!retryable || i === attempts - 1) throw err;
      const cap = Math.min(20000, 2 ** i * 100);
      await new Promise((r) => setTimeout(r, Math.random() * cap));
    }
  }
}

Answer

Contain first, investigate second, and do both in parallel with communication. Containment: set the access key status to Inactive rather than deleting it immediately, because deletion destroys the ability to correlate CloudTrail events by key ID during the investigation, then delete it once scoping is done. If the credential belonged to a role, revoke every issued session by attaching an inline deny policy conditioned on aws:TokenIssueTime before now, because deactivating a key does nothing to already-minted STS sessions.

AWS also applies an automated quarantine policy when it detects a key exposed publicly, but you must never rely on that as your control. Investigation: query CloudTrail with LookupEvents filtered on that access key ID to build a timeline of what it did, in which regions and against which resources. Check GuardDuty findings, in particular the credential exfiltration and anomalous behaviour families, and use Detective or Athena over the CloudTrail S3 bucket for anything beyond the ninety day event history window.

Then hunt for persistence, because a competent attacker plants a way back in: new IAM users or access keys, new roles trusting an external account, modified trust policies, new regions with running EC2 instances for cryptomining, changed S3 bucket policies, disabled CloudTrail or GuardDuty, and new Lambda functions or EventBridge rules. Recovery: rotate every secret the key could reach, not just the key itself, and rebuild anything you cannot prove is clean. Prevention is the part interviewers actually score: eliminate long-lived keys in favour of IAM Identity Center and OIDC federation for CI, run secret scanning as a pre-commit hook and in the pipeline, enforce IMDSv2 so an SSRF cannot lift instance credentials, and use permissions boundaries so a leaked credential is limited by design.

# 1. Contain, do not delete yet
aws iam update-access-key --user-name build-bot --access-key-id AKIAEXAMPLE --status Inactive

# 2. Kill sessions already issued from a compromised role
#    Attach as an inline policy on the role, with a timestamp of "now"
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Deny",
    "Action": "*",
    "Resource": "*",
    "Condition": { "DateLessThan": { "aws:TokenIssueTime": "2026-08-11T09:15:00Z" } }
  }]
}

# 3. Scope the blast radius
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=AccessKeyId,AttributeValue=AKIAEXAMPLE \
  --start-time 2026-08-04T00:00:00Z --max-results 200

# 4. Prevention: no v1 metadata, so SSRF cannot steal instance credentials
aws ec2 modify-instance-metadata-options --instance-id i-0123456789abcdef0 \
  --http-tokens required --http-put-response-hop-limit 1 --http-endpoint enabled

Answer

VPC peering is a point-to-point, non-transitive link. Two peered VPCs can talk, but traffic cannot route through one to reach a third, so connecting forty VPCs fully meshed needs 780 peering connections and an unmanageable pile of route table entries. It is free of hourly charges and has no bandwidth bottleneck, so it remains fine for two or three VPCs that talk a lot.

Transit Gateway is the hub and spoke answer at this scale. Each VPC attaches once, the gateway routes transitively, and multiple gateway route tables let you segment traffic, for example letting every spoke reach shared services while preventing production from reaching development. It also terminates Site-to-Site VPN and Direct Connect gateways, and peers across regions for a multi-region backbone.

The cost model is per attachment per hour plus per GB processed, which is real money at forty attachments, so it should be a deliberate choice. PrivateLink solves a different problem: exposing one service, not connecting whole networks. The consumer creates an interface endpoint that maps to a Network Load Balancer in the provider account, traffic flows one way from consumer to provider, and crucially there is no routing relationship at all, which means overlapping CIDR ranges are irrelevant.

That is the answer to the overlap question: you cannot peer or attach overlapping VPCs to a Transit Gateway, and the practical options are to renumber, which nobody wants to do, or to expose the specific services over PrivateLink and stop trying to merge the networks. Add hybrid DNS on top with Route 53 Resolver inbound and outbound endpoints plus forwarding rules, and private hosted zones associated to the VPCs that need them.

# Hub and spoke, with a separate route table per segment
aws ec2 create-transit-gateway --description gs-core \
  --options AmazonSideAsn=64512,DefaultRouteTableAssociation=disable,DefaultRouteTablePropagation=disable

aws ec2 create-transit-gateway-vpc-attachment \
  --transit-gateway-id tgw-0abc --vpc-id vpc-prod \
  --subnet-ids subnet-tgw-1a subnet-tgw-1b

# Prod spoke associates with the prod table but only propagates to shared services
aws ec2 associate-transit-gateway-route-table \
  --transit-gateway-route-table-id tgw-rtb-prod --transit-gateway-attachment-id tgw-attach-01
aws ec2 enable-transit-gateway-route-table-propagation \
  --transit-gateway-route-table-id tgw-rtb-shared --transit-gateway-attachment-id tgw-attach-01

# Overlapping CIDRs: expose the service instead of routing the network
aws ec2 create-vpc-endpoint-service-configuration \
  --network-load-balancer-arns arn:aws:elasticloadbalancing:ap-south-1:111122223333:loadbalancer/net/gs-api/9f1 \
  --acceptance-required

Answer

There are two supported mechanisms and you should know both. IRSA, IAM Roles for Service Accounts, works by giving the cluster an OIDC identity provider registered in IAM. You annotate a Kubernetes service account with a role ARN, the pod receives a projected web identity token, and the SDK exchanges it through sts:AssumeRoleWithWebIdentity for temporary credentials.

The role trust policy must reference the cluster OIDC issuer and pin both the sub claim to the exact namespace and service account and the aud claim to sts.amazonaws.com, otherwise any service account in the cluster can assume it. EKS Pod Identity is the newer mechanism: an agent runs on the nodes, you create an association between a namespace, a service account and a role through the EKS API, and there is no per-cluster OIDC provider or trust policy editing at all, which makes role reuse across clusters far simpler. Pod Identity is the better default for new clusters where it is supported.

The IP exhaustion question comes from the VPC CNI. Unlike overlay networks, the Amazon VPC CNI gives every pod a real, routable VPC IP address from the subnet, which is why service meshes and security groups work naturally, and also why a /24 node subnet supports far fewer pods than people expect. Each instance type also caps ENIs and IPs per ENI, so pod density is bounded by instance size.

The fixes are prefix delegation, where the CNI allocates /28 prefixes instead of individual addresses and multiplies density; custom networking with a secondary non-routable CIDR such as 100.64.0.0/10 for pods; or an IPv6 cluster, which removes the problem entirely. Pair this with Karpenter for node provisioning, which consolidates workloads and bin-packs onto Spot far more aggressively than Cluster Autoscaler.

# IRSA trust policy: pin BOTH sub and aud, or any pod can assume the role
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.ap-south-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "oidc.eks.ap-south-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E:sub": "system:serviceaccount:payments:api-sa",
        "oidc.eks.ap-south-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E:aud": "sts.amazonaws.com"
      }
    }
  }]
}

# Service account side
apiVersion: v1
kind: ServiceAccount
metadata:
  name: api-sa
  namespace: payments
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/payments-api

# Or skip OIDC entirely with Pod Identity
# aws eks create-pod-identity-association --cluster-name gs-prod \
#   --namespace payments --service-account api-sa \
#   --role-arn arn:aws:iam::111122223333:role/payments-api

# Multiply pod density on the VPC CNI
# kubectl set env ds aws-node -n kube-system ENABLE_PREFIX_DELEGATION=true