Same-account S3 Access Point authentication (IRSA or EKS Pod Identity)

Use this guide when the EKS cluster, the Acceldata Data Plane, the S3 bucket, the Access Point, and the IAM role all exist in the same AWS account.

The goal is that ADOC reaches S3 only through an Access Point. The Access Point allows the IAM role, and that role is bound to the Data Plane's Kubernetes service accounts using one of two authentication methods: IAM Roles for Service Accounts (IRSA), or EKS Pod Identity.

Complete the shared setup once, then choose exactly one authentication option:

• Option 1: IRSA
• Option 2: EKS Pod Identity

For a scenario where the bucket and Access Point live in a different AWS account from the cluster, see Cross-account S3 Access Point authentication. This is also a different mechanism from Cross‑Account Access Setup, which grants direct cross-account bucket access without an Access Point.

Placeholders

Service accounts to bind

Confirm the live service account names in your namespace before you begin:

kubectl -n <namespace> get sa

Bind the IAM role to these service accounts:

• analysis-service
• analysis-standalone-service
• spark-scheduler
• torch-monitors
• analysis-sql-service
• spark-history-server
• acceldata-dataplane-spark

Skip any service account that does not exist in your deployment.

Shared setup

Step 1: Create the S3 bucket

1. In S3, create bucket <bucket> in <region>.
2. Keep Block Public Access enabled.
3. Upload sample files under <prefix>, for example allowed/sample.csv.

Step 2: Create the Access Point

1. Open bucket <bucket> and create Access Point <ap-name>.
2. Choose network Internet, or the VPC your cluster uses.
3. Keep Block Public Access enabled.
4. Copy the Access Point ARN and alias after creation.

Step 3: Create the IAM role with Access Point-only permissions

Create IAM role <iam-role>. Its trust policy depends on the authentication option you choose in Step 4 or Step 5, but the S3 permissions are the same for both options.

Attach this inline policy. Use Access Point ARNs only. To keep the role strictly Access Point-only, do not add arn:aws:s3:::<bucket> to the role.

Json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "HeadAndListAccessPoint",
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": "arn:aws:s3:<region>:<account-id>:accesspoint/<ap-name>"
    },
    {
      "Sid": "ReadViaAccessPoint",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:GetObjectVersion"],
      "Resource": "arn:aws:s3:<region>:<account-id>:accesspoint/<ap-name>/object/<prefix>*"
    }
  ]
}

Do not add an s3:prefix condition to s3:ListBucket. ADOC's Test Connection uses HeadBucket, which does not send a prefix. A prefix condition causes Test Connection to fail even when object reads work.

Step 4: Add the Access Point policy

On Access Point <ap-name>, set a policy that allows only <iam-role>:

Json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowListBucketViaAp",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<account-id>:role/<iam-role>"
      },
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:<region>:<account-id>:accesspoint/<ap-name>"
    },
    {
      "Sid": "AllowGetObjectViaAp",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<account-id>:role/<iam-role>"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:<region>:<account-id>:accesspoint/<ap-name>/object/<prefix>*"
    }
  ]
}

Step 5: Add the bucket policy

The Access Point policy only works if the bucket policy also allows the request. An Access Point policy by itself is not enough — if the bucket policy is empty and only the Access Point ARN is allowed in IAM, head-bucket returns 403.

Add this statement to the bucket policy. If the bucket already has a policy, keep the existing statements and add this one.

Json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowRoleOnlyThroughThisAccessPoint",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<account-id>:role/<iam-role>"
      },
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation",
        "s3:GetObject",
        "s3:GetObjectVersion"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket>",
        "arn:aws:s3:::<bucket>/*"
      ]
    }
  ]
}

The bucket policy is required in addition to the Access Point policy.

Option 1: IRSA

Confirm the cluster OIDC provider

aws eks describe-cluster --name <cluster> --region <region> \
  --query 'cluster.identity.oidc.issuer' --output text

If the result is empty, associate the provider:

eksctl utils associate-iam-oidc-provider --cluster <cluster> --region <region> --approve

Configure the trust policy

In IAM, open <iam-role> and go to Trust relationships. Configure Web Identity for this cluster's OIDC provider, with audience sts.amazonaws.com. Set sub to every service account listed above.

Json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<account-id>:oidc-provider/<oidc-host>"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "<oidc-host>:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "<oidc-host>:sub": [
            "system:serviceaccount:<namespace>:analysis-service",
            "system:serviceaccount:<namespace>:analysis-standalone-service",
            "system:serviceaccount:<namespace>:spark-scheduler",
            "system:serviceaccount:<namespace>:torch-monitors",
            "system:serviceaccount:<namespace>:analysis-sql-service",
            "system:serviceaccount:<namespace>:spark-history-server",
            "system:serviceaccount:<namespace>:acceldata-dataplane-spark"
          ]
        }
      }
    }
  ]
}

A Pod Identity association on the same service account overrides IRSA. For an IRSA-only setup, remove any existing associations first:

aws eks list-pod-identity-associations --cluster-name <cluster> \
  --region <region> --query 'associations[?namespace==`<namespace>`].[associationId,serviceAccount,roleArn]' --output table

aws eks delete-pod-identity-association --cluster-name <cluster> \
  --region <region> --association-id <association-id>

Bind the role to service accounts

NS=<namespace>
ROLE=arn:aws:iam::<account-id>:role/<iam-role>
for sa in analysis-service analysis-standalone-service spark-scheduler \
 torch-monitors analysis-sql-service spark-history-server acceldata-dataplane-spark
do
 kubectl -n "$NS" annotate sa "$sa" eks.amazonaws.com/role-arn="$ROLE" --overwrite
done

Skip service accounts that do not exist. Restart the deployments, skipping any deployment that is already broken.

kubectl -n <namespace> rollout restart deploy

Wait until the deployments are ready, then confirm IRSA on a pod:

kubectl -n <namespace> exec deploy/<analysis-standalone-deploy> -- env \
 | grep -E 'AWS_ROLE_ARN|AWS_WEB_IDENTITY_TOKEN_FILE|AWS_CONTAINER_CREDENTIALS'

Pass:AWS_ROLE_ARN=arn:aws:iam::<account-id>:role/<iam-role> is present, and AWS_CONTAINER_CREDENTIALS_FULL_URI is absent. Authentication in ADOC is AWS IAM Roles For Service Accounts.

Option 2: EKS Pod Identity

Check the Pod Identity agent

kubectl -n kube-system get ds eks-pod-identity-agent

The agent must be running.

Configure the trust policy

In IAM, open <iam-role> and go to Trust relationships. Add this statement. If the role already has the IRSA trust statement, keep it and add this one — do not change the Access Point policy, bucket policy, or S3 inline policy.

Json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "pods.eks.amazonaws.com"
      },
      "Action": [
        "sts:AssumeRole",
        "sts:TagSession"
      ]
    }
  ]
}

Bind associations

Do not annotate the service accounts for Pod Identity.

CLUSTER=<cluster>
REGION=<region>
NS=<namespace>
ROLE=arn:aws:iam::<account-id>:role/<iam-role>

for sa in analysis-service analysis-standalone-service spark-scheduler \
  torch-monitors analysis-sql-service spark-history-server acceldata-dataplane-spark
do
  aws eks create-pod-identity-association \
    --cluster-name "$CLUSTER" --region "$REGION" \
    --namespace "$NS" --service-account "$sa" \
    --role-arn "$ROLE"
done

Skip service accounts that do not exist, then restart the deployments.

kubectl -n <namespace> rollout restart deploy

Confirm Pod Identity on a pod:

kubectl -n <namespace> exec deploy/<analysis-standalone-deploy> -- env \
 | grep -E 'AWS_ROLE_ARN|AWS_WEB_IDENTITY_TOKEN_FILE|AWS_CONTAINER_CREDENTIALS'

Pass:AWS_CONTAINER_CREDENTIALS_FULL_URI is set and AWS_ROLE_ARN is absent, because Pod Identity overrides IRSA. Authentication in ADOC is EKS Pod Identity. If you already added an S3 data source using IRSA credentials, create a new ADOC data source rather than reusing that one.

Prove AWS access before configuring ADOC

Delete any leftover test pod first. Logs from an earlier run are not proof the current configuration works.

kubectl -n <namespace> delete pod ap-head --ignore-not-found
kubectl -n <namespace> run ap-head --restart=Never \
 --image=public.ecr.aws/aws-cli/aws-cli:2.17.55 \
 --overrides='{"spec":{"serviceAccountName":"analysis-standalone-service"}}' \
 --command -- aws s3api head-bucket --bucket <ap-arn> --region <region>

Wait until the pod status is Completed, then run:

kubectl -n <namespace> logs ap-head

Pass: the logs return JSON containing BucketRegion. A 403 response points to an IAM, Access Point, or bucket policy problem, not to ADOC. Do not exec into this pod — it has already exited. An Access Point-only role on torch-monitors may return 403 for existing SQS-based monitors; this is expected.

Add the ADOC datasource and validate

In ADOC, add a new AWS S3 datasource. For the general steps to open the Add Data Source wizard, seeAmazon S3. Use these values for an Access Point connection.

Add datasource / Test Connection

Never enter s3://... in Bucket Name. This is the literal string AWS's HeadBucket call uses. Select Test Connection. A successful result shows the datasource connected.

Asset / Observability configuration

After the connection succeeds, create an asset with any name and set:

Run the crawl, then run the profile.

GET /validate on analysis-service is a health probe. The relevant jobs for datasource testing use type CONNECTION_VALIDATION.

Troubleshooting