AWS: Use assumed-role credentials for REST SigV4 signing by GabrielBBaldez · Pull Request #16794 · apache/iceberg

GabrielBBaldez · 2026-06-12T13:58:56Z

What changed

RESTSigV4AuthSession signs catalog requests with AwsProperties#restCredentialsProvider(), whose decision chain only handled three cases:

explicit static rest.* keys → StaticCredentialsProvider
a custom client.credentials-provider → that provider
otherwise → DefaultCredentialsProvider

There was no branch for client.assume-role.*. So when the catalog is configured with AssumeRoleAwsClientFactory, the assumed role is applied to the S3/Glue/KMS/DynamoDB clients (via applyAssumeRoleConfigurations) but never to the SigV4 REST signing path — REST calls silently fall back to the default credential chain and diverge from the other AWS clients.

This adds an assume-role branch between the custom-provider and default-chain steps that returns an StsAssumeRoleCredentialsProvider built from the existing client.assume-role.* properties (arn, region, session name, timeout, external id, tags), mirroring AssumeRoleAwsClientFactory#createCredentialsProvider. Explicit static rest.* keys and client.credentials-provider keep precedence.

This is the "fastest path" described in the issue. Happy to instead extract the STS provider construction into a shared utility (option 3 in the issue) if reviewers prefer to avoid the small duplication with AssumeRoleAwsClientFactory.

Closes #16667

Testing

New unit tests in TestAwsProperties:

assume-role configured → restCredentialsProvider() returns an StsAssumeRoleCredentialsProvider
no credentials configured → falls back to DefaultCredentialsProvider
explicit static rest.* keys → still returns StaticCredentialsProvider (precedence preserved)

./gradlew :iceberg-aws:test :iceberg-aws:spotlessJavaCheck :iceberg-aws:checkstyleMain :iceberg-aws:checkstyleTest passes locally.

RESTSigV4AuthSession signs requests with AwsProperties#restCredentialsProvider(), whose decision chain only handled static rest.* keys, a custom client.credentials-provider, and otherwise the default credential chain. When the catalog is configured with AssumeRoleAwsClientFactory, the assumed role is applied to the S3/Glue/KMS/DynamoDB clients but never to the SigV4 REST signing path, so REST calls silently fall back to the default credential chain and diverge from the other AWS clients. Add an assume-role branch between the custom-provider and default-chain steps that returns an StsAssumeRoleCredentialsProvider built from the existing client.assume-role.* properties, mirroring AssumeRoleAwsClientFactory#createCredentialsProvider.

CTTY · 2026-06-18T22:18:15Z

+        !Strings.isNullOrEmpty(this.clientAssumeRoleRegion),
+        "Cannot assume role %s to sign REST requests: %s is required",
+        this.clientAssumeRoleArn,
+        CLIENT_ASSUME_ROLE_REGION);


I think we should return here and fall back to DefaultCredentialsProvider. We can print out a warning message here

Done in the latest push, thanks!

…ing for REST signing

CTTY

Just took a pass, generally lgtm! left some feedbacks

CTTY · 2026-07-06T22:30:03Z

    return DefaultCredentialsProvider.builder().build();
  }

+  protected StsAssumeRoleCredentialsProvider assumeRoleCredentialsProvider() {


This should be package private/no accessibility modifier?

StsAssumeRoleCredentialsProvider assumeRoleCredentialsProvider() {

Changed to package private

CTTY · 2026-07-06T22:34:33Z

-    this.clientAssumeRoleSessionName = properties.get(CLIENT_ASSUME_ROLE_SESSION_NAME);
+    this.clientAssumeRoleSessionName =
+        properties.getOrDefault(
+            CLIENT_ASSUME_ROLE_SESSION_NAME, String.format("iceberg-aws-%s", UUID.randomUUID()));


I think we should still initialize this lazily. Auto-gen this is actually a behavior change.

We can check and generate this in createAssumeRoleRequest

I reverted back to lazily generating the session name in the latest change, but I created a private UUID so that the randomized session name is stable across lazy generations for the lifetime of AwsProperties.

This matches the random uuid used in other places like BaseMetadataTable and the test for AliyunOSSExtension.

GabrielBBaldez · 2026-07-09T16:59:23Z

Awesome! Thanks, @NathanCYee and @CTTY, for all the teamwork and collaboration. It was a pleasure working with you guys!

… aws to refer to rest sigv4 catalog configuration

…andom session name

…ate. Update AssumeRoleAwsClientFactory to use shared function to reduce code duplication

CTTY

LGTM!

kevinjqliu · 2026-07-10T17:11:49Z

+        .stsClient(
+            StsClient.builder()
+                .applyMutation(httpClientProperties::applyHttpClientConfigurations)
+                .region(Region.of(this.clientAssumeRoleRegion))


Codex flagged this:

AwsProperties.java now sets the STS client region from client.assume-role.region, but the existing contract says that property applies to AWS clients except STS.

| `client.assume-role.region` | null, requires user input | All AWS clients except the STS client will use the given region instead of the default region chain |

We might want to update one of the places to match behavior (cc @CTTY)

You're right this is a behavioral change.

There was a documented update that the STS service changed from only being served out of us-east-1 to becoming regional by default for non opt-in regions.

April 18, 2025: AWS has made changes to the AWS Security Token Service (AWS STS) global endpoint (sts.amazonaws.com) in Regions enabled by default to enhance its resiliency and performance. AWS STS requests to the global endpoint are automatically served in the same AWS Region as your workloads. These changes will not be deployed to opt-in Regions. We recommend that you use the appropriate AWS STS regional endpoints. For more information, see AWS STS global endpoint changes in the IAM User Guide.

The old behavior to use the global endpoint is now considered legacy in the SDK.

All new SDK major versions releasing after July 2022 will default to regional. New SDK major versions might remove this setting and use regional behavior. To reduce future impact regarding this change, we recommend you start using regional in your application when possible.

According to the same documentation according to documentation the AWS 2.x SDK already defaults to regional except for AssumeRole which routes to global.

If no Region is configured, the AssumeRole and AssumeRoleWithWebIdentity will use the global STS endpoint.

If we are getting ahead of future service disruption, using regional STS would be the forward looking approach but it may temporarily break compatibility for some users in opt-in regions who have not turned on the STS regional endpoint.

#10289 flagged the SDK behavior change as already have taken effect by default. This would then be an Iceberg documentation inaccuracy that did not track this new behavior.

Created #17158 to remove the exclusion in the docs

github-actions Bot added the AWS label Jun 12, 2026

GabrielBBaldez mentioned this pull request Jun 12, 2026

AWS: restCredentialsProvider should return StsAssumeRoleCredentialsProvider when assume role configurations are set #16667

Closed

3 tasks

GabrielBBaldez force-pushed the aws-assume-role-rest-credentials branch from c17cfe7 to c64c9eb Compare June 12, 2026 14:14

GabrielBBaldez force-pushed the aws-assume-role-rest-credentials branch from c64c9eb to 2c13a21 Compare June 12, 2026 14:22

CTTY reviewed Jun 18, 2026

View reviewed changes

AWS: Fall back to default credentials when assume-role region is miss…

9768e85

…ing for REST signing

GabrielBBaldez requested a review from CTTY June 24, 2026 13:50

github-actions Bot added the docs label Jul 1, 2026

NathanCYee force-pushed the aws-assume-role-rest-credentials branch 3 times, most recently from 754a2ac to 0a331df Compare July 1, 2026 20:56

CTTY reviewed Jul 6, 2026

View reviewed changes

NathanCYee force-pushed the aws-assume-role-rest-credentials branch 2 times, most recently from 0df785f to ae521e6 Compare July 9, 2026 16:58

Docs: Add documentation on SigV4 REST credential configuration

942b277

NathanCYee force-pushed the aws-assume-role-rest-credentials branch from ae521e6 to a744bea Compare July 9, 2026 17:56

NathanCYee added 3 commits July 9, 2026 11:13

Docs: Add assume-role configurations to sigv4 docs. Add disclaimer in…

aded6a7

… aws to refer to rest sigv4 catalog configuration

AWS: Use stable UUID per AwsProperties instance for lazily creating r…

de271ce

…andom session name

AWS: Make assumeRoleCredentialsProvider in AwsProperties package-priv…

72aaa35

…ate. Update AssumeRoleAwsClientFactory to use shared function to reduce code duplication

NathanCYee force-pushed the aws-assume-role-rest-credentials branch from a744bea to 72aaa35 Compare July 9, 2026 18:17

CTTY approved these changes Jul 9, 2026

View reviewed changes

CTTY merged commit d411012 into apache:main Jul 9, 2026
38 checks passed

kevinjqliu reviewed Jul 10, 2026

View reviewed changes

NathanCYee mentioned this pull request Jul 10, 2026

Docs: Update documentation on assume role STS behavior to reflect regional endpoint change #17158

Open

Uh oh!

Conversation

GabrielBBaldez commented Jun 12, 2026

What changed

Testing

Uh oh!

CTTY Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

GabrielBBaldez Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

CTTY left a comment

Choose a reason for hiding this comment

Uh oh!

CTTY Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

NathanCYee Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

CTTY Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

NathanCYee Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GabrielBBaldez commented Jul 9, 2026

Uh oh!

CTTY left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kevinjqliu Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NathanCYee Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NathanCYee Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NathanCYee Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

NathanCYee Jul 9, 2026 •

edited

Loading

kevinjqliu Jul 10, 2026 •

edited

Loading

NathanCYee Jul 10, 2026 •

edited

Loading

NathanCYee Jul 10, 2026 •

edited

Loading