Skip to content

Conversation

@CalvinKirs
Copy link
Member

For S3-compatible object storages such as COS and OBS, authentication does not fully follow the AWS native credential provider chain. In these systems, anonymous access is a valid and commonly used mode when no AK/SK is configured.

However, when aws_credentials_provider_version is set to v2, BE currently falls back to the AWS SDK v2 default credential provider chain if no credentials are explicitly provided. This behavior is AWS-specific and is not applicable to S3-compatible storage systems like COS and OBS, which may not support or require the AWS credential resolution chain.

Problem

When aws_credentials_provider_version = v2 and no AK/SK is configured:

  • BE attempts to resolve credentials using the AWS SDK v2 provider chain
  • This may lead to unexpected authentication failures or unnecessary environment dependency
  • The behavior is inconsistent with S3-compatible storage expectations

Expected Behavior

For S3-compatible object storage:

  • If AK/SK is explicitly provided, use the configured credentials
  • If no credentials are provided, fall back to anonymous credentials
  • Do NOT trigger the AWS SDK v2 default credential provider chain

…dentials are absent

For S3-compatible object storages such as COS and OBS, authentication does not fully follow
the AWS native credential provider chain. In these systems, anonymous access is a valid and
commonly used mode when no AK/SK is configured.

However, when `aws_credentials_provider_version` is set to `v2`, BE currently falls back to
the AWS SDK v2 default credential provider chain if no credentials are explicitly provided.
This behavior is AWS-specific and is not applicable to S3-compatible storage systems like
COS and OBS, which may not support or require the AWS credential resolution chain.

Problem
-------
When `aws_credentials_provider_version = v2` and no AK/SK is configured:
- BE attempts to resolve credentials using the AWS SDK v2 provider chain
- This may lead to unexpected authentication failures or unnecessary environment dependency
- The behavior is inconsistent with S3-compatible storage expectations

Expected Behavior
-----------------
For S3-compatible object storage:
- If AK/SK is explicitly provided, use the configured credentials
- If no credentials are provided, fall back to anonymous credentials
- Do NOT trigger the AWS SDK v2 default credential provider chain
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants