Skip to content

Credentials: error handling for unsupported providers#707

Open
nishika26 wants to merge 8 commits intomainfrom
bug/creds_unsupported_provider
Open

Credentials: error handling for unsupported providers#707
nishika26 wants to merge 8 commits intomainfrom
bug/creds_unsupported_provider

Conversation

@nishika26
Copy link
Collaborator

@nishika26 nishika26 commented Mar 23, 2026

Summary

Target issue is #708

Checklist

Before submitting a pull request, please ensure that you mark these task.

  • Ran fastapi run --reload app/main.py or docker compose up in the repository root and test.
  • If you've fixed a bug or added code that is tested and has test cases.

Notes

  • Documentation

    • Added comprehensive guides and JSON request examples for credential configuration with newly supported providers: Google, Sarvamai, and ElevenLabs.
  • Provider Updates

    • Removed AWS from supported providers for now.
  • Improvements

    • Enhanced credential validation error handling with clearer, more actionable error messages.

@nishika26 nishika26 self-assigned this Mar 23, 2026
@coderabbitai
Copy link

coderabbitai bot commented Mar 23, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b0b73883-1084-473f-9915-4dc7033c98d3

📥 Commits

Reviewing files that changed from the base of the PR and between 5fcfa88 and 9451f33.

📒 Files selected for processing (1)
  • backend/app/utils.py

📝 Walkthrough

Walkthrough

Changes address provider management and error handling across the codebase. AWS provider removed from supported list. Documentation updated with additional providers (Google, Sarvamai, ElevenLabs). Validation logic refactored to fail fast on first error rather than accumulating multiple errors. Error handling improved to convert ValueError exceptions to HTTPException with status code 400. Error messages simplified by removing Pydantic-specific prefixes.

Changes

Cohort / File(s) Summary
Documentation Updates
backend/app/api/docs/credentials/create.md, backend/app/api/docs/onboarding/onboarding.md
Added and expanded documentation for supported providers (Google, Sarvamai, ElevenLabs), included JSON request examples, and updated credential payload examples.
Provider Configuration
backend/app/core/providers.py
Removed AWS provider enum member and corresponding configuration entry, leaving OpenAI, Langfuse, Google, Sarvamai, and ElevenLabs as active providers.
Error Handling & Validation
backend/app/crud/credentials.py, backend/app/models/onboarding.py, backend/app/utils.py
Wrapped provider validation calls in try/except blocks to convert ValueError to HTTPException with status 400. Changed credential list validator to fail immediately on first invalid item instead of collecting errors. Enhanced error message processing to strip Pydantic-specific prefixes.
Test Updates
backend/app/tests/api/routes/test_onboarding.py, backend/app/tests/core/test_providers.py, backend/app/tests/crud/test_credentials.py
Updated test assertions to expect HTTPException instead of ValueError, adjusted error message matching to reflect new fail-fast validation behavior, and removed AWS-related test cases.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

documentation, enhancement

Suggested reviewers

  • Prajna1999
  • AkhileshNegi
  • vprashrex

Poem

🐰 hop hop, a change most grand,
AWS bids farewell to the land,
New providers bloom—Google and friends,
Fast-fail validation now starts and ends,
Error messages clear, no Pydantic in sight,
The codebase hops forward, shining bright!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 30.77% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main objective of the PR: improving error handling for unsupported credential providers, specifically by removing AWS and clarifying validation error messages.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch bug/creds_unsupported_provider

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
backend/app/core/providers.py (1)

2-2: Use built-in generics instead of deprecated typing.Dict and typing.List.

Per Python 3.11+ best practices, use dict and list directly instead of importing from typing.

Suggested fix
-from typing import Dict, List
+from typing import Dict, List  # Keep for backward compatibility or remove entirely

And update usages:

-PROVIDER_CONFIGS: Dict[Provider, ProviderConfig] = {
+PROVIDER_CONFIGS: dict[Provider, ProviderConfig] = {
-def get_supported_providers() -> List[str]:
+def get_supported_providers() -> list[str]:
-    required_fields: List[str]
+    required_fields: list[str]
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@backend/app/core/providers.py` at line 2, Replace deprecated typing generics
by using built-in generics: remove "from typing import Dict, List" and update
any type annotations that reference Dict and List to use lowercase dict and list
(e.g., Dict[str, Any] -> dict[str, Any], List[int] -> list[int]). Locate usages
in this module (and any functions/classes that reference Dict or List) such as
type annotations in functions, return types, and variable annotations and
convert them to the built-in forms to comply with Python 3.11+ best practices.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@backend/app/core/providers.py`:
- Around line 19-20: The test
test_validate_provider_credentials_missing_fields() in
backend/app/tests/core/test_providers.py still calls
validate_provider_credentials("aws", ...) even though "aws" was removed from the
Provider enum in backend/app/core/providers.py; remove the AWS-specific test
case (the validate_provider_credentials("aws", ...) assertion and any related
setup for AWS) so the test only covers existing providers (e.g., the remaining
Provider values used in the file) and update any expected error counts/messages
accordingly.

In `@backend/app/models/onboarding.py`:
- Around line 96-115: The field validator in onboarding.py raises TypeError for
malformed credential entries, which bypasses Pydantic/CRUD handling; change
those TypeError raises to ValueError so Pydantic validators and
backend/app/crud/credentials.py (which only catches ValueError) will convert
these to proper 400 responses—specifically replace the two TypeError usages in
the validator loop (the "Credential must be a dict..." and the "Value for
provider '... must be an object/dict." messages) with ValueError while keeping
the same messages and leave validate_provider(provider_key) as-is.

---

Nitpick comments:
In `@backend/app/core/providers.py`:
- Line 2: Replace deprecated typing generics by using built-in generics: remove
"from typing import Dict, List" and update any type annotations that reference
Dict and List to use lowercase dict and list (e.g., Dict[str, Any] -> dict[str,
Any], List[int] -> list[int]). Locate usages in this module (and any
functions/classes that reference Dict or List) such as type annotations in
functions, return types, and variable annotations and convert them to the
built-in forms to comply with Python 3.11+ best practices.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d9290652-293c-4f5a-bd3f-6662b6a9690b

📥 Commits

Reviewing files that changed from the base of the PR and between 029f710 and f749d69.

📒 Files selected for processing (6)
  • backend/app/api/docs/credentials/create.md
  • backend/app/api/docs/onboarding/onboarding.md
  • backend/app/core/providers.py
  • backend/app/crud/credentials.py
  • backend/app/models/onboarding.py
  • backend/app/utils.py

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (3)
backend/app/tests/crud/test_credentials.py (1)

209-210: Consider moving the import to the top of the file.

HTTPException is imported identically inside both test_invalid_provider (line 209) and test_langfuse_credential_validation (line 261). Moving this import to the top-level alongside the other imports (lines 4-18) would eliminate duplication and follow typical Python conventions.

Suggested change

Add to the existing imports at the top of the file:

from app.core.exception_handlers import HTTPException

Then remove the local imports at lines 209-210 and 261-262.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@backend/app/tests/crud/test_credentials.py` around lines 209 - 210, Move the
duplicate local import of HTTPException to the module-level imports: add "from
app.core.exception_handlers import HTTPException" alongside the other top-level
imports, then remove the two in-function imports inside test_invalid_provider
and test_langfuse_credential_validation so both tests use the single top-level
HTTPException symbol.
backend/app/tests/api/routes/test_onboarding.py (1)

304-335: Consider renaming the test function to reflect fail-fast behavior.

The function name test_onboard_project_aggregates_multiple_credential_errors is misleading since the validation now uses fail-fast behavior and no longer aggregates errors. The docstring was updated but the function name still suggests aggregation.

Consider renaming to something like:

-def test_onboard_project_aggregates_multiple_credential_errors(
+def test_onboard_project_fails_on_first_credential_error(
     client: TestClient, superuser_token_headers: dict[str, str], db: Session
 ) -> None:
     """Test onboarding reports credential validation errors (fails on first error)."""

Regarding the unused db parameter flagged by static analysis: this is common in pytest fixtures where the parameter ensures proper test isolation/setup even if not directly referenced in the test body.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@backend/app/tests/api/routes/test_onboarding.py` around lines 304 - 335,
Rename the test function
test_onboard_project_aggregates_multiple_credential_errors to reflect the new
fail-fast validation (for example
test_onboard_project_fails_fast_on_credential_error) and update any docstring if
needed; keep the unused pytest fixture parameter db in the signature so test
isolation/setup remains intact (do not remove db even if static analysis flags
it) and ensure the test body (onboard_data, client.post call and the final
assertion checking "Unsupported provider") continues to reference the same
behavior.
backend/app/models/onboarding.py (1)

85-94: Add return type annotation to the validator method.

Per coding guidelines, all function parameters and return values should have type hints. The validator is missing its return type.

     `@field_validator`("credentials")
     `@classmethod`
-    def _validate_credential_list(cls, v: list[dict[str, dict[str, str]]] | None):
+    def _validate_credential_list(cls, v: list[dict[str, dict[str, str]]] | None) -> list[dict[str, dict[str, str]]] | None:
         if v is None:
             return v
 
         if not isinstance(v, list):
             raise ValueError(
                 "Credential must be a list of single-key dicts (e.g., {'openai': {...}})."
             )

As per coding guidelines: "Always add type hints to all function parameters and return values in Python code".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@backend/app/models/onboarding.py` around lines 85 - 94, The field validator
method _validate_credential_list is missing a return type annotation; update its
signature (the `@field_validator-decorated` classmethod) to declare the return
type as the same as the input (e.g., -> list[dict[str, dict[str, str]]] | None)
so the function signature includes types for parameters and return value, and
ensure any required typing imports are present.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@backend/app/models/onboarding.py`:
- Around line 85-94: The field validator method _validate_credential_list is
missing a return type annotation; update its signature (the
`@field_validator-decorated` classmethod) to declare the return type as the same
as the input (e.g., -> list[dict[str, dict[str, str]]] | None) so the function
signature includes types for parameters and return value, and ensure any
required typing imports are present.

In `@backend/app/tests/api/routes/test_onboarding.py`:
- Around line 304-335: Rename the test function
test_onboard_project_aggregates_multiple_credential_errors to reflect the new
fail-fast validation (for example
test_onboard_project_fails_fast_on_credential_error) and update any docstring if
needed; keep the unused pytest fixture parameter db in the signature so test
isolation/setup remains intact (do not remove db even if static analysis flags
it) and ensure the test body (onboard_data, client.post call and the final
assertion checking "Unsupported provider") continues to reference the same
behavior.

In `@backend/app/tests/crud/test_credentials.py`:
- Around line 209-210: Move the duplicate local import of HTTPException to the
module-level imports: add "from app.core.exception_handlers import
HTTPException" alongside the other top-level imports, then remove the two
in-function imports inside test_invalid_provider and
test_langfuse_credential_validation so both tests use the single top-level
HTTPException symbol.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 029dfd91-b5fa-4d35-93f9-feb98c216dd0

📥 Commits

Reviewing files that changed from the base of the PR and between f749d69 and c4d4f0c.

📒 Files selected for processing (4)
  • backend/app/models/onboarding.py
  • backend/app/tests/api/routes/test_onboarding.py
  • backend/app/tests/core/test_providers.py
  • backend/app/tests/crud/test_credentials.py
💤 Files with no reviewable changes (1)
  • backend/app/tests/core/test_providers.py

@codecov
Copy link

codecov bot commented Mar 27, 2026

Codecov Report

❌ Patch coverage is 76.47059% with 12 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
backend/app/crud/credentials.py 59.09% 9 Missing ⚠️
backend/app/models/onboarding.py 75.00% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant