refactor!: Introduce fully typed clients

[vdusek]

Summary

This is a major refactoring that introduces fully typed Pydantic models throughout the client library. The models are generated from the OpenAPI specifications. All API responses now return typed objects instead of raw dictionaries.

This follows up on apify/apify-docs#2182.

Issues

• Closes: Write integration tests #21
• Closes: Introduce fully typed Py client #481

Packages

• Add direct dependency on Pydantic.
• Removes the dependency on apify-shared.
• Add dev dependency datamodel-code-generator for model generation.

Key changes

• Uses datamodel-code-generator tool configured via pyproject.toml to generate Pydantic models based on the OpenAPI specs.
• Refactors the whole codebase to adopt the new generated models.
• All resource clients now return typed Pydantic models (Actor, Task, Run, etc.).
• Adds response wrappers for validating and extracting API response data.
• Updates list methods to return typed pagination models.
• Documentation examples now use typed attribute access.
• Updates the SDK to use the new typed client.
  • See the corresponding PR in apify/apify-sdk-python for details - chore: Adapt to apify-client v3 [WIP] apify-sdk-python#719.
  • It will be merged later.

Architecture

• Get rid of 3/4/5 levels of inheritance.
• Get rid of inline imports because of circular dependencies.
  • I had to utilize ClientRegistry to be able to achieve that (because of resource clients-siblings imports).

Breaking changes

• Client methods now return Pydantic models instead of dicts.
  • Access patterns change from dict-style (result['key']) to attribute-style (result.key).

Test plan

• Updated test concurrency to 16 workers.
• A lot of new tests were implemented - coverage ~95%.
  • Unit tests - do not call production API, only for testing utils or other functionality using mocks.
  • Integration tests - call production API.
• Thanks to the new tests, I was able to do a lot of fixes in the OpenAPI specs.

Next steps

• Explore the generation of resource clients using openapi-python-client.
• Fully automate model updates based on changes in apify-api/openapi.
• This will be released as part of the Apify client v3.0.

[codecov]

Codecov Report

❌ Patch coverage is 94.01914% with 125 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.16%. Comparing base (aece206) to head (a745fa3).

Files with missing lines	Patch %	Lines
...apify_client/_resource_clients/_resource_client.py	83.33%	24 Missing ⚠️
src/apify_client/_resource_clients/run.py	91.60%	11 Missing ⚠️
src/apify_client/_resource_clients/user.py	80.39%	10 Missing ⚠️
src/apify_client/_resource_clients/dataset.py	91.39%	8 Missing ⚠️
.../apify_client/_resource_clients/key_value_store.py	93.54%	8 Missing ⚠️
src/apify_client/_resource_clients/actor.py	93.18%	6 Missing ⚠️
src/apify_client/_resource_clients/schedule.py	88.00%	6 Missing ⚠️
...apify_client/_resource_clients/webhook_dispatch.py	70.00%	6 Missing ⚠️
src/apify_client/_apify_client.py	95.78%	4 Missing ⚠️
...rc/apify_client/_resource_clients/actor_env_var.py	88.88%	4 Missing ⚠️
... and 15 more

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #604       +/-   ##
===========================================
+ Coverage   76.02%   95.16%   +19.13%     
===========================================
  Files          42       44        +2     
  Lines        2482     4588     +2106     
===========================================
+ Hits         1887     4366     +2479     
+ Misses        595      222      -373     

Flag	Coverage Δ
integration	93.63% <91.86%> (+24.86%)	⬆️
unit	70.61% <50.86%> (+5.95%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.