[Python] Enable pyarrow AzureFileSystem for Windows

[davlee1972]

Describe the enhancement requested

There are several resolved issues around enabling and packaging pyarrow.fs.AzureFileSystem for linux, but with the latest pyarrow 18.0 release there still isn't any packaging for windows..

#44347

No AzureFileSystem in windows.

ImportError: The pyarrow installation is not built with support for 'AzureFileSystem'

C:\>python
Python 3.9.18 (main, Sep 11 2023, 14:09:26) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyarrow as pa
>>> import pyarrow.fs as fs
>>> pa.__version__
'18.0.0'
>>> dir(fs)
['AwsDefaultS3RetryStrategy', 'AwsStandardS3RetryStrategy', 'FSSpecHandler', 'FileInfo', 'FileSelector', 'FileStats', 'FileSystem', 'FileSystemHandler', 'FileType', 'GcsFileSystem', 'HadoopFileSystem', 'LocalFileSystem', 'PyFileSystem', 'S3FileSystem', 'S3LogLevel', 'S3RetryStrategy', 'SubTreeFileSystem', '_MockFileSystem', '__builtins__', '__cached__', '__doc__', '__file__', '__getattr__', '__loader__', '__name__', '__package__', '__spec__', '_copy_files', '_copy_files_selector', '_ensure_filesystem', '_filesystem_from_str', '_is_path_like', '_not_imported', '_resolve_filesystem_and_path', '_stringify_path', 'atexit', 'copy_files', 'ensure_s3_finalized', 'ensure_s3_initialized', 'finalize_s3', 'initialize_s3', 'resolve_s3_region']
>>> hasattr(fs, "AzureFileSystem")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\xxxxx\Anaconda3\lib\site-packages\pyarrow\fs.py", line 75, in __getattr__
    raise ImportError(
ImportError: The pyarrow installation is not built with support for 'AzureFileSystem'

This works fine in linux:

(base) bash-4.2$ python
Python 3.9.19 (main, May  6 2024, 19:43:03)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyarrow as pa
>>> import pyarrow.fs as fs
>>> pa.__version__
'18.0.0'
>>> dir(fs)
['AwsDefaultS3RetryStrategy', 'AwsStandardS3RetryStrategy', 'AzureFileSystem', 'FSSpecHandler', 'FileInfo', 'FileSelector', 'FileStats', 'FileSystem', 'FileSystemHandler', 'FileType', 'GcsFileSystem', 'HadoopFileSystem', 'LocalFileSystem', 'PyFileSystem', 'S3FileSystem', 'S3LogLevel', 'S3RetryStrategy', 'SubTreeFileSystem', '_MockFileSystem', '__builtins__', '__cached__', '__doc__', '__file__', '__getattr__', '__loader__', '__name__', '__package__', '__spec__', '_copy_files', '_copy_files_selector', '_ensure_filesystem', '_filesystem_from_str', '_is_path_like', '_not_imported', '_resolve_filesystem_and_path', '_stringify_path', 'atexit', 'copy_files', 'ensure_s3_finalized', 'ensure_s3_initialized', 'finalize_s3', 'initialize_s3', 'resolve_s3_region']
>>> hasattr(fs, "AzureFileSystem")
True

Component(s)

Packaging, Python

[raulcd]

This is the PR that enabled AzureFileSystem on macOS and Linux wheels, and there is a brief mention about the Windows failure. We should investigate and create a PR:

• GH-39352: [FS][Azure] Enable azure in builds #39971

[raulcd]

It does seem that building Azure FS for Windows fails also for conda:

• [C++] Error when building Arrow-CPP w/ Azure Filesystem for Conda-Forge #41990

Probably requires fixing before trying to enable on the wheels.

[github-actions]

This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.

[h-vetinari]

Not stale

[beschmitt]

Still relevant and not fixed.

[h-vetinari]

Should be fixed soon: #48971