Added batch_size support to sliced inference with deterministic outputs and full test coverage

[Haimzis]

Summary

This PR introduces batch_size support for get_sliced_prediction, enabling batched slice inference while keeping the API simple and backward compatible.

This work is based on earlier community efforts and addresses the reliability concerns reported there. In particular, this implementation is a stable fork of the ideas discussed in:

• Enable batch inference for image slices #1298
• GPU Batch Inference Implementation for SAHI #1227

Because this is a high-impact feature for production environment, we implemented and validated our own version end-to-end.

What Changed

Quoted from the commits in this branch:

• 38c45d4 — “add batch_size support for sliced inference with backward-compatible fallback and tests”
• c517427 — “Fix sliced progress callback and add robust batch-size equivalence test”

Concretely:

• Added batch_size: int = 1 to get_sliced_prediction(...)
• Added input validation (batch_size >= 1)
• Added grouped slice processing logic (N slices per batch)
• Added robust fallback to per-slice inference if batch inference fails
• Extended Ultralytics inference path to accept both single image and image lists
• Fixed sliced progress callback reporting to reflect processed slices correctly
• Kept default behavior unchanged when batch_size is not provided

Why This Matters

Previous attempts introduced important ideas but had runtime/test failures reported by users (shape mismatches, conversion issues, empty outputs, and NotImplementedError paths in some model tests).
This PR focuses on correctness first: same outputs, safer fallback behavior, and minimal API change for users.

Usage

Only one additional argument is needed:

from sahi.predict import get_sliced_prediction

result = get_sliced_prediction(
    image=image,
    detection_model=detection_model,
    slice_height=512,
    slice_width=512,
    overlap_height_ratio=0.1,
    overlap_width_ratio=0.2,
    batch_size=4,  # new
)

If batch_size is omitted, behavior remains equivalent to current single-slice processing (batch_size=1).

New Test Added

A new regression/equivalence test was added in tests/test_predict.py:

• test_get_sliced_prediction_batch_size_exact_output

What it verifies:

• Runs sliced inference with batch_size=1, batch_size=4, and batch_size=8
• Serializes predictions (category, bbox, score, segmentation) into a stable comparable form
• Asserts predictions are non-empty
• Asserts outputs are exactly identical across batch sizes

This guarantees batching changes performance characteristics without changing detection results.

Validation

All tests were run successfully for this branch, including the new batch-size equivalence coverage.

We also tested GPU utilization across multiple batch_size configurations in different runtime setups. In all tested setups, increasing batch_size improved GPU utilization and reduced per-slice inference overhead, while preserving output consistency (as covered by the batch-size equivalence test).

Backward Compatibility

• No breaking API changes
• Default path unchanged (batch_size=1)
• Safe fallback preserves functionality when model/backend batch behavior is unavailable or raises errors

[Haimzis]

Hi @onuralpszr, thanks for the update! I’d appreciate it if you could help assign a reviewer for this PR.