feat: accelerate inference

[begumcig]

Description

Related Issue

Fixes #(issue number)

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Additional Notes

---

Note

Revamps device management across engine/evaluation/metrics with accelerate-aware utilities, adds a new image sharpness metric, updates tutorials/docs, and expands tests for multi-device compatibility.

• Engine/Runtime:
  • Add robust device utilities: device_to_string, split_device, find_bytes_free_per_gpu, enhanced set_to_best_available_device, _resolve_cuda_device, and move_to_device with accelerate device map support and GPTQ consistency.
  • get_device now preserves indices and accelerate state; unify batch device moves via move_batch_to_device in InferenceHandler.
• Evaluation/Task:
  • Task now auto-selects devices, separates inference vs. stateful metric devices (stateful_metric_device, low_memory), and builds metrics accordingly.
  • EvaluationAgent ensures model/task device consistency, preserves/restores device map, and runs metrics on correct devices.
• Metrics:
  • Introduce runs_on/device validation and movement in StatefulMetric/BaseMetric and metric implementations.
  • Update time/energy/memory/architecture/CLIP metrics to use new device flow; add sync timing across multiple CUDA devices.
  • New SharpnessMetric for blind IQA and export via metrics/__init__.py.
• Docs/Tutorials:
  • Update metric customization guide with device support (runs_on, move_to_device).
  • Notebooks: switch to move_to_device(...) helper.
• Tests:
  • Add comprehensive device-compatibility and accelerate-distributed tests; adjust existing tests to new APIs.
• Tooling:
  • Pre-commit: tweak trufflehog invocation and refine local grep hook.

^{Written by Cursor Bugbot for commit 72c8c4a. This will update automatically on new commits. Configure here.}

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB