[feat] Add configurable timeout for Llama Stack requests

[amito]

Description

Introduces a timeout configuration parameter (default 180s) to accommodate long-running RAG queries and prevent premature request failures.

I did some work on POCing Lightspeed for a project in Red Hat OpenShift AI (the POC can be found in my repo).
Ran into some issues with slightly larger models where I timed out on RAG inference calls and had to increase Llama Stack timeouts. The change here implements this timeout extension. I thought it can be useful to others, hence the PR.

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up service version
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Konflux configuration change
• Unit tests improvement
• Integration tests improvement
• End to end tests improvement

Tools used to create PR

Identify any AI code assistants used in this PR (for transparency and review context)

• Assisted-by: Claude Code
• Generated by: N/A

Related Tickets & Documents

• Related Issue #
• Closes #

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Fixed unit tests and made sure they work
• I have built the lightspeed-core image and deployed it on my OpenShift cluster to test this change.

Summary by CodeRabbit

• New Features

  • Added configurable request timeout setting for Llama Stack service requests, with a default of 180 seconds.

• Tests

  • Updated test expectations to verify the new timeout configuration is properly serialized.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

A timeout configuration field is added to the Llama Stack configuration with a default of 180 seconds, logged during client initialization, and passed to the AsyncLlamaStackClient constructor. Related test expectations are updated to reflect the new configuration field.

Changes

Cohort / File(s)	Summary
Timeout Feature src/models/config.py, src/client.py, tests/unit/models/config/test_dump_configuration.py	Adds a timeout field (PositiveInt, default 180 seconds) to LlamaStackConfiguration, logs the timeout value in client initialization, passes it to AsyncLlamaStackClient, and updates test expectations to include the new field in serialized output.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and accurately summarizes the main change: adding a configurable timeout for Llama Stack requests. It is specific, concise, and directly reflects the primary purpose of the pull request.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[tisnik]

LGTM

[tisnik]

/ok-to-test

[tisnik]

/ok-to-test