feat: supports image compressing

[a61995987]

feat:新增本地图片预压缩机制 避免用户向agent发送手机原图 体积过大造成的413错误
fix:增强 _ensure_img_caption 的容错性避免解析图片失败时造成的框架整体性崩溃

Modifications / 改动点

修改文件:astrbot\core\utils\media_utils.py

1. 新增compress_image与_compress_image_sync函数 用于对传入的本地图片进行压缩 原样返回远程图片url

修改文件:astrbot\core\astr_main_agent.py

1. 在_ensure_img_caption函数中引入新增的compress_image压缩函数
2. 在_process_quote_message函数中引入新增的compress_image压缩函数
3. 在build_main_agent函数中引入新增的compress_image压缩函数

fix: 增强 _ensure_img_caption 的容错性避免解析图片失败时造成的框架整体性崩溃:

1. 修改_ensure_img_caption函数 增加finally块并清空image_urls 避免处理错误时导致的框架整体性崩溃
2. 在该函数中原有的except中增加一个图片解析失败的文本组件 用于提醒agent

• This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

---

Checklist / 检查清单

• 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
  / 如果 PR 中有新加入的功能，已经通过 Issue / 邮件等方式和作者讨论过。

• 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
  / 我的更改经过了良好的测试，并已在上方提供了“验证步骤”和“运行截图”。

• 🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in requirements.txt and pyproject.toml.
  / 我确保没有引入新依赖库，或者引入了新依赖库的同时将其添加到 requirements.txt 和 pyproject.toml 文件相应位置。

• 😮 My changes do not introduce malicious code.
  / 我的更改没有引入恶意代码。

Summary by Sourcery

Introduce local image pre-compression and improve robustness of image caption handling to prevent failures from large or unparseable images.

New Features:

• Add asynchronous local image compression for user-uploaded images to reduce payload size before sending to providers.

Bug Fixes:

• Harden image caption generation to handle parsing failures gracefully without crashing the framework and inform the agent when image parsing fails.

Enhancements:

• Apply image compression across image captioning and quoted image handling flows to consistently limit image size before processing.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

此拉取请求引入了一项新功能，用于对本地图片进行预压缩处理，旨在优化图片上传和处理流程，有效避免因图片文件过大而引发的错误。同时，它还通过改进 _ensure_img_caption 函数的错误处理机制，显著增强了图片解析的容错能力，从而提升了系统的整体稳定性和用户体验。

Highlights

• 本地图片预压缩机制: 新增了本地图片预压缩机制，以避免用户上传过大图片（尤其是手机原图）导致服务器返回 413 错误，优化了图片处理流程。
• 增强图片解析容错性: 增强了 _ensure_img_caption 函数的容错性，通过增加 finally 块和错误提示，避免在图片解析失败时导致框架整体崩溃。

Changelog

• astrbot/core/astr_main_agent.py
  • 新增了 _compress_image_internal 异步函数，用于处理本地图片压缩，并将其应用于 _ensure_img_caption、_process_quote_message 和 build_main_agent 等多个图片处理入口。
  • 新增了 _do_compress_sync 函数，负责实际的同步图片压缩逻辑，并通过 asyncio.to_thread 在单独的线程中执行，避免阻塞主事件循环。
  • 增强了 _ensure_img_caption 函数的错误处理，增加了 finally 块以确保在任何情况下都清空 image_urls，并在图片解析失败时添加了用户可见的错误提示。

Activity

• 目前没有与此拉取请求相关的评论、审查或进度更新。

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[sourcery-ai]

Hey - I've found 3 issues, and left some high level feedback:

• In _compress_image_internal, ensure that temp_dir exists before writing (e.g., os.makedirs(temp_dir, exist_ok=True)) to avoid runtime errors when the data/temp directory has not been created yet.
• The repeated calls to _compress_image_internal for the same image path within a single request (e.g., quoting, captions, attachments) can cause duplicate work and extra temp files; consider compressing once per image and reusing the resulting path.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- In `_compress_image_internal`, ensure that `temp_dir` exists before writing (e.g., `os.makedirs(temp_dir, exist_ok=True)`) to avoid runtime errors when the `data/temp` directory has not been created yet.
- The repeated calls to `_compress_image_internal` for the same image path within a single request (e.g., quoting, captions, attachments) can cause duplicate work and extra temp files; consider compressing once per image and reusing the resulting path.

## Individual Comments

### Comment 1
<location path="astrbot/core/astr_main_agent.py" line_range="456" />
<code_context>
             image_caption_provider,
             cfg,
-            req.image_urls,
+            [await _compress_image_internal(url) for url in req.image_urls],
             plugin_context,
         )
</code_context>
<issue_to_address>
**suggestion (performance):** Consider parallelizing image compression calls to avoid sequential awaits on multiple images.

This list comprehension awaits `_compress_image_internal` sequentially for each URL, which can be a bottleneck with many images. Consider using `await asyncio.gather(*(_compress_image_internal(url) for url in req.image_urls))` so compressions run concurrently and reduce total latency.

Suggested implementation:

```python
import asyncio
from PIL import Image as PILImage

from astrbot.core import logger, sp
from astrbot.core.agent.handoff import HandoffTool
from astrbot.core.agent.mcp_client import MCPTool

```

```python
        caption = await _request_img_caption(
            image_caption_provider,
            cfg,
            await asyncio.gather(
                *(_compress_image_internal(url) for url in req.image_urls)
            ),
            plugin_context,
        )

```
</issue_to_address>

### Comment 2
<location path="astrbot/core/astr_main_agent.py" line_range="1195-1196" />
<code_context>
+    if max(img.size) > max_size:
+        img.thumbnail((max_size, max_size), PILImage.Resampling.LANCZOS)
+
+    timestamp = int(time.time() * 1000)
+    save_path = os.path.join(temp_dir, f"compressed_{timestamp}.jpg")
+    img.save(save_path, "JPEG", quality=85, optimize=True)
+    return save_path
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Timestamp-based filenames can collide under concurrency; consider a more robust unique name.

This scheme can overwrite files if two images are processed in the same millisecond. Generate a stronger unique suffix (e.g., `uuid.uuid4().hex`, optionally combined with the timestamp) to avoid collisions when multiple compress operations run concurrently.

Suggested implementation:

```python
import time
import uuid

```

```python
    unique_suffix = f"{int(time.time() * 1000)}_{uuid.uuid4().hex}"
    save_path = os.path.join(temp_dir, f"compressed_{unique_suffix}.jpg")

```
</issue_to_address>

### Comment 3
<location path="astrbot/core/astr_main_agent.py" line_range="1201" />
<code_context>
+    return save_path
+
+# 压缩用户上传的大体积图片 未来可以提取为通用工具
+async def _compress_image_internal(url_or_path: str) -> str:
+    try:
+        data = None
</code_context>
<issue_to_address>
**issue (complexity):** Consider refactoring the new image compression logic into smaller helpers and flatter call sites to keep behavior while simplifying control flow and dependencies.

You can keep the same behavior but reduce complexity by:

### 1. Split I/O vs CPU-bound logic and flatten branches

Right now `_compress_image_internal` does path classification, file I/O, size checks and dispatch to `_do_compress_sync`. You can keep functionality but make the logic more linear and testable by separating “load bytes” from “compress bytes”:

```python
# keep in this module or move to a utils module later
def _load_image_bytes(path_or_url: str) -> bytes | None:
    if path_or_url.startswith("http"):
        return None  # remote: no compression

    if path_or_url.startswith("data:image"):
        _, encoded = path_or_url.split(",", 1)
        return base64.b64decode(encoded)

    if os.path.exists(path_or_url):
        if os.path.getsize(path_or_path) < 1024 * 1024:
            return None  # small file: no compression
        with open(path_or_path, "rb") as f:
            return f.read()

    return None
```

```python
def _do_compress_sync(data: bytes, temp_dir: str) -> str:
    img = PILImage.open(io.BytesIO(data))
    if img.mode in ("RGBA", "P"):
        img = img.convert("RGB")

    max_size = 1280
    if max(img.size) > max_size:
        img.thumbnail((max_size, max_size), PILImage.Resampling.LANCZOS)

    timestamp = int(time.time() * 1000)
    save_path = os.path.join(temp_dir, f"compressed_{timestamp}.jpg")
    img.save(save_path, "JPEG", quality=85, optimize=True)
    return save_path
```

```python
async def _compress_image_internal(url_or_path: str) -> str:
    try:
        data = _load_image_bytes(url_or_path)
        if data is None:
            return url_or_path

        temp_dir = _get_temp_dir()
        return await asyncio.to_thread(_do_compress_sync, data, temp_dir)
    except Exception as e:
        logger.error("图片压缩失败: %s", e)
        return url_or_path
```

This keeps behavior (same conditions for “return original” vs compress) but removes nested branching and mixed responsibilities in one function.

### 2. Extract temp-dir computation

The inline `os.path.dirname(os.path.dirname(...))` chain makes the compression helper harder to read and change. A small helper removes that noise:

```python
def _get_temp_dir() -> str:
    root_dir = os.path.dirname(
        os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
    )
    return os.path.join(root_dir, "data", "temp")
```

Use `_get_temp_dir()` in `_compress_image_internal` (as above). Behavior is identical.

### 3. Flatten nested `await` call sites

The nested `await` in arguments is hard to scan. You can keep the same semantics while improving readability:

```python
# before
llm_resp = await prov.text_chat(
    prompt=IMAGE_CAPTION_DEFAULT_PROMPT,
    image_urls=[
        await _compress_image_internal(
            await image_seg.convert_to_file_path()
        )
    ],
)

# after
path = await image_seg.convert_to_file_path()
compressed_path = await _compress_image_internal(path)
llm_resp = await prov.text_chat(
    prompt=IMAGE_CAPTION_DEFAULT_PROMPT,
    image_urls=[compressed_path],
)
```

Likewise in the other places:

```python
# before
image_path = await _compress_image_internal(
    await comp.convert_to_file_path()
)

# after
path = await comp.convert_to_file_path()
image_path = await _compress_image_internal(path)
```

and in the quoted reply loop:

```python
path = await reply_comp.convert_to_file_path()
image_path = await _compress_image_internal(path)
```

### 4. Optionally move compression helpers to a dedicated utility

To decouple `astr_main_agent.py` from image-specific concerns without changing behavior, you could move the helpers into e.g. `astrbot.core.utils.image_compression` and keep a single high-level API in this file:

```python
# astr_main_agent.py
from astrbot.core.utils.image_compression import compress_image

# keep local name for clarity with minimal changes to call sites
_compress_image_internal = compress_image
```

This keeps all current call sites working, but moves the PIL/base64/temp-dir details out of the main agent module.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

此拉取请求引入了本地图片预压缩机制，并增强了图片解析的容错性。压缩逻辑在各种图片处理路径中得到了良好且一致的应用。在异步应用中使用 asyncio.to_thread 处理同步图片操作是正确的做法，可以有效避免阻塞事件循环。此外，_ensure_img_caption 函数中改进的错误处理也提高了系统的健壮性。

[gemini-code-assist]

为了支持 _do_compress_sync 函数中建议的 uuid.uuid4().hex 用法，需要导入 uuid 模块。

Suggested change

	import zoneinfo
	import zoneinfo
	import uuid

[gemini-code-assist]

为了支持 _compress_image_internal 函数中建议的 get_astrbot_temp_path() 用法，需要从 astrbot.core.utils.astrbot_path 导入此函数。

Suggested change

	from astrbot.core import logger, sp
	from astrbot.core import logger, sp
	from astrbot.core.utils.astrbot_path import get_astrbot_temp_path

[gemini-code-assist]

新创建的压缩图片文件被保存到临时目录 (data/temp)，但目前没有明确的清理机制。随着时间的推移，这个目录可能会积累大量文件，导致磁盘空间耗尽，尤其是在处理大量图片时。建议实现一个临时文件清理策略，例如定期清理旧文件、使用上下文管理器确保文件在使用后删除，或者设置临时目录的最大大小限制。

[LIghtJUNction]

temp_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(file)))), "data/temp")

不要写这种代码，可读性约等于0
要么用函数获取，要么全部都用pathlib的Path路径类来写。

那种写法看起来就清爽多了

[Copilot]

Pull request overview

This PR adds on-the-fly compression for large user-provided images before they’re passed into image-captioning and chat flows, aiming to reduce payload size and improve robustness when handling large image attachments.

Changes:

• Compresses local images (and data:image... inputs) above a size threshold using Pillow via asyncio.to_thread.
• Routes image attachments and quoted-image captioning through the new compression helper.
• Adjusts image-caption failure handling by appending a fallback text part and clearing image_urls.

---

You can also share your feedback on Copilot code review. Take the survey.

[a61995987]

temp_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(file)))), "data/temp")

不要写这种代码，可读性约等于0 要么用函数获取，要么全部都用pathlib的Path路径类来写。

那种写法看起来就清爽多了

感谢检查 已修改:
fix:修改temp_dir指向 现在使用框架内置的get_astrbot_temp_path来获取临时目录
fix:分离image_path的计算行为 提高可读性
fix:使用uuid来生成压缩后的图片 而非时间戳

[outlook84]

麻烦给个开关，以后忘记有这茬还想不起来图片为什么被压缩了。

[a61995987]

麻烦给个开关，以后忘记有这茬还想不起来图片为什么被压缩了。

等PR通过后我会新提交一个前端变更PR 不然这个PR可能会太大

[Soulter]

压缩相关的代码可以放在 astrbot/core/utils/media_utils.py。减少 mainagent 的代码长度。

[a61995987]

缩相关的代码可以放在 astrbot/core/utils/media_u

收到 修改后我会重新@您

[a61995987]

@Soulter 已修改 请审查 主要变更如下:
fix: 分离图片压缩相关函数到媒体文件处理工具中(astrbot\core\utils\media_utils.py 323-385行)
fix: 修改原有的列表推导式以提高可读性(astrbot\core\astr_main_agent.py 450-459行)

另外 在图片压缩相关的函数中预留了配置项的读取 请问需要增加独立的前端配置吗？

[LIghtJUNction]

一个小建议：尽量不要用中文作为分支名

[LIghtJUNction]

temp_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(file)))), "data/temp")
不要写这种代码，可读性约等于0 要么用函数获取，要么全部都用pathlib的Path路径类来写。
那种写法看起来就清爽多了

感谢检查 已修改: fix:修改temp_dir指向 现在使用框架内置的get_astrbot_temp_path来获取临时目录 fix:分离image_path的计算行为 提高可读性 fix:使用uuid来生成压缩后的图片 而非时间戳

我更推荐使用pathlib
我在get_astrbot_temp_path所在那个文件里面，放了一个专门用来获取路径的类。
可以用一下。

[a61995987]

temp_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(file)))), "data/temp")
不要写这种代码，可读性约等于0 要么用函数获取，要么全部都用pathlib的Path路径类来写。
那种写法看起来就清爽多了

感谢检查 已修改: fix:修改temp_dir指向 现在使用框架内置的get_astrbot_temp_path来获取临时目录 fix:分离image_path的计算行为 提高可读性 fix:使用uuid来生成压缩后的图片 而非时间戳

我更推荐使用pathlib 我在get_astrbot_temp_path所在那个文件里面，放了一个专门用来获取路径的类。 可以用一下。

收到 已修改 我没有在astrbot\core\utils\astrbot_path.py看到任何类 目前使用Path(get_astrbot_temp_path())方式来获取临时目录路径 并在_compress_image_sync中使用save_path = temp_dir / f"compressed_{new_uuid}.jpg"拼接生成的文件路径 这样可能更直观

另:我会在以后的PR中注意分支名称 这次似乎没法改了。。
@LIghtJUNction