Skip to content

fix: set packaged Windows runtime build env for pip native builds#6575

Merged
zouyonghe merged 10 commits intoAstrBotDevs:masterfrom
zouyonghe:fix/windows-packaged-pip-build-env
Mar 18, 2026
Merged

fix: set packaged Windows runtime build env for pip native builds#6575
zouyonghe merged 10 commits intoAstrBotDevs:masterfrom
zouyonghe:fix/windows-packaged-pip-build-env

Conversation

@zouyonghe
Copy link
Copy Markdown
Member

@zouyonghe zouyonghe commented Mar 18, 2026

Summary

  • normalize packaged Windows runtime paths before native-extension builds
  • inject packaged runtime include/lib directories into the in-process pip environment on Windows desktop builds
  • add a regression test covering packaged desktop runtime env injection

Verification

  • uv run pytest tests/test_pip_installer.py

Summary by Sourcery

Adjust in-process pip installation to respect packaged Windows desktop runtimes and ensure native builds see the correct include/lib paths while keeping the process environment consistent.

Bug Fixes:

  • Normalize Windows runtime and UNC paths before deriving native build include/lib directories for pip installations.
  • Inject packaged Windows runtime include and lib directories into the in-process pip environment on Windows desktop builds only when those directories exist, without affecting non-Windows or non-packaged runs.
  • Ensure temporary modification of INCLUDE and LIB during in-process pip runs is thread-safe and restores original environment values afterward.

Tests:

  • Add comprehensive tests covering Windows path normalization, temporary environment handling, runtime build env construction, and pip in-process env injection behavior across packaged/non-packaged and Windows/non-Windows scenarios.

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 18, 2026
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves an issue where native Python extension builds fail within a packaged Windows runtime environment due to improperly configured INCLUDE and LIB environment variables. By introducing robust path normalization, dynamic identification of runtime directories, and a safe mechanism for environment variable injection, the changes ensure that pip can successfully compile native components. This enhancement significantly improves the reliability of installing Python packages with native dependencies in desktop applications.

Highlights

  • Windows Path Normalization: Implemented a utility function _normalize_windows_native_build_path to correctly handle and normalize various Windows path formats, including UNC and device paths, ensuring compatibility for native builds.
  • Packaged Runtime Environment Injection: Introduced _build_packaged_windows_runtime_build_env to identify and prepare the include and libs directories from the packaged Python runtime, which are then injected into the INCLUDE and LIB environment variables for native extension compilation.
  • Temporary Environment Management: Added a _temporary_environ context manager to safely apply and revert environment variable modifications, ensuring that changes made for pip builds do not persist beyond the build process.
  • Regression Test for Windows Builds: Included a new asynchronous test case test_run_pip_in_process_injects_windows_runtime_build_env to validate that the INCLUDE and LIB environment variables are correctly set and restored when pip executes within a packaged Windows runtime.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@dosubot dosubot bot added the area:core The bug / feature is about astrbot's core, backend label Mar 18, 2026
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 3 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/test_pip_installer.py" line_range="229-230" />
<code_context>
     ]


+@pytest.mark.asyncio
+async def test_run_pip_in_process_injects_windows_runtime_build_env(monkeypatch):
+    runtime_executable = (
+        r"\\?\C:\Users\buding\AppData\Local\AstrBot\backend\python\python.exe"
</code_context>
<issue_to_address>
**suggestion (testing):** Add tests for cases where INCLUDE/LIB are not set in the environment before injection

The current test only covers the case where INCLUDE/LIB already exist. To fully cover `_prepend_windows_env_path`, please add a test where INCLUDE/LIB are not set initially, and assert that:

- Inside `_run_pip_in_process`, INCLUDE/LIB are set only to the packaged paths (no extra semicolons).
- After the call, both variables are removed from `os.environ` again (i.e., `_temporary_environ` deletes keys that were not originally present).

Suggested implementation:

```python
    observed_env = {}

    def fake_pip_main(args):
        del args
        observed_env["INCLUDE"] = pip_installer_module.os.environ.get("INCLUDE")

```

To cover the case where `INCLUDE`/`LIB` are not set before injection, please **add** the following new test (e.g. directly below `test_run_pip_in_process_injects_windows_runtime_build_env`):

```python
@pytest.mark.asyncio
async def test_run_pip_in_process_injects_windows_runtime_build_env_without_existing_paths(
    monkeypatch,
):
    # Simulate the packaged runtime python (same pattern as the other test)
    runtime_executable = (
        r"\\?\C:\Users\buding\AppData\Local\AstrBot\backend\python\python.exe"
    )
    include_dir = r"C:\Users\buding\AppData\Local\AstrBot\backend\python\include"
    libs_dir = r"C:\Users\buding\AppData\Local\AstrBot\backend\python\libs"

    # Make sure INCLUDE / LIB are *not* present before the call
    monkeypatch.delenv("INCLUDE", raising=False)
    monkeypatch.delenv("LIB", raising=False)

    observed_env: dict[str, str | None] = {}

    def fake_pip_main(args: list[str]) -> None:
        # Capture the environment as seen inside _run_pip_in_process
        del args
        observed_env["INCLUDE"] = pip_installer_module.os.environ.get("INCLUDE")
        observed_env["LIB"] = pip_installer_module.os.environ.get("LIB")

    # Make _run_pip_in_process call our fake instead of the real pip main
    monkeypatch.setattr(pip_installer_module, "_pip_main", fake_pip_main, raising=True)

    # Run with a Windows runtime executable so that _prepend_windows_env_path is used
    await pip_installer_module._run_pip_in_process(
        ["install", "dummy-package"],
        python_executable=runtime_executable,
    )

    # Inside the pip process, INCLUDE / LIB should be set only to the packaged paths
    assert observed_env["INCLUDE"] == include_dir
    assert observed_env["LIB"] == libs_dir
    # No extra semicolons when there were no original values
    assert ";" not in observed_env["INCLUDE"]
    assert ";" not in observed_env["LIB"]

    # After the call, the temporary environment should be cleaned up:
    # keys that did not exist originally must be removed again.
    assert "INCLUDE" not in pip_installer_module.os.environ
    assert "LIB" not in pip_installer_module.os.environ
```

You may need to adjust the following details to match the existing test suite:

1. **Name of the internal pip entry point**:
   * If the existing test monkeypatches a different symbol than `_pip_main` (for example `pip_installer_module.pip_main` or `pip_installer_module._run_pip`), update the `monkeypatch.setattr(...)` call to match.

2. **Signature of `_run_pip_in_process`**:
   * If the current test calls `_run_pip_in_process` positionally (e.g. `await _run_pip_in_process(["install"], runtime_executable)`) instead of using `python_executable=...`, mirror that calling convention.

3. **Environment access**:
   * If the other tests use `os.environ` directly instead of `pip_installer_module.os.environ`, you can replace `pip_installer_module.os.environ` with `os.environ` in both the capturing and the final assertions to stay consistent.

Once wired to the correct symbols, this test will verify that:
- `INCLUDE`/`LIB` are set to *only* the packaged paths inside `_run_pip_in_process` when they were previously absent.
- `_temporary_environ` removes these keys from the environment after the call.
</issue_to_address>

### Comment 2
<location path="tests/test_pip_installer.py" line_range="234-235" />
<code_context>
+    runtime_executable = (
+        r"\\?\C:\Users\buding\AppData\Local\AstrBot\backend\python\python.exe"
+    )
+    include_dir = r"C:\Users\buding\AppData\Local\AstrBot\backend\python\include"
+    libs_dir = r"C:\Users\buding\AppData\Local\AstrBot\backend\python\libs"
+    existing_include = r"C:\VS\include"
+    existing_lib = r"C:\VS\lib"
</code_context>
<issue_to_address>
**suggestion (testing):** Cover scenarios where only include or only libs directories exist

Right now `os.path.isdir` is mocked to return `True` for both include and libs, so we only cover the happy path where both are present. Please add tests (or parametrize this one) to also cover:

- Only include exists: INCLUDE updated, LIB unchanged.
- Only libs exists: LIB updated, INCLUDE unchanged.

This will verify `_build_packaged_windows_runtime_build_env` handles partially present runtimes correctly.

Suggested implementation:

```python
@pytest.mark.asyncio
@pytest.mark.parametrize(
    "include_exists, libs_exists",
    [
        (True, True),   # both include and libs exist
        (True, False),  # only include exists
        (False, True),  # only libs exist
    ],
)
async def test_run_pip_in_process_injects_windows_runtime_build_env(
    monkeypatch, include_exists, libs_exists
):
    runtime_executable = (
        r"\\?\C:\Users\buding\AppData\Local\AstrBot\backend\python\python.exe"
    )
    include_dir = r"C:\Users\buding\AppData\Local\AstrBot\backend\python\include"
    libs_dir = r"C:\Users\buding\AppData\Local\AstrBot\backend\python\libs"
    existing_include = r"C:\VS\include"
    existing_lib = r"C:\VS\lib"
    observed_env = {}

    def fake_pip_main(args):
        del args
        observed_env["INCLUDE"] = pip_installer_module.os.environ.get("INCLUDE")
        observed_env["LIB"] = pip_installer_module.os.environ.get("LIB")
        return 0

    # Only report the runtime include / libs directories as existing according to the
    # current parametrized scenario; delegate all other paths to the real isdir.
    real_isdir = pip_installer_module.os.path.isdir

    def fake_isdir(path: str) -> bool:
        if path == include_dir:
            return include_exists
        if path == libs_dir:
            return libs_exists
        return real_isdir(path)

    monkeypatch.setenv("ASTRBOT_DESKTOP_CLIENT", "1")
    monkeypatch.setenv("INCLUDE", existing_include)
    monkeypatch.setenv("LIB", existing_lib)

```

To fully implement the scenarios, adjust the remainder of this test (the part not shown in the snippet) as follows:

1. **Apply the monkeypatch for `os.path.isdir` and the fake pip main:**
   - After the `monkeypatch.setenv(...)` calls in this test, add:
   ```python
   monkeypatch.setattr(pip_installer_module.os.path, "isdir", fake_isdir)
   monkeypatch.setattr(pip_installer_module, "pip_main", fake_pip_main)
   ```
   (or whatever attribute name the test previously patched; preserve the existing naming convention and target.)

2. **Trigger the code under test as before:**
   - Keep the existing call that was invoking `_build_packaged_windows_runtime_build_env` indirectly (likely via `pip_installer_module.run_pip_in_process(...)`) so the behavior is tested exactly as before.

3. **Update the assertions to be scenario‑aware:**
   - After invoking the function under test, compute the expected values:
   ```python
   expected_include = existing_include
   expected_lib = existing_lib

   if include_exists:
       expected_include = existing_include + pip_installer_module.os.pathsep + include_dir
   if libs_exists:
       expected_lib = existing_lib + pip_installer_module.os.pathsep + libs_dir
   ```
   - Then assert:
   ```python
   assert observed_env["INCLUDE"] == expected_include
   assert observed_env["LIB"] == expected_lib
   ```
   This will validate:
   - when both exist: both INCLUDE and LIB are extended,
   - when only include exists: INCLUDE is extended, LIB unchanged,
   - when only libs exists: LIB is extended, INCLUDE unchanged.
</issue_to_address>

### Comment 3
<location path="tests/test_pip_installer.py" line_range="246-250" />
<code_context>
+        observed_env["LIB"] = pip_installer_module.os.environ.get("LIB")
+        return 0
+
+    monkeypatch.setenv("ASTRBOT_DESKTOP_CLIENT", "1")
+    monkeypatch.setenv("INCLUDE", existing_include)
+    monkeypatch.setenv("LIB", existing_lib)
+    monkeypatch.setattr(pip_installer_module.sys, "platform", "win32")
+    monkeypatch.setattr(pip_installer_module.sys, "executable", runtime_executable)
+    monkeypatch.setattr(
+        pip_installer_module.os.path,
</code_context>
<issue_to_address>
**suggestion (testing):** Add tests for non-packaged or non-Windows runtimes to ensure no env injection occurs

Specifically, consider tests where:
- `sys.platform != 'win32'` (e.g., `'linux'`) so `_run_pip_in_process` leaves `INCLUDE`/`LIB` untouched.
- `ASTRBOT_DESKTOP_CLIENT` is unset or set to a non-packaged value so no `INCLUDE`/`LIB` updates occur.

In those cases, assert that the env passed to `fake_pip_main` matches the original values (or remains absent).

Suggested implementation:

```python
    existing_include = r"C:\VS\include"
    existing_lib = r"C:\VS\lib"
    observed_env = {}

    def fake_pip_main(args):
        del args
        observed_env["INCLUDE"] = pip_installer_module.os.environ.get("INCLUDE")
        observed_env["LIB"] = pip_installer_module.os.environ.get("LIB")
        return 0

    monkeypatch.setenv("ASTRBOT_DESKTOP_CLIENT", "1")
    monkeypatch.setenv("INCLUDE", existing_include)
    monkeypatch.setenv("LIB", existing_lib)
    monkeypatch.setattr(pip_installer_module.sys, "platform", "win32")
    monkeypatch.setattr(pip_installer_module.sys, "executable", runtime_executable)
    monkeypatch.setattr(
        pip_installer_module.os.path,
        "isdir",
        lambda path: path in {include_dir, libs_dir},
    )
    monkeypatch.setattr(
        "astrbot.core.utils.pip_installer._get_pip_main",
        lambda: fake_pip_main,
    )

    installer = PipInstaller("")
    result = await installer._run_pip_in_process(["install", "demo-package"])

    # New tests: ensure no env injection on non-Windows and non-packaged runtimes.

    async def test_run_pip_in_process_does_not_modify_env_on_non_windows(
        monkeypatch, pip_installer_module, runtime_executable
    ):
        existing_include = "/usr/local/include"
        existing_lib = "/usr/local/lib"
        observed_env = {}

        def fake_pip_main(args):
            del args
            observed_env["INCLUDE"] = pip_installer_module.os.environ.get("INCLUDE")
            observed_env["LIB"] = pip_installer_module.os.environ.get("LIB")
            return 0

        # Even if the desktop flag is set, a non-Windows platform should prevent injection
        monkeypatch.setenv("ASTRBOT_DESKTOP_CLIENT", "1")
        monkeypatch.setenv("INCLUDE", existing_include)
        monkeypatch.setenv("LIB", existing_lib)
        monkeypatch.setattr(pip_installer_module.sys, "platform", "linux")
        monkeypatch.setattr(pip_installer_module.sys, "executable", runtime_executable)
        monkeypatch.setattr(
            "astrbot.core.utils.pip_installer._get_pip_main",
            lambda: fake_pip_main,
        )

        installer = PipInstaller("")
        await installer._run_pip_in_process(["install", "demo-package"])

        # INCLUDE/LIB should be untouched for non-Windows platforms
        assert observed_env["INCLUDE"] == existing_include
        assert observed_env["LIB"] == existing_lib

    async def test_run_pip_in_process_does_not_inject_env_when_not_packaged(
        monkeypatch, pip_installer_module, runtime_executable
    ):
        observed_env = {}

        def fake_pip_main(args):
            del args
            observed_env["INCLUDE"] = pip_installer_module.os.environ.get("INCLUDE")
            observed_env["LIB"] = pip_installer_module.os.environ.get("LIB")
            return 0

        # Simulate non-packaged runtime: no ASTRBOT_DESKTOP_CLIENT flag
        monkeypatch.delenv("ASTRBOT_DESKTOP_CLIENT", raising=False)
        monkeypatch.delenv("INCLUDE", raising=False)
        monkeypatch.delenv("LIB", raising=False)
        monkeypatch.setattr(pip_installer_module.sys, "platform", "win32")
        monkeypatch.setattr(pip_installer_module.sys, "executable", runtime_executable)
        monkeypatch.setattr(
            "astrbot.core.utils.pip_installer._get_pip_main",
            lambda: fake_pip_main,
        )

        installer = PipInstaller("")
        await installer._run_pip_in_process(["install", "demo-package"])

        # For non-packaged runtimes, INCLUDE/LIB should not be injected at all
        assert observed_env["INCLUDE"] is None
        assert observed_env["LIB"] is None

```

The new tests I added assume:
1. `pip_installer_module` and `runtime_executable` are existing fixtures already used by the surrounding tests.
2. `PipInstaller` is imported at module level in `tests/test_pip_installer.py`.

If fixture or import names differ in your codebase, adjust the parameter names of the new tests and the imports accordingly. Also ensure the new async test functions are at module scope (no extra indentation) alongside the existing `_run_pip_in_process` tests.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread tests/test_pip_installer.py Outdated
Comment thread tests/test_pip_installer.py Outdated
Comment thread tests/test_pip_installer.py Outdated
Comment on lines +246 to +250
monkeypatch.setenv("ASTRBOT_DESKTOP_CLIENT", "1")
monkeypatch.setenv("INCLUDE", existing_include)
monkeypatch.setenv("LIB", existing_lib)
monkeypatch.setattr(pip_installer_module.sys, "platform", "win32")
monkeypatch.setattr(pip_installer_module.sys, "executable", runtime_executable)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Add tests for non-packaged or non-Windows runtimes to ensure no env injection occurs

Specifically, consider tests where:

  • sys.platform != 'win32' (e.g., 'linux') so _run_pip_in_process leaves INCLUDE/LIB untouched.
  • ASTRBOT_DESKTOP_CLIENT is unset or set to a non-packaged value so no INCLUDE/LIB updates occur.

In those cases, assert that the env passed to fake_pip_main matches the original values (or remains absent).

Suggested implementation:

    existing_include = r"C:\VS\include"
    existing_lib = r"C:\VS\lib"
    observed_env = {}

    def fake_pip_main(args):
        del args
        observed_env["INCLUDE"] = pip_installer_module.os.environ.get("INCLUDE")
        observed_env["LIB"] = pip_installer_module.os.environ.get("LIB")
        return 0

    monkeypatch.setenv("ASTRBOT_DESKTOP_CLIENT", "1")
    monkeypatch.setenv("INCLUDE", existing_include)
    monkeypatch.setenv("LIB", existing_lib)
    monkeypatch.setattr(pip_installer_module.sys, "platform", "win32")
    monkeypatch.setattr(pip_installer_module.sys, "executable", runtime_executable)
    monkeypatch.setattr(
        pip_installer_module.os.path,
        "isdir",
        lambda path: path in {include_dir, libs_dir},
    )
    monkeypatch.setattr(
        "astrbot.core.utils.pip_installer._get_pip_main",
        lambda: fake_pip_main,
    )

    installer = PipInstaller("")
    result = await installer._run_pip_in_process(["install", "demo-package"])

    # New tests: ensure no env injection on non-Windows and non-packaged runtimes.

    async def test_run_pip_in_process_does_not_modify_env_on_non_windows(
        monkeypatch, pip_installer_module, runtime_executable
    ):
        existing_include = "/usr/local/include"
        existing_lib = "/usr/local/lib"
        observed_env = {}

        def fake_pip_main(args):
            del args
            observed_env["INCLUDE"] = pip_installer_module.os.environ.get("INCLUDE")
            observed_env["LIB"] = pip_installer_module.os.environ.get("LIB")
            return 0

        # Even if the desktop flag is set, a non-Windows platform should prevent injection
        monkeypatch.setenv("ASTRBOT_DESKTOP_CLIENT", "1")
        monkeypatch.setenv("INCLUDE", existing_include)
        monkeypatch.setenv("LIB", existing_lib)
        monkeypatch.setattr(pip_installer_module.sys, "platform", "linux")
        monkeypatch.setattr(pip_installer_module.sys, "executable", runtime_executable)
        monkeypatch.setattr(
            "astrbot.core.utils.pip_installer._get_pip_main",
            lambda: fake_pip_main,
        )

        installer = PipInstaller("")
        await installer._run_pip_in_process(["install", "demo-package"])

        # INCLUDE/LIB should be untouched for non-Windows platforms
        assert observed_env["INCLUDE"] == existing_include
        assert observed_env["LIB"] == existing_lib

    async def test_run_pip_in_process_does_not_inject_env_when_not_packaged(
        monkeypatch, pip_installer_module, runtime_executable
    ):
        observed_env = {}

        def fake_pip_main(args):
            del args
            observed_env["INCLUDE"] = pip_installer_module.os.environ.get("INCLUDE")
            observed_env["LIB"] = pip_installer_module.os.environ.get("LIB")
            return 0

        # Simulate non-packaged runtime: no ASTRBOT_DESKTOP_CLIENT flag
        monkeypatch.delenv("ASTRBOT_DESKTOP_CLIENT", raising=False)
        monkeypatch.delenv("INCLUDE", raising=False)
        monkeypatch.delenv("LIB", raising=False)
        monkeypatch.setattr(pip_installer_module.sys, "platform", "win32")
        monkeypatch.setattr(pip_installer_module.sys, "executable", runtime_executable)
        monkeypatch.setattr(
            "astrbot.core.utils.pip_installer._get_pip_main",
            lambda: fake_pip_main,
        )

        installer = PipInstaller("")
        await installer._run_pip_in_process(["install", "demo-package"])

        # For non-packaged runtimes, INCLUDE/LIB should not be injected at all
        assert observed_env["INCLUDE"] is None
        assert observed_env["LIB"] is None

The new tests I added assume:

  1. pip_installer_module and runtime_executable are existing fixtures already used by the surrounding tests.
  2. PipInstaller is imported at module level in tests/test_pip_installer.py.

If fixture or import names differ in your codebase, adjust the parameter names of the new tests and the imports accordingly. Also ensure the new async test functions are at module scope (no extra indentation) alongside the existing _run_pip_in_process tests.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a fix for setting up the build environment for pip native builds within a packaged Windows runtime. It adds helper functions to normalize Windows paths and to temporarily modify environment variables. These are then used in _run_pip_in_process to inject the INCLUDE and LIB directories from the packaged runtime into pip's environment.

The changes are accompanied by a solid regression test that verifies the environment variables are correctly set during the pip execution and restored afterward.

My main concern, detailed in a specific comment, is a potential race condition due to the modification of the global os.environ in an asynchronous context, especially if the critical section spans an await point. I've recommended using a lock to serialize access and prevent issues with concurrent pip installations.

Comment thread astrbot/core/utils/pip_installer.py Outdated
Comment on lines +1000 to +1003
with _temporary_environ(build_env):
result_code, output_lines = await asyncio.to_thread(
_run_pip_main_streaming, pip_main, args
)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Modifying the global os.environ within an async function across an await point can lead to race conditions if _run_pip_in_process is called concurrently. One coroutine could overwrite the environment variables set by another, leading to incorrect build environments for native extensions.

To make this operation safe, you should serialize the execution of this critical section. I recommend adding an asyncio.Lock to the PipInstaller class and acquiring it before modifying the environment.

For example:

# In PipInstaller.__init__:
self._in_process_lock = asyncio.Lock()

# In _run_pip_in_process:
async def _run_pip_in_process(self, args: list[str]) -> int:
    async with self._in_process_lock:
        # ... the rest of the method's logic

This will ensure that concurrent calls do not interfere with each other's environment.

@zouyonghe
Copy link
Copy Markdown
Member Author

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues, and left some high level feedback:

  • The hard-coded absolute Windows paths in the tests (e.g., C:\Users\buding\...) make the intent a bit less clear; consider using more generic placeholder paths or constructing them via ntpath.join to better express that only the structure (not the specific user/profile) matters.
  • The four new async tests around _run_pip_in_process share quite a bit of setup (fake pip_main, runtime_executable, monkeypatching sys and _get_pip_main); extracting a small helper or fixture for this common scaffolding would reduce duplication and make each test’s behavioral differences easier to see.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The hard-coded absolute Windows paths in the tests (e.g., `C:\Users\buding\...`) make the intent a bit less clear; consider using more generic placeholder paths or constructing them via `ntpath.join` to better express that only the structure (not the specific user/profile) matters.
- The four new async tests around `_run_pip_in_process` share quite a bit of setup (fake `pip_main`, `runtime_executable`, monkeypatching `sys` and `_get_pip_main`); extracting a small helper or fixture for this common scaffolding would reduce duplication and make each test’s behavioral differences easier to see.

## Individual Comments

### Comment 1
<location path="astrbot/core/utils/pip_installer.py" line_range="285-294" />
<code_context>
+
+
+@contextlib.contextmanager
+def _temporary_environ(updates: dict[str, str]):
+    if not updates:
+        yield
+        return
+
+    missing = object()
+    previous_values = {key: os.environ.get(key, missing) for key in updates}
+
+    try:
+        os.environ.update(updates)
+        yield
+    finally:
+        for key, previous_value in previous_values.items():
+            if previous_value is missing:
+                os.environ.pop(key, None)
</code_context>
<issue_to_address>
**issue (bug_risk):** Process-wide os.environ mutation may impact other concurrent tasks

Because this context manager mutates `os.environ` while `_run_pip_main_streaming` runs in a background thread, other threads/async tasks may briefly see these INCLUDE/LIB changes. To avoid cross-task interference, consider running pip in a subprocess with an explicit `env`, or otherwise confine the environment changes strictly to the pip invocation.
</issue_to_address>

### Comment 2
<location path="tests/test_pip_installer.py" line_range="238-245" />
<code_context>
+        (False, True),
+    ],
+)
+async def test_run_pip_in_process_injects_windows_runtime_build_env(
+    monkeypatch, include_exists, libs_exists
+):
+    runtime_executable = (
+        r"\\?\C:\Users\buding\AppData\Local\AstrBot\backend\python\python.exe"
+    )
+    include_dir = r"C:\Users\buding\AppData\Local\AstrBot\backend\python\include"
+    libs_dir = r"C:\Users\buding\AppData\Local\AstrBot\backend\python\libs"
+    existing_include = r"C:\VS\include"
+    existing_lib = r"C:\VS\lib"
</code_context>
<issue_to_address>
**suggestion (testing):** Consider adding tests for UNC and forward-slash paths to cover `_normalize_windows_native_build_path` behavior

These tests only cover the `\?\C:\...` executable style and the derived include/libs dirs. `_normalize_windows_native_build_path` also has special handling for `\?\UNC\...` / `\??\UNC\...` and forward slashes, which aren’t exercised. Please either parametrize with UNC and `/`-separator paths and assert INCLUDE/LIB are still normalized correctly, or add unit tests that call `_normalize_windows_native_build_path` directly with those cases.

Suggested implementation:

```python
@pytest.mark.parametrize(
    ("path", "expected"),
    [
        (
            r"\\?\C:\Users\buding\AppData\Local\AstrBot\backend\python\include",
            r"C:\Users\buding\AppData\Local\AstrBot\backend\python\include",
        ),
        (
            r"\\?\UNC\server\share\include",
            r"\\server\share\include",
        ),
        (
            r"\\??\UNC\server\share\libs",
            r"\\server\share\libs",
        ),
        (
            r"C:/Users/buding/AppData/Local/AstrBot/backend/python/libs",
            r"C:\Users\buding\AppData\Local\AstrBot\backend\python\libs",
        ),
    ],
)
def test_normalize_windows_native_build_path_variants(path, expected):
    assert (
        pip_installer_module._normalize_windows_native_build_path(path) == expected
    )


@pytest.mark.asyncio
@pytest.mark.parametrize(
    ("include_exists", "libs_exists"),

```

If `_normalize_windows_native_build_path`’s behavior differs from these expectations (e.g. it preserves certain prefixes or normalizes UNC paths differently), adjust the `expected` values to match its actual, intended normalization rules. Also ensure `pip_installer_module` is imported in this test file (it appears to be already, as it’s used in `test_run_pip_in_process_injects_windows_runtime_build_env`).
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread astrbot/core/utils/pip_installer.py Outdated
Comment thread tests/test_pip_installer.py Outdated
@zouyonghe
Copy link
Copy Markdown
Member Author

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/test_pip_installer.py" line_range="87-89" />
<code_context>
+    return observed_env
+
+
 @pytest.mark.asyncio
 async def test_install_targets_site_packages_for_desktop_client(monkeypatch, tmp_path):
     monkeypatch.setenv("ASTRBOT_DESKTOP_CLIENT", "1")
</code_context>
<issue_to_address>
**suggestion (testing):** Add coverage for the case where neither include nor libs directories exist in the packaged runtime.

The parametrized `test_run_pip_in_process_injects_windows_runtime_build_env` only exercises cases where at least one of `WINDOWS_RUNTIME_INCLUDE_DIR` or `WINDOWS_RUNTIME_LIBS_DIR` exists. We should also assert behavior when `_build_packaged_windows_runtime_build_env()` returns `{}` so we know the build env is not mutated and existing `INCLUDE`/`LIB` values are preserved.

For example:

```python
@pytest.mark.asyncio
async def test_run_pip_in_process_does_not_inject_when_runtime_dirs_missing(monkeypatch):
    observed_env = _configure_run_pip_in_process_capture(
        monkeypatch,
        platform="win32",
        packaged_runtime=True,
        include_value=EXISTING_WINDOWS_INCLUDE_DIR,
        lib_value=EXISTING_WINDOWS_LIB_DIR,
        existing_runtime_dirs=set(),
    )

    installer = PipInstaller("")
    result = await installer._run_pip_in_process(["install", "demo-package"])

    assert result == 0
    assert observed_env == {
        "INCLUDE": EXISTING_WINDOWS_INCLUDE_DIR,
        "LIB": EXISTING_WINDOWS_LIB_DIR,
    }
    assert pip_installer_module.os.environ["INCLUDE"] == EXISTING_WINDOWS_INCLUDE_DIR
    assert pip_installer_module.os.environ["LIB"] == EXISTING_WINDOWS_LIB_DIR
```

This ensures the temporary env injection is a no-op when the runtime provides no build directories.

Suggested implementation:

```python
    monkeypatch.setattr(
        "astrbot.core.utils.pip_installer._get_pip_main",
        lambda: fake_pip_main,
    )
    return observed_env


@pytest.mark.asyncio
async def test_run_pip_in_process_does_not_inject_when_runtime_dirs_missing(monkeypatch):
    observed_env = _configure_run_pip_in_process_capture(
        monkeypatch,
        platform="win32",
        packaged_runtime=True,
        include_value=EXISTING_WINDOWS_INCLUDE_DIR,
        lib_value=EXISTING_WINDOWS_LIB_DIR,
        existing_runtime_dirs=set(),
    )

    installer = PipInstaller("")
    result = await installer._run_pip_in_process(["install", "demo-package"])

    assert result == 0
    assert observed_env == {
        "INCLUDE": EXISTING_WINDOWS_INCLUDE_DIR,
        "LIB": EXISTING_WINDOWS_LIB_DIR,
    }
    assert pip_installer_module.os.environ["INCLUDE"] == EXISTING_WINDOWS_INCLUDE_DIR
    assert pip_installer_module.os.environ["LIB"] == EXISTING_WINDOWS_LIB_DIR


    else:

```

1. Ensure `EXISTING_WINDOWS_INCLUDE_DIR` and `EXISTING_WINDOWS_LIB_DIR` are defined constants in this test module (they are likely already used by `test_run_pip_in_process_injects_windows_runtime_build_env`; if not, define them near the top of the file).
2. Ensure `pip_installer_module` and `PipInstaller` are already imported/available in this file as used in the other tests.
3. If the `else:` shown in the snippet is actually part of another function or fixture, please confirm that this new test is placed outside of that function’s body (i.e., at top-level indentation) to remain consistent with the rest of the test layout.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread tests/test_pip_installer.py
@zouyonghe
Copy link
Copy Markdown
Member Author

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • There is a subtle race in how build_env is constructed: _build_packaged_windows_runtime_build_env reads os.environ outside _PIP_IN_PROCESS_ENV_LOCK, so another thread could modify INCLUDE/LIB between computing build_env and calling _run_pip_main_with_temporary_environ; consider either building build_env inside the locked section or basing the prepend logic on the environment snapshot within _temporary_environ to keep the mutation atomic.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- There is a subtle race in how `build_env` is constructed: `_build_packaged_windows_runtime_build_env` reads `os.environ` outside `_PIP_IN_PROCESS_ENV_LOCK`, so another thread could modify `INCLUDE`/`LIB` between computing `build_env` and calling `_run_pip_main_with_temporary_environ`; consider either building `build_env` inside the locked section or basing the prepend logic on the environment snapshot within `_temporary_environ` to keep the mutation atomic.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@zouyonghe
Copy link
Copy Markdown
Member Author

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/test_pip_installer.py" line_range="289-298" />
<code_context>
     ]


+@pytest.mark.parametrize(
+    ("path", "expected"),
+    [
+        (
+            WINDOWS_PACKAGED_RUNTIME_EXECUTABLE,
+            WINDOWS_RUNTIME_EXECUTABLE,
+        ),
+        (
+            r"\\?\UNC\server\share\include",
+            r"\\server\share\include",
+        ),
+        (
+            r"\??\UNC\server\share\libs",
+            r"\\server\share\libs",
+        ),
+        (
+            "C:/astrbot-test/backend/python/libs",
+            WINDOWS_RUNTIME_LIBS_DIR,
+        ),
+    ],
+)
+def test_normalize_windows_native_build_path_variants(path, expected):
+    assert pip_installer_module._normalize_windows_native_build_path(path) == expected
+
</code_context>
<issue_to_address>
**suggestion (testing):** Consider adding normalization tests for already-normalized and `\\?\C:\...` style non-UNC paths.

These tests already cover UNC and forward‑slash paths. To better exercise `_normalize_windows_native_build_path`, please also add:

- An input that is already normalized (e.g. `WINDOWS_RUNTIME_EXECUTABLE`) to verify it’s returned unchanged.
- A non‑UNC extended path like `\\?\C:\...` or `\??\C:\...` to cover the branch that strips extended‑length prefixes without going through the UNC logic.

This will ensure both prefix‑handling branches and the no-op path are properly tested and guard against regressions in Windows extended path handling.
</issue_to_address>

### Comment 2
<location path="tests/test_pip_installer.py" line_range="358-383" />
<code_context>
+
+
+@pytest.mark.asyncio
+async def test_run_pip_in_process_injects_windows_runtime_build_env_without_existing_paths(
+    monkeypatch,
+):
+    observed_env = _configure_run_pip_in_process_capture(
+        monkeypatch,
+        platform="win32",
+        packaged_runtime=True,
+        existing_runtime_dirs={
+            WINDOWS_RUNTIME_INCLUDE_DIR,
+            WINDOWS_RUNTIME_LIBS_DIR,
+        },
+    )
+
+    installer = PipInstaller("")
+    result = await installer._run_pip_in_process(["install", "demo-package"])
+
+    assert result == 0
+    assert observed_env == {
+        "INCLUDE": WINDOWS_RUNTIME_INCLUDE_DIR,
+        "LIB": WINDOWS_RUNTIME_LIBS_DIR,
+    }
+    assert ";" not in observed_env["INCLUDE"]
+    assert ";" not in observed_env["LIB"]
+    assert "INCLUDE" not in pip_installer_module.os.environ
+    assert "LIB" not in pip_installer_module.os.environ
+
+
</code_context>
<issue_to_address>
**suggestion (testing):** Consider a test where only one of the runtime include/lib directories exists and no existing INCLUDE/LIB env vars are set.

The existing tests cover: (1) both runtime dirs present with no pre-existing INCLUDE/LIB, (2) combinations of `include_exists`/`libs_exists` when INCLUDE/LIB are already set, and (3) the case where both runtime dirs are missing. What’s missing is when only one of `include`/`libs` exists and both env vars are unset. Extending this test parametrically (or adding a small new one) to assert that only the existing dir is injected and that no trailing `;` is added would close this gap and better guard against subtle string-formatting issues in `_build_packaged_windows_runtime_build_env`.

```suggestion
@pytest.mark.asyncio
@pytest.mark.parametrize(
    "include_exists, libs_exists",
    [
        (True, True),   # both runtime dirs present
        (True, False),  # only include dir present
        (False, True),  # only libs dir present
    ],
)
async def test_run_pip_in_process_injects_windows_runtime_build_env_without_existing_paths(
    monkeypatch,
    include_exists,
    libs_exists,
):
    existing_runtime_dirs = set()
    if include_exists:
        existing_runtime_dirs.add(WINDOWS_RUNTIME_INCLUDE_DIR)
    if libs_exists:
        existing_runtime_dirs.add(WINDOWS_RUNTIME_LIBS_DIR)

    observed_env = _configure_run_pip_in_process_capture(
        monkeypatch,
        platform="win32",
        packaged_runtime=True,
        existing_runtime_dirs=existing_runtime_dirs,
    )

    installer = PipInstaller("")
    result = await installer._run_pip_in_process(["install", "demo-package"])

    assert result == 0

    expected_env = {}
    if include_exists:
        expected_env["INCLUDE"] = WINDOWS_RUNTIME_INCLUDE_DIR
    if libs_exists:
        expected_env["LIB"] = WINDOWS_RUNTIME_LIBS_DIR

    assert observed_env == expected_env

    if include_exists:
        assert ";" not in observed_env["INCLUDE"]
    if libs_exists:
        assert ";" not in observed_env["LIB"]

    # Ensure process env is not polluted
    assert "INCLUDE" not in pip_installer_module.os.environ
    assert "LIB" not in pip_installer_module.os.environ
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +289 to +298
@pytest.mark.parametrize(
("path", "expected"),
[
(
WINDOWS_PACKAGED_RUNTIME_EXECUTABLE,
WINDOWS_RUNTIME_EXECUTABLE,
),
(
r"\\?\UNC\server\share\include",
r"\\server\share\include",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Consider adding normalization tests for already-normalized and \\?\C:\... style non-UNC paths.

These tests already cover UNC and forward‑slash paths. To better exercise _normalize_windows_native_build_path, please also add:

  • An input that is already normalized (e.g. WINDOWS_RUNTIME_EXECUTABLE) to verify it’s returned unchanged.
  • A non‑UNC extended path like \\?\C:\... or \??\C:\... to cover the branch that strips extended‑length prefixes without going through the UNC logic.

This will ensure both prefix‑handling branches and the no-op path are properly tested and guard against regressions in Windows extended path handling.

Comment on lines +358 to +383
@pytest.mark.asyncio
async def test_run_pip_in_process_injects_windows_runtime_build_env_without_existing_paths(
monkeypatch,
):
observed_env = _configure_run_pip_in_process_capture(
monkeypatch,
platform="win32",
packaged_runtime=True,
existing_runtime_dirs={
WINDOWS_RUNTIME_INCLUDE_DIR,
WINDOWS_RUNTIME_LIBS_DIR,
},
)

installer = PipInstaller("")
result = await installer._run_pip_in_process(["install", "demo-package"])

assert result == 0
assert observed_env == {
"INCLUDE": WINDOWS_RUNTIME_INCLUDE_DIR,
"LIB": WINDOWS_RUNTIME_LIBS_DIR,
}
assert ";" not in observed_env["INCLUDE"]
assert ";" not in observed_env["LIB"]
assert "INCLUDE" not in pip_installer_module.os.environ
assert "LIB" not in pip_installer_module.os.environ
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Consider a test where only one of the runtime include/lib directories exists and no existing INCLUDE/LIB env vars are set.

The existing tests cover: (1) both runtime dirs present with no pre-existing INCLUDE/LIB, (2) combinations of include_exists/libs_exists when INCLUDE/LIB are already set, and (3) the case where both runtime dirs are missing. What’s missing is when only one of include/libs exists and both env vars are unset. Extending this test parametrically (or adding a small new one) to assert that only the existing dir is injected and that no trailing ; is added would close this gap and better guard against subtle string-formatting issues in _build_packaged_windows_runtime_build_env.

Suggested change
@pytest.mark.asyncio
async def test_run_pip_in_process_injects_windows_runtime_build_env_without_existing_paths(
monkeypatch,
):
observed_env = _configure_run_pip_in_process_capture(
monkeypatch,
platform="win32",
packaged_runtime=True,
existing_runtime_dirs={
WINDOWS_RUNTIME_INCLUDE_DIR,
WINDOWS_RUNTIME_LIBS_DIR,
},
)
installer = PipInstaller("")
result = await installer._run_pip_in_process(["install", "demo-package"])
assert result == 0
assert observed_env == {
"INCLUDE": WINDOWS_RUNTIME_INCLUDE_DIR,
"LIB": WINDOWS_RUNTIME_LIBS_DIR,
}
assert ";" not in observed_env["INCLUDE"]
assert ";" not in observed_env["LIB"]
assert "INCLUDE" not in pip_installer_module.os.environ
assert "LIB" not in pip_installer_module.os.environ
@pytest.mark.asyncio
@pytest.mark.parametrize(
"include_exists, libs_exists",
[
(True, True), # both runtime dirs present
(True, False), # only include dir present
(False, True), # only libs dir present
],
)
async def test_run_pip_in_process_injects_windows_runtime_build_env_without_existing_paths(
monkeypatch,
include_exists,
libs_exists,
):
existing_runtime_dirs = set()
if include_exists:
existing_runtime_dirs.add(WINDOWS_RUNTIME_INCLUDE_DIR)
if libs_exists:
existing_runtime_dirs.add(WINDOWS_RUNTIME_LIBS_DIR)
observed_env = _configure_run_pip_in_process_capture(
monkeypatch,
platform="win32",
packaged_runtime=True,
existing_runtime_dirs=existing_runtime_dirs,
)
installer = PipInstaller("")
result = await installer._run_pip_in_process(["install", "demo-package"])
assert result == 0
expected_env = {}
if include_exists:
expected_env["INCLUDE"] = WINDOWS_RUNTIME_INCLUDE_DIR
if libs_exists:
expected_env["LIB"] = WINDOWS_RUNTIME_LIBS_DIR
assert observed_env == expected_env
if include_exists:
assert ";" not in observed_env["INCLUDE"]
if libs_exists:
assert ";" not in observed_env["LIB"]
# Ensure process env is not polluted
assert "INCLUDE" not in pip_installer_module.os.environ
assert "LIB" not in pip_installer_module.os.environ

@zouyonghe
Copy link
Copy Markdown
Member Author

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The _build_packaged_windows_runtime_build_env helper currently reads from os.environ via _prepend_windows_env_path, which makes it tightly coupled to process state; consider passing in a base env mapping (e.g. Mapping[str, str]) so it can build upon a snapshot and be more reusable/testable outside of process-global mutations.
  • The Windows path normalization logic in _normalize_windows_native_build_path handles several prefix variants inline; extracting the prefix sets (e.g. extended-path vs UNC) into named constants or a small lookup structure would make the intent clearer and reduce the chance of inconsistencies if more cases are added later.
  • The _configure_run_pip_in_process_capture test helper is fairly involved and reused across multiple tests; you might consider turning parts of it into pytest fixtures (e.g. a fixture that sets up a packaged Windows runtime env) to make individual test cases more focused on behavior rather than setup details.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `_build_packaged_windows_runtime_build_env` helper currently reads from `os.environ` via `_prepend_windows_env_path`, which makes it tightly coupled to process state; consider passing in a base env mapping (e.g. `Mapping[str, str]`) so it can build upon a snapshot and be more reusable/testable outside of process-global mutations.
- The Windows path normalization logic in `_normalize_windows_native_build_path` handles several prefix variants inline; extracting the prefix sets (e.g. extended-path vs UNC) into named constants or a small lookup structure would make the intent clearer and reduce the chance of inconsistencies if more cases are added later.
- The `_configure_run_pip_in_process_capture` test helper is fairly involved and reused across multiple tests; you might consider turning parts of it into pytest fixtures (e.g. a fixture that sets up a packaged Windows runtime env) to make individual test cases more focused on behavior rather than setup details.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@zouyonghe
Copy link
Copy Markdown
Member Author

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/utils/pip_installer.py" line_range="254-255" />
<code_context>
+    # the worker.
+    with _PIP_IN_PROCESS_ENV_LOCK:
+        if env_updates is None:
+            env_updates = _build_packaged_windows_runtime_build_env(
+                base_env=dict(os.environ)
+            )
+        with _temporary_environ(env_updates):
</code_context>
<issue_to_address>
**issue (bug_risk):** Using dict(os.environ) loses Windows case-insensitive behavior for env vars and may skip existing INCLUDE/LIB entries.

On Windows, `os.environ` is case-insensitive, but `dict(os.environ)` is not. If the environment uses a different casing (e.g., `include` instead of `INCLUDE`), `_prepend_windows_env_path` will miss the existing value and overwrite rather than prepend. Please pass `os.environ` directly (or use a copy that preserves its mapping type) and only copy if you need to mutate it independently.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread astrbot/core/utils/pip_installer.py Outdated
@zouyonghe
Copy link
Copy Markdown
Member Author

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/test_pip_installer.py" line_range="316-323" />
<code_context>
+        ),
+    ],
+)
+def test_normalize_windows_native_build_path_variants(path, expected):
+    assert pip_installer_module._normalize_windows_native_build_path(path) == expected
+
</code_context>
<issue_to_address>
**suggestion (testing):** Consider adding a case for already-normal UNC paths in `_normalize_windows_native_build_path`

Since this test already covers extended prefixes and forward-slash normalization, consider adding a case with an already-normal UNC path (e.g. `r"\\server\share\include"`) to verify `_normalize_windows_native_build_path` is a no-op in that scenario and to guard against future double-normalization or incorrect prefixing.

```suggestion
        (
            r"\??\UNC\server\share\libs",
            r"\\server\share\libs",
        ),
        # Already-normal UNC path should be a no-op
        (
            r"\\server\share\include",
            r"\\server\share\include",
        ),
        (
            "C:/astrbot-test/backend/python/libs",
            WINDOWS_RUNTIME_LIBS_DIR,
        ),
```
</issue_to_address>

### Comment 2
<location path="astrbot/core/utils/pip_installer.py" line_range="243" />
<code_context>
     return result_code, stream.lines


+def _run_pip_main_with_temporary_environ(
+    pip_main,
+    args: list[str],
</code_context>
<issue_to_address>
**issue (complexity):** Consider inlining the new environment-setup helpers into the main functions to keep all pip in-process env mutation and Windows build-path logic localized and easier to follow.

The added helpers around the pip env setup can be collapsed without changing behavior, which should make the flow easier to follow.

You can keep all current semantics but:

1. Inline the case-insensitive env lookup + prepend logic into `_build_packaged_windows_runtime_build_env`, and remove `_get_windows_env_value` / `_prepend_windows_env_path`.

```python
def _build_packaged_windows_runtime_build_env(
    *, base_env: Mapping[str, str] | None = None,
) -> dict[str, str]:
    if sys.platform != "win32" or not is_packaged_desktop_runtime():
        return {}

    if base_env is None:
        base_env = os.environ

    runtime_executable = _normalize_windows_native_build_path(sys.executable)
    runtime_dir = ntpath.dirname(runtime_executable)
    if not runtime_dir:
        return {}

    include_dir = _normalize_windows_native_build_path(
        ntpath.join(runtime_dir, "include")
    )
    libs_dir = _normalize_windows_native_build_path(
        ntpath.join(runtime_dir, "libs")
    )

    if not (os.path.isdir(include_dir) or os.path.isdir(libs_dir)):
        return {}

    # Case-insensitive view of the existing env
    upper_to_key: dict[str, str] = {k.upper(): k for k in base_env.keys()}

    def _prepend(name: str, path: str) -> str:
        existing_key = upper_to_key.get(name)
        existing = base_env.get(existing_key) if existing_key is not None else None
        return f"{path};{existing}" if existing else path

    env_updates: dict[str, str] = {}
    if os.path.isdir(include_dir):
        env_updates["INCLUDE"] = _prepend("INCLUDE", include_dir)
    if os.path.isdir(libs_dir):
        env_updates["LIB"] = _prepend("LIB", libs_dir)
    return env_updates
```

This removes two global helpers and concentrates the Windows build-path logic in one place.

2. Inline `_temporary_environ` into `_run_pip_main_with_temporary_environ` so that the lock, env mutation, and restore logic live in a single function. This keeps the behavior but reduces indirection and makes the mutation window explicit:

```python
def _run_pip_main_with_temporary_environ(
    pip_main,
    args: list[str],
    env_updates: dict[str, str] | None = None,
) -> tuple[int, list[str]]:
    with _PIP_IN_PROCESS_ENV_LOCK:
        if env_updates is None:
            env_updates = _build_packaged_windows_runtime_build_env(base_env=os.environ)

        if not env_updates:
            return _run_pip_main_streaming(pip_main, args)

        missing = object()
        previous_values = {k: os.environ.get(k, missing) for k in env_updates}

        try:
            os.environ.update(env_updates)
            return _run_pip_main_streaming(pip_main, args)
        finally:
            for key, previous in previous_values.items():
                if previous is missing:
                    os.environ.pop(key, None)
                else:
                    os.environ[key] = previous
```

With this, `_temporary_environ` can be removed entirely, and the only remaining “pip in-process + env” entry point is `_run_pip_main_with_temporary_environ`.

The `_normalize_windows_native_build_path` function is already self-contained and reasonably clear. Unless you have concrete scenarios requiring the separate UNC vs extended-prefix handling, you might later consider simplifying the prefix handling, but the main complexity win comes from reducing the number of small helpers around the env setup as above.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +316 to +323
(
r"\??\UNC\server\share\libs",
r"\\server\share\libs",
),
(
"C:/astrbot-test/backend/python/libs",
WINDOWS_RUNTIME_LIBS_DIR,
),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Consider adding a case for already-normal UNC paths in _normalize_windows_native_build_path

Since this test already covers extended prefixes and forward-slash normalization, consider adding a case with an already-normal UNC path (e.g. r"\\server\share\include") to verify _normalize_windows_native_build_path is a no-op in that scenario and to guard against future double-normalization or incorrect prefixing.

Suggested change
(
r"\??\UNC\server\share\libs",
r"\\server\share\libs",
),
(
"C:/astrbot-test/backend/python/libs",
WINDOWS_RUNTIME_LIBS_DIR,
),
(
r"\??\UNC\server\share\libs",
r"\\server\share\libs",
),
# Already-normal UNC path should be a no-op
(
r"\\server\share\include",
r"\\server\share\include",
),
(
"C:/astrbot-test/backend/python/libs",
WINDOWS_RUNTIME_LIBS_DIR,
),

Comment thread astrbot/core/utils/pip_installer.py
Keep the in-process pip environment mutation and case-insensitive INCLUDE/LIB handling localized so packaged Windows builds are easier to follow. Add a UNC no-op regression case to guard path normalization.
@zouyonghe
Copy link
Copy Markdown
Member Author

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/utils/pip_installer.py" line_range="243" />
<code_context>
     return result_code, stream.lines


+def _run_pip_main_with_temporary_environ(
+    pip_main,
+    args: list[str],
</code_context>
<issue_to_address>
**issue (complexity):** Consider extracting a reusable context manager for environment mutation, simplifying the env-building helpers, and documenting path normalization to keep the new behavior while reducing surface-area complexity.

You can reduce the added complexity by (1) encapsulating the env mutation in a small context manager and (2) simplifying the env‑building helpers. This keeps behavior but shrinks surface area.

### 1. Inline env computation and use a context manager

`_run_pip_main_with_temporary_environ` doesn’t need an `env_updates` parameter yet, and the save/restore loop can be pushed into a reusable helper:

```python
@contextlib.contextmanager
def _temporary_environ(updates: Mapping[str, str]):
    # process-wide: serialize env mutations
    with _PIP_IN_PROCESS_ENV_LOCK:
        if not updates:
            # no-op context
            yield
            return

        missing = object()
        previous_values = {k: os.environ.get(k, missing) for k in updates}
        try:
            os.environ.update(updates)
            yield
        finally:
            for key, previous_value in previous_values.items():
                if previous_value is missing:
                    os.environ.pop(key, None)
                else:
                    os.environ[key] = previous_value


def _run_pip_main_with_temporary_environ(
    pip_main,
    args: list[str],
) -> tuple[int, list[str]]:
    env_updates = _build_packaged_windows_runtime_build_env(base_env=os.environ)
    # Fast path: nothing to do
    if not env_updates:
        return _run_pip_main_streaming(pip_main, args)

    with _temporary_environ(env_updates):
        return _run_pip_main_streaming(pip_main, args)
```

This keeps the lock, preserves the save/restore semantics, but makes the call site trivial and removes the unused `env_updates` abstraction.

### 2. Simplify env building by extracting a case‑insensitive helper

You can keep the `upper_to_key` optimization but move the lookup into a focused helper. That flattens `_build_packaged_windows_runtime_build_env` and makes the intent explicit:

```python
def _get_case_insensitive(env: Mapping[str, str], upper_map: dict[str, str], name: str) -> str | None:
    direct = env.get(name)
    if direct is not None:
        return direct
    existing_key = upper_map.get(name.upper())
    if existing_key is not None:
        return env.get(existing_key)
    return None


def _build_packaged_windows_runtime_build_env(
    *,
    base_env: Mapping[str, str] | None = None,
) -> dict[str, str]:
    if sys.platform != "win32" or not is_packaged_desktop_runtime():
        return {}

    base_env = os.environ if base_env is None else base_env

    runtime_executable = _normalize_windows_native_build_path(sys.executable)
    runtime_dir = ntpath.dirname(runtime_executable)
    if not runtime_dir:
        return {}

    include_dir = _normalize_windows_native_build_path(ntpath.join(runtime_dir, "include"))
    libs_dir = _normalize_windows_native_build_path(ntpath.join(runtime_dir, "libs"))
    include_exists = os.path.isdir(include_dir)
    libs_exists = os.path.isdir(libs_dir)
    if not (include_exists or libs_exists):
        return {}

    upper_to_key = {key.upper(): key for key in base_env}
    env_updates: dict[str, str] = {}

    if include_exists:
        existing = _get_case_insensitive(base_env, upper_to_key, "INCLUDE")
        env_updates["INCLUDE"] = f"{include_dir};{existing}" if existing else include_dir

    if libs_exists:
        existing = _get_case_insensitive(base_env, upper_to_key, "LIB")
        env_updates["LIB"] = f"{libs_dir};{existing}" if existing else libs_dir

    return env_updates
```

This removes the inline `_prepend` closure and the multi‑step lookup, while keeping the case‑insensitive behavior and early‑return conditions intact.

### 3. Make `_normalize_windows_native_build_path` self‑documenting

A minimal docstring and comments make the mixed prefix/normalization logic easier to follow without changing behavior:

```python
def _normalize_windows_native_build_path(path: str) -> str:
    """Normalize a Windows path coming from native APIs or sys.executable.

    Handles:
    - UNC extended prefixes (\\?\UNC\, \??\UNC\) -> standard UNC form (\\server\share\...)
    - Other extended prefixes (\\?\ , \??\ ) by stripping the prefix
    - Then normalizes separators and dot components.
    """
    normalized = path.replace("/", "\\")

    # Extended UNC: \\?\UNC\server\share\... -> \\server\share\...
    for prefix in _WINDOWS_UNC_PATH_PREFIXES:
        if normalized.startswith(prefix):
            return ntpath.normpath(f"\\\\{normalized[len(prefix):]}")

    # Other extended prefixes: strip and then normalize
    for prefix in _WINDOWS_EXTENDED_PATH_PREFIXES:
        if normalized.startswith(prefix):
            normalized = normalized[len(prefix):]
            break

    return ntpath.normpath(normalized)
```

These changes keep all existing behavior but reduce branching, remove an unused parameter, and localize the “tricky” parts into small, well‑named helpers.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread astrbot/core/utils/pip_installer.py
Keep packaged Windows pip environment handling easier to follow by reusing a temporary environment context manager, isolating case-insensitive INCLUDE/LIB lookup, and documenting native path normalization behavior.
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Mar 18, 2026
@zouyonghe
Copy link
Copy Markdown
Member Author

@sourcery-ai review

Copy link
Copy Markdown

@SourceryAI SourceryAI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Hi @zouyonghe! 👋

Thanks for trying out Sourcery by commenting with @sourcery-ai review! 🚀

Install the sourcery-ai bot to get automatic code reviews on every pull request ✨

Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@zouyonghe zouyonghe merged commit 40076b6 into AstrBotDevs:master Mar 18, 2026
6 checks passed
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/test_pip_installer.py" line_range="503-459" />
<code_context>
+
+
+@pytest.mark.asyncio
+async def test_run_pip_in_process_does_not_inject_when_runtime_dirs_missing(
+    configure_run_pip_in_process_capture,
+):
+    observed_env = configure_run_pip_in_process_capture(
+        platform="win32",
+        packaged_runtime=True,
+        include_value=EXISTING_WINDOWS_INCLUDE_DIR,
+        lib_value=EXISTING_WINDOWS_LIB_DIR,
+        existing_runtime_dirs=set(),
+    )
+
+    installer = PipInstaller("")
+    result = await installer._run_pip_in_process(["install", "demo-package"])
+
+    assert result == 0
+    assert observed_env == {
+        "INCLUDE": EXISTING_WINDOWS_INCLUDE_DIR,
+        "LIB": EXISTING_WINDOWS_LIB_DIR,
+    }
+    assert pip_installer_module.os.environ["INCLUDE"] == EXISTING_WINDOWS_INCLUDE_DIR
+    assert pip_installer_module.os.environ["LIB"] == EXISTING_WINDOWS_LIB_DIR
+
+
</code_context>
<issue_to_address>
**suggestion (testing):** Add a variant where INCLUDE/LIB are initially unset and runtime dirs are missing

This already covers the case where runtime dirs are missing but existing `INCLUDE`/`LIB` values are present. Please also add a variant where `INCLUDE` and `LIB` are absent from `os.environ` and `existing_runtime_dirs` is an empty set, and assert that the in-process pip run does not create these keys. That will exercise the "unset env + missing runtime dirs" scenario.

Suggested implementation:

```python
WINDOWS_RUNTIME_LIBS_DIR = ntpath.join(WINDOWS_RUNTIME_ROOT, "libs")
EXISTING_WINDOWS_INCLUDE_DIR = ntpath.join(r"C:\toolchain", "include")
EXISTING_WINDOWS_LIB_DIR = ntpath.join(r"C:\toolchain", "lib")


@pytest.mark.asyncio
async def test_run_pip_in_process_does_not_create_env_when_unset_and_runtime_dirs_missing(
    configure_run_pip_in_process_capture,
):
    observed_env = configure_run_pip_in_process_capture(
        platform="win32",
        packaged_runtime=True,
        include_value=None,
        lib_value=None,
        existing_runtime_dirs=set(),
    )

    installer = PipInstaller("")
    result = await installer._run_pip_in_process(["install", "demo-package"])

    assert result == 0
    assert "INCLUDE" not in observed_env
    assert "LIB" not in observed_env
    assert "INCLUDE" not in pip_installer_module.os.environ
    assert "LIB" not in pip_installer_module.os.environ

```

This assumes `configure_run_pip_in_process_capture` treats `include_value=None` and `lib_value=None` as “do not set these env vars initially”. If in your fixture the absence is represented differently (e.g. by omitting the arguments or using a sentinel), adjust the call accordingly. Also, this test relies on `pip_installer_module` already being imported/available as used in the existing tests.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

expected_lib = f"{WINDOWS_RUNTIME_LIBS_DIR};{EXISTING_WINDOWS_LIB_DIR}"
assert observed_env == {"INCLUDE": expected_include, "LIB": expected_lib}
assert pip_installer_module.os.environ["INCLUDE"] == EXISTING_WINDOWS_INCLUDE_DIR
assert pip_installer_module.os.environ["LIB"] == EXISTING_WINDOWS_LIB_DIR
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Add a variant where INCLUDE/LIB are initially unset and runtime dirs are missing

This already covers the case where runtime dirs are missing but existing INCLUDE/LIB values are present. Please also add a variant where INCLUDE and LIB are absent from os.environ and existing_runtime_dirs is an empty set, and assert that the in-process pip run does not create these keys. That will exercise the "unset env + missing runtime dirs" scenario.

Suggested implementation:

WINDOWS_RUNTIME_LIBS_DIR = ntpath.join(WINDOWS_RUNTIME_ROOT, "libs")
EXISTING_WINDOWS_INCLUDE_DIR = ntpath.join(r"C:\toolchain", "include")
EXISTING_WINDOWS_LIB_DIR = ntpath.join(r"C:\toolchain", "lib")


@pytest.mark.asyncio
async def test_run_pip_in_process_does_not_create_env_when_unset_and_runtime_dirs_missing(
    configure_run_pip_in_process_capture,
):
    observed_env = configure_run_pip_in_process_capture(
        platform="win32",
        packaged_runtime=True,
        include_value=None,
        lib_value=None,
        existing_runtime_dirs=set(),
    )

    installer = PipInstaller("")
    result = await installer._run_pip_in_process(["install", "demo-package"])

    assert result == 0
    assert "INCLUDE" not in observed_env
    assert "LIB" not in observed_env
    assert "INCLUDE" not in pip_installer_module.os.environ
    assert "LIB" not in pip_installer_module.os.environ

This assumes configure_run_pip_in_process_capture treats include_value=None and lib_value=None as “do not set these env vars initially”. If in your fixture the absence is represented differently (e.g. by omitting the arguments or using a sentinel), adjust the call accordingly. Also, this test relies on pip_installer_module already being imported/available as used in the existing tests.

KBVsent pushed a commit to KBVsent/AstrBot that referenced this pull request Mar 19, 2026
…trBotDevs#6575)

* Fix Windows packaged runtime pip build env

* test(pip): cover packaged runtime env injection edges

* refactor(pip): tighten packaged runtime env handling

* test(pip): cover missing runtime build dirs

* fix(pip): build runtime env inside locked section

* test(pip): expand windows path normalization coverage

* refactor(pip): build runtime env from snapshots

* fix(pip): preserve windows env key semantics

* refactor(pip): simplify windows runtime env handling

Keep the in-process pip environment mutation and case-insensitive INCLUDE/LIB handling localized so packaged Windows builds are easier to follow. Add a UNC no-op regression case to guard path normalization.

* refactor(pip): streamline runtime env mutation helpers

Keep packaged Windows pip environment handling easier to follow by reusing a temporary environment context manager, isolating case-insensitive INCLUDE/LIB lookup, and documenting native path normalization behavior.
KBVsent pushed a commit to KBVsent/AstrBot that referenced this pull request Mar 21, 2026
…trBotDevs#6575)

* Fix Windows packaged runtime pip build env

* test(pip): cover packaged runtime env injection edges

* refactor(pip): tighten packaged runtime env handling

* test(pip): cover missing runtime build dirs

* fix(pip): build runtime env inside locked section

* test(pip): expand windows path normalization coverage

* refactor(pip): build runtime env from snapshots

* fix(pip): preserve windows env key semantics

* refactor(pip): simplify windows runtime env handling

Keep the in-process pip environment mutation and case-insensitive INCLUDE/LIB handling localized so packaged Windows builds are easier to follow. Add a UNC no-op regression case to guard path normalization.

* refactor(pip): streamline runtime env mutation helpers

Keep packaged Windows pip environment handling easier to follow by reusing a temporary environment context manager, isolating case-insensitive INCLUDE/LIB lookup, and documenting native path normalization behavior.
xkeyC added a commit to xkeyC/AstrBot that referenced this pull request Mar 28, 2026
* perf: onebot, satori docs improvement

* ci: add pr check

* chore: Delete .github/workflows/pr-checklist-check.yml

* feat: localize session management group & interval method texts (AstrBotDevs#6471)

* fix(ui): localize session management group texts

Replace hardcoded Chinese strings in SessionManagementPage with i18n
lookups for group management labels, dialogs, and action feedback.

Add and align translation keys in en-US, ru-RU, and zh-CN for group
management and batch operation messages to ensure consistent multilingual
UI behavior.

* fix(ui): localize interval method hint text

* fix: SQLite 'database is locked' by adding busy timeout (AstrBotDevs#6474)

The async engine is created without a busy timeout, so concurrent
writes (agent responses, metrics, session updates) fail instantly
with 'database is locked' instead of waiting for the lock.

Add connect_args={'timeout': 30} for SQLite engines so the driver
waits up to 30 seconds for the write lock. Combined with the existing
WAL journal mode, this handles the typical concurrent write bursts
from agent + metrics + session operations.

Fixes AstrBotDevs#6443

* fix: parse multiline frontmatter description in SKILL.md (AstrBotDevs#6460)

* fix(skills): support multiline frontmatter descriptions

* fix(skills): 修复多行 frontmatter 描述解析

* style(skills): clean up frontmatter parser follow-ups

---------

Co-authored-by: RhoninSeiei <RhoninSeiei@users.noreply.github.com>

* chore(deps): bump the github-actions group with 2 updates (AstrBotDevs#6461)

Bumps the github-actions group with 2 updates: [ncipollo/release-action](https://github.com/ncipollo/release-action) and [actions/github-script](https://github.com/actions/github-script).


Updates `ncipollo/release-action` from 1.20.0 to 1.21.0
- [Release notes](https://github.com/ncipollo/release-action/releases)
- [Commits](ncipollo/release-action@v1.20.0...v1.21.0)

Updates `actions/github-script` from 7 to 8
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](actions/github-script@v7...v8)

---
updated-dependencies:
- dependency-name: ncipollo/release-action
  dependency-version: 1.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
- dependency-name: actions/github-script
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: remove deprecated version field from compose.yml (AstrBotDevs#5495)

The version field is no longer required in Docker Compose v2 and has been deprecated.

* fix: reading skills on Windows (AstrBotDevs#6490)

There is an issue with reading the skill directory on the Windows system, which results in a high probability of files under the skill directory being unrecognizable, now fix it.

* fix: subagent lookup failure when using default persona (AstrBotDevs#5672)

* fix: resolve subagent persona lookup for 'default' and unify resolution logic

- Add PersonaManager.get_persona_v3_by_id() to centralize v3 persona resolution
- Handle 'default' persona_id mapping to DEFAULT_PERSONALITY in subagent orchestrator
- Fix HandoffTool.default_description using agent_name parameter correctly
- Add tests for default persona in subagent config and tool deduplication

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: simplify get_default_persona_v3 using get_persona_v3_by_id

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: whatevertogo <whatevertogo@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Soulter <37870767+Soulter@users.noreply.github.com>

* fix: register_agent decorator NameError (AstrBotDevs#5765)

* fix: 修改 register_agent 以避免运行时导入 AstrAgentContext

* test: improve register_agent test robustness

- Add fixture for llm_tools cleanup to avoid test interference
- Use multiple import patterns to make guard more robust to refactors
- Add assertion to verify decorated coroutine is wired as handoff handler

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* 删除测试文件: 移除 register_agent 装饰器的运行时行为测试

---------

Co-authored-by: whatevertogo <whatevertogo@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Soulter <905617992@qq.com>

* fix: only pass dimensions when explicitly configured in embedding config (AstrBotDevs#6432)

* fix: only pass dimensions param when explicitly configured

Models like bge-m3 don't support the dimensions parameter in the
embedding API, causing HTTP 400 errors. Previously dimensions was
always sent with a default value of 1024, even when the user never
configured it. Now dimensions is only included in the request when
embedding_dimensions is explicitly set in provider config.

Closes AstrBotDevs#6421

Signed-off-by: JiangNan <1394485448@qq.com>

* fix: handle invalid dimensions config and align get_dim return

- Add try-except around int() conversion in _embedding_kwargs to
  gracefully handle invalid embedding_dimensions config values
- Update get_dim() to return 0 when embedding_dimensions is not
  explicitly configured, so callers know dimensions weren't specified
  and can handle it accordingly
- Both methods now share consistent logic for reading the config

Signed-off-by: JiangNan <1394485448@qq.com>

* fix: improve logging for invalid embedding_dimensions configuration

---------

Signed-off-by: JiangNan <1394485448@qq.com>
Co-authored-by: Soulter <905617992@qq.com>

* perf: Implement Pydantic data models for the KOOK adapter to enhance data retrieval and message schema validation (AstrBotDevs#5719)

* refactor: 给kook适配器添加kook事件数据类

* format: 使用StrEnum替换kook适配器中的(str,enum)

* docs: add aiocqhttp and satori protocol documentation; remove outdated lagrange and napcat guides

* refactor: downgrade StrEnum to (str, Enum) in kook_type for backward compatibility  (AstrBotDevs#6512)

我那时候搓 AstrBotDevs#5719 的时候 AstrBotDevs#5729 已经合并了, 既然ruff的py限制版本里是`3.12`,那我那时候干脆用的StrEnum,现在发现那个pr revert了,那我也降级回旧Enum写法好了

* feat: install plugin using metadata name and validate importable identifiers (AstrBotDevs#6530)

* feat: install plugin using metadata name and validate importable identifiers

* fix: cleanup temporary upload extraction directory on plugin install failure

* Update astrbot/core/star/star_manager.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* fix: avoid unnecessary install when repository directory already exists

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* fix: restrict workflows to upstream repo (AstrBotDevs#6531)

* Clarify FileUpload/DownloadTool descriptions to fix LLM tool selection (AstrBotDevs#6527)

Multiple models (Gemini 3, GPT-5.2, Claude Sonnet, Kimi K2.5) consistently
pick FileDownloadTool when they should pick FileUploadTool. The old
descriptions used "upload/download" which is ambiguous from the LLM's
perspective — it doesn't know which side is "local" vs "remote".

Rewrite descriptions to use explicit directional language:
- Upload: "Transfer FROM host INTO sandbox" + "when user sends a file"
- Download: "Transfer FROM sandbox OUT to host" + "ONLY when user asks
  to retrieve/export"

Also improve parameter descriptions with the same directional clarity.

Fixes AstrBotDevs#6497

Co-authored-by: Yufeng He <40085740+universeplayer@users.noreply.github.com>

* perf(dashboard): subset MDI icon font and self-host Google Fonts (AstrBotDevs#6532)

* perf(dashboard): subset MDI icon font and self-host Google Fonts

* perf(dashboard): subset MDI icon font and self-host Google Fonts

* perf(dashboard): subset MDI icon font and self-host Google Fonts

* perf(dashboard): subset MDI icon font cr fix

* chore: update lockfile

* enhance:更改未完成更新的文档用词问题(多处“消息平台”已更名为“机器人”) (AstrBotDevs#6568)

* Update kubernetes.md

* Update discord.md

* Update kubernetes.md

* Update AstrBot setup instructions in Kubernetes doc

* fix: set packaged Windows runtime build env for pip native builds (AstrBotDevs#6575)

* Fix Windows packaged runtime pip build env

* test(pip): cover packaged runtime env injection edges

* refactor(pip): tighten packaged runtime env handling

* test(pip): cover missing runtime build dirs

* fix(pip): build runtime env inside locked section

* test(pip): expand windows path normalization coverage

* refactor(pip): build runtime env from snapshots

* fix(pip): preserve windows env key semantics

* refactor(pip): simplify windows runtime env handling

Keep the in-process pip environment mutation and case-insensitive INCLUDE/LIB handling localized so packaged Windows builds are easier to follow. Add a UNC no-op regression case to guard path normalization.

* refactor(pip): streamline runtime env mutation helpers

Keep packaged Windows pip environment handling easier to follow by reusing a temporary environment context manager, isolating case-insensitive INCLUDE/LIB lookup, and documenting native path normalization behavior.

* feat (doc) : Add doc for shipyard-neo sandbox driver (AstrBotDevs#6590)

* fix(ui): localize session management group texts

Replace hardcoded Chinese strings in SessionManagementPage with i18n
lookups for group management labels, dialogs, and action feedback.

Add and align translation keys in en-US, ru-RU, and zh-CN for group
management and batch operation messages to ensure consistent multilingual
UI behavior.

* fix(ui): localize interval method hint text

* docs(sandbox): document shipyard neo setup

Expand the Chinese sandbox guide to cover Shipyard Neo as the
recommended driver and distinguish it from legacy Shipyard.

Add deployment and configuration guidance for standalone and
compose-based setups, include a full annotated config example,
and clarify profile selection, TTL behavior, workspace paths,
and persistence semantics.

* docs(sandbox): recommend standalone shipyard neo

Clarify that Shipyard Neo is best deployed on a separate,
better-provisioned host for long-term use.

Update the setup steps and AstrBot connection guidance, and
remove the earlier combined Docker Compose deployment flow.

* docs(sandbox): expand shipyard neo guide

Document Shipyard Neo as the recommended sandbox driver and
clarify how it differs from the legacy Shipyard setup.

Add guidance for deployment, performance requirements, Bay
configuration, profile selection, TTL behavior, workspace
persistence, and browser capability support.

Also reorganize the sandbox configuration section and keep the
legacy Shipyard instructions for compatibility.

* docs(sandbox): fix shipyard neo doc links

Update the sandbox guides in English and Chinese to link
directly to the upstream `config.yaml` example.

Replace duplicated TTL and persistence notes with references
to the dedicated sections to keep the guide concise and easier
to maintain.

* docs(sandbox): clarify section references in guides (AstrBotDevs#6591)

* fix: prevent wecom ai bot long connection replies from disappearing (AstrBotDevs#6606)

* fix: prevent empty fallback replies from clearing wecom ai bot output

* fix: 优化消息发送逻辑,避免发送空消息

---------

Co-authored-by: shijianhuai <shijianhuai@simuwang.com>
Co-authored-by: Soulter <905617992@qq.com>

* fix(wecom-aibot): significantly improve streaming readability and speed via add throttling (AstrBotDevs#6610)

* fix(wecom-ai): add 0.5s interval for streaming responses

* fix(wecom-ai): correct event type checking and add spacing in WecomAIBotMessageEvent

* feat: context token counting support for multimodal content (images, audio, and chain-of-thought) (AstrBotDevs#6596)

EstimateTokenCounter 之前只计算 TextPart,完全忽略 ImageURLPart、
AudioURLPart 和 ThinkPart。多模态对话中图片占 500-2000 token,
不被计入会导致 context 压缩触发过晚,API 先报 context_length_exceeded。

改动:
- ImageURLPart 按 765 token 估算(OpenAI vision 低/高分辨率中位数)
- AudioURLPart 按 500 token 估算
- ThinkPart 的文本内容正常计算
- 10 个新测试覆盖各类型单独和混合场景

Co-authored-by: Yufeng He <40085740+universeplayer@users.noreply.github.com>

* fix(openai): Token usage not working when using MoonshotAI official API (AstrBotDevs#6618)

fixes: AstrBotDevs#6614

* fix: update hint for ID whitelist configuration to clarify behavior when empty (AstrBotDevs#6611)

* fix: update hint for ID whitelist configuration to clarify behavior when empty

* fix: update whitelist hint

---------

Co-authored-by: machina <1531829828@qq.com>
Co-authored-by: Soulter <905617992@qq.com>

* fix: 截断器丢失唯一 user 消息导致智谱等 provider 返回 400 (AstrBotDevs#6581)

* fix: 截断器丢失唯一 user 消息导致 API 400

修复 AstrBotDevs#6196

当对话只有一条 user 消息(长 tool chain 场景:system → user → assistant
→ tool → assistant → tool → ...),三个截断方法都会把这条 user 消息丢掉,
导致智谱、Gemini 等要求 user 消息的 provider 返回 400。

改动:
- 提取 `_split_system_rest()` 去掉三个方法里重复的 system/non-system 拆分
- 新增 `_ensure_user_message()`:截断后如果没有 user 了,从原始消息里补回
  第一条 user,避免违反 API 格式要求
- 删掉 `truncate_by_dropping_oldest_turns` 里把没有 user 就清空全部消息的逻辑
- 5 个新测试覆盖单 user + 长 tool chain 场景,3 个旧测试更新断言

* style: format code

---------

Co-authored-by: Yufeng He <40085740+universeplayer@users.noreply.github.com>
Co-authored-by: RC-CHN <1051989940@qq.com>

* fix: prevent truncation logic from removing the only user message in long tool-calling conversations (AstrBotDevs#6198)

* fix: 压缩算法删除 user 消息 Bug 修复

* perf: improve truncate algo

---------

Co-authored-by: Soulter <905617992@qq.com>

* feat: add Kimi Coding Plan provider with Anthropic API compatibility (AstrBotDevs#6559)

* Add Kimi Code provider

* Add icon mapping for Kimi Code provider

* Clarify Kimi CodingPlan provider labeling

* Refine Kimi Code header handling

* modified docker compose

* fix: correct Kimi Coding Plan label and update API base URL

---------

Co-authored-by: Soulter <905617992@qq.com>

* fix(openai): improve logging for proxy and API base configuration (AstrBotDevs#6669)

fix: AstrBotDevs#6558

* fix(dashboard): simplify persona selector layout for mobile screens (AstrBotDevs#5907)

* fix: Follow-up logic persists after /stop trigger (AstrBotDevs#6656)

/stop 设置 agent_stop_requested 标记,但 runner 直到当前工具调用
超时才从 _ACTIVE_AGENT_RUNNERS 注销。在此窗口期内,用户发的新消息
被 try_capture_follow_up() 当作 follow-up 吞掉。

在 follow-up 捕获前检查 stop 标记:一旦用户请求停止,就不再把后续
消息注入到正在终止的 agent 上下文中。

Fixes AstrBotDevs#6626

* fix: auto-restart telegram polling loop on failure (AstrBotDevs#6648)

* fix: auto-restart telegram polling loop on failure (AstrBotDevs#373)

* fix: auto-restart telegram polling loop on failure

* fix: harden telegram polling restart lifecycle

* fix(telegram): 根据建议优化轮询鲁棒性并处理 Token 失效错误

* fix: 补全配置元数据及 i18n

* feat: add xiaomi MiMo TTS & STT providers (AstrBotDevs#6643)

* feat: add mimo tts provider support

* fix: handle empty mimo tts choices

* feat: add mimo stt provider support

* chore: rename "OpenAI" provider to "OpenAI Compatible" (AstrBotDevs#6707)

* fix: prevent accidental removal of MCP external tools due to name collisions with disabled built-in tools (AstrBotDevs#5925)

* fix: 解决 MCP 工具与内置工具重名时的连坐问题

- 修改 get_func 方法:优先返回已激活的工具
- 修改 get_full_tool_set 方法:使用 add_tool 防止同名冲突
- 修改 add_tool 方法:优先保留已激活的工具

Fixes AstrBotDevs#5821

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: address PR review feedback for tool conflict resolution

- Fix inconsistency: get_func now uses reversed() to match ToolSet.add_tool's
  "last-active-wins" logic, preventing potential "tool hijacking" issues
- Improve readability: replace double negative condition with clearer logic
- Add compatibility: use getattr with default for tools without 'active' attribute
- Remove unnecessary deepcopy: MCPTool runtime objects should not be deep copied
- Update docstring: accurately describe the actual tool resolution behavior

Addresses review comments from sourcery-ai, gemini-code-assist, and Copilot.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add tests for tool conflict resolution (issue AstrBotDevs#5821)

Add comprehensive tests for ToolSet.add_tool, get_func, and get_full_tool_set
to verify the conflict resolution behavior when MCP tools share names with
built-in tools.

Test cases:
- ToolSet.add_tool: active/inactive priority, last-one-wins for same state
- get_func: returns last active tool, fallback to last matching tool
- get_full_tool_set: deduplication logic, no deepcopy, MCP overrides disabled builtin

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: 修复工具冲突处理逻辑,确保未激活工具不被错误移除

---------

Co-authored-by: whatevertogo <whatevertogo@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add a toggle to disable thinking mode in Ollama (AstrBotDevs#5941)

* feat: add ollama thinking toggle

* fix: simplify hint for ollama_disable_thinking configuration

---------

Co-authored-by: Gargantua <22532097@zju.edu.cn>
Co-authored-by: Soulter <905617992@qq.com>

* fix: preserve PATHEXT for stdio mcp servers on windows (AstrBotDevs#5822)

* fix: preserve PATHEXT for stdio mcp servers on windows

* chore: delete test_mcp_client.py

---------

Co-authored-by: Soulter <905617992@qq.com>

* fix(core): interrupt subagent tool waits on stop (AstrBotDevs#5850)

* fix(core): interrupt subagent tool waits on stop

* test: relax subagent handoff timeout

* test: cover stop-aware tool interruption

* refactor: unify runner stop state

* refactor: simplify tool executor interruption

* fix: preserve tool interruption propagation

* refactor: tighten interruption helpers

---------

Co-authored-by: idiotsj <idiotsj@users.noreply.github.com>

* fix(agent): reject follow-up messages after stop request (AstrBotDevs#6704)

* fix: reject follow-up messages after stop requested (AstrBotDevs#6626)

Once a user sends /stop, follow-up messages should no longer be
accepted for that runner. Previously, there was a race window where
messages sent after stop could still be queued as follow-ups.

This fix gates the follow_up() method to check both done() and
_stop_requested before accepting a new follow-up message.

Acceptance criteria met:
- After /stop, later follow-up messages return None (rejected)
- Post-stop follow-ups are not added to _pending_follow_ups
- No post-stop text is injected into tool results
- Graceful-stop behavior otherwise unchanged
- Follow-ups submitted before stop retain current behavior

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: add regression tests for issue AstrBotDevs#6626 follow-up rejection

Add focused tests that verify the complete tool-result injection path
for follow-up messages after stop is requested:

- test_follow_up_rejected_and_runner_stops_without_execution: Verifies
  that when stop is requested before any execution, follow-ups are
  rejected and the runner stops gracefully without executing tools.

- test_follow_up_merged_into_tool_result_before_stop: Verifies that
  follow-ups queued before stop are properly merged into tool results
  via _merge_follow_up_notice().

- test_follow_up_after_stop_not_merged_into_tool_result: Regression
  test that simulates the race condition from issue AstrBotDevs#6626. Verifies
  that only pre-stop follow-ups are merged into tool results, and
  post-stop follow-ups are rejected at the admission point.

These tests validate the fix in ToolLoopAgentRunner.follow_up() that
checks both self.done() and self._stop_requested before accepting
new follow-up messages.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(agent): update stop request check in ToolLoopAgentRunner

---------

Co-authored-by: ccsang <ccsang@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Soulter <905617992@qq.com>

* fix: skills-like re-query missing extra_user_content_parts causes image_caption not to be injected (AstrBotDevs#6710)

当使用 skills-like tool mode 时,_resolve_tool_exec 的 re-query 调用没有
传递 extra_user_content_parts,导致图片描述等附加内容丢失。

fixes AstrBotDevs#6702

Co-authored-by: Soulter <37870767+Soulter@users.noreply.github.com>

* perf(webchat): enhance message handling with proactive saving and streaming completion (AstrBotDevs#6698)

* fix(config): respect disabled system functions in web search tools (AstrBotDevs#6584)

Co-authored-by: BillionClaw <billionclaw@cl OSS.dev>

* fix(agent): pass tool_call_timeout to subagent handsoff, cron and background task execution, and increase default timeout from 60 to 120 (AstrBotDevs#6713)

* fix(agent): pass tool_call_timeout to SubAgent handoff execution

- Add tool_call_timeout parameter to _execute_handoff method
- Pass run_context.tool_call_timeout to ctx.tool_loop_agent
- Add unit test to verify tool_call_timeout is correctly passed
- Fixes AstrBotDevs#6711: SubAgent MCP tool call timeout now respects configured timeout

The SubAgent handoff execution was using the default 60-second timeout
instead of the configured tool_call_timeout from provider settings.
This change ensures that SubAgent MCP tool calls respect the user's
configured timeout settings.

* test: add unit test for tool_call_timeout in SubAgent handoff

* fix: restore deleted test and fix test assertion

- Restore test_collect_handoff_image_urls_filters_extensionless_missing_event_file
- Fix test_collect_handoff_image_urls_keeps_extensionless_existing_event_file assertion
- Keep new test_execute_handoff_passes_tool_call_timeout_to_tool_loop_agent

* refactor: simplify tool_call_timeout passing in _execute_handoff

- Pass run_context.tool_call_timeout directly to ctx.tool_loop_agent
- Remove unnecessary local variable assignment
- Addresses review feedback from Sourcery AI

* fix(config): increase default tool call timeout from 60 to 120 seconds

---------

Co-authored-by: LehaoLin <linlehao@cuhk.edu.cn>
Co-authored-by: Soulter <905617992@qq.com>

* docs: update README.md to add separator in links section

* fix(skills): use actual sandbox path from cache instead of hardcoded workspace root (AstrBotDevs#6331)

* fix(skills): use actual sandbox path from cache instead of hardcoded workspace root

Fixes AstrBotDevs#6273

When using Shipyard booter, the sandbox workspace directory is
`/home/ship_{session_id}/workspace/` instead of the hardcoded `/workspace`.
This caused Agent to fail reading SKILL.md files with 'No such file or directory'.

Changes:
- In build_skills_prompt: prefer skill.path (from sandbox cache) over
  hardcoded SANDBOX_WORKSPACE_ROOT for sandbox_only skills
- In list_skills: always prefer sandbox_cached_paths over hardcoded path
  for sandbox_only skills

The actual path is resolved at sandbox scan time via Path.resolve() in
_build_scan_command, which returns the correct absolute path based on
the sandbox's actual working directory.

* docs: add comment explaining show_sandbox_path behavior for sandbox_only skills

Address Sourcery AI review comment:
- Clarify that show_sandbox_path is implicitly True for sandbox_only skills
- Explain why the flag is effectively ignored (no local path exists)

* refactor: simplify path_str fallback using or operator

Address review feedback: use single-line fallback instead of if-not pattern.

* style: format skill_manager.py with ruff

Fix ruff format-check failure

* fix(skills): sanitize cached sandbox skill paths

Normalize sandbox cache paths before reading or writing them so invalid,
empty, or mismatched entries fall back to a safe default SKILL.md path.

This avoids using malformed cached paths, keeps path rendering
consistent, and ensures sandbox skill listings always point to the
expected workspace location.

---------

Co-authored-by: ccsang <ccsang@users.noreply.github.com>
Co-authored-by: RC-CHN <1051989940@qq.com>

* fix: ensure Gemini array schemas always include items (AstrBotDevs#6051)

Co-authored-by: Stable Genius <259448942+stablegenius49@users.noreply.github.com>

* fix(webchat): render standalone HTML replies as code (AstrBotDevs#6074)

Co-authored-by: Stable Genius <259448942+stablegenius49@users.noreply.github.com>
Co-authored-by: Soulter <37870767+Soulter@users.noreply.github.com>

* fix: fall back on Windows skill file encodings (AstrBotDevs#6058)

Co-authored-by: stablegenius49 <185121704+stablegenius49@users.noreply.github.com>

* fix(lark): Defer card creation and renew on tool call break (AstrBotDevs#6743)

* fix(lark): defer streaming card creation and renew card on tool call break

- Defer CardKit streaming card creation until the first text token
  arrives, preventing an empty card from rendering before content.
- Handle `type="break"` signal in send_streaming: close the current
  card and lazily create a new one for post-tool-call text, so the
  new card appears below the tool status message in correct order.
- Only emit "break" signal when show_tool_use is enabled; when tool
  output is hidden, the AI response continues on the same card.

* style: format ruff

* fix: cr bug

* fix: cr

* fix: convert Feishu opus files for Whisper API STT (AstrBotDevs#6078)

* fix: convert lark opus files for whisper api

* chore: ruff format

---------

Co-authored-by: Stable Genius <259448942+stablegenius49@users.noreply.github.com>
Co-authored-by: Soulter <905617992@qq.com>

* fix: skip empty knowledge-base embedding batches (AstrBotDevs#6106)

Co-authored-by: stablegenius49 <185121704+stablegenius49@users.noreply.github.com>

* feat(skill_manager): normalize and rename legacy skill markdown files to `SKILL.md` (AstrBotDevs#6757)

* feat(skill_manager): normalize and rename legacy skill markdown files to `SKILL.md`

* fix(vec_db): format debug log message for empty batch insert

* feat(extension): add category filtering for market plugins and enhance UI components (AstrBotDevs#6762)

* chore: bump version to 4.21.0

* feat: supports weixin personal account (AstrBotDevs#6777)

* feat: supports weixin personal account

* feat(weixin): update documentation for personal WeChat integration and add QR code image

* feat(weixin): refactor send method to streamline message handling

* fix(weixin): correct AES key encoding in media payload construction

* feat(weixin): update weixin_oc_base_url description for clarity in config metadata

* feat(weixin): enhance WeChat integration with QR code support and configuration updates

* feat(weixin): implement WeixinOCClient for improved media handling and API requests

* feat(platform): update platform status refresh interval to 5 seconds

* fix(platform.tg_adapter): import Forbidden instead of deprecated Unauthorized (AstrBotDevs#6765) (AstrBotDevs#6769)

* feat: skip search when the entire knowledge base is empty (AstrBotDevs#6750)

* feat:增加知识库全为空时的跳过检索

* apply bot suggestions

* style:reformat code

* feat: fix preserve escaped newlines in frontmatter & update tests & ci workflows (AstrBotDevs#6783)

* Feat(webui): support pinning and dragging for installed plugins (AstrBotDevs#6649) (AstrBotDevs#6776)

* refactor(persona): replace local folder components with shared folder components

* feat(webui): implement draggable reordering with animation for pinned plugins

* refactor(webui): extract PinnedPluginItem into a standalone component

* fix: handle potential None values for token usage metrics in OpenAI provider (AstrBotDevs#6788)

Such as: unsupported operand type(s) for -: 'int' and 'NoneType'

fixes: AstrBotDevs#6772

* feat: supports image compressing (AstrBotDevs#6794)

* feat: supports image compressing (AstrBotDevs#6463)

Co-authored-by: Soulter <905617992@qq.com>

* feat: 增加图像压缩最大尺寸至1280

* Update astrbot/core/astr_main_agent.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* feat: 增强临时文件管理,添加图像压缩路径跟踪与清理功能

* feat: 更新图片压缩功能提示,移除对 chat_completion 提供商的限制说明

---------

Co-authored-by: Chen <42998804+a61995987@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* fix: keep all CallToolResult content items (AstrBotDevs#6149)

Co-authored-by: Soulter <37870767+Soulter@users.noreply.github.com>

* chore: bump version to 4.22.0

* docs: update wechat app version requirements for WeChat adapter and add instructions for profile photo/remark modifications

* chore: gitignore .env warker.js

* fix: remove privacy data from test case (AstrBotDevs#6803)

Co-authored-by: whatevertogo <whatevertogo@users.noreply.github.com>

* fix: align mimo tts style payload with official docs (AstrBotDevs#6814)

* feat(dashboard): add log and cache cleanup in settings (AstrBotDevs#6822)

* feat(dashboard): add log and cache cleanup in settings

* refactor: simplify storage cleaner log config handling

* fix: Repair abnormal indentation

* fix(storage): harden cleanup config handling

Use typed config value access to avoid treating invalid values as
enabled flags or log paths during storage cleanup.

Also stop exposing raw backend exceptions in the dashboard storage
status API and direct users to server logs for details.

---------

Co-authored-by: RC-CHN <1051989940@qq.com>

* fix(t2i): sync active template across all configs (AstrBotDevs#6824)

* fix(t2i): sync active template across all configs

apply template activation and reset to every config profile instead of only
the default one, and reload each pipeline scheduler so changes take effect
consistently in multi-config setups

add a dashboard test that creates extra configs and verifies active template
updates and scheduler reload coverage across all config ids

* fix(t2i): reload all schedulers on template changes

extract a shared helper to reload pipeline schedulers for every config.
when syncing or resetting the active template, persist each config and
then reload all schedulers to keep mappings consistent.

also reload all schedulers when updating the currently active template,
and add dashboard tests to verify cross-config sync and scheduler
replacement behavior.

* fix: cannot use tools in siliconflow provider (AstrBotDevs#6829)

* fix: cannot use tools in siliconflow provider

* fix: handle empty choices in ChatCompletionStreamState

* fix: correct voice message support status in WeChat adapter documentation

* feat(lark): add collapsible reasoning panel support and enhance message handling (AstrBotDevs#6831)

* feat(lark): add collapsible reasoning panel support and enhance message handling

* feat(lark): refactor collapsible panel creation for improved readability and maintainability

* chore: ruff format

* perf: validate config_path before checking existence (AstrBotDevs#6722)

Add a check for empty config_path in check_exist method

* chore(deps): bump pnpm/action-setup in the github-actions group (AstrBotDevs#6862)

Bumps the github-actions group with 1 update: [pnpm/action-setup](https://github.com/pnpm/action-setup).


Updates `pnpm/action-setup` from 4.4.0 to 5.0.0
- [Release notes](https://github.com/pnpm/action-setup/releases)
- [Commits](pnpm/action-setup@v4.4.0...v5.0.0)

---
updated-dependencies:
- dependency-name: pnpm/action-setup
  dependency-version: 5.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: wrong index in ObjectEditor updateKey causing false 'key exists' error

* fix: wrong index in ObjectEditor updateKey causing false 'key exists' error

* fix: same index mismatch issue in updateJSON

* fix(ui): stabilize ObjectEditor pair keys

Use generated ids for key-value pairs instead of array indexes to
prevent mismatch issues during editing and rendering.

Also replace duplicate-key alerts with toast warnings for a more
consistent UI experience.

---------

Co-authored-by: RC-CHN <1051989940@qq.com>

* feat(api): add GET file endpoint and update file route to support multiple methods (AstrBotDevs#6874)

* fix(openapi): rename route view function

* fix(ui): include vuetify radiobox icons (AstrBotDevs#6892)

Add the radiobox icons used indirectly by Vuetify internals
to the required MDI subset so they are kept during font
generation.

Regenerate the subset CSS and font files to prevent missing
radio button icons at runtime.

* fix(tests): update scanUsedIcons tests to include required radio icons (AstrBotDevs#6894)

* doc: Update docs/zh/platform/lark.md (AstrBotDevs#6897)

* 补充飞书配置群聊机器人的部分

- 移除了 im:message:send 权限,因为似乎飞书已经移除了该权限
- 新增关于飞书群聊如何配置权限的部分

* Update docs/zh/platform/lark.md

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Feat(webui): show plugin author on cards & pinned item (AstrBotDevs#5802) (AstrBotDevs#6875)

* feat: 为卡片视图增加作者信息

* feat:置顶列表面板新增作者名称与插件名称

* docs(compshare): correct typos (AstrBotDevs#6878)

* Fix(WebUi): allow batch resetting provider config to "follow" (iss#6749) (AstrBotDevs#6825)

* feat(webui): use explicit 'follow' status for provider settings and improve batch operation logic

* fix: allow batch resetting provider config to "follow config"

* fix(AstrBotDevs#6749): use a unique constant for 'follow' status to avoid collisions with provider IDs

* fix: remove config.use_reloader = True

* refactor(ui): extract follow config sentinel constant

---------

Co-authored-by: RC-CHN <1051989940@qq.com>

* fix: keep weixin_oc polling after inbound timeouts (AstrBotDevs#6915)

* fix: keep weixin_oc polling after inbound timeouts

* Delete tests/test_weixin_oc_adapter.py

---------

Co-authored-by: Soulter <37870767+Soulter@users.noreply.github.com>

* fix(i18n): update OpenAI embedding hint for better compatibility guidance

fixes: AstrBotDevs#6855

* feat: auto-append /v1 to embedding_api_base in OpenAI embedding provider (AstrBotDevs#6863)

* fix: auto-append /v1 to embedding_api_base in OpenAI embedding provider (AstrBotDevs#6855)

When users configure `embedding_api_base` without the `/v1` suffix,
the OpenAI SDK does not auto-complete it, causing request path errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: ensure API base URL for OpenAI embedding ends with /v1 or /v4

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Soulter <905617992@qq.com>

* Fix payload handling for msg_id in QQ API (AstrBotDevs#6604)

Remove msg_id from payload to prevent errors with proactive tool-call path and avoid permission issues.

Co-authored-by: Naer <88199249+V-YOP@users.noreply.github.com>

* fix(provider): add missing index field to streaming tool_call deltas (AstrBotDevs#6661) (AstrBotDevs#6692)

* fix(provider): add missing index field to streaming tool_call deltas

- Fix AstrBotDevs#6661: Streaming tool_call arguments lost when OpenAI-compatible proxy omits index field
- Gemini and some proxies (e.g. Continue) don't include index field in tool_call deltas
- Add default index=0 when missing to prevent ChatCompletionStreamState.handle_chunk() from rejecting chunks

Fixes AstrBotDevs#6661

* fix(provider): use enumerate for multi-tool-call index assignment

- Use enumerate() to assign correct index based on list position
- Iterate over all choices (not just the first) for completeness
- Addresses review feedback from sourcery-ai and gemini-code-assist

---------

Co-authored-by: Yaohua-Leo <3067173925@qq.com>
Co-authored-by: Soulter <905617992@qq.com>

* feat(skills): enhance skill installation to support multiple top-level folders and add duplicate handling, and Chinese skill name support (AstrBotDevs#6952)

* feat(skills): enhance skill installation to support multiple top-level folders and add duplicate handling

closes: AstrBotDevs#6949

* refactor(skill_manager): streamline skill name normalization and validation logic

* fix(skill_manager): update skill name regex to allow underscores in skill names

* fix(skill_manager): improve skill name normalization and validation logic

* chore: bump version to 4.22.1

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: JiangNan <1394485448@qq.com>
Co-authored-by: Soulter <905617992@qq.com>
Co-authored-by: LIghtJUNction <lightjunction.me@gmail.com>
Co-authored-by: Ruochen Pan <1051989940@qq.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Rhonin Wang <33801807+RhoninSeiei@users.noreply.github.com>
Co-authored-by: RhoninSeiei <RhoninSeiei@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: YYMa <118096301+YuanyuanMa03@users.noreply.github.com>
Co-authored-by: linzhengtian <907305684@qq.com>
Co-authored-by: whatevertogo <149563971+whatevertogo@users.noreply.github.com>
Co-authored-by: whatevertogo <whatevertogo@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Soulter <37870767+Soulter@users.noreply.github.com>
Co-authored-by: jnMetaCode <1394485448@qq.com>
Co-authored-by: shuiping233 <49360196+shuiping233@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: 鸦羽 <Raven95676@gmail.com>
Co-authored-by: Yufeng He <40085740+universeplayer@users.noreply.github.com>
Co-authored-by: camera-2018 <40380042+camera-2018@users.noreply.github.com>
Co-authored-by: 糯米茨 <143102889+nuomicici@users.noreply.github.com>
Co-authored-by: エイカク <1259085392z@gmail.com>
Co-authored-by: Ruochen Pan <sorainygreen@gmail.com>
Co-authored-by: Scofield <59475095+shijianhuai@users.noreply.github.com>
Co-authored-by: shijianhuai <shijianhuai@simuwang.com>
Co-authored-by: machina <53079908+machinad@users.noreply.github.com>
Co-authored-by: machina <1531829828@qq.com>
Co-authored-by: leonforcode <leonbeyourside01@gmail.com>
Co-authored-by: daniel5u <danielsuuuuuu@gmail.com>
Co-authored-by: letr <123731298+letr007@users.noreply.github.com>
Co-authored-by: Helian Nuits <sxp20061207@163.com>
Co-authored-by: RichardLiu <97330937+RichardLiuda@users.noreply.github.com>
Co-authored-by: _Kerman <kermanx@qq.com>
Co-authored-by: Gargantua <124801228+catDforD@users.noreply.github.com>
Co-authored-by: Gargantua <22532097@zju.edu.cn>
Co-authored-by: 晴空 <3103908461@qq.com>
Co-authored-by: SJ <idiotgyz@gmail.com>
Co-authored-by: idiotsj <idiotsj@users.noreply.github.com>
Co-authored-by: qingyun <codingtsunami@gmail.com>
Co-authored-by: ccsang <ccsang@users.noreply.github.com>
Co-authored-by: BillionToken <hydr0codone@proton.me>
Co-authored-by: BillionClaw <billionclaw@cl OSS.dev>
Co-authored-by: LIU Yaohua <12531035@mail.sustech.edu.cn>
Co-authored-by: LehaoLin <linlehao@cuhk.edu.cn>
Co-authored-by: Stable Genius <stablegenius043@gmail.com>
Co-authored-by: Stable Genius <259448942+stablegenius49@users.noreply.github.com>
Co-authored-by: stablegenius49 <185121704+stablegenius49@users.noreply.github.com>
Co-authored-by: Lockinwize Lolite <mzwing@mzwing.eu.org>
Co-authored-by: Waterwzy <2916963017@qq.com>
Co-authored-by: M1LKT <144798909+M1LKT@users.noreply.github.com>
Co-authored-by: Chen <42998804+a61995987@users.noreply.github.com>
Co-authored-by: Frank <97429702+tsubasakong@users.noreply.github.com>
Co-authored-by: bread <104435263+bread-ovO@users.noreply.github.com>
Co-authored-by: Stardust <1441308506a@gmail.com>
Co-authored-by: Vorest <147138388+Vorest3679@users.noreply.github.com>
Co-authored-by: GH <BoneAsh@iCloud.com>
Co-authored-by: Zeng Qingwen <143274079+fishwww-ww@users.noreply.github.com>
Co-authored-by: Rainor_da! <51012640+1zzxy1@users.noreply.github.com>
Co-authored-by: Izayoi9 <105905446+Izayoi9@users.noreply.github.com>
Co-authored-by: naer-lily <88199249+naer-lily@users.noreply.github.com>
Co-authored-by: Naer <88199249+V-YOP@users.noreply.github.com>
Co-authored-by: Yaohua-Leo <3067173925@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core The bug / feature is about astrbot's core, backend size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants