Skip to content

feat: add stronger local model#185

Closed
AlexanderMelde wants to merge 1 commit intodevfrom
feature/stronger-model
Closed

feat: add stronger local model#185
AlexanderMelde wants to merge 1 commit intodevfrom
feature/stronger-model

Conversation

@AlexanderMelde
Copy link
Copy Markdown
Collaborator

What does this PR do?

This pull request adds support for several new Qwen model variants to the model loader, including their configuration, context sizes, and metadata. These additions expand the available models, especially for users needing efficient models with lower VRAM requirements or improved reasoning capabilities.

New model support and configuration:

  • Added three new Qwen models to the F16_TO_F32_MODEL_MAP for automatic mapping between quantization formats: Qwen2.5-3B-Instruct-q4f16_1-MLC, Qwen3.5-4B-Instruct-q4f16_1-MLC, and Qwen3.5-35B-A3B-q4f16_1-MLC.
  • Registered new context sizes (32k tokens) for both f16 and f32 versions of the new Qwen models in MODEL_CONTEXT_SIZES.

Model metadata and capabilities:

  • Added detailed metadata for three new models in the ModelLoader class:
    • Qwen3.5-35B-A3B-q4f16_1-MLC: Large MoE model with high reasoning quality and low VRAM usage, marked as recommended.
    • Qwen3.5-4B-Instruct-q4f16_1-MLC: Balanced model for agentic tasks.
    • Qwen2.5-3B-Instruct-q4f16_1-MLC: Reliable small model for structured tasks.

Type

  • New ability
  • New workflow
  • Bug fix
  • Enhancement
  • Docs

How to test

Screenshots (if UI changes)

@pluginslab
Copy link
Copy Markdown
Owner

Closing for now. Qwen 3.5 uses a hybrid Gated DeltaNet architecture that TVM/MLC cannot compile yet, so these model IDs likely do not exist in the current @mlc-ai/web-llm prebuilt list. Selecting them from the dropdown would fail at runtime.

We will revisit this when Qwen 3.5 becomes available in WebLLM. Thanks for the contribution!

@pluginslab pluginslab closed this Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants