Skip to content

[bugfix] qwen25-VL support lora #14638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Fridge003 merged 3 commits into from
Dec 10, 2025
Merged

[bugfix] qwen25-VL support lora #14638

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b1f192d
qwen25-VL support lora
SYChen123 Dec 8, 2025
2dc0dee
Merge branch 'main' into qwenvl-lora
SYChen123 Dec 8, 2025
79b1267
Merge branch 'main' into qwenvl-lora
SYChen123 Dec 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions python/sglang/srt/models/qwen2_5_vl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
# limitations under the License.
"""Inference-only Qwen2-VL model compatible with HuggingFace weights."""
import logging
import re
from functools import partial
from typing import Iterable, List, Optional, Tuple, Type

Expand Down Expand Up @@ -534,6 +535,13 @@ def get_image_feature(self, items: List[MultimodalDataItem]) -> torch.Tensor:
image_embeds = self.visual(pixel_values, grid_thw=image_grid_thw)
return image_embeds

_lora_pattern = re.compile(
r"^model\.layers\.(\d+)\.(?:self_attn|mlp)\.(?:qkv_proj|o_proj|down_proj|gate_up_proj)$"
)
Comment on lines +538 to +540
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current regular expression is slightly too broad. It allows for combinations of modules that do not exist in the model architecture, such as self_attn.down_proj. While this won't cause issues with the current model structure because named_modules() will only yield valid module names, making the regex more specific will improve its correctness and maintainability, especially for future model changes.

A more precise regex would explicitly group the allowed projections under their respective parent modules (self_attn or mlp).

Suggested change
_lora_pattern = re.compile(
r"^model\.layers\.(\d+)\.(?:self_attn|mlp)\.(?:qkv_proj|o_proj|down_proj|gate_up_proj)$"
)
_lora_pattern = re.compile(
r"^model\.layers\.(\d+)\.(?:(?:self_attn\.(?:qkv_proj|o_proj))|(?:mlp\.(?:gate_up_proj|down_proj)))$"
)


def should_apply_lora(self, module_name: str) -> bool:
return bool(self._lora_pattern.match(module_name))

def get_video_feature(self, items: List[MultimodalDataItem]) -> torch.Tensor:
# in qwen-vl, last dim is the same
pixel_values = torch.cat([item.feature for item in items], dim=0).type(
Expand Down
Loading