Aquileo | unsloth/Qwen3.6-27B-NVFP4 · MTP?

unsloth
/

Qwen3.6-27B-NVFP4

Image-Text-to-Text

8-bit precision

compressed-tensors

Model card Files Files and versions

MTP?

#3

by tyapo - opened 28 days ago

i always get 0% acceptance rate on MTP (vllm). other models work. is it supposed to be supported?

•

edited 27 days ago

I see the same thing. I think the MTP heads are missing. I am running with SGLang.

I have quantized a version of this model for NVFP4 here: https://huggingface.co/NeuralNet-Hub/Qwen3.6-27B-NVFP4

Any feedback is appreciated.

Unsloth AI org 15 days ago

See https://huggingface.co/unsloth/Qwen3.6-27B-NVFP4/discussions/4 - sorry on the delay!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment