RedHatAI
/

diffusiongemma-26B-A4B-it-FP8-dynamic

diffusion_gemma

compressed-tensors

Model card Files Files and versions

Update README.md

#1

by lwilkinson - opened 1 day ago

base: refs/heads/main

←

from: refs/pr/1

Discussion Files changed

Files changed (1) hide show

README.md +0 -1

README.md CHANGED Viewed

@@ -19,7 +19,6 @@ It was evaluated on several tasks to assess its quality in comparison to the unq
 VLLM_USE_V2_MODEL_RUNNER=1
 vllm serve  RedHatAI/diffusiongemma-26B-A4B-it-FP8-dynamic \
     --trust-remote-code \
-    --attention-backend TRITON_ATTN \
     --max-num-seqs 4 \
     --hf-overrides '{"diffusion_sampler": "entropy_bound", "diffusion_entropy_bound": 0.1}' \
     --default-chat-template-kwargs '{"enable_thinking": true}'

 VLLM_USE_V2_MODEL_RUNNER=1
 vllm serve  RedHatAI/diffusiongemma-26B-A4B-it-FP8-dynamic \
     --trust-remote-code \
     --max-num-seqs 4 \
     --hf-overrides '{"diffusion_sampler": "entropy_bound", "diffusion_entropy_bound": 0.1}' \
     --default-chat-template-kwargs '{"enable_thinking": true}'