danielhanchen commited on
Commit
d32b2d9
·
verified ·
1 Parent(s): b796e7f

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +31 -3
  2. config.json +1 -1
README.md CHANGED
@@ -5,7 +5,12 @@ base_model:
5
  - Qwen/Qwen3-VL-4B-Thinking
6
  license: apache-2.0
7
  pipeline_tag: image-text-to-text
 
8
  ---
 
 
 
 
9
  <div>
10
  <p style="margin-top: 0;margin-bottom: 0;">
11
  <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
@@ -79,10 +84,10 @@ This is the weight repository for Qwen3-VL-4B-Thinking.
79
 
80
  **Multimodal performance**
81
 
82
- ![](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/table_thinking_vl_4b.jpg)
83
 
84
  **Pure text performance**
85
- ![](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/table_thinking_text_4b.jpg)
86
 
87
  ## Quickstart
88
 
@@ -114,7 +119,7 @@ model = Qwen3VLForConditionalGeneration.from_pretrained(
114
  # device_map="auto",
115
  # )
116
 
117
- processor = AutoProcessor.from_pretrained("Qwen/Qwen/Qwen3-VL-4B-Thinking")
118
 
119
  messages = [
120
  {
@@ -137,6 +142,7 @@ inputs = processor.apply_chat_template(
137
  return_dict=True,
138
  return_tensors="pt"
139
  )
 
140
 
141
  # Inference: Generation of the output
142
  generated_ids = model.generate(**inputs, max_new_tokens=128)
@@ -149,6 +155,28 @@ output_text = processor.batch_decode(
149
  print(output_text)
150
  ```
151
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
152
 
153
 
154
  ## Citation
 
5
  - Qwen/Qwen3-VL-4B-Thinking
6
  license: apache-2.0
7
  pipeline_tag: image-text-to-text
8
+ library_name: transformers
9
  ---
10
+ > [!NOTE]
11
+ > Includes Unsloth **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
12
+ >
13
+
14
  <div>
15
  <p style="margin-top: 0;margin-bottom: 0;">
16
  <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
 
84
 
85
  **Multimodal performance**
86
 
87
+ ![](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-VL/qwen3vl_4b_8b_vl_thinking.jpg)
88
 
89
  **Pure text performance**
90
+ ![](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_4b_8b_text_thinking.jpg)
91
 
92
  ## Quickstart
93
 
 
119
  # device_map="auto",
120
  # )
121
 
122
+ processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-4B-Thinking")
123
 
124
  messages = [
125
  {
 
142
  return_dict=True,
143
  return_tensors="pt"
144
  )
145
+ inputs = inputs.to(model.device)
146
 
147
  # Inference: Generation of the output
148
  generated_ids = model.generate(**inputs, max_new_tokens=128)
 
155
  print(output_text)
156
  ```
157
 
158
+ ### Generation Hyperparameters
159
+ #### VL
160
+ ```bash
161
+ export greedy='false'
162
+ export top_p=0.95
163
+ export top_k=20
164
+ export repetition_penalty=1.0
165
+ export presence_penalty=0.0
166
+ export temperature=1.0
167
+ export out_seq_length=40960
168
+ ```
169
+
170
+ #### Text
171
+ ```bash
172
+ export greedy='false'
173
+ export top_p=0.95
174
+ export top_k=20
175
+ export repetition_penalty=1.0
176
+ export presence_penalty=1.5
177
+ export temperature=1.0
178
+ export out_seq_length=32768 (for aime, lcb, and gpqa, it is recommended to set to 81920)
179
+ ```
180
 
181
 
182
  ## Citation
config.json CHANGED
@@ -37,7 +37,7 @@
37
  "vocab_size": 151936
38
  },
39
  "tie_word_embeddings": true,
40
- "transformers_version": "4.57.0",
41
  "unsloth_fixed": true,
42
  "video_token_id": 151656,
43
  "vision_config": {
 
37
  "vocab_size": 151936
38
  },
39
  "tie_word_embeddings": true,
40
+ "transformers_version": "4.57.1",
41
  "unsloth_fixed": true,
42
  "video_token_id": 151656,
43
  "vision_config": {