Scene-Builder bear logo

Scene-Builder-Qwen3-VL-8B-Instruct

Learning to Design Scenes and Critique Their Quality

Scene-Designer: Learning Design Intelligence for Agentic 3D Scene Generation

OmniAI Group of ZJU ACES Lab

Paper on arXiv Project page GitHub repository Scene-OTC-Bench Base model

Scene-Builder is the supervised fine-tuning (SFT) model from Scene-Designer: Learning Design Intelligence for Agentic 3D Scene Generation, built on Qwen3-VL-8B-Instruct. It learns from complementary scene-construction and scene-critique experience and serves as the starting point for subsequent Scene-GRPO training.

This repository releases the SFT checkpoint with full model weights and inference configurations.

Overview: Design Intelligence

Overview of Design Intelligence: complementary scene design and scene critique capabilities

We define design intelligence as the ability to both design a scene and critique its quality.

Scene design concerns constructing physically valid, functionally correct, and layout-coherent scenes, while efficiently correcting errors without unnecessary trial and error when problems arise. Scene critique focuses on assessing scene quality along the same dimensions and distinguishing better designs from poorer ones. These two capabilities are closely connected, since effective design requires reliable critique to guide construction, while meaningful critique requires an understanding of what constitutes good scene design.

Video Introduction

Watch or download the English introduction · Explore scene demos

The video introduces the broader Scene-Designer project, including design intelligence, data construction, and Critic-guided learning. The weights released here are the Scene-Builder SFT stage, before Scene-GRPO.

From Experience to Scene-Builder

The project constructs two complementary sources of supervision:

  • Scene-OTA-35K: scene-construction experience with Observation–Thought–Action interactions. Causal Trajectory Consolidation preserves important construction and correction decisions while removing ineffective corrections.
  • Scene-OTC-15K: scene-critique experience with Observation–Thought–Critic supervision, using scene perturbations to construct comparisons with ordered quality degradation.

Joint SFT develops scene construction and scene critique capabilities, yielding Scene-Builder. The subsequent Scene-GRPO stage uses a frozen Critic to compare candidate designs and assess parent-to-child improvement, producing Scene-Designer.

For evaluating scene critique, Scene-OTC-Bench provides 94 two-scene comparison questions and 55 four-scene ranking questions.

Checkpoint at a Glance

Property Value
Model Scene-Builder-Qwen3-VL-8B-Instruct
Base model Qwen/Qwen3-VL-8B-Instruct
Training stage Supervised fine-tuning (SFT)
Architecture Qwen3VLForConditionalGeneration
Format Full model weights in four safetensors shards
Precision BF16

Files and Usage

The repository includes the final model weights, shard index, model and generation configurations, tokenizer, chat template, and image/video preprocessing configurations. Training checkpoints, optimizer states, and logs are excluded.

Download the model with the Hugging Face CLI:

hf download XuanyuLiu/Scene-Builder-Qwen3-VL-8B-Instruct --local-dir ./Scene-Builder-Qwen3-VL-8B-Instruct

Load it in a Transformers environment that supports Qwen3-VL:

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "XuanyuLiu/Scene-Builder-Qwen3-VL-8B-Instruct"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    dtype="auto",
    device_map="auto",
)

End-to-end scene generation additionally requires the project's scene-construction tools and execution environment. This repository provides the model weights and processing assets.

Citation

@misc{liu2026scenedesignerlearningdesignintelligence,
      title={Scene-Designer: Learning Design Intelligence for Agentic 3D Scene Generation},
      author={Xuanyu Liu and Hongyan Feng and Miao Pan and Sunlai Chen and Lizhou Liang and Bing Sun and Wenqi Zhang and Xuhong Zhang},
      year={2026},
      eprint={2610.23121},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.23121},
}

Contact

Xuanyu Liu: 22651306@zju.edu.cn.

Downloads last month
101
Safetensors
Model size
9B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for XuanyuLiu/Scene-Builder-Qwen3-VL-8B-Instruct

Finetuned
(761)
this model

Paper for XuanyuLiu/Scene-Builder-Qwen3-VL-8B-Instruct