JarvisPei nielsr HF Staff commited on
Commit
ef0999c
·
1 Parent(s): 7d0de4c

Add paper link and pipeline tag (#1)

Browse files

- Add paper link and pipeline tag (4242c338bd594a76155c19aa72bf293678d1302b)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +22 -8
README.md CHANGED
@@ -1,20 +1,23 @@
1
  ---
2
- license: mit
3
  base_model: ML-GSAI/LLaDA2.1-mini
4
  library_name: peft
 
 
5
  tags:
6
- - lora
7
- - diffusion-language-model
8
- - long-context
9
- - masked-diffusion
10
- - memdlm
11
  ---
12
 
13
  # LLaDA2.1-mini-MemDLM
14
 
15
  > **Research artifact.** This adapter is released for academic and research purposes. It was trained on a limited dataset as a proof-of-concept for the MemDLM method and is not intended for production use.
16
 
17
- A LoRA adapter for [ML-GSAI/LLaDA2.1-mini](https://huggingface.co/ML-GSAI/LLaDA2.1-mini), trained with [MemDLM](https://github.com/JarvisPei/MemDLM) to enhance long-context understanding in discrete diffusion language models.
 
 
18
 
19
  ## Usage
20
 
@@ -43,6 +46,17 @@ bash examples/llada21/eval_run.sh \
43
  --adapter_model_name_or_path JarvisPei/LLaDA2.1-mini-MemDLM
44
  ```
45
 
 
 
 
 
 
 
 
 
 
 
 
46
  ## License
47
 
48
- MIT
 
1
  ---
 
2
  base_model: ML-GSAI/LLaDA2.1-mini
3
  library_name: peft
4
+ pipeline_tag: text-generation
5
+ license: mit
6
  tags:
7
+ - lora
8
+ - diffusion-language-model
9
+ - long-context
10
+ - masked-diffusion
11
+ - memdlm
12
  ---
13
 
14
  # LLaDA2.1-mini-MemDLM
15
 
16
  > **Research artifact.** This adapter is released for academic and research purposes. It was trained on a limited dataset as a proof-of-concept for the MemDLM method and is not intended for production use.
17
 
18
+ This repository contains a LoRA adapter for [ML-GSAI/LLaDA2.1-mini](https://huggingface.co/ML-GSAI/LLaDA2.1-mini), trained using the method introduced in the paper [MemDLM: Memory-Enhanced DLM Training](https://huggingface.co/papers/2603.22241).
19
+
20
+ MemDLM (Memory-Enhanced Diffusion Language Model) narrows the train-inference gap in Diffusion Language Models (DLMs) by embedding a simulated denoising process into training via Bi-level Optimization. This mechanism enhances long-context understanding and retrieval performance by offloading memorization pressure to model parameters.
21
 
22
  ## Usage
23
 
 
46
  --adapter_model_name_or_path JarvisPei/LLaDA2.1-mini-MemDLM
47
  ```
48
 
49
+ ## Citation
50
+
51
+ ```bibtex
52
+ @article{pei2026memdlm,
53
+ title = {MemDLM: Memory-Enhanced DLM Training},
54
+ author = {Zehua Pei and Hui-Ling Zhen and Weizhe Lin and Sinno Jialin Pan and Yunhe Wang and Mingxuan Yuan and Bei Yu},
55
+ year = {2026},
56
+ journal = {arXiv preprint arXiv:2603.22241},
57
+ }
58
+ ```
59
+
60
  ## License
61
 
62
+ MIT