Not-For-All-Audiences

Instructions to use cgato/Thespis-13b-v0.6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cgato/Thespis-13b-v0.6 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cgato/Thespis-13b-v0.6")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cgato/Thespis-13b-v0.6")
model = AutoModelForCausalLM.from_pretrained("cgato/Thespis-13b-v0.6", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cgato/Thespis-13b-v0.6 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cgato/Thespis-13b-v0.6"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cgato/Thespis-13b-v0.6",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/cgato/Thespis-13b-v0.6

SGLang

How to use cgato/Thespis-13b-v0.6 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cgato/Thespis-13b-v0.6" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cgato/Thespis-13b-v0.6",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cgato/Thespis-13b-v0.6" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cgato/Thespis-13b-v0.6",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use cgato/Thespis-13b-v0.6 with Docker Model Runner:
```
docker model run hf.co/cgato/Thespis-13b-v0.6
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

This model is a personal project. It uses a vanilla chat template and is focused on providing multiturn sfw and nsfw RP experience.

This model works best with internet style RP using standard markup with asterisks surrounding actions and no quotes around dialogue. Currently in the middle of adding new features and working on making sure initial responses are strong.

It uses the following data:

2200 samples from Pure-Dove Dataset ( 90 token length or greater. )
2200 samples from Claude Multiround 30k ( 90 token length or greater. )
700 samples from Airoboros 3.1 ( Writing samples longer than 1500 tokens only. )
900 samples from the Augmental Dataset ( 90 token length or greater )
6000 samples of hand curated RP conversation with various characters.

Works with standard chat format for Ooba or SillyTavern.

Prompt Format: Chat ( The default Ooba template and Silly Tavern Template )

{System Prompt}

Username: {Input}
BotName: {Response}
Username: {Input}
BotName: {Response}

Ooba ( Set it to Chat, select a character and go. )

Silly Tavern Settings ( Default )

Turn Template (for Ooba Instruct if making a Discord bot or Some other Many to one Chat):

You can either bake usernames into the prompt directly for ease of use or programatically add them if running through the API to use as a chatbot.

User string: ( Leave empty if populating username into prompt through a script. Put in your username if its a 1 on 1 convo.) Ex. "DiscordUser1: "
Bot String: ( The bots name, followed by a colon and a space.) Ex. "Mayo: "
Context: ( Your bots system prompt, follow by a newline. )

<|user|><|user-message|>\n<|bot|><|bot-message|>\n

Examples

SFW RP

Narration Example (Trigger words "What do I see?", "Where am I?". Its still very much a work in progress.)

Downloads last month: 23

Safetensors

Model size

13B params

Tensor type

BF16

Model tree for cgato/Thespis-13b-v0.6

Quantizations

3 models