Nocturne
Collection
Balance size model with good quality. • 6 items • Updated • 1
How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "DoppelReflEx/MiniusLight-24B" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "DoppelReflEx/MiniusLight-24B",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
What is this?
A nice, simple Slerp merge of 2 Mistral "Small" model and well-known HuggingFace users, TheDrummer/Cydonia-24B-v2 & PocketDoc/Dans-PersonalityEngine-V1.2.0-24b.
This version is the best merge version and recipe I have tried with a good eval scores. Strong in ERP, RP, Story Writing and orther purpose.
Overall, nice to try model, if you want to try. :)
Other information
Chat Template? ChatML, of course!
Merge Method
Detail YAML Config
{
models:
- model: TheDrummer/Cydonia-24B-v2
- model: PocketDoc/Dans-PersonalityEngine-V1.2.0-24b
merge_method: slerp
base_model: TheDrummer/Cydonia-24B-v2
parameters:
t: [0.1, 0.3, 0.6, 0.3, 0.1]
dtype: bfloat16
}
Detail YAML Config
{
models:
- model: TheDrummer/Cydonia-24B-v2
- model: PocketDoc/Dans-PersonalityEngine-V1.2.0-24b
merge_method: slerp
base_model: TheDrummer/Cydonia-24B-v2
parameters:
t: [0.1, 0.3, 0.6, 0.3, 0.1]
dtype: bfloat16
}
- Downloads last month
- 17
Model tree for DoppelReflEx/MiniusLight-24B
Merge model
this model
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "DoppelReflEx/MiniusLight-24B" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DoppelReflEx/MiniusLight-24B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'