Instructions to use concedo/KobbleTinyV2-1.1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use concedo/KobbleTinyV2-1.1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="concedo/KobbleTinyV2-1.1B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("concedo/KobbleTinyV2-1.1B")
model = AutoModelForCausalLM.from_pretrained("concedo/KobbleTinyV2-1.1B")

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use concedo/KobbleTinyV2-1.1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "concedo/KobbleTinyV2-1.1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "concedo/KobbleTinyV2-1.1B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/concedo/KobbleTinyV2-1.1B

SGLang

How to use concedo/KobbleTinyV2-1.1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "concedo/KobbleTinyV2-1.1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "concedo/KobbleTinyV2-1.1B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "concedo/KobbleTinyV2-1.1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "concedo/KobbleTinyV2-1.1B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use concedo/KobbleTinyV2-1.1B with Docker Model Runner:
```
docker model run hf.co/concedo/KobbleTinyV2-1.1B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

KobbleTinyV2-1.1B

This is a finetune of https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T trained on a small 50mb subset of the Kobble Dataset. Training was done in under 2 hours on a single Nvidia RTX 2060 Mobile GPU with qLora (LR 1.5e-4, rank 8, alpha 16, batch size 2, gradient acc. 4, 2048 ctx).

You can obtain the GGUF quantization of this model here: https://huggingface.co/concedo/KobbleTinyV2-1.1B-GGUF

Update: KobbleTiny has been upgraded to V2! The old V1 is still available at this link.

Try it live now: https://concedo-koboldcpp-kobbletiny.hf.space/

Dataset and Objectives

The Kobble Dataset is a semi-private aggregated dataset made from multiple online sources and web scrapes. It contains content chosen and formatted specifically to work with KoboldAI software and Kobold Lite.

Dataset Categories:

Instruct: Single turn instruct examples presented in the Alpaca format, with an emphasis on uncensored and unrestricted responses.
Chat: Two participant roleplay conversation logs in a multi-turn raw chat format that KoboldAI uses.
Story: Unstructured fiction excerpts, including literature containing various erotic and provocative content.

Prompt template: Alpaca

### Instruction:
{prompt}

### Response:

Note: No assurances will be provided about the origins, safety, or copyright status of this model, or of any content within the Kobble dataset.
If you belong to a country or organization that has strict AI laws or restrictions against unlabelled or unrestricted content, you are advised not to use this model.

Downloads last month: 63

Safetensors

Model size

1B params

Tensor type

F16

Model tree for concedo/KobbleTinyV2-1.1B

Adapters

1 model

Finetunes

3 models

Merges

2 models

Quantizations

3 models