Instructions to use naksyu/lime-v2-think-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use naksyu/lime-v2-think-gguf with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="naksyu/lime-v2-think-gguf",
	filename="lime-v2-think-q6_k.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use naksyu/lime-v2-think-gguf with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf naksyu/lime-v2-think-gguf:Q6_K
# Run inference directly in the terminal:
llama-cli -hf naksyu/lime-v2-think-gguf:Q6_K

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf naksyu/lime-v2-think-gguf:Q6_K
# Run inference directly in the terminal:
llama-cli -hf naksyu/lime-v2-think-gguf:Q6_K

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf naksyu/lime-v2-think-gguf:Q6_K
# Run inference directly in the terminal:
./llama-cli -hf naksyu/lime-v2-think-gguf:Q6_K

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf naksyu/lime-v2-think-gguf:Q6_K
# Run inference directly in the terminal:
./build/bin/llama-cli -hf naksyu/lime-v2-think-gguf:Q6_K

Use Docker

docker model run hf.co/naksyu/lime-v2-think-gguf:Q6_K

LM Studio
Jan

vLLM

How to use naksyu/lime-v2-think-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "naksyu/lime-v2-think-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "naksyu/lime-v2-think-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/naksyu/lime-v2-think-gguf:Q6_K

Ollama
How to use naksyu/lime-v2-think-gguf with Ollama:
```
ollama run hf.co/naksyu/lime-v2-think-gguf:Q6_K
```

Unsloth Studio new

How to use naksyu/lime-v2-think-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for naksyu/lime-v2-think-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for naksyu/lime-v2-think-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for naksyu/lime-v2-think-gguf to start chatting

Pi new

How to use naksyu/lime-v2-think-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf naksyu/lime-v2-think-gguf:Q6_K

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "naksyu/lime-v2-think-gguf:Q6_K"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use naksyu/lime-v2-think-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf naksyu/lime-v2-think-gguf:Q6_K

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default naksyu/lime-v2-think-gguf:Q6_K

Run Hermes

hermes

Docker Model Runner
How to use naksyu/lime-v2-think-gguf with Docker Model Runner:
```
docker model run hf.co/naksyu/lime-v2-think-gguf:Q6_K
```

Lemonade

How to use naksyu/lime-v2-think-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull naksyu/lime-v2-think-gguf:Q6_K

Run and chat with the model

lemonade run user.lime-v2-think-gguf-Q6_K

List all available models

lemonade list

lime-v2-think-gguf

English

Lime V2 Think GGUF is a GGUF release of a Qwen3.5-9B based Lime V2 fine-tune for Korean-first chat and local inference.

This model aims for a calm, practical, and emotionally grounded assistant style instead of exaggerated or overly cute behavior. Lime V2 is meant to stay warm and supportive while also being proactive and useful in real conversations.

Main characteristics:

Korean-first conversational tuning
calm and stable tone
practical and proactive assistant behavior
local GGUF deployment for llama.cpp-style runtimes
thinking-enabled chat behavior

Important: Jinja Template

This model is sensitive to the chat template.

If your runtime auto-detects a generic template, reasoning behavior and overall output quality can degrade. In some runtimes, replies may look structurally wrong, reasoning may not appear correctly, or the final answer quality may drop.

Please manually review and override the Jinja chat template if needed.

Use the provided chat_template.jinja from this repo when your runtime does not apply the correct template automatically.

Lime V2 Think expects the following thinking format:

<think> ... </think>

If the output looks broken, empty, or oddly formatted, fix the Jinja template first before judging the model itself.

Files Included

lime-v2-think-q6_k.gguf
chat_template.jinja
training_metadata.json
tokenizer_config.json

Korean

Lime V2 Think GGUF는 Qwen3.5-9B 기반 Lime V2 파인튜닝 모델을 GGUF로 변환한 버전이며, 한국어 중심 대화와 로컬 추론 환경을 주된 목표로 합니다.

이 모델은 과하게 감정적이거나 지나치게 귀여운 스타일보다, 차분하고 실용적이며 정서적으로 안정된 어시스턴트 성향을 지향합니다. Lime V2는 따뜻함을 유지하면서도 실제 대화에서 먼저 정리하고 도와줄 수 있는 방향으로 설계되었습니다.

주요 특징:

한국어 중심 대화 튜닝
차분하고 안정적인 말투
실용적이고 능동적인 어시스턴트 성향
llama.cpp 계열 런타임용 GGUF 배포
thinking 기반 추론 응답 지원

중요 안내: Jinja Template

이 모델은 chat template 영향이 큽니다.

런타임이 일반 템플릿을 자동 적용하면 reasoning 동작이나 전체 응답 품질이 깨질 수 있습니다. 환경에 따라 생각 블록이 이상하게 보이거나, 응답 구조가 어색해지거나, 최종 답변 품질이 떨어질 수 있습니다.

가능하면 Jinja chat template를 직접 확인하고, 필요하면 수동으로 override해서 사용하세요.

자동 인식이 정확하지 않다면 이 저장소에 포함된 chat_template.jinja를 그대로 적용하는 것을 권장합니다.

Lime V2 Think는 아래 형식의 thinking 블록을 기대합니다.

<think> ... </think>

출력이 비어 보이거나, 구조가 이상하거나, 응답 형식이 어색하다면 모델 자체를 판단하기 전에 먼저 Jinja template를 수정하는 것을 권장합니다.

Downloads last month: 2

GGUF

Model size

9B params

Architecture

qwen35

Hardware compatibility

6-bit

Collection including naksyu/lime-v2-think-gguf

Lime

Collection

라임 모음 • 5 items • Updated 9 days ago