Instructions to use AesSedai/MiMo-V2.5-Pro-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AesSedai/MiMo-V2.5-Pro-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="AesSedai/MiMo-V2.5-Pro-GGUF",
	filename="IQ2_S/MiMo-V2.5-Pro-IQ2_S-00001-of-00008.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use AesSedai/MiMo-V2.5-Pro-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use AesSedai/MiMo-V2.5-Pro-GGUF with Ollama:
```
ollama run hf.co/AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M
```

Unsloth Studio

How to use AesSedai/MiMo-V2.5-Pro-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/MiMo-V2.5-Pro-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/MiMo-V2.5-Pro-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AesSedai/MiMo-V2.5-Pro-GGUF to start chatting

How to use AesSedai/MiMo-V2.5-Pro-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AesSedai/MiMo-V2.5-Pro-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use AesSedai/MiMo-V2.5-Pro-GGUF with Docker Model Runner:
```
docker model run hf.co/AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M
```

Lemonade

How to use AesSedai/MiMo-V2.5-Pro-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AesSedai/MiMo-V2.5-Pro-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.MiMo-V2.5-Pro-GGUF-Q4_K_M

List all available models

lemonade list

MiMo-V2.5-Pro-GGUF / kld_data /llm_quantization_data.csv

AesSedai

Upload folder using huggingface_hub

9cf941e verified 23 days ago

raw

history blame contribute delete

4.7 kB

	model_name,file_size_gb,bpw,Mean KLD_mean,0.1% KLD,0.1% Δp,1.0% KLD,1.0% Δp,10.0% KLD,10.0% Δp,25.0% Δp,5.0% KLD,5.0% Δp,75.0% Δp,90.0% KLD,90.0% Δp,95.0% KLD,95.0% Δp,99.0% KLD,99.0% Δp,99.9% KLD,99.9% Δp,"Cor(ln(PPL(Q)), ln(PPL(base)))",Maximum KLD,Maximum Δp,Mean KLD_std,Mean PPL(Q)-PPL(base)_mean,Mean PPL(Q)-PPL(base)_std,Mean PPL(Q)/PPL(base)_mean,Mean PPL(Q)/PPL(base)_std,Mean PPL(Q)_mean,Mean PPL(Q)_std,Mean PPL(base)_mean,Mean PPL(base)_std,Mean ln(PPL(Q)/PPL(base))_mean,Mean ln(PPL(Q)/PPL(base))_std,Mean Δp_mean,Mean Δp_std,Median KLD,Median Δp,Minimum KLD,Minimum Δp,RMS Δp_mean,RMS Δp_std,Same top p_mean,Same top p_std,common_fit_params,common_memory_breakdown_print,common_params_fit_impl,file_path,file_size_gib,ggml_cuda_init,is_self_ref,kl_divergence,llama_context,llama_kv_cache,llama_kv_cache_iswa,llama_model_loader,llama_perf_context_print,load,load_tensors,mixture,print_info,sched_reserve,system_info
	MiMo-V2.5-Pro-IQ2_S (aes_sedai),319.3845055488,2.5,0.318892,-0.0,-98.711,8e-06,-87.442,0.000407,-27.962,-6.701,9.6e-05,-50.864,0.236,0.820311,6.47,1.518651,14.452,3.672543,39.369,7.725067,81.684,88.1,22.687212,99.974,0.001943,0.875452,0.011448,1.274163,0.003421,4.06863,0.022989,3.193178,0.01668,0.24229,0.002685,-5.988,0.053,0.074109,-0.218,-7e-06,-99.995,21.266,0.083,81.418,0.101,1.01,-12467330513.0,350060.773503,kld/MiMo-V2.5-Pro/wiki-test-raw/aes_sedai/MiMo-V2.5-Pro-IQ2_S.md,297.45,8778000.0,False,584512819216.0,9.31,128.0,512.0,-269.0,35.0,0.9311,723031.34,Q6_K / IQ2_XS / IQ2_XS / IQ2_S,256.0,123.314,4.848561200112841e+50
	MiMo-V2.5-Pro-IQ3_S (aes_sedai),376.69010669568,2.95,0.207702,-1e-06,-94.709,6e-06,-74.189,0.000248,-18.727,-4.525,6.1e-05,-35.434,0.223,0.502377,5.476,0.964881,12.356,2.557175,36.131,5.971176,79.827,92.4,23.693077,99.956,0.001453,0.451066,0.007627,1.141259,0.002365,3.644244,0.019594,3.193178,0.01668,0.132132,0.002072,-3.971,0.043,0.044173,-0.133,-8.2e-05,-99.625,16.985,0.076,85.29,0.092,4.67,-12467330513.0,404708.773503,kld/MiMo-V2.5-Pro/wiki-test-raw/aes_sedai/MiMo-V2.5-Pro-IQ3_S.md,350.82,8778000.0,False,584512819216.0,9.31,128.0,512.0,-2138.0,35.0,0.9311,726991.34,Q6_K / IQ2_S / IQ2_S / IQ3_S,256.0,143.244,4.848561200112841e+50
	MiMo-V2.5-Pro-IQ4_XS (aes_sedai),488.54179250176,3.82,0.098682,-3e-06,-79.287,1e-06,-45.616,9.3e-05,-8.833,-1.702,2.3e-05,-17.211,0.443,0.208776,4.879,0.406225,10.578,1.283414,30.7,4.557108,73.555,96.43,23.030434,99.965,0.001013,0.124394,0.004633,1.038956,0.001457,3.317572,0.017483,3.193178,0.01668,0.038217,0.001403,-1.22,0.029,0.017301,-0.014,-5.1e-05,-98.512,11.146,0.065,90.351,0.077,8.27,-14639490513.0,511161.773503,kld/MiMo-V2.5-Pro/wiki-test-raw/aes_sedai/MiMo-V2.5-Pro-IQ4_XS.md,454.99,8778000.0,False,584512819216.0,9.31,128.0,512.0,-469.0,35.0,0.9311,734896.64,Q8_0 / IQ3_S / IQ3_S / IQ4_XS,256.0,124.034,4.848561200112841e+50
	MiMo-V2.5-Pro-Q4_K_M (aes_sedai),629.1912340275201,4.92,0.055113,-4e-06,-61.027,0.0,-27.122,4.2e-05,-4.799,-0.733,1.1e-05,-9.575,0.518,0.093684,4.075,0.180473,8.266,0.667101,24.306,5.00226,67.323,97.89,22.75626,99.992,0.000954,0.027583,0.003461,1.008638,0.001086,3.220761,0.016918,3.193178,0.01668,0.008601,0.001077,-0.207,0.021,0.007783,-0.001,-0.000168,-99.833,7.952,0.062,93.376,0.064,0.81,-14639490513.0,645297.773503,kld/MiMo-V2.5-Pro/wiki-test-raw/aes_sedai/MiMo-V2.5-Pro-Q4_K_M.md,585.98,8778000.0,False,584512819216.0,9.31,128.0,512.0,-569.0,35.0,0.9311,744616.64,Q8_0 / Q4_K / Q4_K / Q5_K,256.0,136.504,4.848561200112841e+50
	MiMo-V2.5-Pro-Q5_K_M (aes_sedai),756.8161872281601,5.92,0.03685,-5e-06,-47.036,-0.0,-20.518,2.9e-05,-3.715,-0.543,8e-06,-7.381,0.467,0.061322,3.458,0.115248,6.968,0.389255,19.843,3.435202,58.434,98.69,24.047461,99.983,0.000815,0.002228,0.002707,1.000698,0.000848,3.195406,0.016711,3.193178,0.01668,0.000698,0.000847,-0.037,0.017,0.005277,-0.0,-2.6e-05,-97.649,6.414,0.058,94.609,0.059,9.51,-7129217125040416.0,7.1257,kld/MiMo-V2.5-Pro/wiki-test-raw/aes_sedai/MiMo-V2.5-Pro-Q5_K_M.md,704.84,8778000.0,False,584512819216.0,9.31,128.0,512.0,-669.0,35.0,0.9311,787801.6,Q8_0 / Q5_K / Q5_K / Q6_K,256.0,122.721,4.848561200112841e+50
	MiMo-V2.5-Pro-Q8_0 (aes_sedai),1087.61456836608,8.5,-0.0,-5.2e-05,-0.004,-3.8e-05,-0.002,-1.3e-05,-0.0,-0.0,-2.2e-05,-0.001,0.0,1.3e-05,0.0,2.2e-05,0.001,3.8e-05,0.002,5.3e-05,0.004,99.96,7.3e-05,0.006,0.0,0.001649,0.000473,1.000516,0.000148,3.194827,0.016728,3.193178,0.01668,0.000516,0.000148,0.0,0.0,0.0,0.0,-7.5e-05,-0.006,0.001,0.0,99.998,0.001,9.24,-1.0347341034317042e+17,7.31277,kld/MiMo-V2.5-Pro/wiki-test-raw/aes_sedai/MiMo-V2.5-Pro-Q8_0.md,1012.92,8778000.0,True,584512819216.0,9.31,128.0,512.0,-80370.0,35.0,0.9311,787361.34,,256.0,382.391,4.848561200112841e+50