Model Depot - ONNX
Collection
Leading Models packaged in ONNX format optimized for use with AI PCs • 26 items • Updated • 1
How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "llmware/slim-sql-onnx" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "llmware/slim-sql-onnx",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
slim-sql-onnx
slim-sql-onnx is a small specialized function calling model that takes as input a table schema and a natural language query, and outputs a SQL statement that corresponds to the query, and can be run against a database table. This is a very small text-to-sql model designed for reasonable accuracy on single tables and relatively straightforward queries, and for easy integration into multi-step processes.
This is an ONNX int4 quantized version of slim-sql-1b-v0, providing a very fast, very small inference implementation, optimized for AI PCs using Intel GPU, CPU and NPU.
Model Description
- Developed by: llmware
- Model type: tinyllama
- Parameters: 1.1 billion
- Model Parent: llmware/slim-sql-1b-v0
- Language(s) (NLP): English
- License: Apache 2.0
- Uses: Text-to-SQL conversion
- RAG Benchmark Accuracy Score: NA
- Quantization: int4
Model Card Contact
- Downloads last month
- 4
Model tree for llmware/slim-sql-onnx
Base model
llmware/slim-sql-1b-v0
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "llmware/slim-sql-onnx" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "llmware/slim-sql-onnx", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'