ZEVORA

Zero-External Vendor Oriented Reasoning Agent — a local-first hybrid AI coding workspace with a private on-device GGUF model and optional cloud providers. Memory, cache, experience, knowledge, and project context stay local by default.

This is a source distribution of the ZEVORA application on Hugging Face. It mirrors the GitHub repository for discoverability and distribution. GitHub is the primary development source: https://github.com/dani12po/zevora

This repository does not contain model weights — see MODELS.md for how ZEVORA uses Qwen3.8-Flash-Next through llama.cpp.

Overview

ZEVORA is a FastAPI-based hybrid AI agent gateway. It runs a local GGUF model via llama.cpp (no API key, no internet required) and optionally falls back to cloud providers for complex, multimodal, or long-context work. It provides a Web UI, an SSE/streaming chat API, an adaptive hybrid router, local memory and cache, approval-gated project tools (MCP), and a CLI controller.

Why ZEVORA

• Local-first: on-device inference keeps your data on your machine.
• Hybrid: cloud capacity when you want or need it; LOCAL_ONLY / CLOUD_ONLY modes.
• Private: prompts, memory, cache, experience, and project context remain local.
• Controlled: all filesystem, terminal, and git actions are approval-gated and workspace-scoped.
• Reproducible: deterministic configuration, verified updates, and SHA-256 checks.

Architecture

User
 │
 ▼
ZEVORA Gateway (FastAPI + Web UI)
 │
 ▼
Agent Core
 │
 ├── Cache
 ├── Memory
 ├── Experience
 ├── Knowledge
 └── Project Context
 │
 ▼
Adaptive Hybrid Router
 │
 ├── ZEVORA Local AI
 │     └── llama.cpp
 │          └── Qwen3.8-Flash-Next GGUF
 │
 └── Cloud Providers
       ├── OpenAI
       ├── Anthropic
       ├── Gemini
       ├── DeepSeek
       ├── xAI
       └── NVIDIA
 │
 ▼
Tools / MCP (approval-gated, workspace-scoped)
 │
 ▼
Verification
 │
 ▼
Final Result

See ARCHITECTURE.md and docs/ for details.

Local AI

ZEVORA Local AI is ZEVORA's on-device inference layer. It loads a single selected GGUF model through llama.cpp lazily, on first local generation. The local model and cloud providers share the same AIProvider abstract contract, registry, discovery, router, and fallback path.

The local data layer keeps the following on your machine:

Component	Purpose
Exact Cache	Return previous responses to identical prompts without inference
Memory	Conversation and project records (SQLite)
Experience	Per-provider routing history that improves model selection
Knowledge Engine	Extracts reusable solution patterns from responses
Project Context	Indexed project metadata for scoped workspace operations
MCP Tools	Filesystem, Git, and terminal access scoped to the selected project

Qwen3.8-Flash-Next

ZEVORA is configured to use the Qwen3.8-Flash-Next model family from the upstream repository:

• Repository: unsloth/Qwen3.8-Flash-Next-GGUF
• Recommended quantization: UD-Q4_K_XL
• Runtime: llama.cpp (via llama-cpp-python)

ZEVORA does not bundle, modify, train, or claim ownership of Qwen weights. See MODELS.md for download instructions and Third-Party Models.

llama.cpp

Local inference uses llama.cpp through the llama-cpp-python binding. On Windows, install the prebuilt CPU wheel inside the project virtual environment:

.venv\Scripts\python.exe -m pip install --prefer-binary "llama-cpp-python>=0.3.14,<0.4" --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu

For GPU acceleration, install the matching CUDA wheel for your hardware and set LOCAL_MODEL_GPU_LAYERS accordingly. If no prebuilt wheel exists for your platform, install Visual C++ Build Tools and follow the upstream llama.cpp Windows source-build instructions.

Installation

Requirements: Python 3.11–3.13

git clone https://github.com/dani12po/zevora
cd ZEVORA
python bootstrap.py
zevora

Open http://127.0.0.1:7432 in your browser. If the zevora command is not on PATH, use python launcher.py.

Quick Start

1. Install Python 3.11+.
2. Clone and run python bootstrap.py.
3. Open the Web UI at http://127.0.0.1:7432.
4. (Optional) Place your chosen GGUF at models/ and configure LOCAL_MODEL_PATH.
5. (Optional) Add a cloud API key in the Providers page for cloud-first complex work.

Configuration

All settings live in .env. Copy .env.example to .env to start:

Copy-Item .env.example .env

API keys are never stored in the database or logs.

Local Model Configuration

Relevant environment variables (see .env.example for the full list):

Variable	Default	Purpose
LOCAL_MODEL_ENABLED	true	Enable local inference
LOCAL_MODEL_RUNTIME	llamacpp	llamacpp, ollama, or openai-compatible
LOCAL_MODEL_REPOSITORY	unsloth/Qwen3.8-Flash-Next-GGUF	Upstream GGUF repository reference
LOCAL_MODEL_QUANT	UD-Q4_K_XL	Preferred quantization
LOCAL_MODEL_PATH	models/zevora-4b-thinking.gguf	Path to the GGUF file
LOCAL_MODEL_NAME	qwen3.8-flash-next	Model identifier shown in the UI/routing
LOCAL_MODEL_DISPLAY_NAME	Qwen3.8-Flash-Next	Human-readable name
LOCAL_MODEL_CONTEXT_LENGTH	8192	Context window
LOCAL_MODEL_MAX_TOKENS	2048	Max output tokens
LOCAL_MODEL_THREADS	0	CPU threads (0 = auto)
LOCAL_MODEL_GPU_LAYERS	0	GPU offload layers
LOCAL_MODEL_BATCH_SIZE	512	Prompt processing batch size
LOCAL_MODEL_TEMPERATURE	0.4	Sampling temperature

Cloud Providers

OpenAI, Anthropic, Gemini, DeepSeek, xAI, and NVIDIA are supported out of the box; custom OpenAI-compatible providers can be registered via the Providers page or config/providers.json. Keys are resolved only at request time and never stored in manifests, logs, or the database.

MCP / Tools

Filesystem, Git, and terminal operations are routed through a constrained local MCP gateway scoped to the selected workspace. Read-only operations follow workspace preferences; mutations and risky commands require explicit approval. Paths outside the selected workspace are blocked even when approval is granted. See docs/MCP_TOOLS.md.

Security Model

• Local GGUF files are verified by SHA-256 before loading when a reference exists.
• Cloud provider base URLs are validated to block SSRF to loopback/link-local/metadata hosts.
• Workspace boundaries are enforced even for approved actions.
• Telemetry and logs are redacted of credentials.
• See SECURITY.md.

Memory / Cache / Experience

These stay local by default (SQLite under data/). EXPERIENCE_LOGGING, MEMORY_ENABLED, and CACHE_ENABLED control them. The cache is keyed with a model signature so responses for one model/quantization are never replayed for another.

Project Context

project_discovery reports frameworks, languages, package manager, and a bounded file tree for the selected workspace. Local knowledge, memory, and project index enrich context before inference; an exact prompt+project match can answer from cache without inference.

Adaptive Routing

AdaptiveHybridRouter picks candidates by capability, cost, history, health, context window, required tools, and task complexity. In AUTO mode, routine coding/debugging work is local-first; complex, architectural, migration, vision, and long-context work is cloud-first. LOCAL_ONLY and CLOUD_ONLY constrain the pool explicitly.

Verification

Approved project actions produce authoritative observations. Failed verification is recorded; a new, explicitly approved action is required before code can change again.

Hardware Requirements

Local inference runs on CPU; GPU offload is optional. The quantization you select should match your hardware:

Quantization class	Typical use
Smaller (Q4_K_M-class)	Low RAM/VRAM, CPU-only, or constrained devices
Larger (UD-Q4_K_XL/higher)	More RAM/VRAM, higher quality at slower speed

Choose the quantization that fits your RAM, VRAM, GPU, CPU, target context length, quality, and speed trade-offs.

Model Selection

Different quantizations exist for the same model. There is no single "best" quantization — it depends on your hardware and goals. Use the model card on unsloth/Qwen3.8-Flash-Next-GGUF to pick the right one, and set LOCAL_MODEL_QUANT and LOCAL_MODEL_PATH in .env.

Development

python -m pip install -r requirements.txt
python -m pytest -q

Testing

The test suite (tests/) covers the gateway, providers, routing, local intelligence, MCP tools, memory, storage, CLI, and security. Run with:

python -m pytest

Hugging Face Distribution

This repository is a sanitized source distribution of ZEVORA for discoverability on Hugging Face. It does not include:

• model weights (GGUF files),
• runtime databases (data/),
• local caches, logs, or user workspaces,
• .env files or any credentials,
• virtual environments or build artifacts.

Only files appropriate for public distribution are published.

GitHub Repository

Primary development happens on GitHub:

• Repository: https://github.com/dani12po/zevora
• Issues / PRs: please open them on GitHub.

License

ZEVORA source code is licensed under the MIT License — see LICENSE. This repository mirrors the GitHub source of truth.

Third-Party Models / Attribution

ZEVORA references, but does not redistribute or claim ownership of, the following third-party assets:

• Qwen3.8-Flash-Next — model weights and its license remain with their respective owners (Qwen / Alibaba Cloud, and the upstream model card).
• Unsloth GGUF distribution — unsloth/Qwen3.8-Flash-Next-GGUF is owned by its author and redistributed by them under their posted license; we reference it, we do not mirror it.

Please refer to the upstream model cards for the exact model licensing terms. We do not invent or re-license these models.

Limitations

• Local inference quality and speed depend on your hardware and chosen quantization.
• Local context is bounded (LOCAL_MODEL_CONTEXT_LENGTH).
• Vision and very large/long-context or complex-architecture work is cloud-first and therefore requires a configured cloud provider.
• Custom runtime providers are not an OS sandbox: approved code retains the host user's filesystem and network privileges. Trust only code from known sources.