faxenoff commited on
Commit
8337da3
Β·
verified Β·
1 Parent(s): 0eeaf19

compiled artefacts from win_x64 (sm_120)

Browse files
README.md CHANGED
@@ -113,9 +113,16 @@ Measured on one laptop: Intel Core Ultra 9 275HX / NVIDIA RTX 5060 Laptop, batch
113
  | lane | per batch | throughput | per word |
114
  |---|--:|--:|--:|
115
  | **TensorRT FP16, RTX 5060 Laptop** | **3.60 ms** | **17 790 words/s** | 0.056 ms |
116
- | **OpenVINO INT8, CPU** | **80.4 ms** | **796 words/s** | 1.26 ms |
 
 
117
  | ONNX Runtime FP32, CPU | 188 ms | 341 words/s | 2.93 ms |
118
 
 
 
 
 
 
119
  The INT8 CPU lane is the intended default β€” 800 words/sec is enough to filter a repository's whole
120
  harvested vocabulary in seconds without touching a GPU, and it is 2.3Γ— the unquantized ONNX path.
121
  The GPU lane exists for hosts that have spare VRAM anyway.
@@ -170,9 +177,9 @@ drops really was ballast.
170
 
171
  ## 7. What is in this repo
172
 
173
- - **OpenVINO INT8** β€” `code-daemon-denoise-v1-s_ov2026.2_{cpu,igpu_lnl}_int8_b64_s40.{xml,bin}` β€” the
174
  default lane (CPU) and an Intel iGPU build.
175
- - **OpenVINO INT4, NPU** β€” `code-daemon-denoise-v1-s_ov2026.2_npu_int4_b16_s40.{xml,bin}` β€” weight-only
176
  INT4 at batch 16 for Intel NPUs.
177
  - **TensorRT FP16** β€” `code-daemon-denoise-v1-s_{win_x64,linux_x64}_trt11.0_sm_120.engine`.
178
  - **TVM Vulkan** β€” `code-daemon-denoise-v1_{win_x64,linux_x64}_tvm0.25_vulkan.{dll,so}` β€” GPU fallback
 
113
  | lane | per batch | throughput | per word |
114
  |---|--:|--:|--:|
115
  | **TensorRT FP16, RTX 5060 Laptop** | **3.60 ms** | **17 790 words/s** | 0.056 ms |
116
+ | **OpenVINO INT8, iGPU** (OV 2026.3) | **56.1 ms** | **1 140 words/s** | 0.88 ms |
117
+ | **OpenVINO INT8, CPU** (OV 2026.3) | **75-83 ms** | **770-850 words/s** | 1.18-1.30 ms |
118
+ | OpenVINO INT4, NPU (OV 2026.3) | 57.8 ms | 277 words/s | 3.61 ms |
119
  | ONNX Runtime FP32, CPU | 188 ms | 341 words/s | 2.93 ms |
120
 
121
+ The NPU number is per batch 16, not 64 β€” it is a low-batch part, so its per-word cost is the
122
+ highest of the three even though its per-batch latency looks similar to the CPU's. Running all
123
+ three Intel devices at once yields ~87 % of the sum of their solo rates (they share one memory
124
+ controller), so a host with no discrete GPU can still denoise ~2 000 words/s.
125
+
126
  The INT8 CPU lane is the intended default β€” 800 words/sec is enough to filter a repository's whole
127
  harvested vocabulary in seconds without touching a GPU, and it is 2.3Γ— the unquantized ONNX path.
128
  The GPU lane exists for hosts that have spare VRAM anyway.
 
177
 
178
  ## 7. What is in this repo
179
 
180
+ - **OpenVINO INT8** β€” `code-daemon-denoise-v1-s_ov2026.3_{cpu,igpu_lnl}_int8_b64_s40.{xml,bin}` β€” the
181
  default lane (CPU) and an Intel iGPU build.
182
+ - **OpenVINO INT4, NPU** β€” `code-daemon-denoise-v1-s_ov2026.3_npu_int4_b16_s40.{xml,bin}` β€” weight-only
183
  INT4 at batch 16 for Intel NPUs.
184
  - **TensorRT FP16** β€” `code-daemon-denoise-v1-s_{win_x64,linux_x64}_trt11.0_sm_120.engine`.
185
  - **TVM Vulkan** β€” `code-daemon-denoise-v1_{win_x64,linux_x64}_tvm0.25_vulkan.{dll,so}` β€” GPU fallback
code-daemon-denoise-v1-s_ov2026.2_cpu_int8_b64_s40.bin β†’ code-daemon-denoise-v1-s_ov2026.3_cpu_int8_b64_s40.bin RENAMED
File without changes
code-daemon-denoise-v1-s_ov2026.2_cpu_int8_b64_s40.xml β†’ code-daemon-denoise-v1-s_ov2026.3_cpu_int8_b64_s40.xml RENAMED
The diff for this file is too large to render. See raw diff
 
code-daemon-denoise-v1-s_ov2026.2_igpu_lnl_int8_b64_s40.bin β†’ code-daemon-denoise-v1-s_ov2026.3_igpu_lnl_int8_b64_s40.bin RENAMED
File without changes
code-daemon-denoise-v1-s_ov2026.2_igpu_lnl_int8_b64_s40.xml β†’ code-daemon-denoise-v1-s_ov2026.3_igpu_lnl_int8_b64_s40.xml RENAMED
The diff for this file is too large to render. See raw diff
 
code-daemon-denoise-v1-s_ov2026.2_npu_int4_b16_s40.bin β†’ code-daemon-denoise-v1-s_ov2026.3_npu_int4_b16_s40.bin RENAMED
File without changes
code-daemon-denoise-v1-s_ov2026.2_npu_int4_b16_s40.xml β†’ code-daemon-denoise-v1-s_ov2026.3_npu_int4_b16_s40.xml RENAMED
The diff for this file is too large to render. See raw diff
 
manifest.json CHANGED
@@ -10,10 +10,10 @@
10
  ],
11
  "runtime_tags": {
12
  "trt": "trt11.0",
13
- "ov": "ov2026.2",
14
  "tvm": "tvm0.25",
15
  "mlx_dir": "model_gpu_mlx0.22"
16
  },
17
- "compiled_at": "2026-08-03T14:52:07Z",
18
  "compiled_by": "models/_compile"
19
  }
 
10
  ],
11
  "runtime_tags": {
12
  "trt": "trt11.0",
13
+ "ov": "ov2026.3",
14
  "tvm": "tvm0.25",
15
  "mlx_dir": "model_gpu_mlx0.22"
16
  },
17
+ "compiled_at": "2026-08-05T08:54:59Z",
18
  "compiled_by": "models/_compile"
19
  }