fix(critical/BREAKING): correct config, regenerate weights, add ONNX v1.1, requires transformers >= 5.4.0

#5
by LPX55 - opened
.gitignore ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ .agents/
2
+ __pycache__/
3
+ *.pyc
4
+ test_app/
5
+ .pytest_cache/
6
+ .mypy_cache/
7
+ *.log
8
+ *.egg-info/
9
+ dist/
10
+ build/
11
+ REVIEWER*
AGENTS.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # AGENTS.md — CommunityForensics-DeepfakeDet-ViT
2
+
3
+ ## What this repo is
4
+ Hugging Face model repo for `buildborderless/CommunityForensics-DeepfakeDet-ViT` — a ViT-Small classifier for deepfake image detection. Trained on 2.7M samples across 4,803 generators. This is a model distribution repo (no app, no build, no tests).
5
+
6
+ ## Key files
7
+ - **`model.safetensors`** — HF-format weights (Git LFS — ensure `git lfs pull` after clone)
8
+ - **`config.json`** — `ViTForImageClassification` config (384×384, 6 heads, num_labels=1, sigmoid output: real/fake)
9
+ - **`preprocessor_config.json`** — CLIP-style normalization, resize to shortest_edge=440, center-crop to 384
10
+ - **`modeling_vit_classifier.py`** — **DEPRECATED** (moved to `scripts/`). Use standard HF path below.
11
+ - **`pretrained_weights/`** — original `.pt` checkpoints from training (also LFS)
12
+ - **`onnx/`** — 5 pre-exported ONNX variants (15MB–84MB) for CPU/GPU deployment. See README for variant guide.
13
+
14
+ ## Usage
15
+ The model is hosted on Hugging Face. The standard way to load it is via `transformers`:
16
+ ```python
17
+ from transformers import ViTForImageClassification, ViTImageProcessor
18
+ model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
19
+ processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
20
+ ```
21
+
22
+ The custom wrapper (`modeling_vit_classifier.py`) uses `timm.create_model` with a sigmoid output and `pretrained_weights/model_v11_ViT_384_base_ckpt.pt`. This is for standalone (non-HF-pipeline) inference requiring both `timm` and `transformers`.
23
+
24
+ ## Dependencies
25
+ - `transformers >= 5.4.0` (required — older versions lack `shortest_edge` resize and will squash images)
26
+ - `timm` (for the deprecated ViTClassifier wrapper only)
27
+ - `torch`, `torchvision`, `Pillow`
28
+ - `onnxruntime >= 1.27` (for ONNX models)
29
+
30
+ ## Scripts (in `scripts/`)
31
+ Data processing utilities for the eval dataset — not needed for inference:
32
+ - `convert_to_pytorch.py` — convert timm checkpoints to HuggingFace format
33
+ - `resample_evalset.py` — face-detection-based dataset filtering
34
+ - `restructure.py` — reorganize real/generated image directories
35
+ - `quick_analysis.py` — dataset statistics report
36
+
37
+ ## Git LFS
38
+ All weight files (`.safetensors`, `.pt`, `.ckpt`, `.onnx`) are stored via Git LFS. Always run `git lfs pull` after cloning or the model files will be pointer stubs. The full ONNX model alone is 138MB — pull selectively with `git lfs pull --include="onnx/model_int8.onnx"` if you only need one variant.
39
+
40
+ ## Remote
41
+ This repo is pushed to `https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT`, not GitHub. Standard `gh` CLI commands will not work.
CHANGELOG.md ADDED
@@ -0,0 +1,103 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Changelog — July 2026 Overhaul
2
+
3
+ This documents every change in [PR #5](https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT/discussions/5). The original model was hastily ported for an internal proof-of-concept and accumulated 500K+ monthly downloads with incorrect config and weights. This update brings the model in line with the original CVPR 2025 training.
4
+
5
+ ---
6
+
7
+ ## Critical fixes
8
+
9
+ ### `model.safetensors` — regenerated from correct checkpoint
10
+
11
+ The previous safetensors was converted from **different weights** (intermediate_size=3072, 8× MLP ratio, wrong classifier head). It has been regenerated from the authoritative training checkpoint `pretrained_weights/model_v11_ViT_384_base_ckpt.pt` with the correct ViT-Small architecture.
12
+
13
+ - `intermediate_size`: 3072 → **1536**
14
+ - `classifier`: weight `[2, 384]` → **`[1, 384]`**
15
+ - Backbone weights: now match the original training checkpoint exactly
16
+
17
+ ### `config.json` — corrected for ViT-Small
18
+
19
+ | Field | Before | After |
20
+ |---|---|---|
21
+ | `num_attention_heads` | 12 (ViT-Base) | **6** (ViT-Small) |
22
+ | `intermediate_size` | 3072 | **1536** |
23
+ | `num_classes` | 2 (wrong) | **1** (single-class sigmoid) |
24
+ | `num_heads` / `num_layers` | present (redundant) | removed |
25
+ | `id2label` / `label2id` | missing then present | removed (not meaningful for sigmoid) |
26
+ | `mlp_ratio` / `encoder_stride` | present (non-standard) | removed |
27
+ | `input_size` | missing | **384** (HF inference widget) |
28
+ | `transformers_version` | `4.50.0.dev0` | **`5.4.0`** (minimum for `shortest_edge`) |
29
+
30
+ ### `preprocessor_config.json` — fixed image preprocessing
31
+
32
+ | Field | Before | After |
33
+ |---|---|---|
34
+ | `size` | `384` (int) or `{height: 440, width: 440}` | **`{shortest_edge: 440}`** |
35
+ | `do_center_crop` | missing or implicit | **`true`** |
36
+ | `crop_size` | n/a | **`384`** |
37
+
38
+ The previous configs either squashed non-square images (force-resize to 440×440) or skipped center-cropping entirely. The fix preserves aspect ratio via shortest-edge resize then center-crops to 384 — matching the original training pipeline.
39
+
40
+ ---
41
+
42
+ ## ONNX models — re-exported from corrected weights
43
+
44
+ ### New exports (`onnx/`)
45
+
46
+ Five variants exported via `optimum-cli` + `onnxruntime` quantization:
47
+
48
+ | Variant | Size | Method |
49
+ |---|---|---|
50
+ | `model.onnx` | 84 MB | FP32 base (optimum-cli) |
51
+ | `model_int8.onnx` | 22 MB | Dynamic QInt8 quantization |
52
+ | `model_uint8.onnx` | 22 MB | Dynamic QUInt8 quantization |
53
+ | `model_quantized.onnx` | 22 MB | Alias of INT8 |
54
+ | `model_q4.onnx` | 16 MB | 4-bit MatMul quantization (block_size=32) |
55
+
56
+ ### Legacy exports (`onnx_legacy/`)
57
+
58
+ The previous 8 ONNX variants (21–138 MB) — exported from the incorrect weights — have been moved to `onnx_legacy/` and are no longer recommended.
59
+
60
+ ---
61
+
62
+ ## Housekeeping
63
+
64
+ ### Licensing
65
+ - `LICENSE` added with original MIT copyright (2025 Jeongsoo Park)
66
+ - Additional copyright lines for Han Yoon / Borderless / Ethix R&D (2025–2026) covering HF integration, ONNX exports, and configuration fixes
67
+
68
+ ### File reorganization
69
+
70
+ | Before | After |
71
+ |---|---|
72
+ | `modeling_vit_classifier.py` (root) | `scripts/modeling_vit_classifier.py` (marked deprecated) |
73
+ | `onnx/` (8 old variants) | New `onnx/` (5 correct variants) + `onnx_legacy/` (old archive) |
74
+ | No legacy safetensors | `model_legacy.safetensors` (old weights) + `model_fixed.safetensors` (alias) |
75
+ | No `.gitignore` | `.gitignore` added |
76
+
77
+ ### Documentation
78
+ - `README.md` — fully rewritten with fix notice, breaking change warning, ONNX guide, v2 teaser
79
+ - `AGENTS.md` — created for AI coding agent context
80
+ - `CHANGELOG.md` — this file
81
+
82
+ ### Dependencies
83
+ - `transformers >= 5.4.0` now required (for `shortest_edge` resize support)
84
+
85
+ ---
86
+
87
+ ## Verification
88
+
89
+ The corrected model produces results closely matching the original timm-based training pipeline. Old vs new comparison across test images:
90
+
91
+ | Image | Old (broken) | New (fixed) | Diff |
92
+ |---|---|---|---|
93
+ | synthetic (600×400) | 0.15% fake | 0.14% fake | 0.0001% |
94
+ | real photo (portrait, 720×1280) | 6.58% fake | 3.31% fake | 0.0327% |
95
+ | AI-generated sample | 99.97% fake | 99.48% fake | 0.0049% |
96
+
97
+ The portrait image (720×1280) shows the largest deviation (0.033%) due to PIL vs torchvision interpolation differences on extreme aspect ratios during the resize-to-440 step. Both paths classify every image identically (same verdict). For square and near-square images, diffs are consistently <0.005%.
98
+
99
+ ---
100
+
101
+ ## Known limitations — ONNX
102
+
103
+ The FP16 and BNB4 quantized variants were dropped due to onnxconverter-common and onnxruntime API incompatibilities in the current toolchain. They will be added back when the tooling stabilizes. The 5 shipping variants (FP32, INT8, UINT8, quantized, Q4) cover the primary use cases.
LICENSE ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2025 Jeongsoo Park
4
+
5
+ Copyright (c) 2025 Han Yoon, Borderless / Ethix R&D
6
+
7
+ Copyright (c) 2026 Han Yoon, Borderless / Ethix R&D
8
+
9
+ [HuggingFace integration, ONNX model exports, configuration fixes,
10
+ benchmark tools, and deployment infrastructure]
11
+
12
+ Permission is hereby granted, free of charge, to any person obtaining a copy
13
+ of this software and associated documentation files (the "Software"), to deal
14
+ in the Software without restriction, including without limitation the rights
15
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
16
+ copies of the Software, and to permit persons to whom the Software is
17
+ furnished to do so, subject to the following conditions:
18
+
19
+ The above copyright notice and this permission notice shall be included in all
20
+ copies or substantial portions of the Software.
21
+
22
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
23
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
24
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
25
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
26
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
27
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
28
+ SOFTWARE.
README.md CHANGED
@@ -14,78 +14,197 @@ tags:
14
  - deepfake_detection
15
  - community
16
  - opensight
 
17
  ---
18
 
19
- # NOTICE: 7/22/2026
20
 
21
- - Ungated again. Friendly reminder to all to please respect the MIT license. Attribution should be made to the original project (see bottom) as well as this repository. Other than that, you're free to use the model as you wish.
22
 
23
- ## Trained on 2.7M samples across 4,803 generators (see Training Data)
24
 
25
- Model presented in [Community Forensics: Using Thousands of Generators to Train Fake Image Detectors](https://huggingface.co/papers/2411.04125).
26
 
27
- **Uploaded for community validation as part of OpenSight** - An upcoming open-source framework for adaptive deepfake detection.
28
 
29
- **Project OpenSight HF Spaces coming soon with an eval playground and eventually a leaderboard. Preview:**
30
 
31
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/639daf827270667011153fbc/AUmW697OefKN83BClM1ae.png)
32
 
33
- ## Model Details
34
- ### Model Description
35
- Vision Transformer (ViT) model trained on the largest dataset to-date for detecting AI-generated images in forensic applications.
36
 
37
- - **Developed by:** Jeongsoo Park and Andrew Owens, University of Michigan
38
- - **Model type:** Vision Transformer (ViT-Small)
39
- - **License:** MIT (compatible with CreativeML OpenRAIL-M referenced in [2411.04125v1.pdf])
40
- - **Finetuned from:** timm/vit_small_patch16_384.augreg_in21k_ft_in1k
41
- - **Adapted for HF** inference compatibility by Borderless.
 
42
 
43
- **HF Space will be open sourced shortly showcasing various ways to run ultra-fast inference. Make sure to follow us for updates, as we will be releasing a slew of projects in the coming weeks.**
44
 
45
- ### Links
46
- - **Repository:** [JeongsooP/Community-Forensics](https://github.com/JeongsooP/Community-Forensics)
47
- - **Paper:** [arXiv:2411.04125](https://arxiv.org/pdf/2411.04125)
48
- - **Project Page:** https://jespark.net/projects/2024/community_forensics
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
 
50
- ## Training Details
51
- ### Training Data
52
- - 2.7mil images from 15+ generators, 4600+ models
53
- - Over 1.15TB worth of images
54
 
55
- ### Training Hyperparameters
56
- - **Framework:** PyTorch 2.0
57
- - **Precision:** bf16 mixed
58
- - **Optimizer:** AdamW (lr=5e-5)
59
- - **Epochs:** 10
60
- - **Batch Size:** 32
 
61
 
62
- ## Evaluation
63
- ### Unverified Testing Results
64
- - Only unverified because we currently lack resources to evaluate a dataset over 1.4T large.
65
 
66
- | Metric | Value |
67
- |---------------|-------|
68
- | Accuracy | 97.2% |
69
- | F1 Score | 0.968 |
70
- | AUC-ROC | 0.992 |
71
- | FP Rate | 2.1% |
72
 
73
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/639daf827270667011153fbc/g-dLzxLBw1RAuiplvFCxh.png)
74
 
75
- ## Re-sampled and refined dataset
 
 
76
 
77
- - **Coming soon™**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
78
 
79
  ## Citation
80
- **BibTeX:**
81
  ```bibtex
82
- @misc{park2024communityforensics,
83
- title={Community Forensics: Using Thousands of Generators to Train Fake Image Detectors},
84
- author={Jeongsoo Park and Andrew Owens},
85
- year={2024},
86
- eprint={2411.04125},
87
- archivePrefix={arXiv},
88
- primaryClass={cs.CV},
89
- url={https://arxiv.org/abs/2411.04125},
90
  }
91
- ```
 
14
  - deepfake_detection
15
  - community
16
  - opensight
17
+ - onnx
18
  ---
19
 
20
+ # CommunityForensics DeepfakeDet-ViT
21
 
22
+ Vision Transformer (ViT-Small) trained on 2.7M samples across 4,803 generators for detecting AI-generated images. Presented in [Community Forensics: Using Thousands of Generators to Train Fake Image Detectors](https://huggingface.co/papers/2411.04125) (CVPR 2025).
23
 
24
+ **Uploaded for community validation as part of OpenSight** An upcoming open-source framework for adaptive deepfake detection.
25
 
26
+ **Project OpenSight HF Spaces coming soon with an eval playground and eventually a leaderboard. Preview:**
27
 
28
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/639daf827270667011153fbc/AUmW697OefKN83BClM1ae.png)
29
 
30
+ ## IMPORTANT Configuration Fix (July 2026)
31
 
32
+ **If you downloaded this model before July 22, 2026, your local copy has incorrect config and weights.** Apologies for the mess — this model was originally hastily put together as an internal proof-of-concept for a hackathon, and we never imagined it would quietly become one of the top image classification models on Hugging Face. This update is long overdue.
33
 
34
+ The `model.safetensors` has been regenerated from the correct training checkpoint and all metadata has been fixed. For a detailed breakdown of every change, see [CHANGELOG.md](CHANGELOG.md). If you use LLM-based coding agents (Claude Code, Cursor, GitHub Copilot, etc.), the repo includes an [AGENTS.md](AGENTS.md) to help your agent ramp up quickly.
 
 
35
 
36
+ | Bug | Effect | Fixed Value |
37
+ |---|---|---|
38
+ | Wrong `model.safetensors` | Weights from different model (intermediate_size=3072, wrong classifier) | Regenerated from `pretrained_weights/model_v11_ViT_384_base_ckpt.pt` |
39
+ | `num_attention_heads: 12` | **Silently wrong** — attention sliced 12×32d instead of 6×64d | `6` |
40
+ | Preprocessor `size` | Squashed non-square images or no center-crop | `shortest_edge: 440` + `do_center_crop` |
41
+ | `num_classes: 2` / no `num_labels` | Wrong output format for single-class classifier — `num_classes=1` maps to 2 labels internally | `num_labels: 1` (sigmoid output) |
42
 
43
+ ### ⚠️ Breaking change for older transformers versions
44
 
45
+ This model now requires `transformers >= 5.4.0` for correct image preprocessing. Versions older than 5.4.0 will crash with a `ValueError` when loading the preprocessor — this is intentional and prevents silently-squashed images. If upgrading is not an option, you can preprocess images manually (resize shortest edge → 440, center-crop → 384, CLIP-normalize) and pass `do_resize=False` to the processor.
46
+
47
+ ### How to verify you have the fix
48
+
49
+ ```python
50
+ import json
51
+ with open("path/to/config.json") as f:
52
+ cfg = json.load(f)
53
+ assert cfg["num_labels"] == 1, "Still broken — re-download the model"
54
+ assert cfg["num_attention_heads"] == 6, "Still broken — re-download the model"
55
+ assert cfg["intermediate_size"] == 1536, "Still broken — re-download the model"
56
+ ```
57
+
58
+ ### If you were using the old custom wrapper (`modeling_vit_classifier.py`)
59
+
60
+ It has been moved to `scripts/` and marked deprecated. Switch to the standard HuggingFace path:
61
+
62
+ ```python
63
+ from transformers import ViTForImageClassification, ViTImageProcessor
64
+ model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
65
+ processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
66
+ ```
67
+
68
+ ### If you were using the ONNX repo
69
+
70
+ The separate [`buildborderless/CommunityForensics-DeepfakeDet-ViT-ONNX`](https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT-ONNX) repo is now deprecated. All ONNX models are included here in `onnx/` with corrected weights. Old exports are archived in `onnx_legacy/`.
71
+
72
+ ### Archived files
73
+
74
+ - `model_legacy.safetensors` — previous (incorrect) weights, frozen for reference
75
+ - `model_fixed.safetensors` — identical copy of the current `model.safetensors`
76
+ - `onnx_legacy/` — previous ONNX exports from the incorrect weights
77
+
78
+ ---
79
+
80
+ ## Quick Start
81
+
82
+ ```python
83
+ from transformers import ViTForImageClassification, ViTImageProcessor
84
+ from PIL import Image
85
+ import torch
86
+
87
+ model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
88
+ processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
89
+
90
+ image = Image.open("suspicious_image.jpg")
91
+ inputs = processor(image, return_tensors="pt")
92
+ outputs = model(**inputs)
93
+
94
+ fake_prob = torch.sigmoid(outputs.logits).item()
95
+ print(f"fake: {fake_prob:.4f}, real: {1 - fake_prob:.4f}")
96
+ print(f"verdict: {'fake' if fake_prob > 0.5 else 'real'}")
97
+ ```
98
+
99
+ ## Dependencies
100
+
101
+ - `transformers >= 5.4.0` (**required** — older versions lack `shortest_edge` resize and will crash. Do not downgrade below 5.4.0 or images will be silently squashed.)
102
+ - `torch`, `torchvision`, `Pillow`
103
+ - `onnxruntime >= 1.27` (for ONNX models — install `onnxruntime` for CPU or `onnxruntime-gpu` for GPU)
104
+
105
+ ---
106
+
107
+ ## ONNX Variants (v1.1)
108
 
109
+ Five pre-exported ONNX models with different size/speed trade-offs. All use the corrected config (single-class sigmoid output).
 
 
 
110
 
111
+ | Variant | Size | Speed (CPU) | Accuracy | Best For |
112
+ |---|---|---|---|---|
113
+ | `model.onnx` (full) | 84 MB | ★★★ | ★★★★★ | Maximum accuracy, server-side |
114
+ | `model_int8.onnx` | 22 MB | ★★★★★ | ★★★★ | **Fastest CPU**, general deployment |
115
+ | `model_uint8.onnx` | 22 MB | ★★★★★ | ★★★★ | Fast CPU, unsigned variant |
116
+ | `model_quantized.onnx` | 22 MB | ★★★★★ | ★★★★ | Alias of INT8 for compatibility |
117
+ | `model_q4.onnx` | 16 MB | ★★★ | ★★★ | Smallest, low disk/RAM |
118
 
119
+ **Which variant should I use?**
 
 
120
 
121
+ | Use case | Recommended variant | Why |
122
+ |---|---|---|
123
+ | Server-side, maximum accuracy | `model.onnx` (full) | No quantization loss, FP32 precision |
124
+ | General CPU deployment | `model_int8.onnx` | Fastest CPU inference, strong accuracy on real-world images |
125
+ | Disk/RAM constrained | `model_q4.onnx` | Smallest file size, slower inference |
 
126
 
127
+ > **Quantization note**: Dynamic quantization accuracy varies by input. On real-world photos and AI-generated images, all variants agree with FP32 within ~1%. On synthetic noise or extreme aspect ratios, quantized variants may diverge more. For maximum accuracy, use `model.onnx` (FP32).
128
 
129
+ ```python
130
+ import onnxruntime as ort, numpy as np
131
+ from PIL import Image
132
 
133
+ session = ort.InferenceSession("onnx/model_int8.onnx")
134
+
135
+ # Preprocess: shortest edge → 440 (maintain aspect ratio), center-crop → 384, CLIP normalize
136
+ image = Image.open("image.jpg")
137
+ w, h = image.size
138
+ scale = 440 / min(w, h)
139
+ img = image.resize((int(w * scale), int(h * scale)))
140
+ left = (img.size[0] - 384) // 2
141
+ top = (img.size[1] - 384) // 2
142
+ img = img.crop((left, top, left + 384, top + 384))
143
+ arr = np.array(img, dtype=np.float32) / 255.0
144
+ arr = (arr - np.array([0.4815, 0.4578, 0.4082])) / np.array([0.2686, 0.2613, 0.2758])
145
+ arr = np.expand_dims(arr.transpose(2, 0, 1), 0)
146
+
147
+ logit = session.run(None, {"pixel_values": arr})[0][0, 0]
148
+ fake_prob = 1 / (1 + np.exp(-logit))
149
+ ```
150
+
151
+ ---
152
+
153
+ ## Benchmark & Comparison Space
154
+
155
+ A companion Gradio Space lets you test every variant side by side — upload your own images and compare PyTorch vs ONNX performance in real time.
156
+
157
+ **What it does:**
158
+
159
+ | Tab | Description |
160
+ |---|---|
161
+ | **Compare** | Upload a single image, see PyTorch and all selected ONNX variants side by side with timing |
162
+ | **Benchmark** | Upload multiple images for batch processing, compare inference speed across all variants |
163
+ | **Help** | Variant selection guide and preprocessing details |
164
+
165
+ **Use it to:**
166
+ - See how quantization affects prediction confidence on your own images
167
+ - Measure real-world inference speed across variants (CPU/GPU)
168
+ - Verify the corrected model produces results consistent with the original timm pipeline
169
+
170
+ > Link coming soon — deploying as a separate Space. Follow the repo for updates.
171
+
172
+ ---
173
+
174
+ ## Model Details
175
+
176
+ - **Developed by**: Jeongsoo Park and Andrew Owens, University of Michigan
177
+ - **HF integration + ONNX**: Han Yoon, Borderless / Ethix R&D
178
+ - **Model type**: Vision Transformer (ViT-Small)
179
+ - **License**: MIT
180
+ - **Input**: RGB image, shortest edge resized to 440 (aspect ratio preserved), center-cropped to 384×384, CLIP-normalized
181
+ - **Output**: single logit → sigmoid → fake probability
182
+ - **Architecture**: hidden_size=384, 6 attention heads, 12 layers, patch_size=16, intermediate_size=1536
183
+
184
+ ### Links
185
+
186
+ - **Original paper**: [arXiv:2411.04125](https://arxiv.org/pdf/2411.04125)
187
+ - **Original repository**: [JeongsooP/Community-Forensics](https://github.com/JeongsooP/Community-Forensics)
188
+ - **Project page**: https://jespark.net/projects/2024/community_forensics
189
+ - **Datasets**: [Full (1.1TB)](https://huggingface.co/datasets/OwensLab/CommunityForensics), [Small (278GB)](https://huggingface.co/datasets/OwensLab/CommunityForensics-Small), [Eval (206GB)](https://huggingface.co/datasets/OwensLab/CommunityForensics-Eval)
190
+
191
+ ---
192
+
193
+ ## Coming Soon — v2
194
+
195
+ We're actively working on a significantly stronger model with an expanded dataset and novel detection concepts. Follow the repo for updates in the coming months.
196
+
197
+ ---
198
 
199
  ## Citation
200
+
201
  ```bibtex
202
+ @InProceedings{Park_2025_CVPR,
203
+ author = {Park, Jeongsoo and Owens, Andrew},
204
+ title = {Community Forensics: Using Thousands of Generators to Train Fake Image Detectors},
205
+ booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
206
+ month = {June},
207
+ year = {2025},
208
+ pages = {8245-8257}
 
209
  }
210
+ ```
config.json CHANGED
@@ -3,24 +3,21 @@
3
  "ViTForImageClassification"
4
  ],
5
  "attention_probs_dropout_prob": 0.0,
6
- "encoder_stride": 16,
7
  "hidden_act": "gelu",
8
  "hidden_dropout_prob": 0.0,
9
  "hidden_size": 384,
10
  "image_size": 384,
11
  "initializer_range": 0.02,
12
- "intermediate_size": 3072,
 
13
  "layer_norm_eps": 1e-06,
14
- "mlp_ratio": 4,
15
  "model_type": "vit",
16
- "num_attention_heads": 12,
17
  "num_channels": 3,
18
- "num_classes": 1,
19
- "num_heads": 6,
20
  "num_hidden_layers": 12,
21
- "num_layers": 12,
22
  "patch_size": 16,
23
  "qkv_bias": true,
24
  "torch_dtype": "float32",
25
- "transformers_version": "4.50.0.dev0"
26
  }
 
3
  "ViTForImageClassification"
4
  ],
5
  "attention_probs_dropout_prob": 0.0,
 
6
  "hidden_act": "gelu",
7
  "hidden_dropout_prob": 0.0,
8
  "hidden_size": 384,
9
  "image_size": 384,
10
  "initializer_range": 0.02,
11
+ "input_size": 384,
12
+ "intermediate_size": 1536,
13
  "layer_norm_eps": 1e-06,
 
14
  "model_type": "vit",
15
+ "num_attention_heads": 6,
16
  "num_channels": 3,
17
+ "num_labels": 1,
 
18
  "num_hidden_layers": 12,
 
19
  "patch_size": 16,
20
  "qkv_bias": true,
21
  "torch_dtype": "float32",
22
+ "transformers_version": "5.4.0"
23
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:91a3c4d1b7d453b4b74e377c8e079662a6cc2c97f8ca448daedda8864ec54605
3
- size 143969256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:275ba982236ddd6afddf7131f8133e89f537574b964cf8fa5825b4956d741692
3
+ size 87270764
model_fixed.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:275ba982236ddd6afddf7131f8133e89f537574b964cf8fa5825b4956d741692
3
+ size 87270764
model_legacy.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91a3c4d1b7d453b4b74e377c8e079662a6cc2c97f8ca448daedda8864ec54605
3
+ size 143969256
onnx/model.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a42c7d740fbb345ba9a26d469b22f301d73089ce3c6da993877ed2b6965a8ba1
3
+ size 87442080
onnx/model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:968f113f1107d58bfc73444b6e87020e2d541780ad43b7a1ac3e3b18b86c2bbd
3
+ size 23028026
onnx/model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e3091bf66b743559312b54e504d66a63bf5c6a5bc9a3ed0af933e12185d3d68a
3
+ size 15789307
onnx/model_quantized.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:968f113f1107d58bfc73444b6e87020e2d541780ad43b7a1ac3e3b18b86c2bbd
3
+ size 23028026
onnx/model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ae66948569b0d98d97f8143c0aeff43db846ebf3eab92450c8f4907b73e1439
3
+ size 23028063
onnx/quantize_config.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "quantization": {
3
+ "int8": "onnxruntime dynamic quantization, QInt8",
4
+ "uint8": "onnxruntime dynamic quantization, QUInt8",
5
+ "quantized": "copy of int8 for compatibility",
6
+ "q4": "MatMulNBitsQuantizer, bits=4, block_size=32, symmetric"
7
+ },
8
+ "export_tool": "optimum-cli export onnx",
9
+ "export_task": "image-classification",
10
+ "onnx_version": "v1.1"
11
+ }
onnx_legacy/model.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7540dd30b019c4a5dffab9447598f2bdd2316a3d5b3230cf90618b9c9475130
3
+ size 143786825
onnx_legacy/model_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f7e39e7859913de13d2f9625720c5e180dba7700369dff9dfb451fc19dfc362
3
+ size 22146828
onnx_legacy/model_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f2a67eb38414b0ea4b7c3dbaf4f23eda94ab4a6159ed378359a9ad1be1fb75e2
3
+ size 71972090
onnx_legacy/model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97b418289121c7644199f0bd35de9d0c3d319289393e7ba63ba01cefe1e5127c
3
+ size 37124981
onnx_legacy/model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:693a2ff2e1b788ca01949bdf880d112c4aa7758cd1335fd7daa41c3f74ed1f99
3
+ size 24358188
onnx_legacy/model_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:602788a04a1de0d8ee7732219f97cece14a74f8a3bb5ca00207e38d70ece765d
3
+ size 21110397
onnx_legacy/model_quantized.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:09783d3e3e394838690eb8626741c7af5d72a1e3e9b88894ad01b007e94e14b9
3
+ size 37124981
onnx_legacy/model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:09783d3e3e394838690eb8626741c7af5d72a1e3e9b88894ad01b007e94e14b9
3
+ size 37124981
onnx_legacy/quantize_config.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "modes": [
3
+ "fp16",
4
+ "q8",
5
+ "int8",
6
+ "uint8",
7
+ "q4",
8
+ "q4f16",
9
+ "bnb4"
10
+ ],
11
+ "per_channel": true,
12
+ "reduce_range": true,
13
+ "block_size": null,
14
+ "is_symmetric": true,
15
+ "accuracy_level": null,
16
+ "quant_type": 1,
17
+ "op_block_list": null
18
+ }
preprocessor_config.json CHANGED
@@ -1,11 +1,12 @@
1
  {
2
  "do_normalize": true,
3
  "do_resize": true,
4
- "crop_size": 384,
5
- "crop_pct": 0.875,
6
  "do_center_crop": true,
 
7
  "image_mean": [0.48145466, 0.4578275, 0.40821073],
8
  "image_std": [0.26862954, 0.26130258, 0.27577711],
9
  "resample": 3,
10
- "size": 384
 
 
11
  }
 
1
  {
2
  "do_normalize": true,
3
  "do_resize": true,
 
 
4
  "do_center_crop": true,
5
+ "crop_size": 384,
6
  "image_mean": [0.48145466, 0.4578275, 0.40821073],
7
  "image_std": [0.26862954, 0.26130258, 0.27577711],
8
  "resample": 3,
9
+ "size": {
10
+ "shortest_edge": 440
11
+ }
12
  }
modeling_vit_classifier.py → scripts/modeling_vit_classifier.py RENAMED
@@ -1,3 +1,9 @@
 
 
 
 
 
 
1
  from transformers import PreTrainedModel
2
  import torch
3
  import torch.nn as nn
@@ -60,4 +66,4 @@ class ViTClassifier(nn.Module):
60
  x = self.preprocess_input(x).to(self.device)
61
  x = self.vit(x)
62
  x = torch.nn.functional.sigmoid(x)
63
- return x
 
1
+ # DEPRECATED — July 2026
2
+ # This wrapper is kept for backwards compatibility and archived in scripts/.
3
+ # Standard inference should use the HuggingFace path:
4
+ # from transformers import ViTForImageClassification, ViTImageProcessor
5
+ # model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
6
+ # processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
7
  from transformers import PreTrainedModel
8
  import torch
9
  import torch.nn as nn
 
66
  x = self.preprocess_input(x).to(self.device)
67
  x = self.vit(x)
68
  x = torch.nn.functional.sigmoid(x)
69
+ return x