SecJudge

Neyman-Pearson security classifier for AI agent tool-use risk detection.

Built for DefenseClaw. Detects credential leaks, prompt injection, PII exposure, malicious commands, data exfiltration, and dangerous tool calls.

Security classification is a Neyman-Pearson problem: false positives (blocking legitimate agent actions) cause engineers to disable the security gate, which is worse than individual missed attacks. SecJudge is designed to maximize recall (attacks caught) subject to a controlled false positive rate.

4-phase training: Supervised Contrastive → CrossEntropy → RLCD → Isotonic Calibration.

Primary Metric: Recall at Fixed FPR

The right question for a security gate is not "what's the F1?" but:

"At a false positive rate engineers will tolerate (≤0.5%), how many attacks do we catch?"

Same-Domain (DefenseClaw Corpus, 1,465 cases)

FPR Budget	Recall (attacks caught)	What this means
≤ 0.1%	83%	Max 1 false block per 1,000 benign actions. Catches 83% of attacks.
≤ 0.5%	92%	Max 1 false block per 200 benign actions. Catches 92% of attacks.
≤ 1.0%	92%	Max 1 false block per 100 benign actions. Catches 92% of attacks.
≤ 5.0%	97%	Max 1 false block per 20 benign actions. Catches 97% of attacks.

Cross-Domain (zero training overlap)

Benchmark	FPR ≤ 0.5%	FPR ≤ 1%	FPR ≤ 5%
Rogue Coding Agent (332 cases)	40%	40%	9%
Gravitee PII (2,000 cases)	46%	46%	7%

These cross-domain results are on data the model has never seen during training.

Comparison: FPR at Fixed Recall

Recall Target	SecJudge FPR (DC)	Interpretation
90% recall	< 0.5%	Catch 90% of attacks with fewer than 3 false blocks per 508 benign
95% recall	< 2%	Catch 95% of attacks with fewer than 10 false blocks per 508 benign
99% recall	< 15%	Catch 99% of attacks — but 1 in 7 benign actions gets flagged

---

Secondary Metrics (F1, for comparison with other models)

Benchmark	SecJudge F1	Jev (TypeSafe Cloud) F1	Recall	Precision
DefenseClaw Corpus (1,465)	0.965	0.843	99.5%	93.6%
NVIDIA Nemotron Agentic IPI (1,272)	0.998	0.847	99.6%	100%
Lakera DEF CON (630)	0.958	0.618	91.9%	100%
Gravitee PII (2,000)	0.932	0.791	88.5%	98.5%
Rogue Coding Agent (332)	0.654	0.793	84.9%	55.3%

Note: F1 treats false positives and false negatives as equally bad. For security applications, recall at controlled FPR (above) is the correct metric.

---

Evaluation Benchmarks

All evaluation datasets have zero overlap with training data.

DefenseClaw Security Suite

The primary benchmark. 957 attacks + 508 benign across 6 sub-corpora:

• regex (595): Credentials (AWS keys, GitHub PATs, bearer tokens), dangerous commands, sensitive paths, C2 beacons, prompt injections, PII + curated benign hard negatives
• injection (160): Prompt injection variants — polite, formal, embedded in paragraphs, Unicode-obfuscated
• pii (160): SSN, passport, driver's license, phone, email, password, IP + spaced/obfuscated variants
• tool_injection (160): Malicious JSON tool-call payloads — destructive commands, exfiltration, credential theft
• exfil (160): Sensitive file access, webhook/C2 upload patterns, SSRF metadata endpoints
• toolcall (230): Semantic tool-call analysis — privileged Docker, kill, nmap, kubectl exec, git hooks

NVIDIA Nemotron-RL-Agentic-IPI-v1

Indirect prompt injection attacks hidden inside realistic enterprise documents across 9 domains (healthcare, IT helpdesk, ecommerce, legal, education, HR, customer service, project management, financial). 40 real tool schemas. A patient chart contains an injection that tricks the agent into sending a referral to an attacker-controlled address.

Lakera/b3-agent-security-benchmark-weak

Real adversarial attacks from a DEF CON-style security competition. Actual humans tried to break 10 AI agent applications using prompt injection, system prompt extraction, data exfiltration, and social engineering. 7 attack models. 3 defense difficulty levels.

rogue-security/coding-agent-security-benchmark

Tool-call JSON payloads from coding agents. 13 violation categories with 5 severity levels: credential theft, dangerous shell commands, PII exposure, malicious code injection, supply chain attacks.

gravitee-io/pii-detection-dataset

PII in realistic documents. 25 entity types: credit cards, US SSN, IP addresses, passwords, email, phone, IBAN, passport numbers. Contexts include medical records, financial transactions, SWIFT messages, XML reports.

---

Model Details

Spec	Value
Architecture	ModernBERT-large + 5-class classification head
Parameters	395,836,421 (395M)
Base model	answerdotai/ModernBERT-large
Output classes	5: NONE, LOW, MEDIUM, HIGH, CRITICAL
Max sequence	512 tokens (architecture supports 8,192)
Inference	25ms/sample (H100), ~50ms (DGX Spark)
Training time	6.6 minutes (8× NVIDIA H100 80GB)
License	Apache-2.0

---

What It Detects

Category	Example	Severity
Credential leaks	AKIA7G4N2K9Q6M8R3T5V, sk-ant-api..., bearer tokens, connection strings	HIGH–CRITICAL
Prompt injection	"Ignore all previous instructions", DAN mode, jailbreaks, instruction override	MEDIUM–HIGH
PII exposure	SSN, credit cards, passports, phone numbers, emails, IP addresses	HIGH–CRITICAL
Malicious commands	rm -rf /, curl | sh, docker --privileged, kill -9 -1	HIGH–CRITICAL
Data exfiltration	/etc/passwd, .ssh/id_rsa, .aws/credentials, SSRF to metadata endpoints	HIGH
Tool call attacks	Reverse shells, privilege escalation, DNS tunneling, git hook injection	HIGH–CRITICAL

What It Correctly Ignores

Benign Content	Why It's Not Flagged
"Explain the format of /etc/passwd for my CS class"	Educational context
"Document 'Enable DAN' in our jailbreak taxonomy"	Writing docs about attacks
rm -rf /tmp/build-output	Scoped deletion (not destructive)
"What's a strong password policy?"	Discussing policy
rg -n 'jailbreak mode' internal/gateway	Searching code for patterns
{"path": "./README.md"}	Normal file operation

---

Usage

Quick Start (recommended)

from secjudge_model import load_secjudge

classify = load_secjudge("nghodki/SecJudge")

result = classify("rm -rf / --no-preserve-root")
print(result)
# {'is_attack': True, 'severity': 'HIGH', 'confidence': 0.999,
#  'raw_score': 0.987,
#  'severity_probabilities': {'NONE': 0.009, 'LOW': 0.001, 'MEDIUM': 0.019, 'HIGH': 0.970, 'CRITICAL': 0.001}}

result = classify("Explain how prompt injection works for my security class")
print(result)
# {'is_attack': False, 'severity': 'NONE', 'confidence': 0.02, ...}

result = classify('{"role": "assistant", "tool_calls": [{"name": "bash", "arguments": {"command": "cat ~/.aws/credentials"}}]}')
print(result)
# {'is_attack': True, 'severity': 'HIGH', 'confidence': 0.999, ...}

load_secjudge() handles everything — model loading, temperature scaling, isotonic calibration, severity prediction. One function, one result dict.

Custom Model Class

from secjudge_model import SecJudgeForSequenceClassification
from transformers import AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("nghodki/SecJudge")
model = SecJudgeForSequenceClassification.from_pretrained_secjudge("nghodki/SecJudge")
model.eval()

enc = tokenizer("rm -rf /", return_tensors="pt")
with torch.no_grad():
    output = model(**enc)

print(output.is_attack)           # [True]
print(output.severity)            # ['HIGH']
print(output.calibrated_score)    # tensor([0.999])
print(output.severity_probs)      # tensor([[0.009, 0.001, 0.019, 0.970, 0.001]])

The forward pass automatically applies temperature scaling, isotonic calibration, severity classification, and attack detection.

Files in This Repository

File	Purpose
model.safetensors	Model weights (395M params)
config.json	HuggingFace model config
tokenizer.json	Tokenizer
secjudge_model.py	Custom model class with calibration in forward pass
secjudge_config.json	Calibration temperatures + training metadata
isotonic_calibrator.pt	PyTorch isotonic lookup table (19KB, no sklearn needed)
isotonic_calibrator.pkl	Sklearn isotonic regressor (legacy)
LICENSE	Apache 2.0

---

Training

Phase 0: Supervised Contrastive Learning (5 epochs, 65s)

Pushes attack and benign examples apart in embedding space before classification training. Uses SupCon loss with temperature 0.07.

Phase 1: CrossEntropy Fine-Tuning (10 epochs, 165s)

Standard binary classification. All 395M parameters updated. AdamW, lr=2e-5, BF16, effective batch 256.

Phase 2: RLCD — Reinforcement Learning for Calibrated Decisions (6 epochs, 114s)

5-class severity head. Proper scoring rules (spherical + ranked probability score) as reward. GRPO exploration with group size 4, sigma 0.4→0.1.

Phase 3: Isotonic Regression (post-training, <1s)

Maps raw model scores to calibrated P(attack). Compresses benign scores toward 0, attack scores toward 1. Creates the score separation needed for low-FPR operation. Stored as pure PyTorch lookup table (no sklearn dependency at inference).

---

Training Data

15,266 samples from 11 sources (50/50 attack/benign balanced):

Dataset	Samples	Type	License	Role
DefenseClaw Security Suite	4,100	Both	Apache-2.0	Primary signal (5× weight)
S-Labs/prompt-injection-dataset	5,900	Both	MIT	Hard negatives (benign with attack keywords)
AnishJoshi/nl2bash-custom	1,400	Benign	—	Benign shell commands
DC JSON-augmented	190	Attack	Apache-2.0	Attacks in tool-call JSON format
DC context-augmented	490	Attack	Apache-2.0	Attacks embedded in benign paragraphs
Magicoder-OSS-Instruct-75K	870	Benign	MIT	Long code content
Attack Example Bank (EN)	770	Attack	Internal	Jailbreak diversity
Trendyol Cybersecurity	680	Benign	Apache-2.0	Security discussions in benign context
3nesdeniz boundary pairs	840	Both	CC-BY-4.0	Paired attack/benign from same scenario
deepset/prompt-injections	330	Both	Apache-2.0	Foundational injection detection
infraset/infraset	225	Benign	Apache-2.0	Sysadmin tasks

---

Limitations

1. Cross-domain recall gap: Same-domain recall at 0.5% FPR is 92%, but cross-domain drops to 40-46%. The model generalizes well on F1 but the strict NP metric exposes the gap.
2. Tool-call JSON: Weakest on rogue-security benchmark. Benign tool outputs with file paths and user data cause false positives.
3. LOW/CRITICAL severity: Training data concentrated on HIGH severity. Finer severity grading needs more labeled data.
4. English only. Non-English security detection is untested.
5. Isotonic calibration is fit-specific: The calibrator is fitted on the validation set and may not transfer perfectly to all distributions. Conformal prediction would provide stronger guarantees.

---

Citation

@misc{secjudge2026,
  title   = {SecJudge: Neyman-Pearson Security Classification for AI Agent Runtimes},
  author  = {Ghodki, Nikhil},
  year    = {2026},
  url     = {https://huggingface.co/nghodki/SecJudge},
  note    = {92\% recall at 0.5\% FPR (same-domain), 40-46\% (cross-domain).
             4-phase: SupCon + CE + RLCD + Isotonic calibration.
             Evaluated on 5,729 cases across 5 independent benchmarks.}
}