mdhvm/anlp-m26-a1-cipher-transformers

Checkpoints for Advanced NLP Assignment 1 — An encoder–decoder Transformer, written from elementary PyTorch operations, is trained to recover plaintext from a repeating-key XOR cipher. Five configurations differ from the base model in exactly one component each — rotary positions, grouped-query attention, RMSNorm, and a token-free Byte Latent Transformer with entropy-based dynamic patching — and are compared at an identical step budget over three seeds.

Loading

import torch, sys
sys.path.insert(0, "src")
from train import CONFIGS, build_model

ck = torch.load("C1/best.pt", map_location="cpu")
# build_model(CONFIGS["C1"], ck["hp"], bundle, device) -- see src/evaluate.py

Training code, tests and the full study: see the accompanying submission.

Files

file	contents
C*/best.pt	model state_dict plus the config and hyperparameters used
C*/results.json	test metrics, efficiency numbers, OOD probes
C*/history.json	per-eval training curves
tokenizers/bpe_*.json	the from-scratch BPE vocabulary and merge list
tables/, figures/	aggregated ablation tables and plots