VQA11y Code

This repository contains the code for VQA11y: data preparation scripts, training utilities, inference pipelines, evaluation tools, and configuration files.

Only code and code-adjacent configuration are included here.

Repository Layout

src/ - Core Python package for accessibility features, data I/O, grounding, inference, perception wrappers, sampling, and VQG utilities.
scripts/ - Command-line tools for data processing, A-CoT generation, GoA inference, evaluation, downloads, exports, and experiments.
vqa11y_eval/ - Evaluation package, model runner registry, metric scripts, and batch inference entry points.
improvements/ - Training and data-preparation code for SFT/DPO experiments.
VQA11y_data/scripts/ - Dataset construction pipeline scripts.
configs/, VQA11y_data/configs/, vqa11y_eval/configs/, improvements/configs/ - YAML/JSON configuration files.
requirements*.txt - Environment files for the main code path and optional components.
setup.py - Editable package install entry point.

Install

git clone https://huggingface.co/VQA11y/VQA11y
cd VQA11y
pip install -r requirements.txt
pip install -e .

Optional components have separate requirement files:

pip install -r requirements-grounded-sam.txt
pip install -r requirements-deepseek-ocr.txt
pip install -r requirements-vla.txt

Environment Variables

export VQA11y_ROOT=/path/to/data
export VQA11y_MODEL_ROOT=/path/to/models

Common Entry Points

Run GoA inference:

python scripts/goa/run_goa.py \
  --model "${VQA11y_MODEL_ROOT}/model-name" \
  --split eval \
  --output outputs/goa_results.jsonl

Run evaluation summary:

python vqa11y_eval/src/scripts/summarize.py \
  --predictions outputs/goa_results.jsonl \
  --output outputs/metrics_summary.json

Run A-CoT generation:

python scripts/acot_pipeline/generate_acot.py \
  --input data/samples.jsonl \
  --output outputs/acot.jsonl \
  --teacher gemini

Run SFT or DPO jobs:

sbatch improvements/scripts/train_sft.slurm qwen3
sbatch improvements/scripts/train_dpo.slurm qwen3

Notes

This code expects you to provide local data and model paths through the environment variables above or through the relevant CLI/config arguments. Large data files and generated outputs are not part of this repository.

Downloads last month: -; Downloads are not tracked for this model. How to track