README.md

---
pretty_name: EventActivityNet
tags:
- event-based-vision
- video-understanding
- activitynet
- hdf5
size_categories:
- 1K<n<10K
---

# EventActivityNet v1.0

EventActivityNet is a generated event voxel tensor dataset derived from
ActivityNet videos together with ActivityNet Captions annotations. It provides
two alternative temporal groupings over the same canonical 3,263-video set.
These are generated tensors, not native event-camera recordings.

| Representation | Public path | Videos | Train / validation | Shards (train / validation) | Event shape | Canonical HDF5 bytes |
|---|---|---:|---:|---:|---|---:|
| 5-bin | `data_5bin/` | 3,263 | 2,316 / 947 | 157 / 62 | `(T5, 5, H, W)` | 4,355,745,895,245 |
| 9-bin | `data_9bin/` | 3,263 | 2,316 / 947 | 157 / 62 | `(T9, 9, H, W)` | 4,214,122,096,103 |

The repository contains approximately 8.57 TB of tar-packaged payload. The
representations use identical video membership, split assignment, and shard
membership. Neither representation is presented as inherently better than the
other.

## Dataset Structure

Each ActivityNet video corresponds to exactly one HDF5 member in each
representation. Train/validation and Large/Medium/Small membership are defined
by manifests; the nested scales do not duplicate payload files.

```text
data_5bin/{train,validation}/
data_9bin/{train,validation}/
metadata/{5bin,9bin}/
metadata/video_metadata.jsonl
annotations/
scales/
docs/
```

## Data Format

Every HDF5 file contains exactly:

- `events`: `(T_B, B, H, W)`, `int16`;
- `voxel_event_start`: `(T_B,)`, `int64`;
- `voxel_event_count`: `(T_B,)`, `int32`.

Here `B` is 5 or 9. For `N` decoded source frames:

```text
T_B = ceil((N - 1) / B)
```

Each event slice represents one adjacent decoded-frame transition. An
`events[t]` tensor groups up to `B` consecutive transition slices. The final
group may be partial; unused bins are zero-filled. Timing uses each video's
released rational source FPS metadata.

See [Dataset Format](docs/DATASET_FORMAT.md) for schema, timing, and memory-safe
loading details.

## Release Scales

| Scale | Videos | Hours | Train | Validation | Classes | Event-friendly |
|---|---:|---:|---:|---:|---:|---:|
| Large | 3,263 | 106.94 | 2,316 | 947 | 200 | 65.31% |
| Medium | 1,537 | 50.00 | 1,074 | 463 | 200 | 64.80% |
| Small | 667 | 20.00 | 473 | 194 | 200 | 64.62% |

Small is a strict subset of Medium, and Medium is a strict subset of Large.

## Included Metadata

- `annotations/activitynet_captions.json`: timestamped ActivityNet Captions descriptions;
- `annotations/activitynet_actions.json`: ActivityNet v1.3 temporal actions;
- `annotations/eventactivitynet_alignment.json`: derived caption/action alignment;
- `annotations/annotation_issues.jsonl`: known source annotation quirks;
- `metadata/video_metadata.jsonl`: shared original-rate timing metadata and per-representation tensor metadata;
- `metadata/5bin/` and `metadata/9bin/`: representation-specific shard manifests, summaries, and checksums;
- `scales/`: Large, Medium, and Small manifests and public statistics.

## Intended Uses

The dataset supports research on generated event voxel representation learning,
activity recognition, caption-aligned activity modeling, and comparison across
nested dataset scales or temporal groupings.

It is out of scope for identifying people, biometric recognition, surveillance
deployment, or consequential decisions about individuals.

## Limitations

- The event voxel tensors are generated from conventional videos rather than recorded by an event camera.
- Source FPS and spatial resolution vary by video.
- Timing is based on decoded frame order and released rational nominal or average FPS; per-frame presentation timestamps are not consumed.
- The subset is curated rather than an unbiased conversion of all ActivityNet videos.

## Checksums

```bash
sha256sum -c metadata/5bin/shard_checksums.sha256
sha256sum -c metadata/9bin/shard_checksums.sha256
```

## Documentation

- [Dataset Card](docs/DATASET_CARD.md)
- [Dataset Format](docs/DATASET_FORMAT.md)
- [Dataset Generation](docs/DATASET_GENERATION.md)
- [Release Notes](docs/RELEASE_NOTES.md)
- [License Notes](docs/LICENSE_NOTES.md)

## Licensing and Citation

ActivityNet and ActivityNet Captions source terms, licenses, citation
obligations, and redistribution restrictions still apply. See
[License Notes](docs/LICENSE_NOTES.md) and [CITATION.cff](CITATION.cff).

## Release Status

Both complete representations passed final integrity and remote-layout audits.
The public payload contains 438 tar shards: 219 per representation.