AI Training Datasets

Manage large dataset archives, streamable Parquet/Arrow shards, and storage durability.