Mosaic is no longer maintained, LitData has some restrictions and webdataset is not random access proof. Took a stab at building a dataloading framework thats performs better or just as well as other frameworks but with a lot of QoL features including data mapping. Random access and easy subsetting.
Mosaic is no longer maintained, LitData has some restrictions and webdataset is not random access proof. Took a stab at building a dataloading framework thats performs better or just as well as other frameworks but with a lot of QoL features including data mapping. Random access and easy subsetting.