DVC data

Features

• TODO

Requirements

• TODO

Installation

You can install DVC data via pip_ from PyPI_:

$ pip install dvc-data

Usage

HashFile

HashFile

Based on dvc-object's Object, this is an object that has a particular hash that can be used to verify its contents. Similar to git's ShaFile.

from dvc_data.hashfile import HashFile

obj = HashFile("/path/to/file", fs, HashInfo("md5", "36eba1e1e343279857ea7f69a597324e")

HashFileDB

Based on dvc-object's ObjectDB, but stores HashFile objects and so is able to verify their contents by their hash_info. Similar to git's ObjectStore.

from dvc_data.hashfile import HashFileDB

odb = HashFileDB(fs, "/path/to/odb")

Index

Index

A trie-like structure that represents data files and directories.

from dvc_data.index import DataIndex, DataIndexEntry

index = DataIndex()
index[("foo",)] = DataIndexEntry(hash_info=hash_info, meta=meta)

Storage

A mapping that describes where to find data contents for index entries. Can be either ObjectStorage for HashFileDB-based storage or FileStorage for backup-like plain file storage.

index.storage_map[("foo",)] = ObjectStorage(...)

Contributing

Contributions are very welcome. To learn more, see the Contributor Guide_.

License

Distributed under the terms of the Apache 2.0 license_, DVC data is free and open source software.

Issues

If you encounter any problems, please file an issue_ along with a detailed description.