datasets.datalad.org

docs.datalad.org

Basic principles — datalad 0.1 documentation

http://docs.datalad.org/en/latest/basics.html

Related efforts and solutions. Data management use cases. Automatic creation and maintenance of datasets by crawling external resources. DataLad was designed so it could be used both as a command-line tool, and as a Python module. Sections following this one ( Command line reference. And Python module reference. DataLad “distribution” is just a superdataset. Which organizes multiple dataset. 8216;s using standard git mechanism of sub-modules. Primarily operates and reports at the level of datasets. Datas...

datalad.org

Articles - DataLad

http://datalad.org/category/articles.html

Git-annex-centric benchmark of the filesystems. For the DataLad project we will establish a large number of git-annex repositories. Majority of those git-annex repositories will not contain any data, but rather a lot of empty files and/or broken symlinks. Some data sharing projects which we are going to cover contain thousands and even . Mon 06 July 2015. Bits from DataLad #1. Summary report of recent activities. Sat 11 October 2014. Tue 16 September 2014. Working with big data without losing track.

datalad.org

About this Project - DataLad

http://datalad.org/pages/about.html

This project aims to adapt the model of open-source software (OSS) distributions to address the technical limitations of today's data-sharing and develop all components of a data distribution. The key concepts are: 1) Leverage - but do not replace - independent, existing, and future data hosting solutions to form a federated platform for data-sharing. 2) Employ software for data tracking and deployment logistics specialized for large data ( git-annex. Data sharing portals such as http:/ openfmri.org.

datalad.org

Working with big data without losing track - DataLad

http://datalad.org/gitannex_based_access.html

Working with big data without losing track. Mon 16 June 2014. Borrowed from the http:/ studyforrest.org. As a good use-case for demonstrating reasoning for use of git-annex by data providers. You can already fetch entire studyforrest dataset using git-annex directly. Here is e.g. a sample list of command to fetch all extracted into CSV timeseries:. Git clone http:/ psydata.ovgu.de/forrest gump/.git studyforrest cd studyforrest git annex get sub*/BOLD/task001 run*/*timeseries*.csv.gz. Thrown over the wall.

datalad.org

Bits from DataLad #1 - DataLad

http://datalad.org/bits_1.html

Bits from DataLad #1. Sat 11 October 2014. Etc We hope you will get to like the name, and we will come up with a beautiful project logo soonish. Meanwhile enjoy a functional colorful qrcode you might find associated with DataLad on those pages. Obtained integration with Travis-CI and coveralls.io. Apparently we are at 85% coverage of the codebase, not bad at all for starters. So you are welcome to submit pull requests without a fear of breaking our. A very important part of this project is also to identi...

datalad.org

Archives - DataLad

http://datalad.org/archives.html

Mon 06 July 2015. Git-annex-centric benchmark of the filesystems. Sat 11 October 2014. Bits from DataLad #1. Tue 16 September 2014. Mon 16 June 2014. Working with big data without losing track. 2015 DataLad Team · Powered by pelican-bootstrap3. Content licensed under a Creative Commons Attribution 4.0 International License. Except where indicated otherwise.

datalad.org

bits - DataLad

http://datalad.org/tag/bits.html

Bits from DataLad #1. Summary report of recent activities. Sat 11 October 2014. 2014 DataLad Team · Powered by pelican-bootstrap3. Content licensed under a Creative Commons Attribution 4.0 International License. Except where indicated otherwise.

datalad.org

git-annex-centric benchmark of the filesystems - DataLad

http://datalad.org/test_fs_analysis.html

Git-annex-centric benchmark of the filesystems. Mon 06 July 2015. On computational clusters we administer/use. Its volume management, data integrity checking, and various features (snapshotting, etc) are very appealing and served us well before. So ZFS was a logical choice to deploy for our DataLad development/storage server(s). But unfortunately, during development we ran into complete system stalls (which thanks to ZoL developers were promptly resolved), and overall very. ZoL/ZFS module: 0.6.4-...BTRFS...

datalad.org

development - DataLad

http://datalad.org/tag/development.html

Git-annex-centric benchmark of the filesystems. For the DataLad project we will establish a large number of git-annex repositories. Majority of those git-annex repositories will not contain any data, but rather a lot of empty files and/or broken symlinks. Some data sharing projects which we are going to cover contain thousands and even . Mon 06 July 2015. Bits from DataLad #1. Summary report of recent activities. Sat 11 October 2014. Tue 16 September 2014. Except where indicated otherwise.