# Object Storage Use Cases Part 3: Storing binary files in Object Storage instead of in your git repository

DevFeed: [Object Storage Use Cases Part 3: Storing binary files in Object Storage instead of in your git repository](<https://devfeed.tech/articles/object-storage-use-cases-part-3-storing-binary-files-in-object-storage-instead-of-in-your-git-repository-19770.md>)

Original publisher: [Read original article](<https://cloud.blog.csc.fi/2019/04/object-storage-use-cases-part-3-storing.html>)

Author: Unknown (noreply@blogger.com)

Published: 2019-04-29T09:06:00Z

Content type: tutorial

Language: en

Sources: [CSC - IT Center For Science - Cloud Team](<https://devfeed.tech/sources/csc-it-center-for-science-cloud-team.md>)

Topics: [Git](<https://devfeed.tech/topics/git.md>), [ceph](<https://devfeed.tech/topics/ceph.md>), [pip](<https://devfeed.tech/topics/pip.md>), [Haskell](<https://devfeed.tech/topics/haskell.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [ceph](<https://devfeed.tech/tags/ceph.md>), [data](<https://devfeed.tech/tags/data.md>), [data-lake](<https://devfeed.tech/tags/data-lake.md>), [git](<https://devfeed.tech/tags/git.md>), [install](<https://devfeed.tech/tags/install.md>), [object-storage](<https://devfeed.tech/tags/object-storage.md>), [python](<https://devfeed.tech/tags/python.md>), [rados](<https://devfeed.tech/tags/rados.md>), [radosgw](<https://devfeed.tech/tags/radosgw.md>), [s3](<https://devfeed.tech/tags/s3.md>), [storage](<https://devfeed.tech/tags/storage.md>)

## AI overview

This post explains why binary files should not be stored directly in Git repositories and explores using CSC's Pouta Cloud Object Storage instead. It documents attempts with git-annex and git-bigstore, including a patch to configure a non-default S3-compatible endpoint.

## Source excerpt

1. It's bad to store binary blobs in git. This is known [1] 2. A great place to store binary files is in object storage 3. Can we use CSC's Pouta Cloud Object Storage service for this? Why yes, yes you can! This is a post in a series about different use cases for object storage. Others include for example hosting a reveal.js presentation and NFS server backups More details about CSC's Pouta Cloud Object Storage Service, it is based on CEPH Rados Object Storage Gateway. What The general solution* to storing binaries in git is to not store them in git and instead create a link to some other place Attempt 1 Install an application that you like. In https://www.perforce.com/blog/storing-large-binary-files-in-git-repositories I searched for "S3", found git-annex which had recent commits so it looked like it wasn't dead. Installation instructions: https://git-annex.branchable.com/install/ The version in EPEL/Ubuntu Xenial does not have S3 support and is from 2014. There's a recent one in Fedora's repos though. Unfortunately, compiling it requires cabal. I don't know Haskell. https://spack.readthedocs.io/en/latest/package_list.html does not have any Haskell, cabal or git-annex available. Next. Attempt 2 Next application in the blog post was https://github.com/lionheart/git-bigstore To install it was much more comfortable for me: pip install "git-bigstore<=2.0" First issue was that looking at https://github.com/lionheart/git-bigstore/blob/master/bigstore/backends/s3.py there was no way to tell it to use another endpoint than the default of AWS S3. A small patch takes care of that, add , endpoint_url='https://object.pouta.csc.fi:443' to the client() boto3 call in the backends/s3.py file from git-bigstore. For a more detailed example see this Pull Request. After spending some time going down some rabbit holes it seems that the Profile option that is supported in git-bigstore does not support endpoints. It can only be used to specify credentials and a few other parameters. Mino