while following the DMTN-023 Pipeline Command-Line Drivers tutorial, I stumbled with the problem of cloning test data. I followed these instructions for installing the git lfs extension and for configuring my ~/.gitconfig and ~/.git-credentials files. The issue appears when I try to clone a test data repository: git-lfs keeps asking my Github password.
Here is what I get:
$ git lfs version
git-lfs/1.2.0 (GitHub; darwin amd64; go 1.6.1; git 9bd3b8e)
$ git clone https://github.com/lsst/testdata_decam.git
Cloning into 'testdata_decam'...
remote: Counting objects: 86, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 86 (delta 2), reused 0 (delta 0), pack-reused 76
Unpacking objects: 100% (86/86), done.
Checking connectivity... done.
Downloading rawData/calib/bpm/2013-07-16/defect01.fits (32.00 MB)
Username for 'https://git-lfs.lsst.codes': airnandez
Password for 'https://airnandez@git-lfs.lsst.codes': <I type my password here>
Username for 'https://git-lfs.lsst.codes': airnandez
Password for 'https://airnandez@git-lfs.lsst.codes': <I type my password here>
Username for 'https://git-lfs.lsst.codes': airnandez
Password for 'https://airnandez@git-lfs.lsst.codes': <I type my password here>
Username for 'https://git-lfs.lsst.codes': airnandez
Password for 'https://airnandez@git-lfs.lsst.codes': <I type my password here>
Username for 'https://git-lfs.lsst.codes': ^Cwarning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'
Exiting because of "interrupt" signal.
Even if I provide my correct password, git-lfs seems not happy with it. I checked that this is the password I use to successfully login to github.com. I observe the same behaviour when trying to clone other repositories with commands such as git clone https://github.com/lsst/ci_hsc.git.
I was wondering if some registration of my github.com account is needed in order to be able to clone LSST test data. Where should I look at to understand what is happening? The -v option of git clone does not provide further information.
Execution environment: OS X 10.11.5, git version 2.6.4 (Apple Git-63)
Thanks Paul. I tried your configuration but I observe the same behaviour. It seems that your configuration should allow you authenticated read-write access to LFS users, according to this document. It makes sense that in that case it does not work for me because I want just read-only access to the repository and very likely my github account has not write authorisation in the repository I need to clone.
I also tried entering a blank user and password, when prompted for authentication for https://git-lfs.lsst.codes but git is not happy: it keeps requesting my username and password. This is weird, since in principle for cloning the repository I was expecting anonymous access.
Here is my configuration.
Excerpts from file ~/.gitconfig:
[filter "lfs"]
required = true
clean = git-lfs clean -- %f
smudge = git-lfs smudge -- %f
# Cache anonymous access to DM Git LFS S3 servers
[credential "https://lsst-sqre-prod-git-lfs.s3-us-west-2.amazonaws.com"]
helper = store
[credential "https://s3.lsst.codes"]
helper = store
# Cache anonymous access to DM Git LFS server
[credential "https://git-lfs.lsst.codes"]
helper = store
Hello everyone. I encountered the similar problem.
I tried to install the latest release of lsst_sims (eups distrib install lsst_sims -t v12_0) and the instalation of sims_maps failed due to failed tests. Tests failed because git_lfs was not installed. Then, I installed it from https://developer.lsst.io/tools/git_lfs.html, as suggested in the error message. After restarted sims installation, it asked me the password for git-lfs.lsst.codes. Here I stopped.
@FabioHernandez sorry you ran into these problems. I appreciate everyones comments. They will help improve our git-lfs implementation.
There are two independent issues happening here. 1) We don’t support the batch API yet. That will be implemented this summer. 2) The git-lfs client doesn’t allow for empty username and passwords. Through some configuration it is possible to work around this. But the configuration is difficult and error prone. I will investigate fixing this later this summer. I don’t know exactly how it will be solved. I expect the solution will involve some creativity or just implementing the fix in the git-lfs client and trying to convince the git-lfs developers to accept it as an optional configuration.
Getting the git-lfs configurations right can be a pain, but the improvements planned for this summer should make it better. In the meantime (for those using the Anaconda distribution) you could try a package I hacked together a few weeks ago:
After running this, git-lfs should be fully configured for use with LSST. This will also install git-lfs (and git itself), so nothing else should be needed.
In fact, I didn’t notice the command I needed to type for setting the lfs.batch option to false, which is correctly documented in the section “Installing Git LFS”. That was the root cause of my issue.
My suggestion is to slightly modify the documentation to replace the instruction:
git config --global lfs.batch false
and to include the option:
[lfs]
batch = false
as one of the required modifications to the ~/.gitconfig file. This way, all the configuration required for git-lfs to work would be concentrated in a single place.