git clone git lfs 出现错误 fatal: model-00001-of-00002.safetensors: smudge filter lfs failed
git clone git lfs 出现错误 fatal: model-00001-of-00002.safetensors: smudge filter lfs failed
error: external filter ‘git-lfs filter-process’ failed
fatal: model-00001-of-00002.safetensors: smudge filter lfs failed。
Error downloading object: model-00001-of-00002.safetensors (66dec18): Smudge error: Error downloading model-00001-of-00002.safetensors (66dec18c9f1705b9387d62f8485f4e7d871ca388718786737ed3c72dbfaac9fb): batch response: Repository or object not found: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf.git/info/lfs/objects/batch
Check that it exists and that you have proper access to it
Errors logged to /root/meta-llama/Llama-2-7b-chat-hf/.git/lfs/logs/20240221T024656.481462682.log Use `git lfs logs last` to view the log. error: external filter 'git-lfs filter-process' failed fatal: model-00001-of-00002.safetensors: smudge filter lfs failed warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry with 'git restore --source=HEAD :/'
详细错误如下：
git-lfs/2.11.0 (GitHub; linux amd64; go 1.13.4)
git version 2.27.0
$ git-lfs filter-process
Error downloading object: model-00001-of-00002.safetensors (66dec18): Smudge error: Error downloading model-00001-of-00002.safetensors (66dec18c9f1705b9387d62f8485f4e7d871ca388718786737ed3c72dbfaac9fb): batch response: Repository or object not found: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf.git/info/lfs/objects/batch
Check that it exists and that you have proper access to it
Repository or object not found: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf.git/info/lfs/objects/batch
Check that it exists and that you have proper access to it
github.com/git-lfs/git-lfs/errors.Errorf
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/errors.go:69
github.com/git-lfs/git-lfs/lfshttp.defaultError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfshttp/errors.go:128
github.com/git-lfs/git-lfs/lfshttp.(*Client).handleResponse
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfshttp/errors.go:51
github.com/git-lfs/git-lfs/lfshttp.(*Client).DoWithRedirect
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfshttp/client.go:326
github.com/git-lfs/git-lfs/lfsapi.(*Client).doWithCreds
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfsapi/auth.go:103
github.com/git-lfs/git-lfs/lfsapi.(*Client).doWithAuth
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfsapi/auth.go:67
github.com/git-lfs/git-lfs/lfsapi.(*Client).DoWithAuth
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfsapi/auth.go:25
github.com/git-lfs/git-lfs/lfsapi.(*Client).DoAPIRequestWithAuth
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfsapi/auth.go:56
github.com/git-lfs/git-lfs/tq.(*tqClient).Batch
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/api.go:74
github.com/git-lfs/git-lfs/tq.Batch
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/api.go:40
github.com/git-lfs/git-lfs/tq.(*TransferQueue).enqueueAndCollectRetriesFor
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:559
github.com/git-lfs/git-lfs/tq.(*TransferQueue).collectBatches.func1
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:453
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1357
batch response
github.com/git-lfs/git-lfs/errors.newWrappedError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/types.go:198
github.com/git-lfs/git-lfs/errors.Wrap
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/errors.go:74
github.com/git-lfs/git-lfs/tq.(*tqClient).Batch
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/api.go:77
github.com/git-lfs/git-lfs/tq.Batch
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/api.go:40
github.com/git-lfs/git-lfs/tq.(*TransferQueue).enqueueAndCollectRetriesFor
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:559
github.com/git-lfs/git-lfs/tq.(*TransferQueue).collectBatches.func1
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:453
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1357
Error downloading model-00001-of-00002.safetensors (66dec18c9f1705b9387d62f8485f4e7d871ca388718786737ed3c72dbfaac9fb)
github.com/git-lfs/git-lfs/errors.newWrappedError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/types.go:198
github.com/git-lfs/git-lfs/errors.Wrapf
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/errors.go:85
github.com/git-lfs/git-lfs/lfs.(*GitFilter).downloadFile
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:115
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:76
github.com/git-lfs/git-lfs/commands.smudge
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:120
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:766
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).ExecuteC
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:852
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:800
github.com/git-lfs/git-lfs/commands.Run
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/run.go:103
main.main
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/git-lfs.go:33
runtime.main
/usr/lib/golang/src/runtime/proc.go:203
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1357
Smudge error
github.com/git-lfs/git-lfs/errors.newWrappedError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/types.go:198
github.com/git-lfs/git-lfs/errors.NewSmudgeError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/types.go:284
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:85
github.com/git-lfs/git-lfs/commands.smudge
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:120
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:766
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).ExecuteC
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:852
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:800
github.com/git-lfs/git-lfs/commands.Run
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/run.go:103
main.main
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/git-lfs.go:33
runtime.main
/usr/lib/golang/src/runtime/proc.go:203
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1357
Current time in UTC:
2024-02-21 02:46:56
ENV:
LocalWorkingDir=/root/meta-llama/Llama-2-7b-chat-hf
LocalGitDir=/root/meta-llama/Llama-2-7b-chat-hf/.git
LocalGitStorageDir=/root/meta-llama/Llama-2-7b-chat-hf/.git
LocalMediaDir=/root/meta-llama/Llama-2-7b-chat-hf/.git/lfs/objects
LocalReferenceDirs=
TempDir=/root/meta-llama/Llama-2-7b-chat-hf/.git/lfs/tmp
ConcurrentTransfers=8
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=/root/meta-llama/Llama-2-7b-chat-hf/.git/lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic,lfs-standalone-file
UploadTransfers=basic,lfs-standalone-file
GIT_DIR=/root/meta-llama/Llama-2-7b-chat-hf/.git
GIT_EXEC_PATH=/usr/libexec/git-core
Client IP addresses:
172.17.0.2
解决方法：
// Skip smudge - We'll download binary files later in a faster batch git lfs install --skip-smudge // Do git clone here git clone ... // Fetch all the binary files in the new clone cd project git lfs pull // Reinstate smudge git lfs install --force
如果是国外的大文件，比如 大模型，最好配置git的代理，如下：
git config --global http.sslVerify "false" git config --global advice.detachedHead false #建议socks5 和http 都配置一下代理 #配置socks5 #git config --global http.proxy socks5 127.0.0.1:7890 #git config --global https.proxy socks5 127.0.0.1:789 #配置http #git config --global http.proxy 127.0.0.1:7890 #git config --global https.proxy 127.0.0.1:7890 #查看设置： #git config --global --get http.proxy #git config --global --get https.proxy #取消办法： #git config --global --unset http.proxy #git config --global --unset https.proxy
就可以正常下载了：
Downloading LFS objects: 20% (1/5), 298 MB | 3.1 MB/s
