DGNum/demarches-normaliennes
14
1
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity

Merge pull request #7104 from betagouv/US/fix-archive-write-append-binary-with-string

Browse source 
fix(parallel_download_queue): appending content to file in chunk leads to some corrupted files.
This commit is contained in:
mfo 2022-04-01 18:26:32 +02:00 committed by GitHub
commit 09b730b992
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
app/lib/download_manager/parallel_download_queue.rb
View file

@ -36,11 +36,12 @@ module DownloadManager
File.write(attachment_path, attachment.file.read, mode: 'wb') File.write(attachment_path, attachment.file.read, mode: 'wb')
else else
request = Typhoeus::Request.new(attachment.url) request = Typhoeus::Request.new(attachment.url)
request.on_body do |chunk|
File.write(attachment_path, chunk, mode: 'a+b')
end
request.on_complete do |response| request.on_complete do |response|
unless response.success? if response.success?
File.open(attachment_path, mode: "wb") do |fd|
fd.write(response.body)
end
else
File.delete(attachment_path) if File.exist?(attachment_path) # -> case of retries failed, must cleanup partialy downloaded file File.delete(attachment_path) if File.exist?(attachment_path) # -> case of retries failed, must cleanup partialy downloaded file
on_error.call(attachment, path_in_download_dir, response.code) on_error.call(attachment, path_in_download_dir, response.code)
end end