Commit graph

30 commits

Author SHA1 Message Date
Paul Chavard
561b83781e refactor(archive): remove dependencie on archive from ArchiveUploader 2022-04-12 14:54:54 +02:00
Paul Chavard
cf8c084a59 refactor: extract download_and_zip in to a shared service 2022-04-12 14:54:54 +02:00
Martin
c1884f231c Revert "Merge pull request #7105 from betagouv/US/fix-dossier.processed_in_month"
This reverts commit a0e144b9a7, reversing
changes made to 49848bd150.
2022-04-05 13:39:37 +02:00
Martin
c07e0fc13e fix(Dossier.processed_in_month): ensure proper usage via method sig instead of defensive programming style 2022-04-05 12:14:07 +02:00
simon lehericey
bd0b88a410 move create_list_of_attachments 2022-04-05 11:55:14 +02:00
Martin
dbcf21a555 feat(archive): extract archive status management within job to simplify the main service as well as to isolate this part for a merge with exports csv/xslx [maybe?]
Update app/dashboards/archive_dashboard.rb

Co-authored-by: LeSim <mail@simon.lehericey.net>
2022-03-31 13:35:49 +02:00
Martin
ab2caaa5f7 fix(ProcedureArchiveService.zip_root_folder): should take archive instance otherwise when we generate many archive for the same procedure, errors may occures 2022-03-30 16:29:54 +02:00
Martin
98c1fb8abc feat(archive): drop old feature 2022-03-18 14:26:09 +01:00
Martin
5739150f15 feat(service/archive_uploader): add an archive uploader class to upload files thru a custom script which handle file encryption of massive file (bigger than 4Go)
Update doc/object-storange-and-data-encryption.md

Co-authored-by: LeSim <mail@simon.lehericey.net>

Update app/services/archive_uploader.rb

Co-authored-by: LeSim <mail@simon.lehericey.net>

Update doc/object-storange-and-data-encryption.md

Co-authored-by: Pierre de La Morinerie <kemenaran@gmail.com>

clean(doc): align document file name and document h1

clean(review): refactore based on various comments

clean(review): refactore based on various comments
2022-03-16 14:56:21 +01:00
Paul Chavard
7937e58caa fix(archives): only export dossiers in archive groupe_instructeurs
fix #6793
2022-01-18 11:16:20 +01:00
Martin
ce1b189dcd refactor(DownloadManager): extract parallel download in dedicated class. move error management in custom class for procedure exports using the didicated class 2022-01-04 16:17:03 +01:00
Martin
2ed9cccba0 extract download all attachments in dedicated class using async/async-http for better perf 2022-01-04 16:17:03 +01:00
Martin
721d926c0d wip(fix): chdir 2021-12-13 16:37:04 +01:00
mfo
148be50c86 Update app/services/procedure_archive_service.rb
Co-authored-by: LeSim <mail@simon.lehericey.net>
2021-12-13 16:37:04 +01:00
Martin
f7bc387e44 feat(flipper): use new way of generating archive only on some procedure flipped with new actor :zip_using_binary 2021-12-13 16:37:04 +01:00
simon lehericey
f0b0e7fd9a Switch to usage of zip unix binary to create archive. Also use a dedicated queue for DelayedJob
use dedicated archives queue

As the used disk space will increase, we want a fined grain control

move zip logic in dedicated method

zip

wip

wip

fix(spec): pass spec in green

tech(improvements): avoid File.delete(folder), favor FileUtils.remove_entry_secure which is safer. Also wrap most of code that open file within blocks so it is cleaned when the block ends. Lastly use  attachement.download to avoid big memory pressure [download in chunk, write in chunk] otherwise big file [124>1GO] are loaded in memory. what if we run multiple jobs/download in parallel ?

fix(spec): try to retry with grace

clean(procedure_archive_service_spec.rb): better retry [avoid to rewrite on open file]

lint(things): everything
2021-12-13 16:37:04 +01:00
Paul Chavard
96cbbc0192 fix(virus scan): prevent virus scan on archives and signatures uploads 2021-09-07 14:21:48 +02:00
Christophe Robillard
031b74759f archives: move bugreport 2021-08-05 09:24:20 +02:00
Christophe Robillard
38e3469e9a add bug report to archive 2021-08-03 14:51:41 +02:00
Christophe Robillard
67c0d93eae Debugging archive generation
debugging exception that occurs sometime when trying to download pj during
archive generation
2021-07-29 11:12:07 +02:00
Christophe Robillard
66cc0dd08d Revert "Revert "Expose dossier PDF export as IO""
This reverts commit 362093eff0.
2021-06-24 21:01:59 +02:00
krichtof
362093eff0
Revert "Expose dossier PDF export as IO" 2021-06-24 19:21:37 +02:00
Paul Chavard
f39944a02e Create procedure sub folder in archives
Co-authored-by: Christophe Robillard <christophe.robillard@beta.gouv.fr>
2021-06-24 17:50:17 +02:00
Christophe Robillard
f40d96fbd2 Revert "Revert "Export de tous les dossier d'une démarche""
This reverts commit d9a588b52e.
2021-04-29 17:29:47 +02:00
krichtof
d9a588b52e
Revert "Export de tous les dossier d'une démarche" 2021-04-29 16:07:18 +02:00
Christophe Robillard
09870c918d prevent race conditions when creating archives 2021-04-28 11:40:58 +02:00
Christophe Robillard
dfbe004122 rename content_type to time_span_type for archives 2021-04-28 11:40:58 +02:00
Christophe Robillard
9134114c2e fix rubocop offenses 2021-04-28 11:40:58 +02:00
Christophe Robillard
8bee53fe77 rend plus robuste la création de l'archive 2021-04-28 11:40:58 +02:00
Christophe Robillard
07cc4fa97f add service to create procedure archive 2021-04-28 11:40:58 +02:00