DGNum/demarches-normaliennes
15
1
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity
15131 commits 250 branches 1166 tags 78 MiB
Commit graph

321 commits

Author SHA1 Message Date
Martin
7fb85c97cb feat(Administrateur::ArchivesControllers): add archives #index and #create for admin 2022-07-04 14:18:46 +02:00
Paul Chavard
bbc44fe339 fix(export): no crash on empty repetitions 2022-06-17 11:33:48 +02:00
Paul Chavard
02675a104d fix(revision): fix tests to use new revison system 2022-05-31 08:58:53 +02:00
simon lehericey
0b90f4d30d is the order important ? 2022-05-18 10:11:31 +02:00
simon lehericey
ae86cdcf9e strange test 2022-05-18 10:11:31 +02:00
Paul Chavard
6909337500 fix(spec): try fix dossier expiration spec 2022-05-17 10:14:43 +02:00
Paul Chavard
9e0b3b642f cleanup(sendinblue): remove sendinblue tracking 2022-05-06 11:14:44 +02:00
simon lehericey
2d42b7d57d spec: fix flaky 2022-05-06 10:34:40 +02:00
simon lehericey
ea361643be types_de_champ -> types_de_champs_public 2022-05-03 11:05:24 +02:00
Martin
79c473ecf9 fix(ProcedureExportService.to_zip): prebatch dossiers only for to_csv,to_xlsx,to_ods. also add spec around to_zip 2022-04-22 05:13:51 +02:00
Martin
a3c5bf58ac bug(ProcedureExportService.to_zip): use in_batches on Array 2022-04-22 05:13:51 +02:00
Paul Chavard
433c01b1e6 Revert "Merge pull request #7137 from betagouv/faster_pdf" 
This reverts commit 9da44bd913, reversing
changes made to ebac71796c.
 2022-04-12 19:22:07 +02:00
Paul Chavard
2832ea0286 refactor(export): return blob from to_* methods 2022-04-12 14:54:54 +02:00
Paul Chavard
561b83781e refactor(archive): remove dependencie on archive from ArchiveUploader 2022-04-12 14:54:54 +02:00
Paul Chavard
cf8c084a59 refactor: extract download_and_zip in to a shared service 2022-04-12 14:54:54 +02:00
simon lehericey
4bbe2db668 only export safe pj 2022-04-12 12:05:08 +02:00
simon lehericey
4d3412daf5 batch it 2022-04-12 11:56:56 +02:00
simon lehericey
e2130cc2d3 add witness dossier 2022-04-07 12:06:13 +02:00
simon lehericey
951d964701 clean pj_service test 2022-04-07 12:06:13 +02:00
simon lehericey
34b0578d70 pj_and_path only take dossier id 2022-04-07 12:06:13 +02:00
simon lehericey
7ac1288905 pj_service take a dossier collection ! 2022-04-07 12:06:13 +02:00
Martin
9e8807d12a feat(ArchiveUploader.upload_with_chunking_wrapper): retry once on error 2022-04-05 15:11:21 +02:00
Martin
c72ba43c55 feat(ArchiveUploader.upload_with_chunking_wrapper): expect to retry call to wrapper once 2022-04-05 15:11:21 +02:00
simon lehericey
1f98f75ccc remove unused method 2022-04-05 11:55:14 +02:00
simon lehericey
4c3b9979e9 fix test 2022-04-05 11:55:14 +02:00
Martin
dbcf21a555 feat(archive): extract archive status management within job to simplify the main service as well as to isolate this part for a merge with exports csv/xslx [maybe?] 
Update app/dashboards/archive_dashboard.rb

Co-authored-by: LeSim <mail@simon.lehericey.net>
 2022-03-31 13:35:49 +02:00
Martin
ab2caaa5f7 fix(ProcedureArchiveService.zip_root_folder): should take archive instance otherwise when we generate many archive for the same procedure, errors may occures 2022-03-30 16:29:54 +02:00
Martin
98c1fb8abc feat(archive): drop old feature 2022-03-18 14:26:09 +01:00
Martin
5739150f15 feat(service/archive_uploader): add an archive uploader class to upload files thru a custom script which handle file encryption of massive file (bigger than 4Go) 
Update doc/object-storange-and-data-encryption.md

Co-authored-by: LeSim <mail@simon.lehericey.net>

Update app/services/archive_uploader.rb

Co-authored-by: LeSim <mail@simon.lehericey.net>

Update doc/object-storange-and-data-encryption.md

Co-authored-by: Pierre de La Morinerie <kemenaran@gmail.com>

clean(doc): align document file name and document h1

clean(review): refactore based on various comments

clean(review): refactore based on various comments
 2022-03-16 14:56:21 +01:00
Jon
97feca6305 feat(ClamAV): add config to disable clamav usage 2022-02-15 09:15:47 +01:00
Paul Chavard
3d8471e064 fix(dossier): do not send notification on expiration when dossier is already deleted 2022-01-19 17:52:53 +01:00
Paul Chavard
7937e58caa fix(archives): only export dossiers in archive groupe_instructeurs 
fix #6793
 2022-01-18 11:16:20 +01:00
Martin
383a92bcd9 fix(parallel_download_queue): tophoeus does not like raise from a request handler [crash straight to first frame] 2022-01-07 14:24:37 +01:00
Martin
3eb1c1a421 feat(async_backend): switch to typhoeus 2022-01-04 16:27:23 +01:00
Martin
ce1b189dcd refactor(DownloadManager): extract parallel download in dedicated class. move error management in custom class for procedure exports using the didicated class 2022-01-04 16:17:03 +01:00
Martin
2ed9cccba0 extract download all attachments in dedicated class using async/async-http for better perf 2022-01-04 16:17:03 +01:00
Christophe Robillard
f9609b0b0b fix zone specs 
by reenabling task before the next spec is run
 2021-12-23 09:58:31 +01:00
Pierre de La Morinerie
4efc1053f9 specs: make dossiers deletion specs more robust 
It would fail when ran on some dates, like "21 Dec. 2021".
 2021-12-21 17:20:32 +01:00
Christophe Robillard
822c754581 add namespace for populate_zones task 2021-12-16 17:20:06 +01:00
Christophe Robillard
882f92268c update zone to procedures 2021-12-16 17:20:06 +01:00
François Vantomme
14e0c68266 feat(api particulier): add MESRI field 2021-12-16 16:45:18 +01:00
Martin
5ad0ee8ca9 fix(spec.flacky): try to use extractor in a more conventional way to extract file content 2021-12-14 14:18:59 +01:00
Martin
f7bc387e44 feat(flipper): use new way of generating archive only on some procedure flipped with new actor :zip_using_binary 2021-12-13 16:37:04 +01:00
simon lehericey
f0b0e7fd9a Switch to usage of zip unix binary to create archive. Also use a dedicated queue for DelayedJob 
use dedicated archives queue

As the used disk space will increase, we want a fined grain control

move zip logic in dedicated method

zip

wip

wip

fix(spec): pass spec in green

tech(improvements): avoid File.delete(folder), favor FileUtils.remove_entry_secure which is safer. Also wrap most of code that open file within blocks so it is cleaned when the block ends. Lastly use  attachement.download to avoid big memory pressure [download in chunk, write in chunk] otherwise big file [124>1GO] are loaded in memory. what if we run multiple jobs/download in parallel ?

fix(spec): try to retry with grace

clean(procedure_archive_service_spec.rb): better retry [avoid to rewrite on open file]

lint(things): everything
 2021-12-13 16:37:04 +01:00
François Vantomme
6ab8ac0662 feat (api particulier): add Pole emploi field 2021-12-08 10:36:25 +01:00
François Vantomme
2110ef072c feat(api particulier): add DGFiP field 2021-12-07 11:03:05 +01:00
Paul Chavard
2ba05bfb4f fix(dossier): use depose_at instead of en_construction_at 2021-12-06 15:49:17 +01:00
Martin
1bb868714c fix(spec/lint/review): lint and fix spec of previous commits, also fix based on tchak feedback 2021-12-06 07:05:17 +01:00
Pierre de La Morinerie
970c3e4b2b specs: avoid executing business logic in Procedure factory 
Calling business logic in a factory is a code-smell, because it
usually requires the object to be saved into database, and may have
unintended consequences when the business logic is changed.

Also, this allows to just build a published procedure, without saving it
to the database.
 2021-11-23 08:07:07 +01:00
Paul Chavard
71939c650b fix(export): remove wrong columns from repetition spreadsheets 
This fix prevent repetition children types de champ from being pulled from cloned procedures. stable_id is stable across revisions but also across cloned procedures.
 2021-11-10 20:45:32 +01:00