Commit graph

600 commits

Author SHA1 Message Date
Martin
1944624082 fix(PiecesJustificativesService.safe_attachment): utilise la nouvelle API pour comparer le resultat du scan antivirus 2023-01-06 10:54:27 +01:00
Paul Chavard
c4a30084fe refactor(virus_scan): remove lock 2023-01-04 12:12:11 +01:00
Paul Chavard
fc4afb9793 fix(pays): some Île are written with î for some reason… 2022-12-26 21:54:45 +00:00
Paul Chavard
f17690b1d3 feat(graphql): expose departement and region 2022-12-21 14:03:02 +01:00
Paul Chavard
4d4c378724 feat(api_geo): add api_geo service 2022-12-21 14:03:02 +01:00
Paul Chavard
227296682b fix(dossier): clone should include more attachments 2022-12-20 21:30:32 +01:00
simon lehericey
fe8bd15939 feat(timestamp): ensure signature is openssl compatible 2022-12-07 19:28:01 +01:00
simon lehericey
de2c5c69be feat(timestamp): use certigna to timestamp 2022-12-07 19:28:01 +01:00
Colin Darie
f4d403a81c fix(apiv1): piece_justificative continue to returns a single attachment 2022-12-06 15:47:06 +01:00
Martin
35dbc4021e poc(batch_operation): show locked files 2022-12-06 11:28:48 +01:00
Colin Darie
7cbe6bbaa4 fix(attachment): clone piece justificatives with multiple attachments 2022-12-05 10:47:10 +01:00
Colin Darie
3affd1660f fix(api-v1): piece justificative serialization 2022-12-05 10:47:10 +01:00
Colin Darie
91f4971df2 feat(champ): has_one_attached => has_many_attached piece_justificative_file 2022-12-05 10:47:10 +01:00
Paul Chavard
c0fd080d0e refactor(operation_log): store data in jsonb instead of files 2022-11-22 18:06:05 +01:00
Paul Chavard
d306583a7b refactor(active_storage): PiecesJustificativesService::FakeAttachment -> ActiveStorage::FakeAttachment 2022-11-22 18:05:26 +01:00
Paul Chavard
0c0618aa79 feat(export): add GeoJSON export 2022-11-18 10:04:35 +01:00
Martin
18b7f4e4fa remaniement(dossier.clone): simplifications, pas de callback/after, pas d'heritage, mm interface pour cloner les champs public/prive, clonage des PJs avec le nouveau service de clone, ajoute une cle etrangere sur la colonne parent_dossier_id 2022-11-14 09:53:19 +01:00
Paul Chavard
e6288c39f2 refactor(dossier): champs -> champs_public 2022-11-10 22:44:39 +00:00
Paul Chavard
95f65900d3 refactor(active_storage): no download on clone 2022-11-10 12:25:55 +00:00
Martin
9562108c60 amelioration(archive.nom-de-fichier): ameliore le nom du fichier d'une archive mensuelle pour eviter les confusion 2022-11-02 16:56:50 +01:00
Paul Chavard
2ed11308a8 feat(graphql): improuve reptition champs schema 2022-11-02 11:09:56 +01:00
Paul Chavard
e7766694d3 fix(messages): retry stale blobs 2022-11-02 09:19:51 +01:00
Martin
d9432ccd3b fix(etablissement.as_degraded_mode): backfill missing data via a cron 2022-09-23 15:12:25 +02:00
Martin
daeab50321 fix(data): backfill etablissement as degraded mode 2022-09-22 18:02:47 +02:00
Colin Darie
95a4e8a907
feat(api_entreprise): create etablissement with fallback when API is down
On créé l'établissement uniquement avec le SIRET, sans que ce soit bloquant
pour compléter le dossier. On demande à l'utilisateur de vérifier
lui-même la concordance du SIRET avec son entreprise.

Cf #7766
2022-09-16 10:43:59 +02:00
Colin Darie
d1c06aa92e feat(api_entreprise): add service api_up?, connected to their status page 2022-09-15 18:46:53 +02:00
Colin Darie
e8210e6c2b fix(agent_connect): constant path
https://sentry.io/organizations/demarches-simplifiees/issues/3248516694/?project=1429550&query=is%3Aunresolved&statsPeriod=14d
2022-08-03 10:19:45 +02:00
Kara Diaby
181e237cf0 add generation csv service 2022-07-29 11:17:22 +02:00
Paul Chavard
03fbe506f6 perf(graphql): benchmark demarches_publiques query 2022-07-26 12:54:10 +02:00
Christophe Robillard
d294feabe3 gzip demarches publiques export 2022-07-21 21:27:27 +02:00
Paul Chavard
8532c71036 perf(dossier.pdf): use DossierPreloader to generate pdf 2022-07-21 15:02:29 +02:00
Paul Chavard
260d8d2fec perf(graphql): add graphql benchmark 2022-07-21 12:29:08 +02:00
Paul Chavard
1fe82a956e fix(dossier.pdf): load all types_de_champ (including repetitions) 2022-07-21 11:56:23 +02:00
simon lehericey
ca8dc34196 preload all tdc once for all 2022-07-20 15:48:16 +02:00
simon lehericey
4dbd44d72a includes stuff 2022-07-20 15:48:16 +02:00
simon lehericey
b9ad3bbce1 extract procedure 2022-07-20 15:48:16 +02:00
simon lehericey
9c73bf1932 batch it 2022-07-20 15:48:16 +02:00
Christophe Robillard
25a1129ae4 use internal_use instead of require_admin for graphql context 2022-07-06 17:59:03 +02:00
Christophe Robillard
886f595350 [graphql] add dossiersCount field
visible only for admin
2022-07-06 17:59:03 +02:00
Christophe Robillard
7684e97494 export demarches publiques 2022-07-06 17:59:03 +02:00
Martin
7fb85c97cb feat(Administrateur::ArchivesControllers): add archives #index and #create for admin 2022-07-04 14:18:46 +02:00
Paul Chavard
ea18c2b9ba chore(build): use vitejs 2022-06-23 15:22:54 +02:00
Paul Chavard
350ed3a11a feat(graphql): expose revision on demarche descriptor type 2022-06-17 12:33:35 +02:00
Paul Chavard
bbc44fe339 fix(export): no crash on empty repetitions 2022-06-17 11:33:48 +02:00
Paul Chavard
564daeffe8 perf(dossier): improuve dossier preloading perf 2022-06-14 09:04:26 +01:00
Paul Chavard
b635952b2f refactor(revision): types_de_champ for export 2022-05-23 18:36:18 +02:00
Paul Chavard
9e0b3b642f cleanup(sendinblue): remove sendinblue tracking 2022-05-06 11:14:44 +02:00
simon lehericey
ea361643be types_de_champ -> types_de_champs_public 2022-05-03 11:05:24 +02:00
Martin
79c473ecf9 fix(ProcedureExportService.to_zip): prebatch dossiers only for to_csv,to_xlsx,to_ods. also add spec around to_zip 2022-04-22 05:13:51 +02:00
Paul Chavard
433c01b1e6 Revert "Merge pull request #7137 from betagouv/faster_pdf"
This reverts commit 9da44bd913, reversing
changes made to ebac71796c.
2022-04-12 19:22:07 +02:00
Paul Chavard
d14e132305 feat(export): add zip format support 2022-04-12 14:54:54 +02:00
Paul Chavard
2832ea0286 refactor(export): return blob from to_* methods 2022-04-12 14:54:54 +02:00
Paul Chavard
87e5e6faeb refactor(export): add procedure attr to export service 2022-04-12 14:54:54 +02:00
Paul Chavard
561b83781e refactor(archive): remove dependencie on archive from ArchiveUploader 2022-04-12 14:54:54 +02:00
Paul Chavard
cf8c084a59 refactor: extract download_and_zip in to a shared service 2022-04-12 14:54:54 +02:00
simon lehericey
4bbe2db668 only export safe pj 2022-04-12 12:05:08 +02:00
simon lehericey
2269585188 preload all tdc once for all 2022-04-12 11:56:56 +02:00
simon lehericey
4c0b157383 includes stuff 2022-04-12 11:56:56 +02:00
simon lehericey
2c195e3ce2 extract procedure 2022-04-12 11:56:56 +02:00
simon lehericey
4d3412daf5 batch it 2022-04-12 11:56:56 +02:00
simon lehericey
1dcfb2509f check nonce 2022-04-11 14:40:02 +02:00
simon lehericey
9938586d96 check state 2022-04-11 13:56:27 +02:00
simon lehericey
e2a54e3ee3 extract bill_ids per batch, and catch them all afterwards to avoid duplicate 2022-04-07 12:06:13 +02:00
simon lehericey
f24f6ee105 extract operation_logs_and_signature from pjs_for_dossiers 2022-04-07 12:06:13 +02:00
simon lehericey
c27d9a01f2 fetch documents by batch to avoid huge memory load 2022-04-07 12:06:13 +02:00
simon lehericey
e1afb35ca2 fetch pjs_for_dossier by dossiers
put all bills in bills folder
2022-04-07 12:06:13 +02:00
simon lehericey
11aedb2dc8 fetch pjs_for_commentaires by dossiers 2022-04-07 12:06:13 +02:00
simon lehericey
5631141a46 fetch pjs_for_champ by dossiers 2022-04-07 12:06:13 +02:00
simon lehericey
34b0578d70 pj_and_path only take dossier id 2022-04-07 12:06:13 +02:00
simon lehericey
7ac1288905 pj_service take a dossier collection ! 2022-04-07 12:06:13 +02:00
Martin
9e8807d12a feat(ArchiveUploader.upload_with_chunking_wrapper): retry once on error 2022-04-05 15:11:21 +02:00
Martin
c1884f231c Revert "Merge pull request #7105 from betagouv/US/fix-dossier.processed_in_month"
This reverts commit a0e144b9a7, reversing
changes made to 49848bd150.
2022-04-05 13:39:37 +02:00
Martin
c07e0fc13e fix(Dossier.processed_in_month): ensure proper usage via method sig instead of defensive programming style 2022-04-05 12:14:07 +02:00
simon lehericey
bd0b88a410 move create_list_of_attachments 2022-04-05 11:55:14 +02:00
simon lehericey
1f98f75ccc remove unused method 2022-04-05 11:55:14 +02:00
simon lehericey
f2fea1f882 faster pjs_for_dossier 2022-04-05 11:55:14 +02:00
simon lehericey
62e0553a4e faster operation logs 2022-04-05 11:55:14 +02:00
simon lehericey
437e871f79 extract operation_logs_and_signatures method 2022-04-05 11:55:14 +02:00
simon lehericey
57f9e5bac3 always allow dossier pjs download (-9 queries) 2022-04-04 17:26:49 +02:00
simon lehericey
dca6e65f8d speed up commentaires 2022-04-01 15:51:43 +02:00
simon lehericey
0555ff68cd speed up pjs_for_champs * 10 2022-04-01 15:51:41 +02:00
Martin
dbcf21a555 feat(archive): extract archive status management within job to simplify the main service as well as to isolate this part for a merge with exports csv/xslx [maybe?]
Update app/dashboards/archive_dashboard.rb

Co-authored-by: LeSim <mail@simon.lehericey.net>
2022-03-31 13:35:49 +02:00
Paul Chavard
44c64669e9 Revert "Merge pull request #6787 from tchak/use-vite"
This reverts commit 5d572727b5, reversing
changes made to 43be4482ee.
2022-03-31 12:07:52 +02:00
Martin
ab2caaa5f7 fix(ProcedureArchiveService.zip_root_folder): should take archive instance otherwise when we generate many archive for the same procedure, errors may occures 2022-03-30 16:29:54 +02:00
Paul Chavard
187e84a010 feat(assets): use vitejs to build javascript 2022-03-29 16:27:08 +02:00
Martin
98c1fb8abc feat(archive): drop old feature 2022-03-18 14:26:09 +01:00
Martin
5739150f15 feat(service/archive_uploader): add an archive uploader class to upload files thru a custom script which handle file encryption of massive file (bigger than 4Go)
Update doc/object-storange-and-data-encryption.md

Co-authored-by: LeSim <mail@simon.lehericey.net>

Update app/services/archive_uploader.rb

Co-authored-by: LeSim <mail@simon.lehericey.net>

Update doc/object-storange-and-data-encryption.md

Co-authored-by: Pierre de La Morinerie <kemenaran@gmail.com>

clean(doc): align document file name and document h1

clean(review): refactore based on various comments

clean(review): refactore based on various comments
2022-03-16 14:56:21 +01:00
Paul Chavard
ec2f2dc78c feat(graphql): expose more dossier informations 2022-03-14 15:58:02 +01:00
Paul Chavard
54b559364a feat(dossier): replace discarded with visible_by_administration 2022-03-10 14:29:40 +01:00
Jon
97feca6305 feat(ClamAV): add config to disable clamav usage 2022-02-15 09:15:47 +01:00
Kara Diaby
5d10158fa6 Instructeur : ne peut plus cliquer sur un dossier supprimé dans la recherche 2022-02-03 11:17:39 +01:00
Paul Chavard
3d8471e064 fix(dossier): do not send notification on expiration when dossier is already deleted 2022-01-19 17:52:53 +01:00
Paul Chavard
7937e58caa fix(archives): only export dossiers in archive groupe_instructeurs
fix #6793
2022-01-18 11:16:20 +01:00
Martin
ce1b189dcd refactor(DownloadManager): extract parallel download in dedicated class. move error management in custom class for procedure exports using the didicated class 2022-01-04 16:17:03 +01:00
Martin
2ed9cccba0 extract download all attachments in dedicated class using async/async-http for better perf 2022-01-04 16:17:03 +01:00
Christophe Robillard
882f92268c update zone to procedures 2021-12-16 17:20:06 +01:00
Martin
721d926c0d wip(fix): chdir 2021-12-13 16:37:04 +01:00
mfo
148be50c86 Update app/services/procedure_archive_service.rb
Co-authored-by: LeSim <mail@simon.lehericey.net>
2021-12-13 16:37:04 +01:00
Martin
f7bc387e44 feat(flipper): use new way of generating archive only on some procedure flipped with new actor :zip_using_binary 2021-12-13 16:37:04 +01:00
simon lehericey
f0b0e7fd9a Switch to usage of zip unix binary to create archive. Also use a dedicated queue for DelayedJob
use dedicated archives queue

As the used disk space will increase, we want a fined grain control

move zip logic in dedicated method

zip

wip

wip

fix(spec): pass spec in green

tech(improvements): avoid File.delete(folder), favor FileUtils.remove_entry_secure which is safer. Also wrap most of code that open file within blocks so it is cleaned when the block ends. Lastly use  attachement.download to avoid big memory pressure [download in chunk, write in chunk] otherwise big file [124>1GO] are loaded in memory. what if we run multiple jobs/download in parallel ?

fix(spec): try to retry with grace

clean(procedure_archive_service_spec.rb): better retry [avoid to rewrite on open file]

lint(things): everything
2021-12-13 16:37:04 +01:00