DGNum/demarches-normaliennes
14
1
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity
17104 commits 4 branches 1166 tags 84 MiB
Commit graph

342 commits

Author SHA1 Message Date
Martin
383a92bcd9 fix(parallel_download_queue): tophoeus does not like raise from a request handler [crash straight to first frame] 2022-01-07 14:24:37 +01:00
Martin
3eb1c1a421 feat(async_backend): switch to typhoeus 2022-01-04 16:27:23 +01:00
Martin
ce1b189dcd refactor(DownloadManager): extract parallel download in dedicated class. move error management in custom class for procedure exports using the didicated class 2022-01-04 16:17:03 +01:00
Martin
2ed9cccba0 extract download all attachments in dedicated class using async/async-http for better perf 2022-01-04 16:17:03 +01:00
François Vantomme
14e0c68266 feat(api particulier): add MESRI field 2021-12-16 16:45:18 +01:00
François Vantomme
115adbec70 style(api particulier): harmonize pole emploi resource name constant 2021-12-16 16:45:17 +01:00
François Vantomme
3d81d4b541 feat(api particulier): add MESRI adapter 2021-12-16 16:45:17 +01:00
François Vantomme
66a9109cd7 feat(api particulier): add MESRI support to sources service 2021-12-16 16:45:17 +01:00
simon lehericey
f0b0e7fd9a Switch to usage of zip unix binary to create archive. Also use a dedicated queue for DelayedJob 
use dedicated archives queue

As the used disk space will increase, we want a fined grain control

move zip logic in dedicated method

zip

wip

wip

fix(spec): pass spec in green

tech(improvements): avoid File.delete(folder), favor FileUtils.remove_entry_secure which is safer. Also wrap most of code that open file within blocks so it is cleaned when the block ends. Lastly use  attachement.download to avoid big memory pressure [download in chunk, write in chunk] otherwise big file [124>1GO] are loaded in memory. what if we run multiple jobs/download in parallel ?

fix(spec): try to retry with grace

clean(procedure_archive_service_spec.rb): better retry [avoid to rewrite on open file]

lint(things): everything
 2021-12-13 16:37:04 +01:00
Paul Chavard
b67818e425 feat(helpscout): add a tag to messages submited through contact form 2021-12-09 11:47:13 +01:00
François Vantomme
4cfc9f7d86 feat(api particulier): add Pole Emploi adapter 2021-12-08 10:36:11 +01:00
François Vantomme
cfb8c184fe perf(api particulier): use filter_map instead of map.compact 
https://docs.rubocop.org/rubocop-performance/cops_performance.html#performancemapcompact
 2021-12-08 10:36:11 +01:00
François Vantomme
a24cf18c9d feat (api particulier): add Pole Emploi support to sources service 2021-12-08 10:36:11 +01:00
Pierre de La Morinerie
c75dba701d lib: replace URI.encode by CGI.escape 
URI.encode was removed in Ruby 3.

Fix an error when adding a phone number to a Helpscout ticket.
 2021-12-07 15:25:26 +01:00
François Vantomme
f0913edebd feat(api particulier): add DGFiP adapter 2021-12-07 11:03:05 +01:00
François Vantomme
97de7c197d feat(api particulier): add DGFIP support to sources service 2021-12-07 11:03:05 +01:00
kara Diaby
9ee9389ba1 add the good retry_on StaleObjectError 2021-11-05 13:32:21 +01:00
simon lehericey
35c7f05a0a source service supports unknown scope 2021-10-12 14:27:20 +02:00
simon lehericey
b69dafc3d4 CNAF in lowercase 2021-10-12 14:26:40 +02:00
simon lehericey
17a2b5dc53 fix a strange encoding error 2021-10-12 14:26:40 +02:00
Pierre de La Morinerie
0b02fce5e4 jobs: move ActiveJobLogSubscriber out of initializers 
This is a class of its own, it doesn't need to be in the initializers.
 2021-10-12 11:40:19 +02:00
kara Diaby
6ef5b5d474 modify downloable file 2021-10-05 15:00:21 +02:00
simon lehericey
57604b9c89 add schema validations 2021-09-22 12:08:24 +02:00
simon lehericey
75043070da add CNAFAdapter 2021-09-22 12:08:24 +02:00
simon lehericey
a7651e3772 add api composition_familiale 2021-09-22 12:08:24 +02:00
simon lehericey
ceb09c5967 api remove useless symbolize 2021-09-22 12:08:24 +02:00
simon lehericey
07962cc2c7 service: match remote keys 2021-09-22 12:08:24 +02:00
simon lehericey
8d747ff16e add sanitize method to service 2021-09-21 11:21:10 -05:00
Pierre de La Morinerie
6656d1130b i18n: refactor admin support locales 
This allow i18n-tasks to better detect dynamic keys.
 2021-09-21 10:44:20 -05:00
Pierre de La Morinerie
f9220891a7 i18n: refactor user support locales 
This allow i18n-tasks to better detect dynamic keys.
 2021-09-21 10:44:20 -05:00
simon lehericey
8327c6fd0c add ApiParticulier::Service::Sources 2021-09-17 13:50:23 +02:00
simon lehericey
87cb16093f fetch token introspection 
Co-authored-by: François VANTOMME <akarzim@gmail.com>
 2021-09-15 14:37:04 +02:00
Christophe Robillard
ddea0b9a45 purge old sendinblue mails 2021-08-20 08:52:39 +02:00
Paul Chavard
44ebd0e8a5 Fix api education wrong data type 2021-06-08 12:25:24 +02:00
Paul Chavard
01c558953b Remove API GEO legacy adapter 2021-05-24 11:50:16 +02:00
Christophe Robillard
01f180022b prend en compte uniquement les pj pour estimer la taille d'un dossier 2021-05-04 12:50:12 +02:00
Pierre de La Morinerie
13d2364fd5 jobs: add Excon::Error::Socket to the list of transient errors 
In ActiveStorage::Purge job we see a lot of SSL errors:

> Connection reset by peer - SSL_connect (Errno::ECONNRESET)

These errors seem transient, and resolve themselves after a while.
 2021-05-04 11:38:22 +02:00
Christophe Robillard
3d04221dab generate pdf export for dossier only on demand 2021-04-29 17:30:38 +02:00
Christophe Robillard
f40d96fbd2 Revert "Revert "Export de tous les dossier d'une démarche"" 
This reverts commit d9a588b52e.
 2021-04-29 17:29:47 +02:00
krichtof
d9a588b52e
Revert "Export de tous les dossier d'une démarche" 2021-04-29 16:07:18 +02:00
Pierre de La Morinerie
2cb6ec7c51 jobs: add more transient errors 
These errors occur frequently during interactions with OVH's
ObjectStorage.
 2021-04-29 14:08:12 +02:00
Pierre de La Morinerie
684af77e35 jobs: extract an RetryOnTranscientErrors concern 2021-04-29 14:08:12 +02:00
Christophe Robillard
fba0d78153 generate pdf export inside pieces_justificatives_service 2021-04-28 11:40:58 +02:00
Pierre de La Morinerie
872f6b0153 lib: add migration helpers for making a column unique 2021-04-13 18:09:59 +02:00
Pierre de La Morinerie
16f695031b manager: add Mailjet emails to the sent emails list 2021-04-13 10:58:30 +02:00
Pierre de La Morinerie
5e8327ff09 manager: refactor SendInBlue unblock action 2021-04-13 10:57:26 +02:00
Pierre de La Morinerie
76e74003ad manager: fix SendInBlue emails 2021-04-13 10:57:26 +02:00
simon lehericey
cd4ad6878c remove insee_api_v3 feature 2021-04-06 18:23:14 +02:00
simon lehericey
d552e364fc retry 5 times on integrity error and then block processing 2021-04-06 18:03:39 +02:00
Christophe Robillard
858bbde2d6 remove dead code 2021-03-16 15:13:17 +01:00
Christophe Robillard
60cc4d3697 add pdf to pjs export 2021-03-16 15:13:17 +01:00
Paul Chavard
228a83f8a5 Add api address external data fetch 2021-02-17 19:10:16 +01:00
simon lehericey
f7ae974f29 Add throttling for api entreprise 2021-02-10 12:40:14 +01:00
Pierre de La Morinerie
150ddab660 zeitwerk: Api -> API 2021-02-09 13:07:30 +01:00
Paul Chavard
97ce8f312b Use external_id to fetch annuaire_education data 2021-01-15 11:50:02 +01:00
Paul Chavard
5c9f2e8783 Add api education adapter and job 2021-01-14 17:57:48 +01:00
Paul Chavard
a2e87cbb56 Add attestation, justificatifs, operation_logs and bill_signatures to dossier export 2020-12-16 16:34:31 +01:00
Fabrice Gangler
3a9f82bd6f Fix(form /contact-admin): use APPLICATION_NAME 
Refs: #5799
 2020-12-15 16:14:51 +01:00
kara Diaby
f748ccfc9e Appelle l'API entreprise avec le token en Header 2020-12-10 17:49:49 +01:00
simon lehericey
b187244a29 extract and refactor api errors 2020-12-04 17:07:30 +01:00
simon lehericey
f56235c7ee retry exponentially on api entreprise timeout 2020-12-03 11:52:01 +01:00
simon lehericey
78a40b5176 Increase api_entreprise_timeout 2020-12-03 10:26:05 +01:00
simon lehericey
9eab310565 [#5675] for all api entreprise job, retry on 502, 503 2020-12-02 18:25:31 +01:00
simon lehericey
02e7de0003 more details when Typhoeus request fails 2020-12-02 18:25:31 +01:00
kara Diaby
bc0f77978f add an api token verification for api entreprise 2020-11-09 16:58:24 +01:00
simon lehericey
c563956a9f retry each day during 5 days to fetch attestation social 2020-10-28 15:53:58 +00:00
Christophe Robillard
61ea73580f use buttons instead of select for contact page 2020-09-22 16:09:15 +02:00
Judith
2f0953692c Internationalization of the Contact page in FR and EN 2020-08-27 11:03:34 +02:00
Christophe Robillard
f77cc06a91 prend en compte le nom de l'enseigne pour l'établissement 2020-08-20 14:23:02 +02:00
clemkeirua
7e085c657d specific deactivation of rubocop DS/ApplicationName rule 2020-07-23 16:20:16 +02:00
clemkeirua
d31b4b6848 ajout d'un test sur le telechargement de zips 2020-07-21 15:45:57 +02:00
clemkeirua
48b96046f1 pad file name with creation date 
During zip creation, when 2 files have an identical name it creates a corrupted zip file
Duplicate files can now be downloaded with all the versions, the file names contain a timestamp
 2020-07-20 15:33:22 +02:00
clemkeirua
def81f9de8 fix zip download for everybody 2020-07-20 11:14:34 +02:00
Paul Chavard
6a24c3f812 Rails app:update 2020-07-07 18:03:56 +02:00
clemkeirua
d5c955080b fix demandes adapter when no demandes 2020-07-07 09:24:23 +02:00
Christophe Robillard
f9b1b8217a more verbose exception when RequestFailed for apientreprise occurs 2020-06-11 17:24:45 +02:00
Christophe Robillard
8df9ae3a8d ask only effectifs for april 2020-06-10 17:41:13 +02:00
Christophe Robillard
1b509c15f8 raise exception with url 2020-05-20 15:53:08 +02:00
Christophe Robillard
d92a0ba8f3 raise BadFormatRequest error 
L'intention est de distinguer ce type d'erreur avec les erreurs serveur
5xx.
Lorsque l'api est appelé par un Job, les erreurs serveur seront retry, car éphémères.
Alors que les erreurs de type BadFormatRequest ou ResourceNotFound n'ont
pas être retried.
 2020-05-20 11:09:54 +02:00
Christophe Robillard
3b46594e60 precise http error code for api calls 2020-05-20 11:09:54 +02:00
Christophe Robillard
e3ee423197 call api_entreprise adapters with siret 2020-05-20 11:09:54 +02:00
Christophe Robillard
dbf04dd0d8 make api entreprise call only if token not expired 2020-05-06 10:57:45 +02:00
Christophe Robillard
85b2e9ef44 create bilans_bdf_adapter 2020-04-30 16:15:16 +02:00
Christophe Robillard
38c68b16e3 create attestation_fiscale_adapter 2020-04-30 11:39:54 +02:00
Christophe Robillard
158b4802ab store attestation sociale and display 2020-04-29 16:25:47 +02:00
Christophe Robillard
d74b14c205 fetch attestation sociale url 2020-04-29 14:25:55 +02:00
Christophe Robillard
b147956a92 create attestation_sociale_adapter 2020-04-29 14:01:48 +02:00
Christophe Robillard
e29af48420 call api_entreprise with specific token if exist 2020-04-28 15:17:11 +02:00
Christophe Robillard
453adf3c74 instructeurs: affiche l'effectif moyen annuel n-1 2020-04-23 09:53:27 +02:00
Christophe Robillard
1165f8483d rename effectif columns 2020-04-21 21:13:07 +02:00
Christophe Robillard
599d8e33e8 create ApiEntreprise::EffectifsAdapter 2020-04-21 14:22:11 +02:00
Paul Chavard
52d2ace823 Remove quartiers prioritaires 2020-04-14 10:29:59 +02:00
Christophe Robillard
91abcd7792 récupère les infos d'établissement avec infos non diffusables 2020-03-17 15:57:02 +00:00
Paul Chavard
c281347da1 Remove restclient 2020-03-17 11:12:14 +01:00
Paul Chavard
577d21de5c Use the new robot api id 2020-03-13 15:36:17 +01:00
Paul Chavard
14295db9ad Revert "Revert "Merge pull request #4552 from tchak/champ-communes"" 
This reverts commit 4373cb22cb.
 2020-01-14 18:46:07 +01:00
clemkeirua
4373cb22cb Revert "Merge pull request #4552 from tchak/champ-communes" 
This reverts commit 4cec26f73a, reversing
changes made to 0ef25ef36c.
 2020-01-13 16:26:27 +01:00
Paul Chavard
e61e39d345 Remove unused code and tests 2020-01-07 11:52:51 +01:00
maatinito
9de9a1fd71 Use time.zone functions instead of server time functions 
(Time.zone.today vs Date.today)
 2019-11-26 14:26:36 +01:00
clemkeirua
9503a363e5 4378 Ajout des PJ de la messagerie dans le zip des PJs du dossier 2019-11-25 10:36:32 +01:00