Commit graph

298 commits

Author SHA1 Message Date
Martin
e765ef16fd fix(parallel_download_queue): appending content to file in those circumstances does not lead to good result. for now write response.body directly to the attachment [may lead to RAM overhead in case of big file... but i do not have another quick fix idea] 2022-04-01 18:22:28 +02:00
simon lehericey
31d52d2483 do not open all fd at once 2022-03-17 22:44:53 +01:00
Pierre de La Morinerie
3a16235868 db: Add a delete_orphans db helper 2022-03-08 14:47:43 +01:00
Pierre de La Morinerie
847abca122 config: simplify mailer configuration again
Move everything to initializers, and replace the email settings
interceptor by a BalancerDeliveryMethod.

It has the advantage that it can be configured entirely from the
`config/environment.rb` file, without an extra file to look at.
2022-02-08 12:44:43 +01:00
Pierre de La Morinerie
04cfc8ed9d config: use alternate delivery methods to configure ActionMailer
Previously `SENDINBLUE_BALANCING` was used only when
`SENDINBLUE_ENABLED` was *disabled* (otherwise only SendInBlue was ever
used).

This commit:

- Ensure that `SENDINBLUE_BALANCING` is used only when SendInBlue is
  *enabled* (which is more intuitive).
- Make it easier to add other delivery methods.
2022-02-08 12:44:43 +01:00
Pierre de La Morinerie
68fd3aea39 config: rename SOURCE to LOGRAGE_SOURCE 2022-01-19 15:43:44 +01:00
Martin
383a92bcd9 fix(parallel_download_queue): tophoeus does not like raise from a request handler [crash straight to first frame] 2022-01-07 14:24:37 +01:00
Martin
3eb1c1a421 feat(async_backend): switch to typhoeus 2022-01-04 16:27:23 +01:00
Martin
ce1b189dcd refactor(DownloadManager): extract parallel download in dedicated class. move error management in custom class for procedure exports using the didicated class 2022-01-04 16:17:03 +01:00
Martin
2ed9cccba0 extract download all attachments in dedicated class using async/async-http for better perf 2022-01-04 16:17:03 +01:00
François Vantomme
14e0c68266 feat(api particulier): add MESRI field 2021-12-16 16:45:18 +01:00
François Vantomme
115adbec70 style(api particulier): harmonize pole emploi resource name constant 2021-12-16 16:45:17 +01:00
François Vantomme
3d81d4b541 feat(api particulier): add MESRI adapter 2021-12-16 16:45:17 +01:00
François Vantomme
66a9109cd7 feat(api particulier): add MESRI support to sources service 2021-12-16 16:45:17 +01:00
simon lehericey
f0b0e7fd9a Switch to usage of zip unix binary to create archive. Also use a dedicated queue for DelayedJob
use dedicated archives queue

As the used disk space will increase, we want a fined grain control

move zip logic in dedicated method

zip

wip

wip

fix(spec): pass spec in green

tech(improvements): avoid File.delete(folder), favor FileUtils.remove_entry_secure which is safer. Also wrap most of code that open file within blocks so it is cleaned when the block ends. Lastly use  attachement.download to avoid big memory pressure [download in chunk, write in chunk] otherwise big file [124>1GO] are loaded in memory. what if we run multiple jobs/download in parallel ?

fix(spec): try to retry with grace

clean(procedure_archive_service_spec.rb): better retry [avoid to rewrite on open file]

lint(things): everything
2021-12-13 16:37:04 +01:00
Paul Chavard
b67818e425 feat(helpscout): add a tag to messages submited through contact form 2021-12-09 11:47:13 +01:00
François Vantomme
4cfc9f7d86 feat(api particulier): add Pole Emploi adapter 2021-12-08 10:36:11 +01:00
François Vantomme
cfb8c184fe perf(api particulier): use filter_map instead of map.compact
https://docs.rubocop.org/rubocop-performance/cops_performance.html#performancemapcompact
2021-12-08 10:36:11 +01:00
François Vantomme
a24cf18c9d feat (api particulier): add Pole Emploi support to sources service 2021-12-08 10:36:11 +01:00
Pierre de La Morinerie
c75dba701d lib: replace URI.encode by CGI.escape
URI.encode was removed in Ruby 3.

Fix an error when adding a phone number to a Helpscout ticket.
2021-12-07 15:25:26 +01:00
François Vantomme
f0913edebd feat(api particulier): add DGFiP adapter 2021-12-07 11:03:05 +01:00
François Vantomme
97de7c197d feat(api particulier): add DGFIP support to sources service 2021-12-07 11:03:05 +01:00
kara Diaby
9ee9389ba1 add the good retry_on StaleObjectError 2021-11-05 13:32:21 +01:00
simon lehericey
35c7f05a0a source service supports unknown scope 2021-10-12 14:27:20 +02:00
simon lehericey
b69dafc3d4 CNAF in lowercase 2021-10-12 14:26:40 +02:00
simon lehericey
17a2b5dc53 fix a strange encoding error 2021-10-12 14:26:40 +02:00
Pierre de La Morinerie
0b02fce5e4 jobs: move ActiveJobLogSubscriber out of initializers
This is a class of its own, it doesn't need to be in the initializers.
2021-10-12 11:40:19 +02:00
kara Diaby
6ef5b5d474 modify downloable file 2021-10-05 15:00:21 +02:00
simon lehericey
57604b9c89 add schema validations 2021-09-22 12:08:24 +02:00
simon lehericey
75043070da add CNAFAdapter 2021-09-22 12:08:24 +02:00
simon lehericey
a7651e3772 add api composition_familiale 2021-09-22 12:08:24 +02:00
simon lehericey
ceb09c5967 api remove useless symbolize 2021-09-22 12:08:24 +02:00
simon lehericey
07962cc2c7 service: match remote keys 2021-09-22 12:08:24 +02:00
simon lehericey
8d747ff16e add sanitize method to service 2021-09-21 11:21:10 -05:00
Pierre de La Morinerie
6656d1130b i18n: refactor admin support locales
This allow i18n-tasks to better detect dynamic keys.
2021-09-21 10:44:20 -05:00
Pierre de La Morinerie
f9220891a7 i18n: refactor user support locales
This allow i18n-tasks to better detect dynamic keys.
2021-09-21 10:44:20 -05:00
simon lehericey
8327c6fd0c add ApiParticulier::Service::Sources 2021-09-17 13:50:23 +02:00
simon lehericey
87cb16093f fetch token introspection
Co-authored-by: François VANTOMME <akarzim@gmail.com>
2021-09-15 14:37:04 +02:00
Christophe Robillard
ddea0b9a45 purge old sendinblue mails 2021-08-20 08:52:39 +02:00
Paul Chavard
44ebd0e8a5 Fix api education wrong data type 2021-06-08 12:25:24 +02:00
Paul Chavard
01c558953b Remove API GEO legacy adapter 2021-05-24 11:50:16 +02:00
Christophe Robillard
01f180022b prend en compte uniquement les pj pour estimer la taille d'un dossier 2021-05-04 12:50:12 +02:00
Pierre de La Morinerie
13d2364fd5 jobs: add Excon::Error::Socket to the list of transient errors
In ActiveStorage::Purge job we see a lot of SSL errors:

> Connection reset by peer - SSL_connect (Errno::ECONNRESET)

These errors seem transient, and resolve themselves after a while.
2021-05-04 11:38:22 +02:00
Christophe Robillard
3d04221dab generate pdf export for dossier only on demand 2021-04-29 17:30:38 +02:00
Christophe Robillard
f40d96fbd2 Revert "Revert "Export de tous les dossier d'une démarche""
This reverts commit d9a588b52e.
2021-04-29 17:29:47 +02:00
krichtof
d9a588b52e
Revert "Export de tous les dossier d'une démarche" 2021-04-29 16:07:18 +02:00
Pierre de La Morinerie
2cb6ec7c51 jobs: add more transient errors
These errors occur frequently during interactions with OVH's
ObjectStorage.
2021-04-29 14:08:12 +02:00
Pierre de La Morinerie
684af77e35 jobs: extract an RetryOnTranscientErrors concern 2021-04-29 14:08:12 +02:00
Christophe Robillard
fba0d78153 generate pdf export inside pieces_justificatives_service 2021-04-28 11:40:58 +02:00
Pierre de La Morinerie
872f6b0153 lib: add migration helpers for making a column unique 2021-04-13 18:09:59 +02:00