Christophe Robillard
609a1aa45b
add archive creation and purge archives jobs
2021-04-28 11:40:58 +02:00
simon lehericey
e636e3a752
add merge_and_update_metadata method
2021-04-06 18:11:26 +02:00
simon lehericey
d552e364fc
retry 5 times on integrity error and then block processing
2021-04-06 18:03:39 +02:00
Pierre de La Morinerie
75a1046315
active_storage: refactor concerns
...
Follow-up of #5953 .
Refactor the concerns with two goals:
- Getting closer from the way ActiveStorage adds its own hooks.
Usually ActiveStorage does this using an `Attachment#after_create`
hook, which then delegates to the blob to enqueue the job.
- Enqueuing each job only once. By hooking on `Attachment#after_create`,
we guarantee each job will be added only once.
We then let the jobs themselves check if they are relevant or not, and
retry or discard themselves if necessary.
We also need to update the tests a bit, because Rails'
`perform_enqueued_jobs(&block)` test helper doesn't honor the `retry_on`
clause of jobs. Instead it forwards the exception to the caller – which
makes the test fail.
Instead we use the inline version of `perform_enqueued_jobs()`, without
a block, which properly ignores errors catched by retry_on.
2021-03-16 11:49:14 +01:00
Pierre de La Morinerie
08094e5384
jobs: retry on ActiveStorage::IntegrityError during virus scan
...
This is the same setting than what ActiveStorage::AnalyzeJob uses.
2021-03-16 10:52:12 +01:00
Paul Chavard
d24ee27cac
Try to reduce the number of external data fetches
...
Only fetch external data if the external_id is the current one and data is not yet fetched
2021-03-11 14:56:37 +01:00
Paul Chavard
21d5da379f
remove unused job
2021-03-11 14:56:37 +01:00
Paul Chavard
7eee14feef
Add ChampFetchExternalDataJob
2021-02-17 19:10:16 +01:00
Pierre de La Morinerie
150ddab660
zeitwerk: Api -> API
2021-02-09 13:07:30 +01:00
Paul Chavard
9156754cf6
After maximum ammount of retries discard the job and log exception
2021-02-05 18:56:27 +01:00
Paul Chavard
41c3a98d7d
Update Raven references to use Sentry
2021-01-28 19:46:36 +01:00
Paul Chavard
0633d43868
Stop crashing if attachment can not be watermarked and mark blob as invalid
2021-01-27 16:49:09 +01:00
simon lehericey
650a598698
ensure a buggy procedure does not block the others from being closed
2021-01-19 10:42:16 +01:00
Paul Chavard
97ce8f312b
Use external_id to fetch annuaire_education data
2021-01-15 11:50:02 +01:00
Paul Chavard
5c9f2e8783
Add api education adapter and job
2021-01-14 17:57:48 +01:00
simon lehericey
1348b883e3
Add cron to complete missing analysis
2020-12-17 11:05:33 +01:00
kara Diaby
f3c4040d2d
add grace period to deleted dossiers
2020-12-16 15:00:26 +01:00
Paul Chavard
3c6e30a326
Show titre identite link only once watermark is added
2020-12-10 16:50:03 +01:00
clemkeirua
513d4f6ff1
move all the cron jobs in a dedicated directory
2020-12-08 13:22:43 +00:00
simon lehericey
b187244a29
extract and refactor api errors
2020-12-04 17:07:30 +01:00
simon lehericey
46c355beb2
Add comments
2020-12-03 11:52:01 +01:00
simon lehericey
f56235c7ee
retry exponentially on api entreprise timeout
2020-12-03 11:52:01 +01:00
simon lehericey
9eab310565
[ #5675 ] for all api entreprise job, retry on 502, 503
2020-12-02 18:25:31 +01:00
Paul Chavard
18dfacb933
Make watermark configurable
2020-11-25 16:19:06 +01:00
Paul Chavard
371179dc5b
Watermark titres identite
2020-11-25 16:19:06 +01:00
simon lehericey
c563956a9f
retry each day during 5 days to fetch attestation social
2020-10-28 15:53:58 +00:00
simon lehericey
ad01d4dee5
really faster stats by precomputing some of them
2020-10-06 16:12:01 +02:00
clemkeirua
245e9e59c7
do not run ApiEntreprise jobs on missing etablissements
2020-09-28 17:36:38 +02:00
Paul Chavard
dea78c49f6
Run jobs in named queues
2020-09-23 08:22:17 +02:00
simon lehericey
5459c2fa7f
Fix uninitialized excon constant
2020-09-08 18:36:24 +02:00
Pierre de La Morinerie
292dc84eb4
jobs: ignore deleted file in Virus Scan
...
We have errors in production where the job starts correctly (i.e. the
blob exists), but `blob.open` fails with a `ActiveStorage::FileNotFound`
error.
When checking later in production, the blob has been deleted.
This points to the blob (and the file) being deleted during the virus
scan job.
In that case, ignore the error (rather than retrying the job).
2020-09-03 11:00:59 +02:00
simon lehericey
49aa426d1b
application job swallow BadRequest error and retry
2020-09-02 16:59:54 +02:00
Paul Chavard
5ceae8235b
Ignore type_de_champ procedure_id
2020-09-02 11:26:26 +02:00
Paul Chavard
ec72fdd164
Remove migration service
2020-09-02 11:26:26 +02:00
Pierre de La Morinerie
3c91cfc83c
jobs: add health-related keywords to dubious procedures scanner
...
We are not certified for hosting health-related data yet.
2020-08-25 12:04:10 +00:00
Christophe Robillard
f2dcb61521
transform job in task for set dossiers_last_updated_at
2020-08-12 16:10:15 +02:00
Pierre de La Morinerie
54a4db1c47
config: flip config.active_storage.queues
...
ActiveStorage jobs are now moved to their own queue.
For consistency, we also move our own analysis jobs (VirusScannerJob)
on the same `:active_storage_analysis` queue.
2020-08-05 16:48:36 +02:00
clemkeirua
952a43058f
last_commentaire_updated_at does not take system email into account
2020-07-30 11:52:41 +02:00
clemkeirua
153c6aebd7
add temp job to set last_updated_at_* values for dossier
...
Co-authored-by: Christophe Robillard <christophe.robillard@beta.gouv.fr>
2020-07-30 10:31:14 +02:00
Paul Chavard
bb5a90da76
fix active_revision_id -> active_revision.id
2020-07-23 15:07:25 +02:00
Paul Chavard
7d421d471c
Fix revision migration job
2020-07-23 09:56:34 +02:00
Paul Chavard
bc09b61684
Add migration service and task
2020-07-21 19:35:30 +02:00
Pierre de La Morinerie
7a67f1a802
jobs: ignore missing blob during virus scan
...
We currently have many failed VirusScannerJob enqueued, because the
underlying blob is missing.
This PR fixes the issue by discarding the job in those cases (because if
the blob is gone, the job is never going to succeed).
The implementation is based on a similar issue encoutered by the
ActiveStorage::AnalyzeJob. See 06f8baf73c
2020-07-13 14:35:41 +02:00
Christophe Robillard
d20d91bc07
no sentry reporting for api-entreprise
2020-07-08 17:58:33 +02:00
Christophe Robillard
954d41ce4e
fetch effectifs mensuels for may month
2020-06-24 10:23:00 +02:00
Paul Chavard
e89f0846bb
Fix job max attempts
2020-06-16 15:57:18 +02:00
Christophe Robillard
f8b170836c
report to sentry for BadFormatRequest and ResourceNotFound exceptions
2020-06-11 23:09:50 +02:00
Christophe Robillard
b90513f456
no sentry report when error 400 for exercices
2020-06-11 18:16:19 +02:00
Paul Chavard
c4896b45ad
Prepare to remove dossier procedure_id
2020-06-11 16:13:44 +02:00
Christophe Robillard
8df9ae3a8d
ask only effectifs for april
2020-06-10 17:41:13 +02:00
