Paul Chavard
a6e3ea4839
fix(dossier): normalize champs commune data
2023-04-07 16:02:13 +02:00
Colin Darie
4288f28ac8
chore(job): same max_run_time between ArchiveCreationJob and ExportJob
2023-03-09 15:37:13 +01:00
Colin Darie
697aeeb348
refactor(job): DossieRebaseJob in new queue so they don't interfere with others
...
Ces jobs sont empilés par milliers d'un coup et peuvent mettre des dizaines
de minutes à s'écouler. On ne veut pas que ça interfère avec d'autres jobs,
notamment les scans antivirus qui, si retardés, empilent des "poll" pour
connaitre le status du fichier.
2023-03-09 15:18:14 +01:00
Colin Darie
86f15b8c05
chore(job): config delayed job for long-living export jobs
2023-03-08 18:37:41 +01:00
Colin Darie
6b87c290cb
fix(cron): PurgeOldEmailEvent in batches
2023-03-08 15:31:10 +01:00
Colin Darie
d20114f978
chore(sentry): use "procedure" tag everywhere for a better debug experience
2023-03-08 13:48:44 +01:00
Colin Darie
43613009d6
fix(job): Cron::WeeklyOverviewJob simplified and respect dolist API rate limiting
2023-03-06 14:39:55 +01:00
Colin Darie
f8f14830a6
fix(missing-antivirus-job): retry when backend is unavailable
2023-03-06 14:39:55 +01:00
Colin Darie
e9585632d9
fix(job): hopefully fix pg timeouts in Cron::PurgeUnattachedBlobsJob
2023-03-02 13:42:04 +01:00
sebastiencarceles
f52554b5a3
review: update value with async fetch
...
1 - spec cover the job which fetches external data
2 - refactor the job with guard clauses
3 - delegate update operation to the champ itself
4 - annuaire education: override the update operation
to let the value be populated by the fetched data
5 - prefilling: don't fetch data synchronously
2023-02-28 14:53:08 +01:00
Paul Chavard
f6b528e497
chore(geo_area): normalize all geo_areas
2023-02-15 11:58:59 +01:00
sebastiencarceles
f0ffae8320
migrate(champs): normalize departements
2023-02-13 16:32:34 +01:00
Colin Darie
a4329b6699
chore(export/job): set procedure_id for sentry
2023-02-09 11:06:54 +01:00
Colin Darie
964f3829c6
fix(export/job): race condition when export was already generated
2023-02-09 11:06:54 +01:00
Martin
f4cc8078b4
amelioration(EmailEvent): purge les email_events vieux de plus de un mois
...
Update app/models/email_event.rb
Co-authored-by: Colin Darie <colin@darie.eu>
Update app/jobs/cron/purge_old_email_event_job.rb
Co-authored-by: Colin Darie <colin@darie.eu>
2023-02-06 10:35:57 +01:00
Lisa Durand
120fbb0ef7
Overload the finished_at
2023-02-01 17:06:04 +01:00
Lisa Durand
d0a131dc55
change the way we set the finished_at on batch operation job to fix alert display bug
2023-02-01 17:06:04 +01:00
Paul Chavard
dd2d3d4626
fix(task): improuve after party backfill_dossiers_repetitions
2023-02-01 15:27:07 +01:00
Paul Chavard
8bf0d9c5ec
fix(dossier): backfill missing repetition champs
2023-02-01 10:36:46 +01:00
Damien Le Thiec
3a8a50a216
Allow prefill pays type de champs ( #8344 )
...
* Allow prefill pays type de champs
* Avoid conditional prefil params for pays champ
* Clean pays data with batch update
* Fix bug and add test batch update pays value
* Improve performance batch_update_pays
* Fix associated country code problem
* Fix after party task name
* Format country name if needed in batch update
2023-01-18 12:52:38 +01:00
Colin Darie
784b0458fe
feat(dolist): consolidate dispatched events with delivered emails
2023-01-18 09:56:40 +01:00
Damien Le Thiec
5e26acb0e1
Make date and datetime prefillable ( #8304 )
...
* Make date and datetime prefillable
* Format in ISO8601 format
2023-01-12 17:42:02 +01:00
Damien Le Thiec
d5bfe84381
Nettoyage des dossiers préremplis abandonnés ( #8341 )
...
* Clean prefilled dossiers
2023-01-12 16:14:42 +01:00
Paul Chavard
dfe65136be
refactor(repetition): add row_id to champs
2023-01-05 18:04:54 +01:00
Martin
018a94f43b
correctif(batch_operation): empeche de changer l'etat d'un dossier en cas d'erreur. et s'assure de delinker le dossier en cas de dossier invalid
2023-01-05 16:20:18 +01:00
Paul Chavard
6e18090fb3
refactor(virus_scan_result): use column instead of metadata on blob
2023-01-04 12:12:11 +01:00
Paul Chavard
d469bca0ae
refactor(watermark): use column instead of metadata on blob
2023-01-04 12:07:08 +01:00
Christophe Robillard
edffd0a7a8
extract ds_opendata_enabled
2022-12-20 12:52:56 +01:00
Christophe Robillard
daa4411abe
Revert "unschedule export demarches opendata"
...
This reverts commit da0738c661
.
`ExportAndPublishDemarchesPubliqjesJob` is now schedulable
2022-12-20 12:52:56 +01:00
Christophe Robillard
672e56a099
schedule job during a no-busy time
2022-12-16 18:02:36 +01:00
Paul Chavard
ed4d5cb36a
refactor(demarche): make declarative demarche processing syncroneous
2022-12-06 11:41:25 +01:00
Martin
ea8bd13bcf
poc(batch_operation.purge): nettoie les batch terminé ou inutiles
2022-12-06 11:28:48 +01:00
Martin
eaf72162da
poc(batch_operation): some rewrite to avoid various conflict (when an instructeur try to create a job with an incompatible dossier regarding the current task). also soome cleanup to isole spec in least involved model
2022-12-06 11:28:48 +01:00
Martin
4266ab93c5
poc(batch_operation): track dossier processed safely
2022-12-06 11:28:48 +01:00
Martin
beb39027d0
poc(batch_operation.ui): implement simple ui to trigger a batch of current page
2022-12-06 11:28:48 +01:00
Martin
7df86c50fb
poc(batch_operations_controller): implement simple [not yet with procedure_presentation] action to create a batch operation
2022-12-06 11:28:48 +01:00
Martin
b11dbafc59
poc(batch_operation_job): implement archive dossiers with async behaviour, lock, run_at, finished_at, failed_dossier_ids, success_dossier_ids
2022-12-06 11:28:48 +01:00
Colin Darie
b13c5e56f6
refactor(piece_justificative): UX follows mockups
2022-12-05 10:47:10 +01:00
Paul Chavard
b89f6e2d53
feat(operation_log): add Cron::DossierOperationLogMoveToColdStorageJob
2022-11-22 18:06:05 +01:00
Kara Diaby
4d6ff768c4
Ajoute les dossiers déposés dans les statistiques Datagouv
2022-11-03 10:24:04 +01:00
Paul Chavard
1fd1878370
feat(jobs): add request_id to jobs payload
2022-10-12 14:32:23 +02:00
Damien Le Thiec
0131a41266
Merge branch 'main' into feature/add_rna_type_de_champs
2022-10-05 12:38:24 +02:00
Damien Le Thiec
0d9f02c88d
First draft autofetch data for RNA
2022-10-04 22:11:01 +02:00
Christophe Robillard
da0738c661
unschedule export demarches opendata
2022-09-30 07:26:48 +02:00
Martin
d9432ccd3b
fix(etablissement.as_degraded_mode): backfill missing data via a cron
2022-09-23 15:12:25 +02:00
Colin Darie
67cbc61e1e
chore(services): update etablissement adresse & geocode when siret changes
2022-09-06 09:23:52 +02:00
Martin
45e0c15eb0
feat(PurgeManagerAdministrateurSessionsJob): also purge instructeur assign_tos
2022-08-23 13:28:10 +02:00
Kara Diaby
ec3118e82c
Add concern for Datagouv schedulable jobs
2022-07-29 12:25:27 +02:00
Kara Diaby
a156529394
modify export descriptif demarches job
2022-07-29 11:17:22 +02:00
Kara Diaby
882ee74f88
Comptes créés avec FC par mois : CRON Job
2022-07-29 11:17:22 +02:00
Kara Diaby
dc1d0ce071
Démarches supprimées par mois : CRON Job
2022-07-29 11:17:22 +02:00
Kara Diaby
4a29162c6c
Démarches closes par mois : CRON Job
2022-07-29 11:17:22 +02:00
Kara Diaby
999059269e
Instructeurs connectés par mois : CRON Job
2022-07-29 11:17:21 +02:00
Kara Diaby
8e9d1eeb6f
Nombre de comptes créés par mois : CRON
2022-07-29 11:17:21 +02:00
Kara Diaby
b48a48980a
Nombre de dossiers créés par mois : CRON
2022-07-29 11:17:21 +02:00
Kara Diaby
9fa54c3cc8
Nombre de démarches créés par mois : CRON
2022-07-29 11:17:21 +02:00
Kara Diaby
e0dc36926e
Nombre d'instructeurs créés par mois : CRON
2022-07-29 11:17:21 +02:00
Kara Diaby
a0a3c6c0ee
Administrateurs crées par mois : CRON Job
2022-07-29 11:17:20 +02:00
Colin Darie
197fa54ea4
fix(archives): purge stuck pending archives
...
Cf 0cb096b3d6440a4da05ca2902ed6fa72a93e6579
2022-07-27 15:26:59 +02:00
Colin Darie
f1957e51f8
fix(exports): purge stuck (pending) exports, not generated or failed
...
Avec l'autre PR, ça reproduit le comportement d'avant, c'est à dire
que quoiqu'il arrive un export est purgé :
- soit 16h après sa génération (on a 16h pour le télécharger)
- soit 12h après sa création, et qu'il est bloqué
Auparavant, tous les exports étaient purgés au bout de 3h quelle que soit
le statut.
2022-07-27 15:26:59 +02:00
Martin
3e56fdd1d7
fix(export): when it takes more than 3 hours, exports are purge before being generated. make it possible to have an export that takes more than 3 hours and share this behaviour with kind of same class archive
2022-07-25 10:48:54 +02:00
Christophe Robillard
906dd4b509
remove useless DATAGOUV_TMP_DIR env
2022-07-21 21:54:15 +02:00
Christophe Robillard
0e3e6eb62a
add schedulable? class method to cron jobs
2022-07-21 21:54:15 +02:00
Christophe Robillard
bed6167010
use specific namespace for datagouv cron jobs
2022-07-21 21:54:15 +02:00
Christophe Robillard
b6c96301ef
export and publish opendata demarches
2022-07-21 21:54:15 +02:00
Martin
7fb85c97cb
feat(Administrateur::ArchivesControllers): add archives #index and #create for admin
2022-07-04 14:18:46 +02:00
mfo
011fcd40d8
Merge pull request #7457 from betagouv/US/fix-webhook
...
fix(WebHookJob): missing find procedure
2022-06-10 17:30:28 +02:00
Martin
165db34c4a
fix(WebHookJob): missing find procedure
2022-06-10 17:28:07 +02:00
Martin
b84c2966c4
fix(DeclarativeProceduresJob.perform_now): avoid to crash other demarche when one fail
2022-06-10 14:59:36 +02:00
Martin
d39b3b09ed
fix(webhook.perform): ensure to enqueue job with current state/updated_at for consistency reason
2022-06-01 15:54:55 +02:00
Christophe Robillard
f14cd7b022
remove warning mail for dubious procedures
2022-05-25 14:44:53 +02:00
Paul Chavard
9e0b3b642f
cleanup(sendinblue): remove sendinblue tracking
2022-05-06 11:14:44 +02:00
simon lehericey
da97ad1858
destroy all admins
2022-05-03 12:14:02 +02:00
Martin
3c0e3f725d
feat(administrateur/procedures#publication): show dubious to administrateur
...
Update app/views/administrateurs/procedures/publication.html.haml
Co-authored-by: Paul Chavard <paul.chavard@beta.gouv.fr>
2022-04-27 15:13:22 +02:00
Martin
dbcf21a555
feat(archive): extract archive status management within job to simplify the main service as well as to isolate this part for a merge with exports csv/xslx [maybe?]
...
Update app/dashboards/archive_dashboard.rb
Co-authored-by: LeSim <mail@simon.lehericey.net>
2022-03-31 13:35:49 +02:00
LeSim
fdc4e5bfbc
Merge pull request #6928 from adullact/feature/6886-watermark-url
...
Watermark depuis une URL distante
2022-02-08 11:45:26 +01:00
Paul Chavard
0268511264
fix(manager): remove admins added for a day
...
fix #6855
2022-02-08 11:18:44 +01:00
Jon
88d3a6e4c3
feat: can have remote watemark file
2022-02-04 18:55:16 +01:00
Paul Chavard
403e71b41f
feat(procedure): only purge procedures without dossiers
2022-01-18 11:22:04 +01:00
simon lehericey
f0b0e7fd9a
Switch to usage of zip unix binary to create archive. Also use a dedicated queue for DelayedJob
...
use dedicated archives queue
As the used disk space will increase, we want a fined grain control
move zip logic in dedicated method
zip
wip
wip
fix(spec): pass spec in green
tech(improvements): avoid File.delete(folder), favor FileUtils.remove_entry_secure which is safer. Also wrap most of code that open file within blocks so it is cleaned when the block ends. Lastly use attachement.download to avoid big memory pressure [download in chunk, write in chunk] otherwise big file [124>1GO] are loaded in memory. what if we run multiple jobs/download in parallel ?
fix(spec): try to retry with grace
clean(procedure_archive_service_spec.rb): better retry [avoid to rewrite on open file]
lint(things): everything
2021-12-13 16:37:04 +01:00
kara Diaby
9ee9389ba1
add the good retry_on StaleObjectError
2021-11-05 13:32:21 +01:00
Paul Chavard
d5d3995dad
fix(dossier): fix dossier.avis cascade
2021-10-26 11:16:29 +02:00
Paul Chavard
1561ea82f6
fix(transfer): manually nullify staled transfers references
2021-10-21 13:54:26 +02:00
Paul Chavard
2a3a9dd822
feat(revisions): rebase dossiers brouillons
2021-10-20 17:26:09 +02:00
Paul Chavard
4a947f9135
feat(manager): add become administrateur button in manager (with 24h expiration)
2021-10-07 11:32:12 +02:00
simon lehericey
e4484eb6d9
discard expert not found job error
2021-10-04 12:57:37 +02:00
Paul Chavard
5985755229
feat(dossier): add purge stale dossier transfers job
2021-09-08 14:39:46 +02:00
Christophe Robillard
ddea0b9a45
purge old sendinblue mails
2021-08-20 08:52:39 +02:00
Paul Chavard
ffa8c0c80a
feat(dossiers): enable dossiers termine expiration behind feature flag
...
feature flag "procedure_process_expired_dossiers_termine" controls if a procedure has expiration
enabled on dossiers termine
re #3796
2021-08-18 16:11:35 +01:00
Christophe Robillard
66cc0dd08d
Revert "Revert "Expose dossier PDF export as IO""
...
This reverts commit 362093eff0
.
2021-06-24 21:01:59 +02:00
krichtof
362093eff0
Revert "Expose dossier PDF export as IO"
2021-06-24 19:21:37 +02:00
Paul Chavard
48dca8bad1
Execute AchiveCreationJob in exports queue
...
Co-authored-by: Christophe Robillard <christophe.robillard@beta.gouv.fr>
2021-06-24 17:49:58 +02:00
Christophe Robillard
f40d96fbd2
Revert "Revert "Export de tous les dossier d'une démarche""
...
This reverts commit d9a588b52e
.
2021-04-29 17:29:47 +02:00
krichtof
d9a588b52e
Revert "Export de tous les dossier d'une démarche"
2021-04-29 16:07:18 +02:00
Pierre de La Morinerie
684af77e35
jobs: extract an RetryOnTranscientErrors
concern
2021-04-29 14:08:12 +02:00
Christophe Robillard
aa407d5607
place le cronjob PurgeStaleArchives dans le bon répertoire
2021-04-28 11:40:58 +02:00
Christophe Robillard
8bee53fe77
rend plus robuste la création de l'archive
2021-04-28 11:40:58 +02:00
Christophe Robillard
609a1aa45b
add archive creation and purge archives jobs
2021-04-28 11:40:58 +02:00
simon lehericey
e636e3a752
add merge_and_update_metadata method
2021-04-06 18:11:26 +02:00
simon lehericey
d552e364fc
retry 5 times on integrity error and then block processing
2021-04-06 18:03:39 +02:00