use dedicated archives queue
As the used disk space will increase, we want a fined grain control
move zip logic in dedicated method
zip
wip
wip
fix(spec): pass spec in green
tech(improvements): avoid File.delete(folder), favor FileUtils.remove_entry_secure which is safer. Also wrap most of code that open file within blocks so it is cleaned when the block ends. Lastly use attachement.download to avoid big memory pressure [download in chunk, write in chunk] otherwise big file [124>1GO] are loaded in memory. what if we run multiple jobs/download in parallel ?
fix(spec): try to retry with grace
clean(procedure_archive_service_spec.rb): better retry [avoid to rewrite on open file]
lint(things): everything
feat(expiration_banner): enhance wording of expiration
feat(dossiers/expiration_banner): enhance wording regarding expiration to include duree_conservation_dossiers_dans_ds + extension_conservation, also add spec on expiration_banner for instructeur
fix(lint): lint haml
fix(spec): enable flipper and allow procedure to receive flipper check when checking banner presence
fix(doc): add missing documentation on readme regarding system testing with a visual feedback
fix(typo): add missing accent
clean(PR): feedback from Tchak, better to wrap feature check for expirability by procedure within dossier.expirable? helper
tech(question): discard_and_keep_track! ; are we really keeping track with default_scope { kept } ?
feat(stats): add DeletedDossier in Stat computations
Revert "tech(question): discard_and_keep_track! ; are we really keeping track with default_scope { kept } ?"
This reverts commit d1155b7eeaaf1a9f80189e59667e109541fcb089.
feat(stats): support deleted_dossiers for last_four_months_hash and cumulative_hash. extract sanitize query & merge hashes in methdos
clean(rubocop): lint with rubocop
Update db/migrate/20211126080118_add_index_to_deleted_at_to_deleted_dossiers.rb
Co-authored-by: LeSim <mail@simon.lehericey.net>
fix(rubocop): avoid uneeded allocation
fix(migration): add concurrent index with expected synthax
fix(brakeman): add ignore message since group date_trunc evaluation is used by only ourself
i18n(france_connect/*): replace wording with i18n
fix(lint): i18n key issue
secu(views/france_connect/particulier/merge.html.haml): sanitize france_connect_email just in case
fix(brakeman): sanitize FCI.email_france_connect when used with html_safe via an I18n.t, also add exception to brakeman
feat(fci.confirmation_code): add confirmation code to france_connect_informations
feat(user_mailer.france_connect_confirmation_code): add confirmation by email mail method/preview/spec, pointing to merge_mail_with_existing_account (reuse existing method)
feat(mail_merge): mail merge
feat(merge.cannot_use_france_connect): same behaviour as callback
clean(fci.confirmation_code): use same token for mail validation as merge
feat(resend_france_connect/particulier/merge_confirmation): resend email with link. also enhance some trads, cleanup halfy finished refacto
clean(tech): finalize story by plugging merge_with_new_account to email validation
fix(deadspec): was removed
fix(spec): broken after last refactoring
lint(rubocop): space before parenthesis
lint(haml-lint): yoohoooo space before =
fix(lint): scss now :D
Update app/assets/stylesheets/buttons.scss
cleanup
feat(france_connect): re-add confirm by email, with an option for confirmation by email instead of only confirmation by email
fixup! Add confirmation by email when merging DC/FC accounts
fix(lint): haml_spec failure
Deep-cloned objects have all their relationships stale. Thus, for a
newly deep-cloned revision, `revision.types_de_champs` returns `[]`,
even when it actually has associated types de champ.
This causes consecutive champs creations and re-ordering to fail in
subtle ways, like:
```
procedure.draft_revision.add_type_de_champ(…)
procedure.publish_revision!
procedure.draft_revision.add_type_de_champ(…)
procedure.draft_revision.move_type_de_champ(…) # this will fail
```
As `publish_revision!` created a new stale revision, moving the type
de champ fails because not all existing champs are found until the
object is refreshed.
We don't hit this path in production, because usually only a single
operation is made in a request.
To fix this, save the new revision before associating it as the draft
procedure.
(Another option would be to `reload` the revision after creation, but
this seems better contained and matches the name of the method.)
It seems better to create associations in an declarative fashion, rather
than using imperative code. This also makes the attribute compatible
with build_stubbed.
Calling business logic in a factory is a code-smell, because it
usually requires the object to be saved into database, and may have
unintended consequences when the business logic is changed.
Also, this allows to just build a published procedure, without saving it
to the database.
This fix prevent repetition children types de champ from being pulled from cloned procedures. stable_id is stable across revisions but also across cloned procedures.
Creating dossiers is faster than creating a procedure, but still slow.
We can create a single dossier in the default case, and only create
several others when the example requires it.
Speeds up this spec from 0m 57s to 0m 49s.
Creating a procedure with all available types de champ is slow. We can
create a simpler procedure in the default case, and only create all
types de champs when the example requires it.
Speeds up this spec from 1m 55s to 0m 57s.
Creating a dossier with available champs populated is slow. We can
create simpler dossiers in the default case, and only populate all
champs when the example requires it.
Speeds up this spec from 2mn 20s to 1m 55s.
Before, every time a password was tested, the dictionaries were parsed
again by zxcvbn.
Parsing dictionaries is slow: it may take up to ~1s. This doesn't matter
that much in production, but it makes tests very slow (because we tend
to create a lot of User records).
With this changes, the initializer tester is shared between calls, class
instances and threads. It is lazily loaded on first use, in order not to
slow down the application boot sequence.
This uses ~20 Mo of memory (only once for all threads), but makes tests
more that twice faster.
For instance, model tests go from **8m 21s** to **3m 26s**.
NB:
An additionnal optimization could be to preload the tester on
boot, before workers are forked, to take advantage of Puma copy-on-write
mechanism. In this way all forked workers would use the same cached
instance.
But:
- We're not actually sure this would work properly. What if Ruby updates
an interval ivar on the class, and this forces the OS to copy the
whole data structure in each fork?
- Puma phased restarts are not compatible with copy-on-write anyway.
So we're avoiding this optimisation for now, and take the extra 20 Mo
per worker.
