Update doc/object-storange-and-data-encryption.md
Co-authored-by: LeSim <mail@simon.lehericey.net>
Update app/services/archive_uploader.rb
Co-authored-by: LeSim <mail@simon.lehericey.net>
Update doc/object-storange-and-data-encryption.md
Co-authored-by: Pierre de La Morinerie <kemenaran@gmail.com>
clean(doc): align document file name and document h1
clean(review): refactore based on various comments
clean(review): refactore based on various comments
By default, `has_and_belongs_to_many` properly deletes the record in
the join table.
However, as the association is declared manually with a
`has_many / through`, it doesn't delete the joined record automatically.
As we also lack a foreign-key contraint on the join table, that means
a dangling record remains in the join table.
To fix this, let's declare it a proper `has_and_belongs_to_many`
association, which will let the join record be deleted automatically
on destroy.
By default, `has_and_belongs_to_many` properly deletes the record in
the join table.
However, as the association is declared manually with a
`has_many / through`, it doesn't delete the joined record automatically.
As we also lack a foreign-key contraint on the join table, that means
a dangling record remains in the join table.
To fix this, let's declare it a proper `has_and_belongs_to_many`
association, which will let the join record be deleted automatically
on destroy.
wip(dossier_created_hook): add tile to administrateurs/procedure#show in order to crud dossier_created_hook
refactor(css.utilities): remove merge helpers.scss within utils.scss (same purpose). use scss each for spacer modifiers
refactor(dossiers/_merci.html): extract partial _merci so we can re-use it in preview of dossier_created_hook.
feat(wip): current progress
Turns out deleting the tasks after 6 months only may not be enough for instances deploying
only once a year, for instance.
This reverts commit 77b65f3aa0.
When filling all the champs of a dossier, we skip the "Address"
champ (because it would attempt to request a blocked URL).
But that means the "Yes/No" radio buttons may be under the sticky
bottom bar, and thus not register the click – which makes the test fail.
Fix the issue by manually scrolling to the radio buttons.
Move everything to initializers, and replace the email settings
interceptor by a BalancerDeliveryMethod.
It has the advantage that it can be configured entirely from the
`config/environment.rb` file, without an extra file to look at.
Before this commit, the monthly dossiers count was serialized into the
Stat record using human-formatted dates, as:
```ruby
s.dossiers_in_the_last_4_months = {
"octobre 2021"=>409592,
"novembre 2021"=>497823,
"décembre 2021"=>38170,
"janvier 2022"=>0
}
```
Turns out the ordering of keys in a serialized hash is not guaranteed.
After a round-trip to the database, the keys will be wrongly sorted.
Instead we want to save raw Date objects, which will preserve the
ordering. The date formatting can be applied at display-time by the
controller.
Fix#6848
Context: we want to validate public and private types_de_champ
separately.
Before we validated the whole revision (and then validators themselves
enumerated all champs, public and private).
Now we validate the actual public types_de_champ, which will let us
validate separately the private types_de_champ.
Errors generated by the `invisible-catcha` gem are reported as
`flash[:error]` (which differs from Rail's usual `flash[:alert]`).
We probably shouldn't use flash[:error] in our own codebase; but we can
handle its use in third-party libraries.
Errors generated by the `invisible-catcha` gem are reported as
`flash[:error]` (which differs from Rail's usual `flash[:alert]`).
Avoid crashing when the flash level passed to this helper is not known.
For an unknown reason, ActiveRecord::Base.connected? does not work anymore in dev. It returns false negative.
This simple workaround can timeout. The caller has to monitor the response time.
fix(profil_controller#update_email): changing email from current_user.email to current_user.email destroy current user. whoops ☠️'
Update config/locales/en.yml
Co-authored-by: Pierre de La Morinerie <pierre.de_la_morinerie@beta.gouv.fr>
Update config/locales/fr.yml
Co-authored-by: Pierre de La Morinerie <pierre.de_la_morinerie@beta.gouv.fr>
Update spec/controllers/users/profil_controller_spec.rb
Update config/locales/fr.yml
Co-authored-by: Pierre de La Morinerie <pierre.de_la_morinerie@beta.gouv.fr>
Update spec/controllers/users/profil_controller_spec.rb
fix(spec): broken due to typo
use dedicated archives queue
As the used disk space will increase, we want a fined grain control
move zip logic in dedicated method
zip
wip
wip
fix(spec): pass spec in green
tech(improvements): avoid File.delete(folder), favor FileUtils.remove_entry_secure which is safer. Also wrap most of code that open file within blocks so it is cleaned when the block ends. Lastly use attachement.download to avoid big memory pressure [download in chunk, write in chunk] otherwise big file [124>1GO] are loaded in memory. what if we run multiple jobs/download in parallel ?
fix(spec): try to retry with grace
clean(procedure_archive_service_spec.rb): better retry [avoid to rewrite on open file]
lint(things): everything
feat(expiration_banner): enhance wording of expiration
feat(dossiers/expiration_banner): enhance wording regarding expiration to include duree_conservation_dossiers_dans_ds + extension_conservation, also add spec on expiration_banner for instructeur
fix(lint): lint haml
fix(spec): enable flipper and allow procedure to receive flipper check when checking banner presence
fix(doc): add missing documentation on readme regarding system testing with a visual feedback
fix(typo): add missing accent
clean(PR): feedback from Tchak, better to wrap feature check for expirability by procedure within dossier.expirable? helper
tech(question): discard_and_keep_track! ; are we really keeping track with default_scope { kept } ?
feat(stats): add DeletedDossier in Stat computations
Revert "tech(question): discard_and_keep_track! ; are we really keeping track with default_scope { kept } ?"
This reverts commit d1155b7eeaaf1a9f80189e59667e109541fcb089.
feat(stats): support deleted_dossiers for last_four_months_hash and cumulative_hash. extract sanitize query & merge hashes in methdos
clean(rubocop): lint with rubocop
Update db/migrate/20211126080118_add_index_to_deleted_at_to_deleted_dossiers.rb
Co-authored-by: LeSim <mail@simon.lehericey.net>
fix(rubocop): avoid uneeded allocation
fix(migration): add concurrent index with expected synthax
fix(brakeman): add ignore message since group date_trunc evaluation is used by only ourself
i18n(france_connect/*): replace wording with i18n
fix(lint): i18n key issue
secu(views/france_connect/particulier/merge.html.haml): sanitize france_connect_email just in case
fix(brakeman): sanitize FCI.email_france_connect when used with html_safe via an I18n.t, also add exception to brakeman
feat(fci.confirmation_code): add confirmation code to france_connect_informations
feat(user_mailer.france_connect_confirmation_code): add confirmation by email mail method/preview/spec, pointing to merge_mail_with_existing_account (reuse existing method)
feat(mail_merge): mail merge
feat(merge.cannot_use_france_connect): same behaviour as callback
clean(fci.confirmation_code): use same token for mail validation as merge
feat(resend_france_connect/particulier/merge_confirmation): resend email with link. also enhance some trads, cleanup halfy finished refacto
clean(tech): finalize story by plugging merge_with_new_account to email validation
fix(deadspec): was removed
fix(spec): broken after last refactoring
lint(rubocop): space before parenthesis
lint(haml-lint): yoohoooo space before =
fix(lint): scss now :D
Update app/assets/stylesheets/buttons.scss
cleanup
feat(france_connect): re-add confirm by email, with an option for confirmation by email instead of only confirmation by email
fixup! Add confirmation by email when merging DC/FC accounts
fix(lint): haml_spec failure
Deep-cloned objects have all their relationships stale. Thus, for a
newly deep-cloned revision, `revision.types_de_champs` returns `[]`,
even when it actually has associated types de champ.
This causes consecutive champs creations and re-ordering to fail in
subtle ways, like:
```
procedure.draft_revision.add_type_de_champ(…)
procedure.publish_revision!
procedure.draft_revision.add_type_de_champ(…)
procedure.draft_revision.move_type_de_champ(…) # this will fail
```
As `publish_revision!` created a new stale revision, moving the type
de champ fails because not all existing champs are found until the
object is refreshed.
We don't hit this path in production, because usually only a single
operation is made in a request.
To fix this, save the new revision before associating it as the draft
procedure.
(Another option would be to `reload` the revision after creation, but
this seems better contained and matches the name of the method.)
It seems better to create associations in an declarative fashion, rather
than using imperative code. This also makes the attribute compatible
with build_stubbed.
Calling business logic in a factory is a code-smell, because it
usually requires the object to be saved into database, and may have
unintended consequences when the business logic is changed.
Also, this allows to just build a published procedure, without saving it
to the database.
This fix prevent repetition children types de champ from being pulled from cloned procedures. stable_id is stable across revisions but also across cloned procedures.
Creating dossiers is faster than creating a procedure, but still slow.
We can create a single dossier in the default case, and only create
several others when the example requires it.
Speeds up this spec from 0m 57s to 0m 49s.
Creating a procedure with all available types de champ is slow. We can
create a simpler procedure in the default case, and only create all
types de champs when the example requires it.
Speeds up this spec from 1m 55s to 0m 57s.
Creating a dossier with available champs populated is slow. We can
create simpler dossiers in the default case, and only populate all
champs when the example requires it.
Speeds up this spec from 2mn 20s to 1m 55s.
Before, every time a password was tested, the dictionaries were parsed
again by zxcvbn.
Parsing dictionaries is slow: it may take up to ~1s. This doesn't matter
that much in production, but it makes tests very slow (because we tend
to create a lot of User records).
With this changes, the initializer tester is shared between calls, class
instances and threads. It is lazily loaded on first use, in order not to
slow down the application boot sequence.
This uses ~20 Mo of memory (only once for all threads), but makes tests
more that twice faster.
For instance, model tests go from **8m 21s** to **3m 26s**.
NB:
An additionnal optimization could be to preload the tester on
boot, before workers are forked, to take advantage of Puma copy-on-write
mechanism. In this way all forked workers would use the same cached
instance.
But:
- We're not actually sure this would work properly. What if Ruby updates
an interval ivar on the class, and this forces the OS to copy the
whole data structure in each fork?
- Puma phased restarts are not compatible with copy-on-write anyway.
So we're avoiding this optimisation for now, and take the extra 20 Mo
per worker.
instead of looking linked user by email because :
- follows FC recommendation to fetch ds account by openid
- the email is not a valid key as many user can share the same FCI email.
The following scenario is now working
A user A (email: 1@mail.com) uses FC to connect to DS
=> It is connected as 1@mail.com
Another user B (email: generic@mail.com) uses FC to connect
=> It is connected as generic@mail.com
The first user A change its FC email to generic@mail.com and connect to DS
=> It is still connected as 1@mail.com
There's a random failure in this spec, where the CI triggers this error:
> Failure/Error: check('checkbox')
>
> Selenium::WebDriver::Error::ElementClickInterceptedError:
> element click intercepted: Element <input required="required" type="checkbox" value="on" name="dossier[champs_attributes][7][value]" id="dossier_champs_attributes_7_value"> is not clickable at point (205, 892). Other element would receive the click: <div class="send-dossier-actions-bar">...</div>
That's because the checkbox is partially overlapped by the sticky
action bar at the bottom of the screen – but only _some of the time_.
This commit attempts to fix the issue by manually scrolling the checkbox
at the center of the screen before clicking it.
