Commit graph

320 commits

Author SHA1 Message Date
Pierre de La Morinerie
d0e87a08cf services: cache zxcvbn dictionaries per-thread
Before, every time a password was tested, the dictionaries were parsed
again by zxcvbn.

Parsing dictionaries is slow: it may take up to ~1s. This doesn't matter
that much in production, but it makes tests very slow (because we tend
to create a lot of User records).

With this changes, the initializer tester is shared between calls, class
instances and threads. It is lazily loaded on first use, in order not to
slow down the application boot sequence.

This uses ~20 Mo of memory (only once for all threads), but makes tests
more that twice faster.

For instance, model tests go from **8m 21s** to **3m 26s**.

NB:
An additionnal optimization could be to preload the tester on
boot, before workers are forked, to take advantage of Puma copy-on-write
mechanism. In this way all forked workers would use the same cached
instance.

But:

- We're not actually sure this would work properly. What if Ruby updates
  an interval ivar on the class, and this forces the OS to copy the
  whole data structure in each fork?
- Puma phased restarts are not compatible with copy-on-write anyway.

So we're avoiding this optimisation for now, and take the extra 20 Mo
per worker.
2021-10-25 12:04:56 +02:00
simon lehericey
d68129b34d add cnaf type de champ 2021-10-12 14:26:40 +02:00
kara Diaby
cbedef996b tests 2021-10-05 15:00:21 +02:00
simon lehericey
9fc195f0e5 add test to dossier_projection_service 2021-09-07 15:17:06 +02:00
Paul Chavard
8e1bfb469f fix(dossier): send expiration notifications 2 weeks prior to supression instead of a month 2021-08-26 11:28:57 +02:00
Christophe Robillard
e06f11f33f add code insee libelle for commune export 2021-08-19 08:04:47 +02:00
Paul Chavard
ffa8c0c80a feat(dossiers): enable dossiers termine expiration behind feature flag
feature flag "procedure_process_expired_dossiers_termine" controls if a procedure has expiration
enabled on dossiers termine

re #3796
2021-08-18 16:11:35 +01:00
Christophe Robillard
031b74759f archives: move bugreport 2021-08-05 09:24:20 +02:00
Christophe Robillard
38e3469e9a add bug report to archive 2021-08-03 14:51:41 +02:00
François Vantomme
17b659539f Feat (API Particulier): new encryption service 2021-07-30 11:18:44 +02:00
Christophe Robillard
66cc0dd08d Revert "Revert "Expose dossier PDF export as IO""
This reverts commit 362093eff0.
2021-06-24 21:01:59 +02:00
krichtof
362093eff0
Revert "Expose dossier PDF export as IO" 2021-06-24 19:21:37 +02:00
Paul Chavard
b73d504f8d Expose dossier PDF export as IO
Co-authored-by: Christophe Robillard <christophe.robillard@beta.gouv.fr>
2021-06-24 17:49:24 +02:00
Christophe Robillard
fdec9b2fd5 archives: don't update dossiers
this fix avoid to touch dossier after attaching pdf_export_for_instructeur
2021-06-18 11:00:57 +02:00
kara Diaby
6ebd57640c tests 2021-06-15 19:36:46 +02:00
François Vantomme
99086a63be Fix (Instructeur): deal nicely with non-guaranteed elements order in arrays
Use of one-dimension arrays comparison & `contain_exactly` RSpec matcher
to avoid this behaviour:

Failures:
  1) InstructeursImportService#import when an email is malformed ignores or corrects
     Failure/Error:
       expect(procedure_groupes).to match_array([
         ["Occitanie", ["paul@mccartney.uk", "ringo@starr.uk"]],
         ["défaut", []]
       ])
       expected collection contained:  [["Occitanie", ["paul@mccartney.uk", "ringo@starr.uk"]], ["défaut", []]]
       actual collection contained:    [["Occitanie", ["ringo@starr.uk", "paul@mccartney.uk"]], ["défaut", []]]
       the missing elements were:      [["Occitanie", ["paul@mccartney.uk", "ringo@starr.uk"]]]
       the extra elements were:        [["Occitanie", ["ringo@starr.uk", "paul@mccartney.uk"]]]
     # ./spec/services/instructeurs_import_service_spec.rb:70:in `block (4 levels) in <main>'
2021-05-27 08:33:52 +02:00
simon lehericey
17617fba43 address champ needs the data column 2021-05-19 15:59:18 +02:00
simon lehericey
a46000dc1f ensure to_s is used on specialized champ 2021-05-19 09:39:20 +02:00
Paul Chavard
e043645a88 cleanup tests 2021-05-18 11:42:10 +02:00
kara Diaby
468e9e849a tests 2021-05-18 11:42:10 +02:00
simon lehericey
9fd1c604a1 display follower instructeur emails in alphabetic order 2021-05-04 16:38:34 +02:00
Christophe Robillard
01f180022b prend en compte uniquement les pj pour estimer la taille d'un dossier 2021-05-04 12:50:12 +02:00
Christophe Robillard
3d04221dab generate pdf export for dossier only on demand 2021-04-29 17:30:38 +02:00
Christophe Robillard
f40d96fbd2 Revert "Revert "Export de tous les dossier d'une démarche""
This reverts commit d9a588b52e.
2021-04-29 17:29:47 +02:00
krichtof
d9a588b52e
Revert "Export de tous les dossier d'une démarche" 2021-04-29 16:07:18 +02:00
simon lehericey
7b94c2de88 remove extra Dossier call in projection 2021-04-29 11:46:00 +02:00
simon lehericey
db23a853a9 fetch all notification at once 2021-04-29 11:23:41 +02:00
Christophe Robillard
dfbe004122 rename content_type to time_span_type for archives 2021-04-28 11:40:58 +02:00
Christophe Robillard
9134114c2e fix rubocop offenses 2021-04-28 11:40:58 +02:00
Christophe Robillard
8bee53fe77 rend plus robuste la création de l'archive 2021-04-28 11:40:58 +02:00
Christophe Robillard
07cc4fa97f add service to create procedure archive 2021-04-28 11:40:58 +02:00
Christophe Robillard
fba0d78153 generate pdf export inside pieces_justificatives_service 2021-04-28 11:40:58 +02:00
simon lehericey
5bfd4ecbbf add DossierProjectionService 2021-04-22 10:46:04 +02:00
Pierre de La Morinerie
9096a3885c specs: fix creating dossiers :en_construction 2021-04-06 12:12:56 +02:00
Pierre de La Morinerie
75a1046315 active_storage: refactor concerns
Follow-up of #5953.

Refactor the concerns with two goals:

- Getting closer from the way ActiveStorage adds its own hooks.
  Usually ActiveStorage does this using an `Attachment#after_create`
  hook, which then delegates to the blob to enqueue the job.
- Enqueuing each job only once. By hooking on `Attachment#after_create`,
  we guarantee each job will be added only once.

We then let the jobs themselves check if they are relevant or not, and
retry or discard themselves if necessary.

We also need to update the tests a bit, because Rails'
`perform_enqueued_jobs(&block)` test helper doesn't honor the `retry_on`
clause of jobs. Instead it forwards the exception to the caller – which
makes the test fail.

Instead we use the inline version of `perform_enqueued_jobs()`, without
a block, which properly ignores errors catched by retry_on.
2021-03-16 11:49:14 +01:00
Pierre de La Morinerie
1c77f59f6b specs: fix test failing on March 1st
Because February only has 28 days.
2021-03-02 12:26:28 +01:00
Christophe Robillard
a288a13805 disable france connect authentication for admin and instructeurs 2021-02-10 18:13:47 +01:00
Pierre de La Morinerie
150ddab660 zeitwerk: Api -> API 2021-02-09 13:07:30 +01:00
Pierre de La Morinerie
88ffa10a56 spec: fix rspec raise_error warning
Rspec warns that if there is for example a SyntaxError, the test will
pass (an error was raised, but it wasn't an ArgumentError, so this is
fine).

Instead check that no error occurs whatsoever.
2021-02-09 10:12:32 +01:00
Pierre de La Morinerie
0d101f3f1a views: remove old design's leftbars 2021-02-02 15:33:09 +01:00
Christophe Robillard
792b53beeb import several instructeurs for a routee procedure
Co-authored-by: simon lehericey <mail@simon.lehericey.net>
2021-01-20 15:03:17 +01:00
Paul Chavard
54f2084aef Add annuaire_education champ 2021-01-14 17:57:48 +01:00
simon lehericey
1159494d37 Add test to ensure no titre identite is exported 2020-12-16 17:31:38 +01:00
kara Diaby
f748ccfc9e Appelle l'API entreprise avec le token en Header 2020-12-10 17:49:49 +01:00
clemkeirua
513d4f6ff1 move all the cron jobs in a dedicated directory 2020-12-08 13:22:43 +00:00
simon lehericey
b187244a29 extract and refactor api errors 2020-12-04 17:07:30 +01:00
Keirua
3fa628235e Nettoyage de I18n.transliterate pour les noms de feuilles d'exports
Co-authored-by: LeSim <mail@simon.lehericey.net>
2020-09-29 15:50:19 +00:00
clemkeirua
9714b607aa ensure valid excel worksheet names 2020-09-29 15:50:19 +00:00
simon lehericey
1eb780b0c9 convert sheet_name to ascii before tuncate to 30 to ensure length < 31 bytes 2020-09-29 14:11:06 +02:00
clemkeirua
c19b653581 higher level length sanitization for worksheet length 2020-09-24 14:59:46 +02:00
kara Diaby
cb4e91c405 Add iban type de champ 2020-09-23 15:56:26 +02:00
kara Diaby
5c68d75107 add titre_identite champ 2020-09-18 14:57:08 +02:00
clemkeirua
2b23eefee2 fix tests that now requir an admin user to have a strong password 2020-09-18 11:16:34 +02:00
Paul Chavard
0a70291b90 update services specs to use revisions 2020-09-02 11:26:26 +02:00
Christophe Robillard
f77cc06a91 prend en compte le nom de l'enseigne pour l'établissement 2020-08-20 14:23:02 +02:00
Christophe Robillard
2c56511204 validate api_entreprise_token 2020-07-08 18:59:22 +02:00
Christophe Robillard
a072d35211 use traitement model
when a dossier is terminated (accepte, refuse or classe_sans_suite),
we store now `processed_at` and `motivation` in a traitement instance
2020-07-08 18:15:15 +02:00
Paul Chavard
c9ab80c880 WIP 2020-07-07 18:03:56 +02:00
Pierre de La Morinerie
43569f687e specs: use fixture_file_upload rather than Rack::Test::UploadedFile
This is mostly for consistency (both styles were used before).

Note: we still have to use `Rack::Test::UploadedFile.new` in factories,
because of https://github.com/thoughtbot/factory_bot/issues/385.
2020-07-01 11:13:45 +02:00
Christophe Robillard
ba5fa9aa4b update ApiEntrepriseService
There is now the `create_etablissement` method which
create etablissement with EtablissementAdapter
and enqueue api_entreprise jobs to retrieve
all informations we can get based on SIRET
2020-05-20 11:09:54 +02:00
Pierre de La Morinerie
9f1407b6d7 expiration: fix the mailer arguments
The mailers expect serializable arguments, but were given
ActiveRecord::Relation objects instead. This made the mailers throw an
exception.

But how was that possible ? This code is tested, and the tests were
green.

Well, the specs spy on the mailer implementation, in order to check that
the mailers methods were properly called. Fair enough.

But if the specs mock the mailer code (instead of calling the original
implementation), we may not notice that the original implementation
rejects our method parameters.

Here this is the case: once we actually call the original implementation
the tests start to fail, because some arguments are not converted from
an ActiveRecord::Relation to a serializable array.

This is fixed by ensuring that the mailer code is executed (and doesn't
throw an exception).
2020-05-18 16:24:08 +02:00
Pierre de La Morinerie
b63fa0e6b8 dossier: remove "Birthdate" column from export 2020-05-18 13:14:28 +00:00
Pierre de La Morinerie
00adb006b4 specs: improve ProcedureExportService spec 2020-05-18 13:14:28 +00:00
clemkeirua
36f381a743 fix flacky test 2020-05-07 10:23:28 +02:00
Christophe Robillard
dba77ec0b8 fix spec with not expired token
and use of `ApiEntrepriseToken#roles`
2020-05-06 10:57:45 +02:00
Paul Chavard
4299c9dbea Use a “safe” reference date in expiration tests 2020-05-05 12:41:45 +02:00
Christophe Robillard
b12226c667 fetch and store bilans bdf 2020-04-30 16:15:16 +02:00
Christophe Robillard
3ddb5a4cb4 store attestation fiscale and display 2020-04-30 11:39:54 +02:00
Christophe Robillard
158b4802ab store attestation sociale and display 2020-04-29 16:25:47 +02:00
Christophe Robillard
d74b14c205 fetch attestation sociale url 2020-04-29 14:25:55 +02:00
Christophe Robillard
4bd4524f67 fix api_entreprise specs 2020-04-28 15:17:11 +02:00
Christophe Robillard
453adf3c74 instructeurs: affiche l'effectif moyen annuel n-1 2020-04-23 09:53:27 +02:00
Paul Chavard
31943f7d9c Add handeling of expired processed dossiers to deletion service 2020-04-22 15:39:10 +02:00
Christophe Robillard
1165f8483d rename effectif columns 2020-04-21 21:13:07 +02:00
Christophe Robillard
db3ef195ad fetch and show effectif mensuel
- fetch and store effectif mensuel (at the creation of the dossier or
when updating a siret field

- show effectif mensuel only for instructeurs
2020-04-21 14:22:11 +02:00
clemkeirua
2d68d7dc59 fix to_tsquery disallowed chars 2020-04-20 15:15:22 +02:00
Paul Chavard
52d2ace823 Remove quartiers prioritaires 2020-04-14 10:29:59 +02:00
clemkeirua
590cee050f add tests for dossier_search_service 2020-04-09 09:42:31 +02:00
Paul Chavard
ca6c97bf0f Refactor expired dossiers deletion service 2020-04-02 15:24:02 +02:00
Paul Chavard
7ba4c513e6 Refactor notify near deletion mailers 2020-04-01 17:40:52 +02:00
Pierre de La Morinerie
4cb747fdb6 specs: always require rails_helper
Test helpers are separated between two files: spec_helper and
rails_helper. This separation is meant to allow tests that do not
require Rails (like testing standalone libs) to boot faster.

The spec_helper file is always loaded, through `--require spec_helper`
in the `.rspec` config file. When needed, the rails_helper file is
expected to be required manually.

This is fine, but:
- Many test files have a redundant `require 'spec_helper'` line;
- Many test files should require `rails_helper`, but don't.

Not requiring `rails_helper` will cause the Rails-concerned section of
the test environment not to be configured–which may cause subtle bugs
(like the test database not being properly initialized).

Moreover, Spring loads all the Rails files on preloading anyway. So the
gains from using only `spec_helper` are thin.

To streamline this process, this commit:
- Configures `.rspec` to require `rails_helper` by default;
- Remove all manual requires to spec_helper or rails_helper.

Reference: https://stackoverflow.com/questions/24145329/how-is-spec-rails-helper-rb-different-from-spec-spec-helper-rb-do-i-need-it
2020-03-31 12:48:32 +02:00
Paul Chavard
c40d689edf Use bigger deltas to allow for month length variations 2020-03-31 11:50:59 +02:00
Paul Chavard
5b5ae5a7a5 Do not send notifications or create operation logs on brouillon demarches 2020-03-25 15:04:52 +01:00
Paul Chavard
5c2eba1dd1 Always add a reason to dossier deletion 2020-03-24 09:30:11 +01:00
Paul Chavard
cd0acb1344 Cleanup dossier mailers 2020-03-19 16:52:18 +01:00
simon lehericey
3dd4ea26c7 add similar tests for send_brouillon_expiration_notices and delete_expired_brouillon_and_notify 2020-03-18 15:25:02 +01:00
simon lehericey
3f6012bff5 Delete complicated test 2020-03-18 15:25:02 +01:00
simon lehericey
74db7ab146 test for #send_en_construction_expiration_notices 2020-03-18 15:25:02 +01:00
simon lehericey
f35c0518b7 test for #delete_expired_en_construction_and_notify 2020-03-18 15:25:02 +01:00
Paul Chavard
fe1cd2a47b Delete and notify en_construction dossiers 2020-03-18 15:25:02 +01:00
Paul Chavard
f35ccc9d92 Move process_expired_dossiers_brouillon to a service 2020-03-18 15:25:02 +01:00
kara Diaby
395af2fbb0 instructors : receive daily emails for declarative folders 2020-03-17 15:54:50 +01:00
Paul Chavard
c281347da1 Remove restclient 2020-03-17 11:12:14 +01:00
kara Diaby
4a70a7fcc8 remove email_notifications_enabled column because of the new daily_email_notifications_enabled 2020-02-27 11:09:19 +01:00
clemkeirua
fcea7c4f85 remove invalid excel worksheet characters 2020-01-29 11:45:56 +01:00
Paul Chavard
eec38bad73 Ajouter les informations de l’instructeur et de l’expert dans l’export des avis
fix #4667
2020-01-28 15:09:41 +01:00
Pierre de La Morinerie
724ff50363 factories: rename dossier for_individual to with_individual
This clarifies the distinction between

```
create :procedure, :for_individual
```

and

```
create :dossier, :with_individual
```
2020-01-21 17:24:56 +01:00
Paul Chavard
14295db9ad Revert "Revert "Merge pull request #4552 from tchak/champ-communes""
This reverts commit 4373cb22cb.
2020-01-14 18:46:07 +01:00
clemkeirua
4373cb22cb Revert "Merge pull request #4552 from tchak/champ-communes"
This reverts commit 4cec26f73a, reversing
changes made to 0ef25ef36c.
2020-01-13 16:26:27 +01:00
Paul Chavard
4bbd16576b Add champ communes 2020-01-07 11:52:51 +01:00