DGNum/demarches-normaliennes
13
1
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity

fix: reduce a join query size

Browse source
This commit is contained in:
simon lehericey 2024-04-12 10:12:26 +02:00
parent cb2dd3dd3a
commit d419508b68
No known key found for this signature in database
GPG key ID: CDE670D827C7B3C5
1 changed files with 13 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
app/jobs/cron/purge_unattached_blobs_job.rb
View file

@ -5,12 +5,19 @@ class Cron::PurgeUnattachedBlobsJob < Cron::CronJob
# .in_batches { _1.each... } is more efficient in this case that in_batches.each_record or find_each # .in_batches { _1.each... } is more efficient in this case that in_batches.each_record or find_each
# because it plucks only ids in a preliminary query, then load records with selected columns in batches by ids. # because it plucks only ids in a preliminary query, then load records with selected columns in batches by ids.
# This is faster than other batch strategies, which load at once selected columns with an ORDER BY in the same query, triggering timeouts. # This is faster than other batch strategies, which load at once selected columns with an ORDER BY in the same query, triggering timeouts.
ActiveStorage::Blob.unattached.select(:id, :service_name, :created_at).in_batches do |relation| #
relation.each do |blob| # .where(created_at: 1.week.ago..1.day.ago) to limit the number of records to be joined
return if blob.created_at > 24.hours.ago # not in where() because it's not an indexed column # to the attachments table because of the unattached scope. Otherwise, it is triggering timeouts.
#
blob.purge_later # the creation of an index on created_at does not seem required yet.
end #
# caveats: the job needs to be run at least once a week to avoid missing blobs
ActiveStorage::Blob
.where(created_at: 1.week.ago..1.day.ago)
.unattached
.select(:id, :service_name)
.in_batches do |relation|
relation.each(&:purge_later)
end end
end end
end end