Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid index_active_storage_attachments_uniqueness violation by attaching multiple files in a single call to attach #3343

Conversation

jpawlyn
Copy link
Contributor

@jpawlyn jpawlyn commented Oct 16, 2024

Description

We recently tried to switch to using direct_upload: true on the Files field. However when creating a new record with multiple files, we got the error duplicate key value violates unique constraint "index_active_storage_attachments_uniqueness".

The fix for this is to attach files in a single call to attach instead of calling attach for each file.

Fixes # (issue)

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works

Screenshots & recording

index-violation

Manual review steps

  1. Step 1
  2. Step 2

Manual reviewer: please leave a comment with output from the test if that's the case.

by attaching multiple files in a single call to attach
Copy link

codeclimate bot commented Oct 16, 2024

Code Climate has analyzed commit 17c2add and detected 0 issues on this pull request.

View more on Code Climate.

Copy link

@camilodelvasto camilodelvasto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@adrianthedev
Copy link
Collaborator

@camilodelvasto can we add a test to check for the bad behavior that was happening before? Also to check for nil values (to see that the compact_blank) works.

Copy link
Contributor

@Paul-Bob Paul-Bob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great! Thank you @jpawlyn for the contribution!

Good to merge after adding some tests

@jpawlyn
Copy link
Contributor Author

jpawlyn commented Oct 17, 2024

Looking great! Thank you @jpawlyn for the contribution!

Good to merge after adding some tests

I haven't been able to re-create the issue in the Avo test suite I'm afraid but I have recreated it in the Active Storage test suite.

Looking at https://github.com/rails/rails/blob/ee661852313ff697d5f3a73a68aab99101ea3ead/activestorage/test/models/attached/many_test.rb#L417-L426 the call to attach is taking a single signed id. When I change that to 2 separate calls to attach ie

  test "attaching an existing blob from a signed ID to a new record" do
    User.new(name: "Jason").tap do |user|
      user.highlights.attach create_blob(filename: "funky.jpg").signed_id
      user.highlights.attach create_blob(filename: "town.jpg").signed_id
      assert_predicate user, :new_record?
      assert_equal "funky.jpg", user.highlights.first.filename.to_s

      user.save!
      assert_equal "funky.jpg", user.reload.highlights.first.filename.to_s
    end
  end

the output is:

Error:
ActiveStorage::ManyAttachedTest#test_attaching_an_existing_blob_from_a_signed_ID_to_a_new_record:
ActiveRecord::RecordNotUnique: SQLite3::ConstraintException: UNIQUE constraint failed: active_storage_attachments.record_type, active_storage_attachments.record_id, active_storage_attachments.name, active_storage_attachments.blob_id
    test/models/attached/many_test.rb:425:in `block (2 levels) in <class:ManyAttachedTest>'
    test/models/attached/many_test.rb:419:in `block in <class:ManyAttachedTest>'

When I switch the test to a single call to attach with multiple signed_ids, the test passes. ie

  test "attaching an existing blob from a signed ID to a new record" do
    User.new(name: "Jason").tap do |user|
      user.highlights.attach create_blob(filename: "funky.jpg").signed_id, create_blob(filename: "town.jpg").signed_id
      assert_predicate user, :new_record?
      assert_equal "funky.jpg", user.highlights.first.filename.to_s

      user.save!
      assert_equal "funky.jpg", user.reload.highlights.first.filename.to_s
    end
  end

I don't see anywhere warning against multiple calls to attach but examples do suggest just one call to attach should be used.

Not sure how easy it is to re-create this in the Avo test suite but please let me know if you have any suggestions.

@jpawlyn jpawlyn force-pushed the fix-active-storage-attachments-uniqueness-violation branch from 24f05e3 to 1b71177 Compare October 29, 2024 07:45
@jpawlyn
Copy link
Contributor Author

jpawlyn commented Oct 29, 2024

@Paul-Bob I tried writing a test but unfortunately it doesn't fail with the original code. Somehow my app running in production and the activestorage test suite re-creates the issue but not for the Avo test suite.

@Paul-Bob
Copy link
Contributor

@Paul-Bob I tried writing a test but unfortunately it doesn't fail with the original code. Somehow my app running in production and the activestorage test suite re-creates the issue but not for the Avo test suite.

Does it fail on Avo local?

@jpawlyn
Copy link
Contributor Author

jpawlyn commented Oct 29, 2024

@Paul-Bob I tried writing a test but unfortunately it doesn't fail with the original code. Somehow my app running in production and the activestorage test suite re-creates the issue but not for the Avo test suite.

Does it fail on Avo local?

When running my app locally, yes it does fail

@Paul-Bob
Copy link
Contributor

When running my app locally, yes it does fail

Ok, thank you for your effort on this @jpawlyn. I'll have a look

@Paul-Bob Paul-Bob merged commit 15836c2 into avo-hq:main Nov 4, 2024
22 checks passed
@Paul-Bob Paul-Bob added the Fix label Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants