Ruby on Rails | Screencasts | Download | Documentation | Weblog | Community | Source

Ticket #6896 (closed defect: fixed)

Opened 3 years ago

Last modified 2 years ago

[PATCH] counter_cache value gets overwritten

Reported by: dcmanges Assigned to: core
Priority: normal Milestone: 1.2.4
Component: ActiveRecord Version: edge
Severity: major Keywords: counter_cache verified
Cc: blog@advany.com

Description

Instances of ActiveRecord::Base store the counter_cache column in attributes. Therefore, if the counter_cache changes between when the object is initialized and when the object is updated, the update SQL query will overwrite the correct counter_cache value.

I have included a patch to demonstrate failing tests, and a also a patch for a proposed solution.

Attachments

counter_cache_overwrite_failing_tests.diff (1.2 kB) - added by dcmanges on 12/28/06 04:09:17.
attr_readonly.diff (7.4 kB) - added by dcmanges on 12/28/06 04:11:01.

Change History

12/28/06 04:09:17 changed by dcmanges

  • attachment counter_cache_overwrite_failing_tests.diff added.

12/28/06 04:11:01 changed by dcmanges

  • attachment attr_readonly.diff added.

12/28/06 04:12:11 changed by dcmanges

attr_readonly.diff adds an attr_readonly method to prevent the counter_cache from being updated - tests included.

01/03/07 04:22:08 changed by josh

  • keywords changed from counter_cache to counter_cache 1.2regression.
  • version changed from edge to 1.2.0rc1.
  • milestone changed from 1.x to 1.2.

Confirmed that the second patch works and is consider a 1.2 regression bug.

01/03/07 04:52:46 changed by david

  • keywords changed from counter_cache 1.2regression to counter_cache.

This has always been the case. It's definitely worth investigating, but it's not a 1.2 regression.

(follow-ups: ↓ 5 ↓ 7 ↓ 8 ) 01/03/07 05:01:12 changed by josh

I also don't think adding an "attr_readonly" is the best solution.

(in reply to: ↑ 4 ) 01/04/07 02:02:37 changed by kpd

It would be nice to have readonly attributes for other reasons as well. They are needed for nested set implementations, and I have seen others asking about making attributes readonly on the rails talk list.

I would really like to see something like this patch committed.

01/04/07 10:58:55 changed by fxn

Yes, in some application I have a clustering of users that is maintained in bulk updates as a certain graph of contacts evolves. Each user has a cluster_id where he belongs to. I had to be super-defensive to avoid those race conditions in client code. This patch would have been the right solution.

(in reply to: ↑ 4 ) 01/07/07 20:36:16 changed by abdurrahmanadvany

  • cc set to blog@advany.com.
  • version changed from 1.2.0rc1 to edge.

Replying to josh:

I also don't think adding an "attr_readonly" is the best solution.

What do you thing is the best solution? How would you approach this problem?

(in reply to: ↑ 4 ) 01/07/07 22:30:50 changed by brainchild

Replying to josh:

I also don't think adding an "attr_readonly" is the best solution.

It might be true that there are possibly better solutions. However this would be an improvment - what we are talking about here is a bug! And I think the severity of a bug should be definitly higher!

If you might ask why to call this a serious bug: A value that is readonly should never be included in the update-statement - that's why it's set to readonly. And this might be for a good reason. Imagine that this column is updated by the database using PLpgSQL, probably because the particular column has a datatype that is not supported by the ruby database driver like all types of arrays for instance (at least for postgres-pr). In this case this very bug is a NO-GO for Rails. You would either need a hack/workaround or to choose a different framework like STRUTS. I don't think this is where Rails wanno go!

Please don't reject this bugfix just because you think there a better solutions that might still be applied later on.

Thank you!

02/27/07 14:53:24 changed by zapnap

This patch would be huge for me, and would go a long ways towards enforcing proper encapsulation at the AR level. Any attribute created to cache a computed value (average_score on an Article over many Ratings, etc), for example, should not have a publicly available mutator. Best solution I can think of and vastly better than what I was working on. I'd love to see this committed.

03/20/07 18:42:23 changed by bitsweat

  • severity changed from normal to major.

03/20/07 20:33:19 changed by kpd

For those of you who need a quick fix, it is worth noting that ActiveRecord already provides perfect read-only behavior, but only for the primary key column!

The crude but (so far) effective hack is to monkey-patch attributes_with_quotes, adding your columns to the line containing 'column.primary', like so:

quoted[name] = quote_value(value, column) unless !include_primary_key && (column.primary || ["protected1", "protected2"].include?(column.name))

I found I also needed to cut-n-paste the 'quote_value' method in to make this work.

(follow-up: ↓ 13 ) 03/21/07 02:30:06 changed by dcmanges

I will clean up this patch, but before I do:

  • Is there a better solution than attr_readonly ?
  • What should be the behavior for the writer?
    • I think the writer should work as normal if the model is a new_record.
    • If not a new record, using the writer could raise an exception or fail validation...

(in reply to: ↑ 12 ; follow-up: ↓ 14 ) 03/22/07 08:14:21 changed by bn

Replying to dcmanges:

* What should be the behavior for the writer? * I think the writer should work as normal if the model is a new_record. * If not a new record, using the writer could raise an exception or fail validation...

This seems like sound behavior to me, though it kind of conflicts with the name "attr_readonly". Maybe "attr_final" or "attr_constant" would be slightly better?

(in reply to: ↑ 13 ) 04/18/07 16:03:51 changed by phallstrom

Replying to bn:

Replying to dcmanges:

* What should be the behavior for the writer? * I think the writer should work as normal if the model is a new_record. * If not a new record, using the writer could raise an exception or fail validation...

This seems like sound behavior to me, though it kind of conflicts with the name "attr_readonly". Maybe "attr_final" or "attr_constant" would be slightly better?

Or perhaps "attr_frozen" to go along with the "freeze" method in Ruby?

(follow-up: ↓ 16 ) 06/13/07 17:01:25 changed by obie

I believe the best solution is for the automatic callbacks that are generated when counter_cache is used to keep the counter attributes on the instance in synch with the db. Right now all they do is cause an UPDATE to happen, which means that the db is now a step ahead of the instance in play. If you happen to modify that instance after doing the increment and save it, the counter_cache values will get clobbered back to their original value.

(in reply to: ↑ 15 ) 06/14/07 03:24:09 changed by dcmanges

Replying to obie:

I believe the best solution is for the automatic callbacks that are generated when counter_cache is used to keep the counter attributes on the instance in synch with the db. Right now all they do is cause an UPDATE to happen, which means that the db is now a step ahead of the instance in play. If you happen to modify that instance after doing the increment and save it, the counter_cache values will get clobbered back to their original value.

Unless locking is involved, you can't keep all instances in sync with the database. I think the best solution is to have a way to exclude an attribute from the update queries (not the counter_cache update queries, but the save/update_attributes queries).

06/14/07 13:12:12 changed by obie

Dan, I know where you're coming from and your solution might be the best, but enhancing that automatic callback the way I described does solve the number one problem caused by this bug and is less controversial:

parent.children.create(blah...)
parent.foo = bar
parent.save! # d'oh, the counter_cache is reset to what it was originally

08/09/07 09:55:42 changed by saimonmoore

I think this ticket should be refreshed a bit. If the current patch is not in a state to be applied what needs to be done to get this fixed. There are a couple of good suggestions here. It be nice if core devs gave their opinion on the matter.

09/28/07 22:28:55 changed by jsierles

+1 on this patch. been using it for a while and does the trick.

09/28/07 23:55:53 changed by bitsweat

(In [7675]) Failing counter cache test. References #6896.

09/29/07 08:01:42 changed by jfgroff

+1: This patch totally saved our app which uses plenty of counter caches to save boatloads of db counting queries. No adverse side-effects at all after a few months live. Can't believe this bug hasn't sent more people screaming to include the patch since it was discovered 9 months ago. Definitely a must for rails2 imho.

09/29/07 09:28:19 changed by abdurrahmanadvany

  • keywords changed from counter_cache to counter_cache verified.

+1 on this patch, works perfectly!, changed to verified

09/30/07 07:09:47 changed by rick

  • status changed from new to closed.
  • resolution set to fixed.

(In [7693]) Add attr_readonly to specify columns that are skipped during a normal ActiveRecord #save operation. Closes #6896 [dcmanges]