Ruby on Rails | Screencasts | Download | Documentation | Weblog | Community | Source

Ticket #9622 (closed defect: fixed)

Opened 2 years ago

Last modified 2 years ago

[PATCH] token_generator throttles inserts to 1/sec

Reported by: jchris Assigned to: core
Priority: normal Milestone: 2.x
Component: Plugins Version: edge
Severity: normal Keywords: tiny verified
Cc:

Description

If you are inserting many undifferentiated (no attributes set) objects quickly (maybe you need their primary keys before you can get on with life) and applying_tokens to them on create, you'll end up spinning your wheels while running selects as fast as you can. This is because #inspect and Time.now won't change from one item to the next. Adding some randomness speeds things up dramatically. The fix is a oneliner in plugins/token_generator/lib/token_generator.rb

      token = Digest::MD5.hexdigest("#{inspect}#{Time.now}").first(size)

should be

      token = Digest::MD5.hexdigest("#{inspect}#{Time.now}#{rand}").first(size)

I can supply a patch, but the fix is dead simple if anyone cares to apply it.

Attachments

add_rand_to_generate_token.diff (502 bytes) - added by rmm5t on 09/22/07 03:39:34.
#{rand} added to md5 input in #generate_token
fix_size_of_generated_token.diff (446 bytes) - added by rmm5t on 10/18/07 19:08:01.
Fix for regression of the token size.

Change History

09/22/07 02:57:10 changed by rmm5t

  • status changed from new to closed.
  • resolution set to worksforme.

I don't think this is necessary. #inspect already contains the memory address of the object. Even if each object has no attributes set, a new address is assigned for each and is visible in the return value of #inspect. So, in essence, appending rand isn't necessary.

In other words, when I use token_generator, I can create objects much faster than 1/sec because the address changes for each. Here's an example script/console session (the Question object includes TokenGenerator):

>> require 'benchmark'
=> []
>> puts Benchmark.measure { 600.times { Question.create } }
  1.590000   0.170000   1.760000 (  3.236665)
=> nil

That's obviously way faster at roughly 185/sec and would be even faster if it weren't actually touching the database.

Feel free to reopen if I misunderstood what you're trying to accomplish.

09/22/07 03:12:12 changed by jchris

>> Track.new.inspect
=> "#<Track id: nil, token: nil, updated_at: nil, created_at: nil, json_cache: nil, merged_into_id: nil, takedown_id: nil, play_count: nil, activity_cache: nil>"

I don't get a memory address on my #inspect (but I have that funky new AR inspect format happening.) Maybe that's been changed (I'm running from Rails r7298), so things aren't an issue anymore.

I any case, with #rand in place, I get benchmarks like yours, without it, this is what I get:

>> require 'benchmark'
=> []
>> puts Benchmark.measure { 30.times { Track.create } }
 18.640000   1.600000  20.240000 ( 28.849825)

I'll try again on an up to date trunk over the weekend, and see if it's been fixed.

09/22/07 03:33:49 changed by rmm5t

  • status changed from closed to reopened.
  • resolution deleted.

jchris, I see. I naively checked this against my only project that uses TokenGenerator (happened to be a Rails 1.2.3 instance). Your right. It looks like ActiveRecord::Base#inspect no longer has any memory addresses.

09/22/07 03:39:34 changed by rmm5t

  • attachment add_rand_to_generate_token.diff added.

#{rand} added to md5 input in #generate_token

09/22/07 03:40:27 changed by rmm5t

  • keywords changed from simple to tiny.
  • summary changed from [SIMPLE] token_generator throttles inserts to 1/sec to [PATCH] token_generator throttles inserts to 1/sec.

+1

09/22/07 03:41:53 changed by rmm5t

It would be nice to be able to write a test for this kind of thing, but the plugin is without tests altogether, and I took the lazy route.

10/18/07 01:19:15 changed by danger

+1 It would be hilarious if the next 'Rails is slow' debater pointed out that Rails only lets you do some things once per second. Let's avoid that :-)

10/18/07 16:40:57 changed by mikong

  • keywords changed from tiny to tiny verified.

I learned a few things verifying this. Thanks :)

+1

10/18/07 18:20:57 changed by bitsweat

  • status changed from reopened to closed.
  • resolution set to fixed.

(In [7965]) Increase randomness of TokenGenerator#generate_token by reusing CGI::Session.generate_unique_id. Closes #9622 [rmm5t, Jeremy Kemper]

10/18/07 19:07:21 changed by rmm5t

  • status changed from closed to reopened.
  • resolution deleted.

Jeremy, I like the use of CGI::Session.generate_unique_id. Don't forget about the size of the token. Patch to fix regression forthcoming.

10/18/07 19:08:01 changed by rmm5t

  • attachment fix_size_of_generated_token.diff added.

Fix for regression of the token size.

10/18/07 19:23:47 changed by bitsweat

  • status changed from reopened to closed.
  • resolution set to fixed.

(In [7967]) Take first(size) chars for token. Closes #9622.