Ruby on Rails | Screencasts | Download | Documentation | Weblog | Community | Source

Changeset 6740

Show
Ignore:
Timestamp:
05/15/07 21:36:21 (1 year ago)
Author:
bitsweat
Message:

Introduce the request.body stream. Lazy-read to parse parameters rather than always setting RAW_POST_DATA. Reduces the memory footprint of large binary PUT requests.

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • trunk/actionpack/CHANGELOG

    r6736 r6740  
    11*SVN* 
     2 
     3* Introduce the request.body stream. Lazy-read to parse parameters rather than always setting RAW_POST_DATA. Reduces the memory footprint of large binary PUT requests.  [Jeremy Kemper] 
    24 
    35* Add some performance enhancements to ActionView. 

  • trunk/actionpack/lib/action_controller/cgi_ext.rb

    r6733 r6740  
     1require 'action_controller/cgi_ext/stdinput' 
    12require 'action_controller/cgi_ext/parameters' 
    23require 'action_controller/cgi_ext/query_extension' 
    …… 
    56 
    67class CGI #:nodoc: 
     8  include ActionController::CgiExt::Stdinput 
    79  include ActionController::CgiExt::Parameters 
     10 
     11  class << self 
     12    alias :escapeHTML_fail_on_nil :escapeHTML 
     13 
     14    def escapeHTML(string) 
     15      escapeHTML_fail_on_nil(string) unless string.nil? 
     16    end 
     17  end 
    818end 

  • trunk/actionpack/lib/action_controller/cgi_ext/parameters.rb

    r6733 r6740  
    11require 'cgi' 
    22require 'strscan' 
    3  
    4 class CGI #:nodoc: 
    5   class << self 
    6     alias :escapeHTML_fail_on_nil :escapeHTML 
    7  
    8     def escapeHTML(string) 
    9       escapeHTML_fail_on_nil(string) unless string.nil? 
    10     end 
    11   end 
    12 end 
    133 
    144module ActionController 
    …… 
    7363        end 
    7464 
    75         def parse_formatted_request_parameters(mime_type, raw_post_data) 
     65        def parse_formatted_request_parameters(mime_type, body) 
    7666          case strategy = ActionController::Base.param_parsers[mime_type] 
    7767            when Proc 
    78               strategy.call(raw_post_data) 
     68              strategy.call(body) 
    7969            when :xml_simple, :xml_node 
    80               raw_post_data.blank? ? {} : Hash.from_xml(raw_post_data).with_indifferent_access 
     70              body.blank? ? {} : Hash.from_xml(body).with_indifferent_access 
    8171            when :yaml 
    82               YAML.load(raw_post_data) 
     72              YAML.load(body) 
    8373          end 
    8474        rescue Exception => e # YAML, XML or Ruby code block errors 
    8575          { "exception" => "#{e.message} (#{e.class})", "backtrace" => e.backtrace, 
    86             "raw_post_data" => raw_post_data, "format" => mime_type } 
     76            "body" => body, "format" => mime_type } 
    8777        end 
    8878 

  • trunk/actionpack/lib/action_controller/cgi_ext/query_extension.rb

    r6733 r6740  
    3232      @multipart = false 
    3333 
    34       # POST and PUT may have params in entity body. If content type is 
    35       # missing for POST, assume urlencoded. If content type is missing 
    36       # for PUT, don't assume anything and don't parse the parameters: 
    37       # it's likely binary data. 
    38       # 
    39       # The other HTTP methods have their params in the query string. 
     34      # POST and PUT may have params in entity body. If content type is missing 
     35      # or non-urlencoded, don't read the body or parse parameters: assume it's 
     36      # binary data. 
    4037      if method == :post || method == :put 
    4138        if boundary = extract_multipart_form_boundary(content_type) 
    …… 
    4340          @params = read_multipart(boundary, content_length) 
    4441        elsif content_type.blank? || content_type !~ %r{application/x-www-form-urlencoded}i 
    45           read_params(method, content_length) 
    4642          @params = {} 
    4743        end 

  • trunk/actionpack/lib/action_controller/cgi_process.rb

    r6733 r6740  
    5959    end 
    6060 
     61    # The request body is an IO input stream. If the RAW_POST_DATA environment 
     62    # variable is already set, wrap it in a StringIO. 
     63    def body 
     64      if raw_post = env['RAW_POST_DATA'] 
     65        StringIO.new(raw_post) 
     66      else 
     67        @cgi.stdinput 
     68      end 
     69    end 
     70 
    6171    def query_parameters 
    6272      @query_parameters ||= 
    …… 
    6777      @request_parameters ||= 
    6878        if ActionController::Base.param_parsers.has_key?(content_type) 
    69           CGI.parse_formatted_request_parameters(content_type, @env['RAW_POST_DATA']) 
     79          CGI.parse_formatted_request_parameters(content_type, body.read) 
    7080        else 
    7181          CGI.parse_request_parameters(@cgi.params) 

  • trunk/actionpack/lib/action_controller/request.rb

    r6517 r6740  
    11module ActionController 
    2   # Subclassing AbstractRequest makes these methods available to the request objects used in production and testing, 
    3   # CgiRequest and TestRequest 
     2  # CgiRequest and TestRequest provide concrete implementations. 
    43  class AbstractRequest 
    54    cattr_accessor :relative_url_root 
    65    remove_method :relative_url_root 
    76 
    8     # Returns the hash of environment variables for this request, 
     7    # The hash of environment variables for this request, 
    98    # such as { 'RAILS_ENV' => 'production' }. 
    109    attr_reader :env 
    1110 
    12     attr_accessor :format 
    13  
    14     # Returns the HTTP request method as a lowercase symbol (:get, for example). Note, HEAD is returned as :get 
    15     # since the two are supposedly to be functionaly equivilent for all purposes except that HEAD won't return a response 
    16     # body (which Rails also takes care of elsewhere). 
     11    # The requested content type, such as :html or :xml. 
     12    attr_writer :format 
     13 
     14    # The HTTP request method as a lowercase symbol, such as :get. 
     15    # Note, HEAD is returned as :get since the two are functionally 
     16    # equivalent from the application's perspective. 
    1717    def method 
    18       @request_method ||= (!parameters[:_method].blank? && @env['REQUEST_METHOD'] == 'POST') ? 
    19         parameters[:_method].to_s.downcase.to_sym : 
    20         @env['REQUEST_METHOD'].downcase.to_sym 
    21        
     18      @request_method ||= 
     19        if @env['REQUEST_METHOD'] == 'POST' && !parameters[:_method].blank? 
     20          parameters[:_method].to_s.downcase.to_sym 
     21        else 
     22          @env['REQUEST_METHOD'].downcase.to_sym 
     23        end 
     24 
    2225      @request_method == :head ? :get : @request_method 
    2326    end 
    …… 
    4346    end 
    4447 
    45     # Is this a HEAD request? HEAD is mapped as :get for request.method, so here we ask the  
    46     # REQUEST_METHOD header directly. Thus, for head, both get? and head? will return true. 
     48    # Is this a HEAD request? request.method sees HEAD as :get, so check the 
     49    # HTTP method directly. 
    4750    def head? 
    4851      @env['REQUEST_METHOD'].downcase.to_sym == :head 
    …… 
    270273    # Must be implemented in the concrete request 
    271274    #++ 
     275 
     276    # The request body is an IO input stream. 
     277    def body 
     278    end 
     279 
    272280    def query_parameters #:nodoc: 
    273281    end 

  • trunk/actionpack/lib/action_controller/test_process.rb

    r6350 r6740  
    4141    end 
    4242 
     43    # Wraps raw_post in a StringIO. 
     44    def body 
     45      StringIO.new(raw_post) 
     46    end 
     47 
     48    # Either the RAW_POST_DATA environment variable or the URL-encoded request 
     49    # parameters. 
    4350    def raw_post 
    44       if raw_post = env['RAW_POST_DATA'] 
    45         raw_post 
    46       else 
    47         params = self.request_parameters.dup 
    48         %w(controller action only_path).each do |k| 
    49           params.delete(k) 
    50           params.delete(k.to_sym) 
    51         end 
    52      
    53         params.map { |k,v| [ CGI.escape(k.to_s), CGI.escape(v.to_s) ].join('=') }.sort.join('&') 
    54       end 
     51      env['RAW_POST_DATA'] ||= url_encoded_request_parameters 
    5552    end 
    5653 
    …… 
    140137        @env["SERVER_PORT"]      = 80 
    141138        @env['REQUEST_METHOD']   = "GET" 
     139      end 
     140 
     141      def url_encoded_request_parameters 
     142        params = self.request_parameters.dup 
     143 
     144        %w(controller action only_path).each do |k| 
     145          params.delete(k) 
     146          params.delete(k.to_sym) 
     147        end 
     148 
     149        params.to_query 
    142150      end 
    143151  end 

  • trunk/actionpack/test/abstract_unit.rb

    r6611 r6740  
    44 
    55require 'yaml' 
     6require 'stringio' 
    67require 'test/unit' 
    78require 'action_controller' 
     9require 'action_controller/cgi_ext' 
    810require 'action_controller/test_process' 
    911 

  • trunk/actionpack/test/controller/raw_post_test.rb

    r6733 r6740  
    11require "#{File.dirname(__FILE__)}/../abstract_unit" 
    2 require 'stringio' 
    3 require 'action_controller/cgi_ext/query_extension' 
    42 
    53class RawPostDataTest < Test::Unit::TestCase 
    …… 
    1210    ENV['REQUEST_METHOD'] = 'POST' 
    1311    ENV['CONTENT_TYPE'] = ' apPlication/x-Www-form-urlEncoded; charset=utf-8' 
    14     assert_equal ['1'], cgi_params['a'] 
    15     assert_has_raw_post_data 
     12    assert_equal ['1'], cgi.params['a'] 
     13    assert_raw_post_data 
    1614  end 
    1715 
    …… 
    1917    ENV['REQUEST_METHOD'] = 'POST' 
    2018    ENV['CONTENT_TYPE'] = '' 
    21     assert_equal ['1'], cgi_params['a'] 
    22     assert_has_raw_post_data 
     19    assert_equal ['1'], cgi.params['a'] 
     20    assert_raw_post_data 
    2321  end 
    2422 
    25   def test_post_with_unrecognized_content_type_reads_body_but_doesnt_parse_params 
     23  def test_post_with_unrecognized_content_type_ignores_body 
    2624    ENV['REQUEST_METHOD'] = 'POST' 
    2725    ENV['CONTENT_TYPE'] = 'foo/bar' 
    28     assert cgi_params.empty? 
    29     assert_has_raw_post_data 
     26    assert cgi.params.empty? 
     27    assert_no_raw_post_data 
    3028  end 
    3129 
    …… 
    3331    ENV['REQUEST_METHOD'] = 'PUT' 
    3432    ENV['CONTENT_TYPE'] = 'application/x-www-form-urlencoded' 
    35     assert_equal ['1'], cgi_params['a'] 
    36     assert_has_raw_post_data 
     33    assert_equal ['1'], cgi.params['a'] 
     34    assert_raw_post_data 
    3735  end 
    3836 
    …… 
    4038    ENV['REQUEST_METHOD'] = 'PUT' 
    4139    ENV['CONTENT_TYPE'] = '' 
    42     assert cgi_params.empty? 
    43     assert_has_raw_post_data 
     40    assert cgi.params.empty? 
     41    assert_no_raw_post_data 
    4442  end 
    4543 
    …… 
    4745    ENV['REQUEST_METHOD'] = 'PUT' 
    4846    ENV['CONTENT_TYPE'] = 'foo/bar' 
    49     assert cgi_params.empty? 
    50     assert_has_raw_post_data 
     47    assert cgi.params.empty? 
     48    assert_no_raw_post_data 
    5149  end 
    5250 
    5351  private 
    54     def cgi_params 
    55       old_stdin, $stdin = $stdin, StringIO.new(@request_body.dup) 
    56       ENV['CONTENT_LENGTH'] = $stdin.size.to_s 
    57       CGI.new.params 
    58     ensure 
    59       $stdin = old_stdin 
     52    def cgi 
     53      unless defined? @cgi 
     54        ENV['CONTENT_LENGTH'] = @request_body.size.to_s 
     55        @cgi = CGI.new('query', StringIO.new(@request_body.dup)) 
     56      end 
     57 
     58      @cgi 
    6059    end 
    6160 
    62     def assert_has_raw_post_data(expected_body = @request_body) 
     61    def assert_raw_post_data 
    6362      assert_not_nil ENV['RAW_POST_DATA'] 
    6463      assert ENV['RAW_POST_DATA'].frozen? 
    65       assert_equal expected_body, ENV['RAW_POST_DATA'] 
     64      assert_equal @request_body, ENV['RAW_POST_DATA'] 
     65 
     66      assert_equal '', cgi.stdinput.read 
     67    end 
     68 
     69    def assert_no_raw_post_data 
     70      assert_nil ENV['RAW_POST_DATA'] 
     71 
     72      assert_equal @request_body, cgi.stdinput.read 
    6673    end 
    6774end 

  • trunk/actionpack/test/controller/test_test.rb

    r6559 r6740  
    1212      raise Test::Unit::AssertionFailedError, "#raw_post is blank" if request.raw_post.blank? 
    1313      render :text => request.raw_post 
     14    end 
     15 
     16    def render_body 
     17      render :text => request.body.read 
    1418    end 
    1519 
    …… 
    9498    get :render_raw_post, params.dup 
    9599 
    96     raw_post = params.map {|k,v| [CGI::escape(k.to_s), CGI::escape(v.to_s)].join('=')}.sort.join('&') 
    97     assert_equal raw_post, @response.body 
     100    assert_equal params.to_query, @response.body 
     101  end 
     102 
     103  def test_body_stream 
     104    params = { :page => { :name => 'page name' }, 'some key' => 123 } 
     105 
     106    get :render_body, params.dup 
     107 
     108    assert_equal params.to_query, @response.body 
    98109  end 
    99110 

  • trunk/actionpack/test/controller/webservice_test.rb

    r6511 r6740  
    11require File.dirname(__FILE__) + '/../abstract_unit' 
    2 require 'stringio' 
    32 
    43class WebServiceTest < Test::Unit::TestCase 
     4  class MockCGI < CGI #:nodoc: 
     5    attr_accessor :stdoutput, :env_table 
    56 
    6   class MockCGI < CGI #:nodoc: 
    7     attr_accessor :stdinput, :stdoutput, :env_table 
    8  
    9     def initialize(env, data = '')       
     7    def initialize(env, data = '') 
    108      self.env_table = env 
    11       self.stdinput = StringIO.new(data) 
    129      self.stdoutput = StringIO.new 
    13       super() 
     10      super(nil, StringIO.new(data)) 
    1411    end 
    1512  end 
    16  
    1713 
    1814  class TestController < ActionController::Base