Form#serialize scrambling collection[][attribute]

Confirmed:

>> ActionController::CgiRequest.parse_query_parameters 'i[][a]=1&i[][b]=2&i[][a]=3&i[][b]=4'
=> {"i"=>[{"a"=>"1", "b"=>"2"}, {"a"=>"3", "b"=>"4"}]}
>> ActionController::CgiRequest.parse_query_parameters 'i[][a]=1&i[][a]=3&i[][b]=2&i[][b]=4'
=> {"i"=>[{"a"=>"1"}, {"a"=>"3", "b"=>"2"}, {"b"=>"4"}]}

Quite nasty. I'll deal with it.

failing test case

I had to deal with this problem, and after some investigation, found that the Ajax request code was marshalling the query string back into a hash and then running toQueryString on it again.

So the workaround is to use:

<%= link_to_remote "stuff", :url => action_path(), :with => "Form.getElements('myform').reject(function (x) { return x.name == '_method);} )" %>

instead of

<%= link_to_remote "stuff", :url => action_path(), :with => "Form.serialize('myform')" %>

(rejecting the _method field is necessary because prototype neglects to set the _method to post/get for a form that already has a _method set to put)

actually, I must have inadvertently changed something that made that work, because with a clean copy of protoype 1.5.1.1 it no longer works.

What is the eta on fixing this? I should have read more carefully because I spent a bunch of time figuring out that the bug has nothing to do with Prototype, when that should have been obvious from mislav's comment. Shouldn't the component change to ActionPack and isn't this a fairly significant bug?

I would also see the bug in normal form submissions if I lay out a nested resource like this:

record1 record2 record3

---

attr1 attr1 attr1 attr2 attr2 attr2

I'll try to fix this as soon as possible, probably next week. It will be included in Prototype 1.6.1

I'm not sure the fix should be in prototype. Perhaps prototype should preserve the order of form fields, but more importantly the Rails query parser should be less braindead.

I've attached a patch which has a rewritten UrlEncodedPairParser that handles array params in both breadth-first and depth-first order (so long as no rows have missing keys when breadth-first)

It fixes this bug, and also incorporates the patch to HashWithIndifferentAccess first posted in Ticket #9030 and fixes that bug too.

lastobelus, can you try to fix your patch to adhere more to Rails coding practices; use more intelligible variable names and use two whitespace indents?

refactored UrlEncodedPairParser to handle array params in breadth-first order, cleaned up

How's this? not sure how the tabs crept in there...

Lastobelus, please continue patching ActionPack on #9030 only. This ticket is for Prototype's parsing mechanism which you can only fix through JavaScript, not Ruby code.

Patch fixes original poster's problem.

Feel fairly certain he won't care whether it is done with "JavaScript", "Ruby" or Javascript that's been painstakingly twisted to look vaguely LIKE ruby. Although, after 4 months he's probably gone back to Hibernate/Struts.

And I COULD fix Prototype's parsing mechanism with ruby, it would just take a lot more typing.

Let us first wait for your ActionPack fix and tests to go through. After some tests have been commited I'll try and match it in Prototype.

I will not implement full parsing in Proto, just enough not to scramble parameters when serializing from string to Object and back to string again.

Ok. The current Prototype serializing will actually work unchanged with the patched parser. Prototype isn't changing the order of fields within an array param, it is just grouping the columns together, so it writes

a[][field1]=1&a[][field1]=2&a[][field2]=x&a[][field2]=y

  instead of

a[][field1]=1&a[][field2]=x&a[][field1]=2&a[][field2]=y

The reason I tackled the parser instead of Prototype is because I felt the parser was more "wrong" than Prototype and that changing the Prototype serialization to maintain physical order through marshalling round trips would be a lot more work and resolve only a subset of the problems. You'd have to use metainformation. Once a sparse array is marshalled, the information about how it was physically laid out is lost. For example, the form that I had that led me to this ticket was also broken with Prototype 1.5.

on #10101 I have added a new patch and a test app.

The new parser handles multiple nesting of arrays:

object[subobject][][subsubobject][]

to accomplish this, an explicit array index syntax was added.

Also, for the checkbox hack to work when arrays can be depth first or breadth first, an explicit syntax for the hidden field name was added.

The new parser and patched form_helper allow using fields_for to create forms for arbitrarily nested arrays of objects.

We are still using Prototype 1.5 in our project, because we are using ext.js and some other widgets which we have not taken the time to upgrade to Prototype 1.6. Because of this, I have not verified the new parser with ajax form submission with prototype 1.6, and last I checked I think there was a small bug in prototype with the serialization, that I fixed by adding a :with_hash option to prototype_helper.rb to avoid double serialization when prototype tries to automagically add the form_authenticity_token.

I will attempt to look at that with the latest edge prototype.js when I have time

We've used the new param parser on our project for a couple months now, and a couple colleagues who want forms for nested objects are using it too.