Support for interleaved migrations

Argh. Due to a last minute change from a decimal to string column to store version info, this is now broken. Working on a fix.

Support for new migrations in the middle of the sequence

I've replaced the broken diff with the fixed one.

Just realized that the 2.1 milestone is reserved for core members.

I'd love to see this make its way into Rails, this has been a significant pain point for us due to our extensive branching.

+1 from me, looks good

Uploading a new diff to fix a minor issue with repeatable schema loads, very small change

Interleaved Migrations patch but with a fix to repeatable schema loading

Interleaved migrations patch with ddollar's repeatable schema loading fix, and a unit test

interleaved_migrations_with_repeatable_loads_and_test.diff contains a unit test that runs migration 3, then 1, checking that nothing raises and that 1 applies.

Finally, we try to revert 2, which never ran, and make sure that it does not revert.

+1

love the idea and the patch is nice and clean.

+1, I would love to see this functionality included.

+1. Tested the patch and it works as advertised. Normal migrations work as expected, and new migrations interleaved into old ones are applied on a migrate.

There are several places in this patch where you do:

 @version = @connection.select_values( 
 	34	          "SELECT version FROM schema_migrations").map(&:to_i).max rescue nil 

That should really just be

select MAX(version) FROM schema_migrations

Also we should probably document the two kinds of tables and how we've moved between them somewhat.

Replying to nzkoz:

There are several places in this patch where you do: {{{ @version = @connection.select_values( 34 "SELECT version FROM schema_migrations").map(&:to_i).max rescue nil }}} That should really just be {{{ select MAX(version) FROM schema_migrations }}}

Sadly, SELECT MAX() here gives me back the numbers in ASCIIabetic order, which is not quite what you want, I think. It would put a migration version of '30' above a timestamp on 2008.

I thought about making version either an int8 column, or a decimal with no precision, but I figured things would get too different from database to database, so I decided instead to just do what jbarnette did to schema_info in [9122] and make it a string column, which sadly needs the .map(&:to_i).max bit.

Also we should probably document the two kinds of tables and how we've moved between them somewhat.

Glad to do it. Where would such documentation go? As comments in the code, or somewhere in the API?

Hmm, I didn't see the fact that it was a stringnow? Out of interest, why is that? The nice thing about those timestamps is that they sort numerically as well as alphabetically, so I can't see why we need to change it?

As for the documentation, stick it alongside the rest of the docs in migration.rb as a new heading.

Can you shed any light on the string column for us jbarnette?

Replying to nzkoz:

Hmm, I didn't see the fact that it was a stringnow? Out of interest, why is that? The nice thing about those timestamps is that they sort numerically as well as alphabetically, so I can't see why we need to change it?

DHH wanted the whole system to be backwards compatible with existing migration files. So migration 3 would happen after today's timestamp, which is no good.

As for the documentation, stick it alongside the rest of the docs in migration.rb as a new heading.

Got it. Will post patch later today.

Replying to nzkoz:

Hmm, I didn't see the fact that it was a stringnow? Out of interest, why is that?

I presume also we didn't want to assume that all databases have int8. But maybe jbarnette can elaborate. I mostly just went with the flow.

Replying to nzkoz:

It's an integer-size-across-databases thing: 20081231060606 is big. :) I preferred the simplicity of storing the same notation used for filename prefixes. I suppose we could store epoch seconds in the DB instead.

The way I see it:

Epoch seconds in the DB is are tricky, because some you'd want to fetch verbatim (like 1, 2, 3, 4), but some you'd want to convert to timestamps. We'd need logic to determine that. It's not hard by any means, just cumbersome (we would need to basically define the jbarnette's epoch, *grin*). Saving the migration files themselves with seconds from epoch as opposed to UTC timestamps would avoid that logic, but if I recall correctly DHH didn't want them to be named after seconds from the epoch.

The other alternative is to add the creation of that table to the adapters, so that they can all create their version of what an int8 column is. Again, too much work for little gain in my opinion.

Let me zoom back out for a second, and ask this: what are we trying to avoid by not making that column a varchar? Ruby iterating over a few thousand values at worst? Considering that it would happen during a task that was never snappy to begin with (rake db:migrate), I don't think it's worth the effort or code size.

That said, I have personal interest in this patch making it in (we branch a lot!), so if either int8, seconds from the epoch or something else is a better solution in the eyes of the core team, I'd be happy to code away.

Oh, and those documentation comments are coming. Busy weekend.

The varchar is the best way to go, IMHO. No need for conversions or heuristics on the 'legacy' migration numbers.

Yeah, looks fine to me, though the repeated '.map(&:to_i).max' stuff should be encapsulated somehow rather than repeated in a few places.

Once that's done and the docs are ready, looks good to me.

Interleaved migrations patch with bugfixes, tests and docs

Behold: documented the destiny of schema_info, and encapsulated the scary .map(&:to_i).max.

Interleaved migrations patch bugfixes, tests and docs

Just realized that there're two of the latest patch "interleaved_migrations_with_tests_and_docs". They're both the same according to md5sum. I posted twice by accident, I guess.

I was just bitching about timestamp migrations, which earned me a PDI on this plugin :) I'll check it out later tonight and see about getting this into 2.1. I'll be in #rails-contrib when i'm working on it.

(In [9244]) Add support for interleaving migrations by storing which migrations have run in the new schema_migrations table. Closes #11493 [jordi]