Ruby on Rails | Screencasts | Download | Documentation | Weblog | Community | Source

Ticket #9632 (closed defect: incomplete)

Opened 10 months ago

Last modified 9 months ago

[PATCH] Multibyte conformance tests always skip utf8proc

Reported by: julik Assigned to: michael@koziarski.com
Priority: normal Milestone: 2.x
Component: ActiveSupport Version: edge
Severity: normal Keywords: multibyte handlers utf8proc
Cc: manfred, nzkoz

Description

Due to a typo on the conformance test and the all-swallowing rescue the conformance tests for utf8proc dud not run properly.

This patch brings this to the naming conventions currently used by utf8proc gem and hooks it up to the tests proper. Unfortunately, utf8proc is not compliant as it seems. Something to email Jan Behrens about, but it would be very nice to get some accelerated unicode goodness.

Attachments

wrong_include.diff (1.6 kB) - added by julik on 09/22/07 19:54:37.

Change History

09/22/07 19:54:37 changed by julik

  • attachment wrong_include.diff added.

09/22/07 19:57:43 changed by julik

  • cc changed from manfred to manfred, nzkoz.

On a sidenote, there probably might be a way to use UCD for CP1251 exclusively on utf8proc and thusly avoid loading largish tables. This would spare quite some RAM.

09/23/07 22:31:24 changed by david

  • owner changed from core to michael@koziarski.com.

09/23/07 22:51:01 changed by manfred

Julian, what do you mean exactly with 'a way to use UCD for CP1251'? If you mean we don't have to load the UCD when using utf8proc, I'm afraid that's not going to work because utf8proc isn't as feature complete as the pure ruby implementation.

Maybe we should write our own C extension based on GLib Unicode functions?

09/23/07 23:06:37 changed by julik

So you answered my question :-) we can't. But maybe we can sort-of approve the fallacies ot utf8proc and partialize the UCD bootstrap for this handler (as well as for an ICU handler that I would like to restore at some point).

This is optional of course, just something to think about. I would write an extension but I dunno if I'm in state to learn C now, and it's questionable if having C parts is good for Rails.

Maybe we can first fix the requires and then I will refile the ticket.

10/15/07 03:59:33 changed by bitsweat

  • status changed from new to closed.
  • resolution set to incomplete.

When I apply the patch and install the utf8proc gem, I still don't see the 

Also testing the utf8 backend

message.