[ruby-core:63977] [CommonRuby - Feature #10084] Add Unicode String Normalization to String class

From: nobu@...
Date: 2014-07-23 13:44:07 UTC
List: ruby-core #63977
Issue #10084 has been updated by Nobuyoshi Nakada.


What will happen for a non-unicode string, raising an exception?

----------------------------------------
Feature #10084: Add Unicode String Normalization to String class
https://bugs.ruby-lang.org/issues/10084#change-47995

* Author: Martin D=C3=BCrst
* Status: Open
* Priority: Normal
* Assignee:=20
* Category:=20
* Target version:=20
----------------------------------------
Unicode string normalization is a frequent operation when comparing or norm=
alizing strings.

This should be available directly on the String class.

The proposed syntax is:

   'string'.normalize       # normalize 'string' according to NFC (most fre=
quent on the Web)
   'string'.normalize :nfc  # normalize 'string' according to NFC; :nfd, :n=
fkc, :nfkd also usable
   'string'.nfc             # shorter variant, but maybe too many methods

There are several "unofficial" but convenient normalization variants that c=
ould be offered, e.g.:
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20
   'string'.normalize :mac  # use MacIntosh file system normalization varia=
nt

Implementations are already available in pure Ruby (easy for other Ruby imp=
lementations; e.g. eprun: https://github.com/duerst/eprun) and in C (unf,=
=E2=80=A6, http://bibwild.wordpress.com/2013/11/19/benchmarking-ruby-unicod=
e-normalization-alternatives/)

---Files--------------------------------
Normalization.pdf (576 KB)


--=20
https://bugs.ruby-lang.org/

In This Thread

Prev Next