javascript - Unicode string with diacritics split by chars

Question

I have this Unicode string: Ааа?Ббб?Ввв?Гг??Дд

And I want to it split by chars. Right now if I try to loop truth all chars I get something like this:
A a a ' Б ...

Is there a way to properly split this string to chars: А а а? ?

Answer 1

To do this properly, what you want is the algorithm for working out the grapheme cluster boundaries, as defined in UAX 29. Unfortunately this requires knowledge of which characters are members of which classes, from the Unicode Character Database, and JavaScript doesn't make that information available(*). So you'd have to include a copy of the UCD with your script, which would make it pretty bulky.

An alternative if you only need to worry about the basic accents used by Latin or Cyrillic would be to take only the Combining Diacritical Marks block (U+0300-U+036F). This would fail for other languages and symbols, but might be enough for what you want to do.

function findGraphemesNotVeryWell(s) {
    var re= /.[u0300-u036F]*/g;
    var match, matches= [];
    while (match= re.exec(s))
        matches.push(match[0]);
    return matches;
}

findGraphemesNotVeryWell('Ааа?Ббб?Ввв?Гг??Дд');
["А", "а", "а?", "Б", "б", "б?", "В", "в", "в?", "Г", "г", "?", "?", "Д", "д"]

(*: there might be a way to extract the information by letting the browser render the string, and measuring the positions of selections in it... but it would surely be very messy and difficult to get working cross-browser.)