Re: [OT] Re: OT: Gawk match() and numbers in scientific notation

Hermann Peifer wrote:
> pk wrote:
>> On Saturday 10 May 2008 10:58, Hermann Peifer wrote:
>>
>> Agreed. That's why I'm trying to find a way to print character classes 
>> and
>> collating sequences, so that when someone comes asking "why doesn't
sort
>> work correctly" or "why doesn't grep/awk/sed. etc. match what I
expect",
>> they can be instructed to run the (non-existent yet) command or script
to
>> see for themselves what the programs' idea of what they waht to do is.
>>
> 
> I combined some code from Ed's and Steffen's scripts and added 
> /usr/bin/printf in the middle part. Unlike bash's builtin printf or 
> gawk's (s)printf: /usr/bin/printf is able to convert Unicode code point 
> values into chars.
> 
> The script is far away from being smart, efficient, or anything like 
> that, but it seems to work with Unicode-aware locales.
> 
> $ LC_ALL=en_GB.UTF-8 ./collating_chars.sh [A-C]
>
AáÁàÀăắằẵẳặĂẮẰẴẲẶâấầẫẩậÂẤẦẪẨẬǎǍåǻÅǺäǟÄǞãÃȧǡȦǠąĄāĀảẢȁȀȃȂạẠḁḀẚªæǽǣÆǼǢbBḃḂḅḄḇḆɓƁcC

> 
> 
> $ LC_ALL=da_DK.UTF-8 ./collating_chars.sh [A-C]
>
AaÁáÀàĂẮẰẴẲẶăắằẵẳặÂẤẦẪẨẬâấầẫẩậǍǎǺǻǞǟÃãȦǠȧǡĄąĀāẢảȀȁȂȃẠạḀḁẚªBbḂḃḄḅḆḇƁɓC
> 
> The example shows that in da_DK.UTF-8 locale, the [A-C] range expands to

>  less characters than in en_GB (and most other) locales. The reason is 
> that A_RING and AE_LIGATURE characters sort after Z, according to Danish

> sorting rules.
> 
> Hermann
> 
> 
> $ cat collating_chars.sh
> #!/bin/bash
> # collate sequence of characters which belong
> # to a given character range or class
> # based on Ed's and Steffen's code
> # seems to work for Unicode locales
> #
> # usage:
> # collating_chars.sh [A-Z]
> # collating_chars.sh [[:upper:]]
> 
> gawk 'BEGIN {
> 
>     for (i=0;i<=32767;i++) {
> 
>         num = sprintf("%X", i)
>         l   = length(num)
> 
>         # construct a format that is
>         # understood by /usr/bin/printf
> 
>         if (i < 16)
>                 num = "\\\\x0" num
>         else if (i < 128)
>                 num = "\\\\x" num
>         else if (l == 2)
>                 num = "\\\\u00" num
>         else if (l == 3)
>                 num = "\\\\u0" num
>         else
>                 num = "\\\\u" num
> 
>         # exclude C1 control chars
>         # as printf doesnt like them
> 
>         if (i < 128 || i > 159)
>                 print num
>     }
> }' |
> 
> # Print Unicode chars with /usr/bin/printf
> while read num ; do /usr/bin/printf "$num\n" ; done  | sort |
> 
> # Collate characters which are matched by the given range or class
> gawk -v re=$1 '$1 ~ re { s = s $1 } END { print s }'
              ^^
better use: re="$1"

Nice script. Works fine. I tested a lot with it and adapted it to my 
personal suites.

--
Steffen