about summary refs log tree commit diff
path: root/localedata
Commit message (Collapse)AuthorAgeFilesLines
* Fix whitespace related license issues.Carlos O'Donell2024-10-074-4/+4
| | | | | | | | | | | | Several copies of the licenses in files contained whitespace related problems. Two cases are addressed here, the first is two spaces after a period which appears between "PURPOSE." and "See". The other is a space after the last forward slash in the URL. Both issues are corrected and the licenses now match the official textual description of the license (and the other license in the sources). Since these whitespaces changes do not alter the paragraph structure of the license, nor create new sentences, they do not change the license.
* Update to Unicode 16.0.0 [BZ #32168]Mike FABIAN2024-09-2714-3098/+14403
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Unicode 16.0.0 Support: Character encoding, character type info, and transliteration tables are all updated to Unicode 16.0.0, using the generator scripts contributed by Mike FABIAN (Red Hat). Changes in CHARMAP and WIDTH: Total added characters in newly generated CHARMAP: 5185 Total removed characters in newly generated WIDTH: 1 Total added characters in newly generated WIDTH: 170 The removed character from WIDTH is U+1171E AHOM CONSONANT SIGN MEDIAL RA. It changed like this: UnicodeData.txt 15.1.0: 1171E;AHOM CONSONANT SIGN MEDIAL RA;Mn;0;NSM;;;;;N;;;;; UnicodeData.txt 16.0.0: 1171E;AHOM CONSONANT SIGN MEDIAL RA;Mc;0;L;;;;;N;;;;; EastAsianWidth.txt 15.1.0: 1171D..1171F ; N # Mn [3] AHOM CONSONANT SIGN MEDIAL LA..AHOM CONSONANT SIGN MEDIAL LIGATING RA EastAsianWidth.txt 16.0.0: 1171E ; N # Mc AHOM CONSONANT SIGN MEDIAL RA I.e it changed from Mn (Mark Nonspacing) to Mc (Mark Spacing combining). So it should now have width 1 instead of 0, therefore it is OK that it was removed from WIDTH, characters not in WIDTH get width 1 by default. Nothing suspicious when browsing the list of the 170 added characters. Changes in ctype: alpha: Added 4452 characters in new ctype which were not in old ctype combining: Added 51 characters in new ctype which were not in old ctype combining_level3: Added 43 characters in new ctype which were not in old ctype graph: Added 5185 characters in new ctype which were not in old ctype lower: Added 25 characters in new ctype which were not in old ctype print: Added 5185 characters in new ctype which were not in old ctype punct: Missing 33 characters of old ctype in new ctype punct: Added 766 characters in new ctype which were not in old ctype tolower: Added 27 characters in new ctype which were not in old ctype totitle: Added 27 characters in new ctype which were not in old ctype toupper: Added 27 characters in new ctype which were not in old ctype upper: Added 27 characters in new ctype which were not in old ctype Nothing suspicous in the additions. About the 33 characters removed from `punct`: U+0363 - U+036F are identical in UnicodeData.txt. Difference in DerivedCoreProperties.txt: DerivedCoreProperties.txt 15.1.0: not there. DerivedCoreProperties.txt 16.0.0: 0363..036F ; Alphabetic # Mn [13] COMBINING LATIN SMALL LETTER A..COMBINING LATIN SMALL LETTER X So that’s the reason why they are added to `alpha` and removed from `punct`. Same for U+1DD3 - U+1DE6, they are identical in UnicodeData.txt but there is a difference in DerivedCoreProperties.txt: DerivedCoreProperties.txt 15.1.0: 1DE7..1DF4 ; Alphabetic # Mn [14] COMBINING LATIN SMALL LETTER ALPHA..COMBINING LATIN SMALL LETTER U WITH DIAERESIS DerivedCoreProperties.txt 16.0.0: 1DD3..1DF4 ; Alphabetic # Mn [34] COMBINING LATIN SMALL LETTER FLATTENED OPEN A ABOVE..COMBINING LATIN SMALL LETTER U WITH DIAERESIS So they became `Alphabetic` and were thus added to `alpha` and removed from `punct`. Resolves: BZ #32168 Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* support: Use macros for *stat wrappersFlorian Weimer2024-08-161-1/+1
| | | | | | | | | | | | Macros will automatically use the correct types, without having to fiddle with internal glibc macros. It's also impossible to get the types wrong due to aliasing because support_check_stat_fd and support_check_stat_path do not depend on the struct stat* types. The changes reveal some inconsistencies in tests. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* support: Add FAIL test failure helperMaciej W. Rozycki2024-07-261-31/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a FAIL test failure helper analogous to FAIL_RET, that does not cause the current function to return, providing a standardized way to report a test failure with a message supplied while permitting the caller to continue executing, for further reporting, cleaning up, etc. Update existing test cases that provide a conflicting definition of FAIL by removing the local FAIL definition and then as follows: - tst-fortify-syslog: provide a meaningful message in addition to the file name already added by <support/check.h>; 'support_record_failure' is already called by 'support_print_failure_impl' invoked by the new FAIL test failure helper. - tst-ctype: no update to FAIL calls required, with the name of the file and the line number within of the failure site additionally included by the new FAIL test failure helper, and error counting plus count reporting upon test program termination also already provided by 'support_record_failure' and 'support_report_failure' respectively, called by 'support_print_failure_impl' and 'adjust_exit_status' also respectively. However in a number of places 'printf' is called and the error count adjusted by hand, so update these places to make use of FAIL instead. And last but not least adjust the final summary just to report completion, with any error count following as reported by the test driver. - test-tgmath2: no update to FAIL calls required, with the name of the file of the failure site additionally included by the new FAIL test failure helper. Also there is no need to track the return status by hand as any call to FAIL will eventually cause the test case to return an unsuccesful exit status regardless of the return status from the test function, via a call to 'adjust_exit_status' made by the test driver. Reviewed-by: DJ Delorie <dj@redhat.com>
* tests-mbwc: Silence gcc 14 -Werror=format-overflow=Stefan Liebler2024-06-271-19/+5
| | | | | | | | | | | | | | | | | | | | | | | | | With gcc 14, I get this warning/werror when building the localedata tests: tests-mbwc/tsp_common.c: In function ‘result.constprop.isra’: tests-mbwc/tsp_common.c:55:43: error: ‘%s’ directive writing up to 92 bytes into a region of size between 0 and 114 [-Werror=format-overflow=] 55 | sprintf (result_rec, "%s:%s:%d:%d:%d:%c:%s\n", func, loc, rec_no, seq_no, | ^~ In file included from ../include/bits/stdio2.h:1, from ../libio/stdio.h:980, from ../include/stdio.h:14, from tests-mbwc/tsp_common.c:10: In function ‘sprintf’, inlined from ‘result.constprop.isra’ at tests-mbwc/tsp_common.c:55:3: ../libio/bits/stdio2.h:30:10: note: ‘__builtin___sprintf_chk’ output between 20 and 234 bytes into a destination of size 132 30 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 31 | __glibc_objsize (__s), __fmt, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 32 | __va_arg_pack ()); | ~~~~~~~~~~~~~~~~~ cc1: all warnings being treated as errors This patch now gets rid of using sprintf and the result_rec buffer and just prints to fp directly.
* Define ISO 639-3 "ltg" (Latgalian) and add ltg_LV localeMike FABIAN2024-06-174-0/+309
| | | | | | | | | Resolves: BZ # 31411 References: https://iso639-3.sil.org/code/ltg https://en.wikipedia.org/wiki/Latgalian_language https://github.com/unicode-org/cldr/blob/main/common/main/ltg.xml
* localedata: Lowercase day and abday in cs_CZMike FABIAN2024-06-111-15/+15
| | | | | | Resolves: BZ # 25119 Also to sync with CLDR
* localedata: add new locales scn_ITDavid Paleino2024-06-072-0/+156
| | | | Signed-off-by: David Paleino <dapal@debian.org>
* localedata: cv_RU: update translationMike FABIAN2024-05-232-209/+102
| | | | Resolves: BZ # 21271
* localedata: Fix several issues with the set of characters considered 0-width ↵Jules Bertholet2024-05-155-1897/+978
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [BZ #31370] = `Default_Ignorable_Code_Point`s should have width 0 = Unicode specifies (https://www.unicode.org/faq/unsup_char.html#3) that characters with the `Default_Ignorable_Code_Point` property > should be rendered as completely invisible (and non advancing, i.e. “zero width”), if not explicitly supported in rendering. Hence, `wcwidth()` should give them all a width of 0, with two exceptions: - the soft hyphen (U+00AD SOFT HYPHEN) is assigned width 1 by longstanding precedent - U+115F HANGUL CHOSEONG FILLER needs a carveout due to the unique behavior of the conjoining Korean jamo characters. One composed Hangul "syllable block" like 퓛 is made up of two to three individual component characters, or "jamo". These are all assigned an `East_Asian_Width` of `Wide` by Unicode, which would normally mean they would all be assigned width 2 by glibc; a combination of (leading choseong jamo) + (medial jungseong jamo) + (trailing jongseong jamo) would then have width 2 + 2 + 2 = 6. However, glibc (and other wcwidth implementations) special-cases jungseong and jongseong, assigning them all width 0, to ensure that the complete block has width 2 + 0 + 0 = 2 as it should. U+115F is meant for use in syllable blocks that are intentionally missing a leading jamo; it must be assigned a width of 2 even though it has no visible display to ensure that the complete block has width 2. However, `wcwidth()` currently (before this patch) incorrectly assigns non-zero width to U+3164 HANGUL FILLER and U+FFA0 HALFWIDTH HANGUL FILLER; this commit fixes that. Unicode spec references: - Hangul: §3.12 https://www.unicode.org/versions/Unicode15.0.0/ch03.pdf#G24646 and §18.6 https://www.unicode.org/versions/Unicode15.0.0/ch18.pdf#G31028 - `Default_Ignorable_Code_Point`: §5.21 https://www.unicode.org/versions/Unicode15.0.0/ch05.pdf#G40095. = Non-`Default_Ignorable_Code_Point` format controls should be visible = The Unicode Standard, §5.21 - Characters Ignored for Display (https://www.unicode.org/versions/Unicode15.0.0/ch05.pdf#G40095) says the following: > A small number of format characters (General_Category = Cf ) > are also not given the Default_Ignorable_Code_Point property. > This may surprise implementers, who often assume > that all format characters are generally ignored in fallback display. > The exact list of these exceptional format characters > can be found in the Unicode Character Database. > There are, however, three important sets of such format characters to note: > > - prepended concatenation marks > - interlinear annotation characters > - Egyptian hieroglyph format controls > > The prepended concatenation marks always have a visible display. > See “Prepended Concatenation Marks” in [*Section 23.2, Layout Controls*](https://www.unicode.org/versions/Unicode15.1.0/ch23.pdf#M9.35858.HeadingBreak.132.Layout.Controls) > for more discussion of the use and display of these signs. > > The other two notable sets of format characters that exceptionally are not ignored > in fallback display consist of the interlinear annotation characters, > U+FFF9 INTERLINEAR ANNOTATION ANCHOR through > U+FFFB INTERLINEAR ANNOTATION TERMINATOR, > and the Egyptian hieroglyph format controls, > U+13430 EGYPTIAN HIEROGLYPH VERTICAL JOINER through > U+1343F EGYPTIAN HIEROGLYPH END WALLED ENCLOSURE. > These characters should have a visible glyph display for fallback rendering, > because if they are not displayed, > it is too easy to misread the resulting displayed text. > See “Annotation Characters” in [*Section 23.8, Specials*](https://www.unicode.org/versions/Unicode15.1.0/ch23.pdf#M9.21335.Heading.133.Specials), > as well as [*Section 11.4, Egyptian Hieroglyphs*](https://www.unicode.org/versions/Unicode15.1.0/ch11.pdf#M9.73291.Heading.1418.Egyptian.Hieroglyphs) > for more discussion of the use and display of these characters. glibc currently correctly assigns non-zero width to the prepended concatenation marks, but it incorrectly gives zero width to the interlinear annotation characters (which a generic terminal cannot interpret) and the Egyptian hieroglyph format controls (which are not widely supported in rendering implementations at present). This commit fixes both these issues as well. = Derive Hangul syllable type from Unicode data = Previosuly, the jungseong and jongseong jamo ranges were hard-coded into the script. With this commit, they are instead parsed from the HangulSyllableType.txt data file published by Unicode. This does not affect the end result. Signed-off-by: Jules Bertholet <julesbertholet@quoi.xyz>
* localedata: fix weekdays in mdf_RU localeMike FABIAN2024-05-081-14/+15
| | | | | | From Кирилл Изместьев <izmestevks@basealt.ru>, see: https://sourceware.org/bugzilla/show_bug.cgi?id=31530#c6 and the following comments.
* localedata: add mdf_RU localeMike FABIAN2024-05-084-0/+207
| | | | Resolves: BZ # 31530
* locale: Handle loading a missing locale twice (Bug 14247)Carlos O'Donell2024-04-223-5/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | Delay setting file->decided until the data has been successfully loaded by _nl_load_locale(). If the function fails to load the data then we must return and error and leave decided untouched to allow the caller to attempt to load the data again at a later time. We should not set decided to 1 early in the function since doing so may prevent attempting to load it again. We want to try loading it again because that allows an open to fail and set errno correctly. On the other side of this problem is that if we are called again with the same inputs we will fetch the cached version of the object and carry out no open syscalls and that fails to set errno so we must set errno to ENOENT in that case. There is a second code path that has to be handled where the name of the locale matches but the codeset doesn't match. These changes ensure that errno is correctly set on failure in all the return paths in _nl_find_locale(). Adds tst-locale-loadlocale to cover the bug. No regressions on x86_64. Co-authored-by: Jeff Law <law@redhat.com> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* Update SHARED-FILES and license for Unicode 15.1.0.Carlos O'Donell2024-02-272-44/+37
| | | | | | | | | | | | | | | In 2018 the license changed to use Unicode-3.0 license. The Unicode License is a permissive MIT type of license. Automation is updated to fetch the correct license file to keep it in sync with the data files. The new license is OSI approved and has an SPDX identifer: https://opensource.org/license/unicode-license-v3 https://spdx.org/licenses/Unicode-3.0.html The FSF and the GNU Project have been contacted to update the license list for this license: https://www.gnu.org/licenses/license-list.html
* localedata: ssy_ER: Fix syntax errorMike FABIAN2024-02-081-1/+1
|
* localedata: hr_HR: change currency to EUR/€Dragan Stanojević (Nevidljivi)2024-02-081-4/+4
| | | | Resolves: BZ # 29845
* Change lv_LV collation to agree with the recent change in CLDRMike FABIAN2024-02-082-36/+122
| | | | | | | Resolves: https://sourceware.org/bugzilla/show_bug.cgi?id=23774 See this change in CLDR committed on 2024-01-29: https://github.com/unicode-org/cldr/commit/635e2d3d0566563e12c8ea58c219d35183fcec12
* localedata: Use consistent values for grouping and mon_groupingMike FABIAN2024-01-25131-229/+223
| | | | | | Resolves: BZ # 31205 Adapt test cases in test-grouping_iterator.c
* localedata: renamed: aa_ER@saaho -> ssy_ERMike FABIAN2024-01-182-5/+17
| | | | Resolves: BZ # 19956
* localedata: add crh_RU, Crimean Tartar language in the Cyrillic script as ↵Mike FABIAN2024-01-182-0/+184
| | | | | | used in Russia. Resolves: BZ # 24386
* localedata: tr_TR, ku_TR: Sync with CLDR: “Turkey” -> “Türkiye”Mike FABIAN2024-01-182-7/+7
| | | | Resolves: BZ # 31257
* localedata: miq_NI: Shorten month names in abmonMike FABIAN2024-01-181-17/+13
| | | | Resolves: BZ # 23172
* localedata: add gbm_IN localeMike FABIAN2024-01-172-0/+178
| | | | Resolves: BZ # 19479
* localedata: anp_IN: Fix abbreviated month namesMike FABIAN2024-01-151-16/+14
| | | | | | | | | Resolves: BZ # 31239 The correct abbreviated month names were apparently given in the comment above `abmon`. But the value of `abmon` was apparently just copied from the value of `mon` and this mistake was hard to see because code point notation <Uxxxx> was used. After converting to UTF-8 it was obvious that there was apparently a copy and paste mistake.
* localedata/unicode-gen/utf8_gen.py: fix Hangul syllable nameMike FABIAN2024-01-142-400/+400
| | | | Resolves: BZ # 29506
* localedata: Remove redundant commentsMike FABIAN2024-01-1387-260/+14
|
* localedata: revert all the remaining locale sources to UTF-8Mike FABIAN2024-01-11205-6090/+6088
|
* localedata: am_ET ber_DZ en_GB en_PH en_US fil_PH kab_DZ om_ET om_KE ti_ET ↵Mike FABIAN2024-01-118-132/+132
| | | | tl_PH: convert to UTF-8
* localedata: resolve cyclic dependenciesMike FABIAN2024-01-1111-1050/+226
| | | | Resolves: BZ # 24006
* localedata: kv_RU: convert to UTF-8Mike FABIAN2024-01-111-57/+57
|
* localedata: add new locale kv_RUMike FABIAN2024-01-112-0/+165
| | | | Resolves: BZ # 30605
* localedata: Sort Makefile variables.Carlos O'Donell2024-01-101-219/+298
| | | | | | Sort Makefile variables using scrips/sort-makefile-lines.py. No regressions on x86_64.
* localedata: su_ID: make lang_name agree with CLDRMike FABIAN2024-01-091-1/+1
|
* localedata: add new locale su_IDMike FABIAN2024-01-092-0/+165
| | | | Resolves: BZ # 27312
* localedata: add new locale zgh_MAMike FABIAN2024-01-092-0/+191
| | | | | | Resolves: BZ # 12908 https://iso639-3.sil.org/code/zgh
* localedata: add tok/UTF-8 to SUPPORTEDMike FABIAN2024-01-091-0/+1
|
* localedata: tok: add yY and nN to yesexpr and noexprMike FABIAN2024-01-091-2/+2
| | | | See: https://sourceware.org/bugzilla/show_bug.cgi?id=31221#c2
* localedata: tok: convert to UTF-8Mike FABIAN2024-01-091-3/+3
|
* localedata: add data for tok (Toki Pona)Janet Blackquill2024-01-091-0/+154
| | | | | | | | | Resolves: BZ # 31221 glibc can recognise its code, but does not have its data. This patch remedies that. Signed-off-by: Janet Blackquill <uhhadd@gmail.com>
* localedata: dz_BT, bo_CN: convert to UTF-8Mike FABIAN2024-01-082-136/+136
|
* localedata: dz_BT, bo_CN: Fix spelling of "phur bu" in both Tibetan and DzongkhaValery Ushakov2024-01-082-4/+4
| | | | Resolves: BZ # 31086
* localedata: bo_CN: Fix spelling errors in Tibetan dataValery Ushakov2024-01-081-20/+23
| | | | Resolves: BZ # 31086
* localedata: bo_CN: Fix incomplete edit in Tibetan yesexprValery Ushakov2024-01-081-1/+1
| | | | Resolves: BZ # 31086
* localedata: dz_BT: Fix spelling errors in Dzongha dataValery Ushakov2024-01-081-23/+23
| | | | Resolves: BZ # 31086
* localedata: unicode-gen: Remove redundant \s* from regexp, fix commentsMike FABIAN2024-01-082-5/+5
|
* localedata: convert the remaining *_RU locales to UTF-8Mike FABIAN2024-01-086-264/+264
|
* localedata: ru_RU, ru_UA: convert to UTF-8Mike FABIAN2024-01-042-75/+75
|
* localedata: es_??: convert to UTF-8Mike FABIAN2024-01-0420-114/+114
|
* localedata: miq_NI: convert to UTF-8Mike FABIAN2024-01-041-7/+7
|
* localedata: fy_DE: make this "Western Frisian" to agree with the language ↵Mike FABIAN2024-01-031-29/+35
| | | | | | code "fy" Resolves: BZ # 14522
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-21 22:33:50 +0000