about summary refs log tree commit diff
path: root/src/multibyte/mbsrtowcs.c
Commit message (Collapse)AuthorAgeFilesLines
* byte-based C locale, phase 1: multibyte character handling functionsRich Felker2015-06-161-0/+19
| | | | | | | | | | | | | | | | | | this patch makes the functions which work directly on multibyte characters treat the high bytes as individual abstract code units rather than as multibyte sequences when MB_CUR_MAX is 1. since MB_CUR_MAX is presently defined as a constant 4, all of the new code added is dead code, and optimizing compilers' code generation should not be affected at all. a future commit will activate the new code. as abstract code units, bytes 0x80 to 0xff are represented by wchar_t values 0xdf80 to 0xdfff, at the end of the surrogates range. this ensures that they will never be misinterpreted as Unicode characters, and that all wctype functions return false for these "characters" without needing locale-specific logic. a high range outside of Unicode such as 0x7fffff80 to 0x7fffffff was also considered, but since C11's char16_t also needs to be able to represent conversions of these bytes, the surrogate range was the natural choice.
* include cleanups: remove unused headers and add feature test macrosSzabolcs Nagy2013-12-121-3/+1
|
* fix buffer overflow in mbsrtowcsRich Felker2013-09-271-1/+1
| | | | | | | | | | | | | | | | | | | issue reported by Michael Forney: "If wn becomes 0 after processing a chunk of 4, mbsrtowcs currently continues on, wrapping wn around to -1, causing the rest of the string to be processed. This resulted in buffer overruns if there was only space in ws for wn wide characters." the original patch submitted added an additional check for !wn after the loop; to avoid extra branching, I instead just changed the wn>=4 check to wn>=5 to ensure that at least one slot remains after the word-at-a-time loop runs. this should not slow down the tail processing on real-world usage, since an extra slot that can't be processed in the word-at-a-time loop is needed for the null termination anyway.
* fix failure of mbsrtowcs to record stop position when dest is fullRich Felker2013-06-291-1/+4
|
* overhaul mbsrtowcsRich Felker2013-04-041-69/+64
| | | | | | | | | | | | | | | | | these changes fix at least two bugs: - misaligned access to the input as uint32_t for vectorized ASCII test - incorrect src pointer after stopping on EILSEQ in addition, the text of the standard makes it unclear whether the mbstate_t object is to be modified when the destination pointer is null; previously it was cleared either way; now, it's only cleared when the destination is non-null. this change may need revisiting, but it should not affect most applications, since calling mbsrtowcs with non-zero state can only happen when the head of the string was already processed with mbrtowc. finally, these changes shave about 20% size off the function and seem to improve performance by 1-5%.
* use restrict everywhere it's required by c99 and/or posix 2008Rich Felker2012-09-061-1/+1
| | | | | | | | to deal with the fact that the public headers may be used with pre-c99 compilers, __restrict is used in place of restrict, and defined appropriately for any supported compiler. we also avoid the form [restrict] since older versions of gcc rejected it due to a bug in the original c99 standard, and instead use the form *restrict.
* cleanup utf-8 multibyte code, use visibility if possibleRich Felker2011-02-271-16/+0
| | | | | | | | | | | | | this code was written independently of musl, with support for a the backwards, nonstandard "31-bit unicode" some libraries/apps might want. unfortunately the extra code (inside #ifdef) makes the source harder to read and makes code that should be simple look complex, so i'm removing it. anyone who wants to use the old code can find it in the history or from elsewhere. also, change the visibility of the __fsmu8 state machine table to hidden, if supported. this should improve performance slightly in shared-library builds.
* cleanup multibyte stuff to remove ugly casts, sanitize the ptr align castsRich Felker2011-02-131-19/+19
|
* initial check-in, version 0.5.0 v0.5.0Rich Felker2011-02-121-0/+121