As mentioned in https://github.com/boa-dev/boa/pull/2848#issuecomment-1518909512, this uses our new default ICU4X data to replace `char::is_start` and `char::is_continue` from the `boa_unicode` crate with the [`icu_properties`](https://crates.io/crates/icu_properties) crate.
Note that this doesn't deprecate `boa_unicode` yet, since that'll require some discussion about how to proceed with a now unused sub-crate.
We currently use `unicode_normalization` to handle the `String.prototype.normalize` method. However, the crate doesn't support UTF-16 as a first class string, so we had to do some hacks by converting the valid parts of a string to UTF-8, normalizing each one, encoding back to UTF-16 and concatenating everything with the unpaired surrogates within. All of this is obviously suboptimal for performance, which is why I leveraged the `icu_normalizer`, which does support UTF-16 input, to replace our current implementation.
Additionally, this allows users to override the default normalization data if the `intl` feature is enabled by providing the required data in the `BoaProvider` data provider.
This PR upgrades ICU to 1.2.
Unfortunately we still have some breaking changes, so this is being handled in https://github.com/unicode-org/icu4x/issues/3332
Co-authored-by: jedel1043 <jedel0124@gmail.com>
This fixes some more ES5 tests that were failing because the functions haven't been implemented.
It changes the following:
- Adds `String::to_locale_case`, which uses ICU4X to convert strings to uppercase or lowercase.
- Refactors `String::to_uppercase` and `String::to_lowercase` into a single `String::to_case` which uses a const generic to distinguish each case.
- Adds utility functions on `JsString` to avoid code repetition.
This Pull Request fixes/closes #1180. (I'll open a tracking issue for the progress)
It changes the following:
- Redesigns the internal API of Intl to (hopefully!) make it easier to implement a service.
- Implements the `Intl.Locale` service.
- Implements the `Intl.Collator` service.
- Implements the `Intl.ListFormat` service.
On the subject of the failing tests. Some of them are caused by missing locale data in the `icu_testdata` crate; we would need to regenerate that with the missing locales, or vendor a custom default data.
On the other hand, there are some tests that are bugs from the ICU4X crate. The repo https://github.com/jedel1043/icu4x-test262 currently tracks the found bugs when running test262. I'll sync with the ICU4X team to try to fix those.
cc @sffc
jedel1043
Iban Eguia Moraza
raskad