This Pull Request restructures the lint deny/warn/allow lists in almost all crates. `boa_engine` will be done in a follow up PR as the changes there are pretty extensive.
Just some quality changes to improve the maintainability of the tester:
- Replaces `anyhow` with `color_eyre` to have a better output on errors/panics.
- Changes the ignore file to a TOML file and replaces all parsing logic with the `toml` crate.
- Adds a `ignored` field on all `Test`s to simplify run logic.
- Replaces the global `IGNORED` with an `ignored` argument on the CLI.
<!---
Thank you for contributing to Boa! Please fill out the template below, and remove or add any
information as you feel necessary.
--->
Not sure if anyone else may be working on something more substantial/in-depth, but I thought I'd post this. 😄
The basic rundown is that this is more of an untested (and in some ways naïve) draft than anything else. It builds rather heavily on `rust-gc`, and tries to keep plenty of the core aspects so as to not break anything too much, and also to minimize overarching changes were it to actually be merged at some point.
This implementation does add ~~a generational divide (although a little unoptimized) to the heap,~~ a GcAlloc/Collector struct with methods, and an ephemeron implementation that allows for the WeakPair and WeakGc pointers.
So, there were some tests that weren't reporting the result of async evaluations correctly. This PR fixes this. It also ignores tests with the `IsHTMLDDA` feature, since we haven't implemented it.
On another note, this also changes the symbols of the test suite to 'F' (failed) and '-' (ignored), which is clearer for colorless terminals.
This Pull Request updates the codebase to the newest version of rustc (1.65.0).
It changes the following:
- Bumps `rust-version` to 1.65.0.
- Rewrites some snippets to use the new let else, ok_or_else and some other utils.
- Removes the `rustdoc::missing_doc_code_examples` allow lint from our codebase. (Context: https://github.com/rust-lang/rust/pull/101732)
This Pull Request replaces `contains`, `contains_arguments`, `has_direct_super` and `function_contains_super` with visitors. (~1000 removed lines!)
Also, the new visitor implementation caught a bug where we weren't setting the home object of async functions, generators and async generators for methods of classes, which caused a stack overflow on `super` calls, and I think that's pretty cool!
Next is `var_declared_names`, `lexically_declared_names` and friends, which will be on another PR.
We were always getting a "VM Implementation" headline in Test262 results for PRs. This was a leftover from the time when we had 2 distinct implementations for the execution. This is no longer the case, so we can remove it to reduce clutter.
This is an experiment that tries to migrate the codebase from eager `Error` objects to lazy ones.
In short words, this redefines `JsResult = Result<JsValue, JsError>`, where `JsError` is a brand new type that stores only the essential part of an error type, and only transforms those errors to `JsObject`s on demand (when having to pass them as arguments to functions or store them inside async/generators).
This change is pretty big, because it unblocks a LOT of code from having to take a `&mut Context` on each call. It also paves the road for possibly making `JsError` a proper variant of `JsValue`, which can be a pretty big optimization for try/catch.
A downside of this is that it exposes some brand new error types to our public API. However, we can now implement `Error` on `JsError`, making our `JsResult` type a bit more inline with Rust's best practices.
~Will mark this as draft, since it's missing some documentation and a lot of examples, but~ it's pretty much feature complete. As always, any comments about the design are very much appreciated!
Note: Since there are a lot of changes which are essentially just rewriting `context.throw` to `JsNativeError::%type%`, I'll leave an "index" of the most important changes here:
- [boa_engine/src/error.rs](https://github.com/boa-dev/boa/pull/2283/files#diff-f15f2715655440626eefda5c46193d29856f4949ad37380c129a8debc6b82f26)
- [boa_engine/src/builtins/error/mod.rs](https://github.com/boa-dev/boa/pull/2283/files#diff-3eb1e4b4b5c7210eb98192a5277f5a239148423c6b970c4ae05d1b267f8f1084)
- [boa_tester/src/exec/mod.rs](https://github.com/boa-dev/boa/pull/2283/files#diff-fc3d7ad7b5e64574258c9febbe56171f3309b74e0c8da35238a76002f3ee34d9)
This Pull Request fixes/closes #2330.
It changes the following:
- Upgrades from clap 3 to clap 4 (note that criterion still uses clap 3, tracked here: https://github.com/bheisler/criterion.rs/issues/596)
- Updates the derive syntax with the new 4.0 syntax
- Adds hints for fish & zsh
Co-authored-by: José Julián Espina <jedel0124@gmail.com>
I think it's time to address the elephant in the room.
This Pull Request will (hopefully!) solve part of #736.
This is a complete rewrite of `JsString`, but instead of storing `u8` bytes it stores `u16` words. The `encode!` macro (renamed to `utf16!` for simplicity) from the `const-utf16` crate allows us to create UTF-16 encoded arrays at compilation time. `JsString` implements `Deref<Target=[u16]>` to unlock the slice methods and possibly make some manipulations easier. However, we would need to create our own library of utilities for `JsString`.
<!---
Thank you for contributing to Boa! Please fill out the template below, and remove or add any
information as you feel neccesary.
--->
This Pull Request removes two dependencies that were not really needed, and fixes#2244 by no longer having the package in the dependency tree.
It changes the following:
- The `structopt` dependency in `boa_tester` has been replaced by `clap` v3, the same way as we did in `boa_cli`. This means that we have one less dependency (at least), and that `clap` v2 is only used as a dev-dependency by `criterion` (which will probably be removed in 0.4, as per https://github.com/bheisler/criterion.rs/issues/596).
- The no-longer-updated `num-format` dependency has been removed from `boa_tester`. We were only using it to add comma thousands separator on results, so I added a simple function to do it (not very performant, but it will only be used a few times when showing results).
Looking at this, I noticed a couple of things:
- The `csv` dependency, used by `criterion` has not been updated in more than a year, and it's using a very old `itoa` dependency. They updated the dependency in the repository in March, but unfortunately, the release is taking some more time than expected, and a tracking issue can be found here: https://github.com/BurntSushi/rust-csv/issues/271
- `cargo update` fails, because the latest update to `tinystr` in the ICU4x breaks ICU4x 0.6. I have reported this here: https://github.com/unicode-org/icu4x/issues/2428 and their recommendation is for us to use a beta version of the library, but I don't think we should go for that, since this is a semver breakage.
The `Context` currently contains a `strict` flag that indicates is global strict mode is active. This is redundant to the strict flag that is set on every function and causes some non spec compliant situations. This pull request removes the strict flag from `Context` and fixes some resulting errors.
Detailed changes:
- Remove strict flag from `Context`
- Make 262 tester compliant with the strict section in [test262/INTERPRETING.md](2e7cdfbe18/INTERPRETING.md (strict-mode))
- Make 262 tester compliant with the `raw` flag in [test262/INTERPRETING.md](2e7cdfbe18/INTERPRETING.md (flags))
- Allow function declarations in strict mode
- Fix parser flag propagation for classes
- Move some early errors from the lexer to the parser
- Add / fix some early errors for 'arguments' and 'eval' identifier usage in strict mode
- Refactor `ArrayLiteral` parser for readability and correct early errors
This Pull Request changes the following:
- Implement redeclaration errors in the parser
- Remove redeclaration errors from the compiler (this is a big step towards #1907)
- Fix some failing tests on the way
This requires a slight change in our public api. The Parser new requires a full `Context` instead of just the `Interner` for parsing new code. This is required, because if multiple scripts are parsed (e.g. every input in the REPL) global variables must be checked for redeclarations.
This removes the only use of the `git2` and `hex` dependencies by reading the test262 submodule commit id directly from the `.git` directory.
Because `git2` depends on a lot of other crates, this removes a bunch of dependencies.
Trying to fix the issue in #1982, I noticed that we didn't have a proper error handling for the boa tester.
This adds the `anyhow` dependency to the tester, which makes it much easier to handle errors and bubble them up with attached context. Thanks to this I was able to easily find out the issue, and I think it could be useful to have it. It gives errors such as this one:
```
Error: could not read the suite test
caused by: error reading sub-suite ./test262/test/built-ins
caused by: error reading sub-suite ./test262/test/built-ins/ShadowRealm
caused by: error reading sub-suite ./test262/test/built-ins/ShadowRealm/WrappedFunction
caused by: error reading test ./test262/test/built-ins/ShadowRealm/WrappedFunction/throws-typeerror-on-revoked-proxy.js
caused by: while scanning a block scalar, found a tab character where an indentation space is expected at line 4 column 3
caused by: while scanning a block scalar, found a tab character where an indentation space is expected at line 4 column 3
```
This Pull Request fixes/closes #1645.
It changes the following:
- Add `features` field to `SuiteResult` structure
- Fetch features from `TestSuite` and propagate them via `SuiteResult`
- Add `FeaturesInfo` structure and serialize it to `features.json`
This fixes an issue with 262 negative tests, that should produce a syntax errors. Currently we only parse the test code is such cases. If the parsing does not return an error, we do not compile the code further. This caused some panics. Most of them are fixed by now, the last ones will be fixed with #1860.
This is an attempt to refactor the environments to be more performant at runtime. The idea is, to shift the dynamic hashmap environment lookups from runtime to compile time.
Currently the environments hold hashmaps that contain binding identifiers, values and additional information that is needed to identify some errors. Because bindings in outer environments are accessible from inner environments, this can lead to a traversal through all environments (in the worst case to the global environment).
This change to the environment structure pushes most of the work that is needed to access bindings to the compile time. At compile time, environments and bindings in the environments are being assigned indices. These indices are then stored instead of the `Sym` that is currently used to access bindings. At runtime, the indices are used to access bindings in a fixed size `Vec` per environment. This brings multiple benefits:
- No hashmap access needed at runtime
- The number of bindings per environment is known at compile time. Environments only need a single allocation, as their size is constant.
- Potential for optimizations with `unsafe` https://doc.rust-lang.org/std/vec/struct.Vec.html#method.get_unchecked
Additionally, this changes the global object to have it's bindings directly stored on the `Realm`. This should reduce some overhead from access trough gc objects and makes some optimizations for the global object possible.
The benchmarks look not that great on the first sight. But if you look closer, I think it is apparent, that this is a positive change. The difference is most apparent on Mini and Clean as they are longer (still not near any real life js but less specific that most other benchmarks):
| Test | Base | PR | % |
|------|--------------|------------------|---|
| Clean js (Compiler) | **1929.1±5.37ns** | 4.1±0.02µs | **+112.53%** |
| Clean js (Execution) | 1487.4±7.50µs | **987.3±3.78µs** | **-33.62%** |
The compile time is up in all benchmarks, as expected. The percentage is huge, but if we look at the real numbers, we can see that this is an issue of orders of magnitude. While compile is up `112.53%`, the real change is `~+2µs`. Execution is only down `33.62%`, but the real time changed by `~-500µs`.
Co-authored-by: Iban Eguia <razican@protonmail.ch>
This removes all the calls to `unwrap()` in the codebase, which made me found a couple of places where it wasn't needed, and could be improved. I also noticed we don't have dependabot updates for the test262 submodule and the interner dependencies, so I added those.
I added lints so that no new unwraps are added.
This PR adds some Clippy lints. Mainly, it adds the list of pedantic lints excluding some lints that were causing too many warnings. I also denied some useful restriction and pedantic lints, to make sure we use `Self` all the possible times (for better maintainability), and that we pass elements by reference where possible, for example, or that the documentation is properly written.
This might even have some small performance gains.
I also added a perfect hash function for the CLI keywords, which should be more efficient than a `HashSet`. This is something we could use elsewhere too.
This builds on top of #1758 to try to bring #1763 to life.
Something that should probably be done here would be to convert `JsString` to a `Sym` internally. Then, further optimizations could be done adding common strings to a custom interner type (those that we know statically).
This is definitely work in progress, but I would like to have feedback on the API, and feel free to contribute.
Co-authored-by: raskad <32105367+raskad@users.noreply.github.com>
This Pull Request is part of #279.
It adds a string interner to Boa, which allows many types to not contain heap-allocated strings, and just contain a `NonZeroUsize` instead. This can move types to the stack (hopefully I'll be able to move `Token`, for example, maybe some `Node` types too.
Note that the internet is for now only available in the lexer. Next steps (in this PR or future ones) would include also using interning in the parser, and finally in execution. The idea is that strings should be represented with a `Sym` until they are displayed.
Talking about display. I have changed the `ParseError` type in order to not contain anything that could contain a `Sym` (basically tokens), which might be a bit faster, but what is important is that we don't depend on the interner when displaying errors.
The issue I have now is in order to display tokens. This requires the interner if we want to know identifiers, for example. The issue here is that Rust doesn't allow using a `fmt::Formatter` (only in nightly), which is making my head hurt. Maybe someone of you can find a better way of doing this.
Then, about `cursor.expect()`, this is the only place where we don't have the expected token type as a static string, so it's failing to compile. We have the option of changing the type definition of `ParseError` to contain an owned string, but maybe we can avoid this by having a `&'static str` come from a `TokenKind` with the default values, such as "identifier" for an identifier. I wanted for you to think about it and maybe we can just add that and avoid allocations there.
Oh, and this depends on the VM-only branch, so that has to be merged before :)
Another thing to check: should the interner be in its own module?
This Pull Request fixes some warnings and clips errors. It conflicts with the VM/non-VM PR, so should probably go in first, so that this branch gets properly updated and we get the list of real warnings/errors there.
This PR fixes some vm implementation code. All our internal tests should now pass with the vm enabled.
There are only a few (~100) 262 tests left that currently break with the vm, that previously worked.
Previously when we had the `context.throw_` methods (like `context.thtrow_type_error()`) they were limited as to where we could call them, e.i. a function that returned `JsResult<JsValue>`. So we had to call the `context.construct_` methods with an explicit `Err()` enum wrap to throw in functions that returned non-jsvalues (which happens a lot).
Now, with this PR the throw methods have a generic `JsResult<R>` return that can return in any `JsResult<T>` returning function. Which cleans the API and makes the user experience a bit better.
```rust
return Err(context.construct_type_error("..."));
// to
return context.throw_type_error("...");
```
<!---
Thank you for contributing to Boa! Please fill out the template below, and remove or add any
information as you feel neccesary.
--->
Hi, first time contributor :)
I was playing with getting the test262 test runner working and found a couple small things that I thought would've made my life a bit easier. I've split them into separate commits. Just let me know if you disagree with any and we can drop that commit. And of course, let me know of any changes you'd like to see :)
The changes are:
- Adds details to the documentation on the `--suite` command to `boa_tester`. I was trying to pass a path starting with `./test262/` for a bit before I looked at the code and saw what it was doing.
- Changes the individual test output when verbosity is > 1. Because the tests are run in parallel, the "Result" line for a given test was frequently not immediately after the "Started" line. This made it hard to determine which test had failed. The new output includes the test name in the result line, and also changes the format of all the individual-test-output lines to begin with the test name.
- Adds a `--disable-parallelism` flag. Even with the adjustments to the test output, it was still a bit hard to follow. Running serially for small sub-suites produces output that can be more easily scanned IMO. I added this as a "disable" flag (as opposed to "enable parallelism") in order to maintain the current default of running in parallel.
Co-authored-by: Grant Orndorff <grant@orndorff.me>
This PR adds conformance results for the VM branch both for PRs and for the conformance results in GitHub pages (even if these are not currently being shown). I'm not 100% sure how this will really work, as I'm not very used to the syntax to concatenate strings.
Co-authored-by: João Borges <rageknify@gmail.com>
Co-authored-by: RageKnify <RageKnify@gmail.com>
* Add strict mode flag to `Context`
* Add strict mode handling of `delete`
* Add strict mode test
* Handle non-strict functions in strict functions
* Enable strict mode for functions defined in a strict context
Iban Eguia Moraza
raskad
José Julián Espina
Kevin
NorbertGarfield
Halid Odat
RageKnify
Jason Williams
Grant Orndorff
praveenbakkal
Bartek Iwańczuk
neeldug