This Pull Request improves our test results display per edition and cleanups our edition detector logic.
It changes the following:
- Adds a new `edition` flag to limit the maximum edition that will be tested.
- Adds a new `versioned` flag to display all tested editions in a table.
- Adds utility methods to `SpecEdition` to detect the edition of a test and get all the available editions.
- Cleanups logic.
Output with this PR ~(We only collect ES5, ES6 and ES13 stats, so all other editions are a WIP)~:
~Marking as a draft since I need to determine the version of the remaining features, but feel free to review everything else.~ Finished!
Slightly related to #2411 since we need an API to pass module files, but more useful for #1760, #1313 and other error reporting issues.
It changes the following:
- Introduces a new `Source` API to store the path of a provided file or `None` if the source is a plain string.
- Improves the display of `boa_tester` to show the path of the tests being run. This also enables hyperlinks to directly jump to the tested file from the VS terminal.
- Adjusts the repo to this change.
Hopefully, this will improve our error display in the future.
This Pull Request restructures the lint deny/warn/allow lists in almost all crates. `boa_engine` will be done in a follow up PR as the changes there are pretty extensive.
Just some quality changes to improve the maintainability of the tester:
- Replaces `anyhow` with `color_eyre` to have a better output on errors/panics.
- Changes the ignore file to a TOML file and replaces all parsing logic with the `toml` crate.
- Adds a `ignored` field on all `Test`s to simplify run logic.
- Replaces the global `IGNORED` with an `ignored` argument on the CLI.
This is an experiment that tries to migrate the codebase from eager `Error` objects to lazy ones.
In short words, this redefines `JsResult = Result<JsValue, JsError>`, where `JsError` is a brand new type that stores only the essential part of an error type, and only transforms those errors to `JsObject`s on demand (when having to pass them as arguments to functions or store them inside async/generators).
This change is pretty big, because it unblocks a LOT of code from having to take a `&mut Context` on each call. It also paves the road for possibly making `JsError` a proper variant of `JsValue`, which can be a pretty big optimization for try/catch.
A downside of this is that it exposes some brand new error types to our public API. However, we can now implement `Error` on `JsError`, making our `JsResult` type a bit more inline with Rust's best practices.
~Will mark this as draft, since it's missing some documentation and a lot of examples, but~ it's pretty much feature complete. As always, any comments about the design are very much appreciated!
Note: Since there are a lot of changes which are essentially just rewriting `context.throw` to `JsNativeError::%type%`, I'll leave an "index" of the most important changes here:
- [boa_engine/src/error.rs](https://github.com/boa-dev/boa/pull/2283/files#diff-f15f2715655440626eefda5c46193d29856f4949ad37380c129a8debc6b82f26)
- [boa_engine/src/builtins/error/mod.rs](https://github.com/boa-dev/boa/pull/2283/files#diff-3eb1e4b4b5c7210eb98192a5277f5a239148423c6b970c4ae05d1b267f8f1084)
- [boa_tester/src/exec/mod.rs](https://github.com/boa-dev/boa/pull/2283/files#diff-fc3d7ad7b5e64574258c9febbe56171f3309b74e0c8da35238a76002f3ee34d9)
Trying to fix the issue in #1982, I noticed that we didn't have a proper error handling for the boa tester.
This adds the `anyhow` dependency to the tester, which makes it much easier to handle errors and bubble them up with attached context. Thanks to this I was able to easily find out the issue, and I think it could be useful to have it. It gives errors such as this one:
```
Error: could not read the suite test
caused by: error reading sub-suite ./test262/test/built-ins
caused by: error reading sub-suite ./test262/test/built-ins/ShadowRealm
caused by: error reading sub-suite ./test262/test/built-ins/ShadowRealm/WrappedFunction
caused by: error reading test ./test262/test/built-ins/ShadowRealm/WrappedFunction/throws-typeerror-on-revoked-proxy.js
caused by: while scanning a block scalar, found a tab character where an indentation space is expected at line 4 column 3
caused by: while scanning a block scalar, found a tab character where an indentation space is expected at line 4 column 3
```
This PR adds some Clippy lints. Mainly, it adds the list of pedantic lints excluding some lints that were causing too many warnings. I also denied some useful restriction and pedantic lints, to make sure we use `Self` all the possible times (for better maintainability), and that we pass elements by reference where possible, for example, or that the documentation is properly written.
This might even have some small performance gains.
I also added a perfect hash function for the CLI keywords, which should be more efficient than a `HashSet`. This is something we could use elsewhere too.
This Pull Request fixes some warnings and clips errors. It conflicts with the VM/non-VM PR, so should probably go in first, so that this branch gets properly updated and we get the list of real warnings/errors there.
* Don't consider panic fixes as "new failures"
* Improved coding style
* Fixed Test262 tester for the testing update
* Ignore JSON module tests
* Test262 uses _FIXTURE at any point of the file name
* Improved CI workflows
This improves several things in the CI workflows:
- More conformant Test262 result generation
- Benchmarks should now show comments for all users
- Added Test262 result comparison comments to Pull Requests
* Fixed typo
* Checking the comment generation
* Fixing conditions to test comments
* Fix a couple of bugs on the comparator
* Fixed format
* Trying to fix comment updating
* Removing commit autor when searching
* Replace the comment instead of appending
* Switched back to the `pull_request_target` event
José Julián Espina
Kevin
raskad
Iban Eguia
tofpie
George Roman
