<!---
Thank you for contributing to Boa! Please fill out the template below, and remove or add any
information as you feel neccesary.
--->
Hi, first time contributor :)
I was playing with getting the test262 test runner working and found a couple small things that I thought would've made my life a bit easier. I've split them into separate commits. Just let me know if you disagree with any and we can drop that commit. And of course, let me know of any changes you'd like to see :)
The changes are:
- Adds details to the documentation on the `--suite` command to `boa_tester`. I was trying to pass a path starting with `./test262/` for a bit before I looked at the code and saw what it was doing.
- Changes the individual test output when verbosity is > 1. Because the tests are run in parallel, the "Result" line for a given test was frequently not immediately after the "Started" line. This made it hard to determine which test had failed. The new output includes the test name in the result line, and also changes the format of all the individual-test-output lines to begin with the test name.
- Adds a `--disable-parallelism` flag. Even with the adjustments to the test output, it was still a bit hard to follow. Running serially for small sub-suites produces output that can be more easily scanned IMO. I added this as a "disable" flag (as opposed to "enable parallelism") in order to maintain the current default of running in parallel.
Co-authored-by: Grant Orndorff <grant@orndorff.me>
This PR adds conformance results for the VM branch both for PRs and for the conformance results in GitHub pages (even if these are not currently being shown). I'm not 100% sure how this will really work, as I'm not very used to the syntax to concatenate strings.
Co-authored-by: João Borges <rageknify@gmail.com>
Co-authored-by: RageKnify <RageKnify@gmail.com>
* Add strict mode flag to `Context`
* Add strict mode handling of `delete`
* Add strict mode test
* Handle non-strict functions in strict functions
* Enable strict mode for functions defined in a strict context
* Don't consider panic fixes as "new failures"
* Improved coding style
* Fixed Test262 tester for the testing update
* Ignore JSON module tests
* Test262 uses _FIXTURE at any point of the file name
* Added some result changes for Test262 result comparisons
* Added a bug and a panic to test the changes in the comparator
* Remove explicit todo!()
* Fixing output for test links
* Force comment
* Reducing failures so that commenting works
* New test
* Fixed a bit of formatting
* Reverted purposedly introduced bugs
* Added a "details" section for the test262 test diff
* Fix bold
When the test262 conformance comments are added to a PR, it will now
show the percentage symbol for the conformance, to show up it's
actually a conformance percentage.
* Improved CI workflows
This improves several things in the CI workflows:
- More conformant Test262 result generation
- Benchmarks should now show comments for all users
- Added Test262 result comparison comments to Pull Requests
* Fixed typo
* Checking the comment generation
* Fixing conditions to test comments
* Fix a couple of bugs on the comparator
* Fixed format
* Trying to fix comment updating
* Removing commit autor when searching
* Replace the comment instead of appending
* Switched back to the `pull_request_target` event
Grant Orndorff
Iban Eguia
praveenbakkal
Jason Williams
raskad
Halid Odat
Bartek Iwańczuk
jedel1043
neeldug
0x7D2B
João Borges
tofpie
Jevan Chan
George Roman