* Close active iterators when returning from async generator
* Fix remaining async generator tests
* Fix remaining async generator tests
* Replace some reserved opcodes
* Fix for await of loop flag
* Add tests for fix
* Remove whitespace
* Fix test display and improve test
* Change `usize` to `u32`
* Fix typo
* Refactor prompt highlighting into RLHelper
By moving the prompt coloring to be done in
Highlighter::highlight_prompt, we sidestep the issue on Windows where
the prompt width is calculated post-coloring AND without ignoring escape
codes.
By including it in the implementation of Highlighter, Editor::readline
now operates on a plain-text prompt, so width calculation is correct.
This commit also re-arranges the trait impl order to match the
definition.
* Remove redundant `num_bindings` field from `CodeBlock`
* Remove redundant num_bindings parameter from push_function_inherits
* Remove redundant num_bindings operand from environment opcodes
* Make pop_compile_environment() return an index
* Move boolean `CodeBlock` flags to bitflags
* Fix ci
* Add doc
This PR implements `Hidden Classes`, I named them as `Shapes` (like Spidermonkey does), calling them maps like v8 seems confusing because there already is a JS builtin, likewise with `Hidden classes` since there are already classes in JS.
There are two types of shapes: `shared` shapes that create the transition tree, and are shared between objects, this is mainly intended for user defined objects this makes more sense because shapes can create transitions trees, doing that for the builtins seems wasteful (unless users wanted to creating an object with the same property names and the same property attributes in the same order... which seems unlikely). That's why I added `unique` shapes, only one object has it. This is similar to previous solution, but this architecture enables us to use inline caching.
There will probably be a performance hit until we implement inline caching.
There still a lot of work that needs to be done, on this:
- [x] Move Property Attributes to shape
- [x] Move Prototype to shape
- [x] ~~Move extensible flag to shape~~, On further evaluation this doesn't give any benefit (at least right now), since it isn't used by inline caching also adding one more transition.
- [x] Implement delete for unique shapes.
- [x] If the chain is too long we should probably convert it into a `unique` shape
- [x] Figure out threshold ~~(maybe more that 256 properties ?)~~ curently set to an arbitrary number (`1024`)
- [x] Implement shared property table between shared shapes
- [x] Add code Document
- [x] Varying size storage for properties (get+set = 2, data = 1)
- [x] Add shapes to more object:
- [x] ordinary object
- [x] Arrays
- [x] Functions
- [x] Other builtins
- [x] Add `shapes.md` doc explaining shapes in depth with mermaid diagrams :)
- [x] Add `$boa.shape` module
- [x] `$boa.shape.id(o)`
- [x] `$boa.shape.type(o)`
- [x] `$boa.shape.same(o1, o2)`
- [x] add doc to `boa_object.md`
This Pull Request fixes/closes #718.
It changes the following:
- Adds a new `boa_runtime` crate, that will only include `console` for now
- Changes the `boa_cli` crate to use the new `boa_runtime` crate for the console, instead of the `console` feature of `boa_engine`
- Removes the `console` feature in `boa_engine`
- Adds a new `boa_testing` helper crate with some useful functions for testing `boa`. This part duplicates the code from `boa_engine`, but I could not make `boa_engine` work with this crate as a dependency due to circular dependencies. Maybe doing it a bit generic could work, but didn't have enough time to check it.
To be checked: wether the WASM example works as expected with the console.
This should hopefully fix more async/futures issues related to resuming execution in the future, since we can leverage generator logic to handle this for us.
It changes the following:
- Refactors `GeneratorContext` to handle context preparation.
- Reuses the functionality of `GeneratorContext` in `Await`.
- Removes `EarlyReturnType` in favour of a single `r#await` bool flag in `CallFrame`.
This allows `thread_local` contexts to have owned `HostHooks` and `JobQueues`.
It changes the following:
- Creates a new `MaybeShared` struct that can hold either a reference or an `Rc`.
- Changes the `job_queue` and `host_hooks` parameters of `Context` to use `MaybeShared`.
This PR also allows us to make `SimpleJobQueue` the default promise runner, which I think it's pretty cool :)
cc @lastmjs
This Pull Request fixes test [`assert-throws-same-realm.js`](eb44f67274/test/harness/assert-throws-same-realm.js).
It changes the following:
- Handles global variables through the global object, instead of the `context`.
- Adds an `active_function` field to the vm, which is used as the `NewTarget` when certain builtins aren't called with `new`.
- Adds a `realm_intrinsics` field to `Function`.
Currently some debugging stuff in JavaScript land is difficult to impossible, like triggering a GC collect, this is not impossible to do in JavaScript the way I triggered it was by creating a huge amount of object `for (let i = 0; i < 100000; ++i) { ({}) }` but this is cumbersome and not guaranteed to trigger a gc.
This PR implements `--debug-object` flag that injects the `$boa` debug object in the context, the object is separated into modules currently `gc`, `function`, `object`.
We can now do `$boa.gc.collect()`, which force triggers a GC collect.
Or sometimes I wanted a trace (the current solution is great, you can trace stuff like `>>> 1 + 1` but that is also it's limitation), it traces everything, I sometimes have a scenario and just want to trace a single function in that scenario, that's why I added the `$boa.function.trace(func, this, ...args)` It only traces the function.
```js
>> $boa.function.trace((a, b) => a + b, undefined, 1, 2)
-------------------------Compiled Output: ''--------------------------
Location Count Opcode Operands
000000 0000 DefInitArg 0000: 'a'
000005 0001 DefInitArg 0001: 'b'
000010 0002 RestParameterPop
000011 0003 GetName 0000: 'a'
000016 0004 GetName 0001: 'b'
000021 0005 Add
000022 0006 Return
000023 0007 PushUndefined
000024 0008 Return
... (cut for brevity) ...
```
It also implements `$boa.function.flowgraph(func, options)`:
```js
$boa.function.flowgraph(func, 'graphviz')
$boa.function.flowgraph(func, { format: 'mermaid', direction: 'TopBottom' })
```
Printing the object pointer:
```js
$boa.object.id({}) // '0x566464F33'
```
It currently implements some functionality which we can grow it with our debugging needs since we are not restricted by a spec we can add whatever we want :)
I was originally going to implement this in #2723 (but the PR is too big), for shapes having functions like:
```js
$boa.shape.type({}) // Shared shape
$boa.shape.id({}) // 0x8578FG355 (objects, shape pointer)
$boa.shape.flowgraph({}) // printing the shape transition chain, like $boa.function.flowgraph
```
Shapes chains are very hard to debug once they are big... so having this type of debugging capability would make it much easier.
This PR implements an optimizer, It currently implements the [constant folding optimization][cfo]. this optimization is responsible for "folding"/evaluating constant expressions.
For example:
```js
let x = ((1 + 2 + -4) * 8) << 4
```
Generates the following instruction(s) (`cargo run -- -t`):
```
000000 0000 PushOne
000001 0001 PushInt8 2
000003 0002 Add
000004 0003 PushInt8 4
000006 0004 Neg
000007 0005 Add
000008 0006 PushInt8 8
000010 0007 Mul
000011 0008 PushInt8 4
000013 0009 ShiftLeft
000014 0010 DefInitLet 0000: 'x'
```
With constant folding it generates the following instruction(s) (`cargo run -- -t -O`):
```
000000 0000 PushInt8 -128
000002 0001 DefInitLet 0000: 'x'
```
It changes the following:
- Implement ~~WIP~~ constant folding optimization, ~~only works with integers for now~~
- Add `--optimize, -O` flag to boa_cli
- Add `--optimizer-statistics` flag to boa_cli for optimizer statistics
- Add `--optimize, -O` flag to boa_tester
After I finish with this, will try to implement other optimizations :)
[cfo]: https://en.wikipedia.org/wiki/Constant_folding
This PR adds the `--strict` flag to the CLI that enables strict mode on file/interactive mode execution.
It's a bit annoying to have to prefix with `'use strict';` when trying to debug in interactive mode
```js
>>> 'use strict'; .... // :(
```
It changes the following:
- Adds `--strict` flag to CLI
- update `README.md`
It changes the following:
- Remove Just-in-time compilation from description in `Cargo.toml`
- Change `FlowgraphFormat` to be terminal friendly
- Update `README.md` cli options section
~~Builds off of #2529.~~ Merged.
This Pull Request allows passing any function returning `impl Future<Output = JsResult<JsValue>>` to the `NativeFunction` constructor, allowing native concurrency hooks into the engine.
It changes the following:
- Adds a `NativeFunction::from_async_fn` function.
- Adds a new `JobQueue::enqueue_future_job` method.
- Adds an example usage on `boa_examples`.
I'm creating this draft PR, since I wanted to have some early feedback, and because I though I would have time to finish it last week, but I got caught up with other stuff. Feel free to contribute :)
The main thing here is that I have divided `eval()`, `parse()` and similar functions so that they can decide if they are parsing scripts or modules. Let me know your thoughts.
Then, I was checking the import & export parsing, and I noticed we are using `TokenKind::Identifier` for `IdentifierName`, so I changed that name. An `Identifier` is an `IdentifierName` that isn't a `ReservedWord`. This means we should probably also adapt all `IdentifierReference`, `BindingIdentifier` and so on parsing. I already created an `Identifier` parser.
Something interesting there is that `await` is not a valid `Identifier` if the goal symbol is `Module`, as you can see in the [spec](https://tc39.es/ecma262/#prod-LabelIdentifier), but currently we don't have that information in the `InputElement` enumeration, we only have `Div`, `RegExp` and `TemplateTail`. How could we approach this?
Co-authored-by: jedel1043 <jedel0124@gmail.com>
Small (ish?) step towards having proper realm records
This PR changes the following:
- Moves `Intrinsics` to `Realm`.
- Cleans up the initialization logic of our intrinsics to not depend on `Context`, unblocking things like #2314.
- Adds hooks to initialize the global object and the global this per the corresponding [`InitializeHostDefinedRealm ( )`](https://tc39.es/ecma262/#sec-initializehostdefinedrealm) hook. Though, this is currently broken because the vm uses `GlobalPropertyMap` instead of the `JsObject` API to initialize global properties.
Slightly related to #2411 since we need an API to pass module files, but more useful for #1760, #1313 and other error reporting issues.
It changes the following:
- Introduces a new `Source` API to store the path of a provided file or `None` if the source is a plain string.
- Improves the display of `boa_tester` to show the path of the tests being run. This also enables hyperlinks to directly jump to the tested file from the VS terminal.
- Adjusts the repo to this change.
Hopefully, this will improve our error display in the future.
Follows from #2528, and should complement #2411 to implement the module import hooks.
~~Similarly to the Intl/ICU4X PR (#2478), this has a lot of trivial changes caused by the new lifetimes. I thought about passing the queue and the hooks by value, but it was very painful having to wrap everything with `Rc` in order to be accessible by the host.
In contrast, `&dyn` can be easily provided by the host and has the advantage of not requiring additional allocations, with the downside of adding two more lifetimes to our `Context`, but I think it's worth.~~ I was able to unify all lifetimes into the shortest one of the three, making our API just like before!
Changes:
- Added a new `HostHooks` trait and a `&dyn HostHooks` field to `Context`. This allows hosts to implement the trait for their custom type, then pass it to the context.
- Added a new `JobQueue` trait and a `&dyn JobQueue` field to our `Context`, allowing custom event loops and other fun things.
- Added two simple implementations of `JobQueue`: `IdleJobQueue` which does nothing and `SimpleJobQueue` which runs all jobs until all successfully complete or until any of them throws an error.
- Modified `boa_cli` to run all jobs until the queue is empty, even if a job returns `Err`. This also prints all errors to the user.
Per the [Standard Library development guide](https://std-dev-guide.rust-lang.org/code-considerations/performance/inline.html):
> You can add `#[inline]`:
>
> - To public, small, non-generic functions.
>
> You shouldn't need `#[inline]`:
> - On methods that have any generics in scope.
> - On methods on traits that don't have a default implementation.
>
> `#[inline]` can always be introduced later, so if you're in doubt they can just be removed.
This PR follows this guideline to reduce the number of `#[inline]` annotations in our code, removing the annotation in:
- Non-public functions
- Generic functions
- Medium and big functions.
Hopefully this shouldn't impact our perf at all, but let's wait to see the benchmark results.
This PR is a WIP implementation of a vm instruction flowgraph generator
This aims to make the vm easier to debug and understand for both newcomers and experienced devs.
For example if we have the following code:
```js
let i = 0;
while (i < 10) {
if (i == 3) {
break;
}
i++;
}
```
It generates the following instructions (which is hard to read, especially jumps):
<details>
```
----------------------Compiled Output: '<main>'-----------------------
Location Count Opcode Operands
000000 0000 PushZero
000001 0001 DefInitLet 0000: 'i'
000006 0002 LoopStart
000007 0003 LoopContinue
000008 0004 GetName 0000: 'i'
000013 0005 PushInt8 10
000015 0006 LessThan
000016 0007 JumpIfFalse 78
000021 0008 PushDeclarativeEnvironment 0, 1
000030 0009 GetName 0000: 'i'
000035 0010 PushInt8 3
000037 0011 Eq
000038 0012 JumpIfFalse 58
000043 0013 PushDeclarativeEnvironment 0, 0
000052 0014 Jump 78
000057 0015 PopEnvironment
000058 0016 GetName 0000: 'i'
000063 0017 IncPost
000064 0018 RotateRight 2
000066 0019 SetName 0000: 'i'
000071 0020 Pop
000072 0021 PopEnvironment
000073 0022 Jump 7
000078 0023 LoopEnd
Literals:
<empty>
Bindings:
0000: i
Functions:
<empty>
```
</details>
And the flow graph is generated:
The beginning of the function is marked by the `start` node (in green) and end (in red). In branching the "yes" branch is marked in green and "no" in red.
~~This only generates in [graphviz format](https://en.wikipedia.org/wiki/DOT_(graph_description_language)) (a widely used format) but it would be nice to also generate to a format that `mermaid.js` can understand and that could be put in articles https://github.com/boa-dev/boa-dev.github.io/issues/26~~
TODO:
- [x] Generate graphviz format
- [x] Generate mermaid format
- [x] Programmatically generate colors push and pop env instructions
- [x] Display nested functions in sub-sub-graphs.
- [x] Put under a feature (`"flowgraph"`)
- [x] Handle try/catch, switch instructions
- [x] CLI option for configuring direction of flow (by default it is top down)
- [x] Handle `Throw` instruction (requires keeping track of try blocks)
- [x] Documentation
- [x] Prevent node name collisions (functions with the same name)
This Pull Request restructures the lint deny/warn/allow lists in almost all crates. `boa_engine` will be done in a follow up PR as the changes there are pretty extensive.
This should hopefully improve our compilation times, both from a clean build and from an incremental compilation snapshot.
Next would be the parser, but it imports `Context`, so it'll require a bit more work.
The number of file changes is obviously big, but almost nothing was changed, I just moved everything to another crate and readjusted the imports of the `parser` module. (Though, I did have to change some details, because there were some functions on the ast that returned `ParseError`s, and the tests had to be moved to the parser)
This is an experiment that tries to migrate the codebase from eager `Error` objects to lazy ones.
In short words, this redefines `JsResult = Result<JsValue, JsError>`, where `JsError` is a brand new type that stores only the essential part of an error type, and only transforms those errors to `JsObject`s on demand (when having to pass them as arguments to functions or store them inside async/generators).
This change is pretty big, because it unblocks a LOT of code from having to take a `&mut Context` on each call. It also paves the road for possibly making `JsError` a proper variant of `JsValue`, which can be a pretty big optimization for try/catch.
A downside of this is that it exposes some brand new error types to our public API. However, we can now implement `Error` on `JsError`, making our `JsResult` type a bit more inline with Rust's best practices.
~Will mark this as draft, since it's missing some documentation and a lot of examples, but~ it's pretty much feature complete. As always, any comments about the design are very much appreciated!
Note: Since there are a lot of changes which are essentially just rewriting `context.throw` to `JsNativeError::%type%`, I'll leave an "index" of the most important changes here:
- [boa_engine/src/error.rs](https://github.com/boa-dev/boa/pull/2283/files#diff-f15f2715655440626eefda5c46193d29856f4949ad37380c129a8debc6b82f26)
- [boa_engine/src/builtins/error/mod.rs](https://github.com/boa-dev/boa/pull/2283/files#diff-3eb1e4b4b5c7210eb98192a5277f5a239148423c6b970c4ae05d1b267f8f1084)
- [boa_tester/src/exec/mod.rs](https://github.com/boa-dev/boa/pull/2283/files#diff-fc3d7ad7b5e64574258c9febbe56171f3309b74e0c8da35238a76002f3ee34d9)
This Pull Request fixes#1371. And yeah, the number of file changes is real...
It changes the following:
- Split the ast `Node` into `Statement` and `Expression`.
- Rewrite the parser and bytecompiler to conform to this change.
- Refactor some ast nodes into reusable structures.
- Rewrite `contains_arguments` and `contains` to ease the transition into a future ast visitor.
List of things that were apparently fixed by this refactor?:
- Implement read-assign operation for private accessors (e.g. `this.#field ||= 5`).
- `var await` declaration now allowed outside `async` functions and inside functions nested in async functions.
- Reject redeclarations of variables declared in the init list of a for loop.
Still missing some documentation adjustments, will try to do it ASAP.
This Pull Request fixes/closes #2330.
It changes the following:
- Upgrades from clap 3 to clap 4 (note that criterion still uses clap 3, tracked here: https://github.com/bheisler/criterion.rs/issues/596)
- Updates the derive syntax with the new 4.0 syntax
- Adds hints for fish & zsh
Co-authored-by: José Julián Espina <jedel0124@gmail.com>
<!---
Thank you for contributing to Boa! Please fill out the template below, and remove or add any
information as you feel neccesary.
--->
This Pull Request removes two dependencies that were not really needed, and fixes#2244 by no longer having the package in the dependency tree.
It changes the following:
- The `structopt` dependency in `boa_tester` has been replaced by `clap` v3, the same way as we did in `boa_cli`. This means that we have one less dependency (at least), and that `clap` v2 is only used as a dev-dependency by `criterion` (which will probably be removed in 0.4, as per https://github.com/bheisler/criterion.rs/issues/596).
- The no-longer-updated `num-format` dependency has been removed from `boa_tester`. We were only using it to add comma thousands separator on results, so I added a simple function to do it (not very performant, but it will only be used a few times when showing results).
Looking at this, I noticed a couple of things:
- The `csv` dependency, used by `criterion` has not been updated in more than a year, and it's using a very old `itoa` dependency. They updated the dependency in the repository in March, but unfortunately, the release is taking some more time than expected, and a tracking issue can be found here: https://github.com/BurntSushi/rust-csv/issues/271
- `cargo update` fails, because the latest update to `tinystr` in the ICU4x breaks ICU4x 0.6. I have reported this here: https://github.com/unicode-org/icu4x/issues/2428 and their recommendation is for us to use a beta version of the library, but I don't think we should go for that, since this is a semver breakage.
This Pull Request fixes/closes #2223.
It changes the following:
- Check the .boa_history file, if it is not exist then create the file before load history function to prevent the crash
This Pull Request overrides #2183#2187, #2189 and #2190.
It changes the following:
- Updates rustyline to 10.0.0 (this uses the new phf 0.11)
- Updates phf to 0.11 to avoid different dependency versions
- Fixes the `Editor` creation, which now returns a `Result`.
The `Context` currently contains a `strict` flag that indicates is global strict mode is active. This is redundant to the strict flag that is set on every function and causes some non spec compliant situations. This pull request removes the strict flag from `Context` and fixes some resulting errors.
Detailed changes:
- Remove strict flag from `Context`
- Make 262 tester compliant with the strict section in [test262/INTERPRETING.md](2e7cdfbe18/INTERPRETING.md (strict-mode))
- Make 262 tester compliant with the `raw` flag in [test262/INTERPRETING.md](2e7cdfbe18/INTERPRETING.md (flags))
- Allow function declarations in strict mode
- Fix parser flag propagation for classes
- Move some early errors from the lexer to the parser
- Add / fix some early errors for 'arguments' and 'eval' identifier usage in strict mode
- Refactor `ArrayLiteral` parser for readability and correct early errors
This Pull Request changes the following:
- Implement redeclaration errors in the parser
- Remove redeclaration errors from the compiler (this is a big step towards #1907)
- Fix some failing tests on the way
This requires a slight change in our public api. The Parser new requires a full `Context` instead of just the `Interner` for parsing new code. This is required, because if multiple scripts are parsed (e.g. every input in the REPL) global variables must be checked for redeclarations.
This removes all the calls to `unwrap()` in the codebase, which made me found a couple of places where it wasn't needed, and could be improved. I also noticed we don't have dependabot updates for the test262 submodule and the interner dependencies, so I added those.
I added lints so that no new unwraps are added.
This PR adds some Clippy lints. Mainly, it adds the list of pedantic lints excluding some lints that were causing too many warnings. I also denied some useful restriction and pedantic lints, to make sure we use `Self` all the possible times (for better maintainability), and that we pass elements by reference where possible, for example, or that the documentation is properly written.
This might even have some small performance gains.
I also added a perfect hash function for the CLI keywords, which should be more efficient than a `HashSet`. This is something we could use elsewhere too.
This builds on top of #1758 to try to bring #1763 to life.
Something that should probably be done here would be to convert `JsString` to a `Sym` internally. Then, further optimizations could be done adding common strings to a custom interner type (those that we know statically).
This is definitely work in progress, but I would like to have feedback on the API, and feel free to contribute.
Co-authored-by: raskad <32105367+raskad@users.noreply.github.com>
This Pull Request is part of #279.
It adds a string interner to Boa, which allows many types to not contain heap-allocated strings, and just contain a `NonZeroUsize` instead. This can move types to the stack (hopefully I'll be able to move `Token`, for example, maybe some `Node` types too.
Note that the internet is for now only available in the lexer. Next steps (in this PR or future ones) would include also using interning in the parser, and finally in execution. The idea is that strings should be represented with a `Sym` until they are displayed.
Talking about display. I have changed the `ParseError` type in order to not contain anything that could contain a `Sym` (basically tokens), which might be a bit faster, but what is important is that we don't depend on the interner when displaying errors.
The issue I have now is in order to display tokens. This requires the interner if we want to know identifiers, for example. The issue here is that Rust doesn't allow using a `fmt::Formatter` (only in nightly), which is making my head hurt. Maybe someone of you can find a better way of doing this.
Then, about `cursor.expect()`, this is the only place where we don't have the expected token type as a static string, so it's failing to compile. We have the option of changing the type definition of `ParseError` to contain an owned string, but maybe we can avoid this by having a `&'static str` come from a `TokenKind` with the default values, such as "identifier" for an identifier. I wanted for you to think about it and maybe we can just add that and avoid allocations there.
Oh, and this depends on the VM-only branch, so that has to be merged before :)
Another thing to check: should the interner be in its own module?
José Julián Espina
Shane Murphy
Haled Odat
Haled Odat
Iban Eguia Moraza
dependabot[bot]
raskad
udhaykumarbala
Magic Panda
Jason Williams
