Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add partial syntax, source maps, removed Keyword from AST and other CLI fixes #107

Merged
merged 25 commits into from
Jan 29, 2024

Conversation

kaleidawave
Copy link
Owner

@kaleidawave kaleidawave commented Jan 15, 2024

  • Removed Keyword from AST
    • Added a map store to parsing state which can store positions for keywords
      • This is used in the LSP which gives documentation about language features (what does a keyword do)
      • Uses binary search to find keywords
      • Only under an option
    • Updated synthesis for changes in structure
  • Fixes to context state across REPL entries
  • Simplified the generator macro interpolation mode

Added partial syntax

Added the ability for type annotations and expression to be missing in the source. This allows the following to be valid AST (under a parse option)

const x = ;
const y = 
const z = 2

Which should allow type checking with invalid source which is vital to the LSP

Source map bindings

This adds source map generation using the positions in the output and source

image

While possible a while ago. It needed this fix kaleidawave/source-map#1

Added printing arguments across new lines for function calling

image

WIP other structures could benefit from this. Implementation feels like it has a bit of overhead. Indent could be better handled

Still to do

  • experiment more with cursors (now markers and partial syntax)
  • Finish the change in the generator macro
  • add source map bindings in
  • (maybe) add prettier printing for function calls etc?
  • try crush some fuzzing issues

@kaleidawave kaleidawave added parser Related to Ezno's syntax parser, AST definitions and output internal-improvements Related to internal publishing, building, workflows, formatting & linting lsp Language service protocol and editor integration compiler-performance Making parsing / type checking / transformations faster labels Jan 15, 2024
@kaleidawave kaleidawave changed the title Removed Keyword from AST and other CLI fixes Add partial syntax, source maps, removed Keyword from AST and other CLI fixes Jan 18, 2024
@kaleidawave
Copy link
Owner Author

Hey @jasikpark you know about fuzzing! Looking at a recent naive run it is showing a mismatch

thread '<unnamed>' panicked at fuzz_targets/module_roundtrip_naive.rs:24:9:
input: `2
.A`
output1: `2.A;`

I was a little confused at first but I think input: ... is printing the input data. Is it possible to print output1 there instead? Thx in advance 🙏

@kaleidawave kaleidawave marked this pull request as ready for review January 19, 2024 17:37
@kaleidawave
Copy link
Owner Author

Really want to get ✅ on one of the fuzzing tests. While most of the time the errors are extreme edge cases, it would still be nice. Unfortunately they don't build on windows so having to do this trial and error remotely.

There also seems to be a regression in parsing performance introduced in this PR :(.

@kaleidawave
Copy link
Owner Author

Merging, will open an issue for the fuzzing tests

@kaleidawave kaleidawave merged commit b6ead3e into main Jan 29, 2024
6 of 8 checks passed
@kaleidawave kaleidawave mentioned this pull request Feb 27, 2024
@kaleidawave kaleidawave deleted the fixes-821 branch April 16, 2024 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler-performance Making parsing / type checking / transformations faster internal-improvements Related to internal publishing, building, workflows, formatting & linting lsp Language service protocol and editor integration parser Related to Ezno's syntax parser, AST definitions and output
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant