Make Custom AST compatible with the unified ecosystem

[horacioh]

Hello!

I'm building an application that consumes a custom AST structure based on Unist.

Right now I'm in the process of transforming the custom AST to other well-known AST structures like mdast and hast. I was wondering if there's a recommended path to create tooling around a custom AST of some best practices recommended to follow when creating compatible tooling with new AST structures.

thanks in advance!

UPDATE: I found this handbook which seems a good starting point, not sure if its outdated or y'all recommend something else?

[ChristianMurphy]

Welcome @horacioh 👋

That is a big topic, a good starting point would be reading the guides in https://unifiedjs.com/learn
Then reading through some of the previous related questions:

• So, I've written an AST spec: what to do next? syntax-tree/unist#36
• Utilities for creation & injection of AST nodes syntax-tree/unist#38
• How do I create a compiler? #159

That should give some pointers.
After reading through, there may still be some more specific questions.
Those would be welcome, as you ask them, it could be good to include context on what you are trying to build.
Things like:

• what is the overall goal of your new project?
• what language(s) are being parsed?
• what types of transformations are needed?
• what it is compiled to?

[horacioh]

thanks for the answer @ChristianMurphy !

regarding the more specific questions:

what is the overall goal of your new project?
I want to make it possible to transform from our custom AST to others like mdast, hast, or even MDX

what language(s) are being parsed?
I'm working with the raw custom AST (in our rich text editor), so the parsing can be done in JavaScript

what types of transformations are needed?
Our custom AST is for our new data-layer protocol, we call it Mttast (Mintter AST)
so the transformations we need for now are hast <-> mttast and mdast <-> mttast

what it is compiled to?
we don't need it right now to be compiled to anything since the structure we manipulate is the actual mttast.

What I'm doing right now is learning about hast-util-to-mdast and understand how the transformation is done there. I'm also documenting all the process so maybe I can help with a guideline for the /learn section in the unifiedjs website.

It's really helpful that all the repos have tests. that is helping me understand the code flow and what every piece is doing!
(also there are some code comments that are helping too!)

[horacioh]

quick question @ChristianMurphy,

a transformer in mutate mode should always return a whole node? I mean if I'm transforming only a Literal node, do I need to wrap it inside a root node to make it "compatible" with other transfromers?

[ChristianMurphy]

@horacioh do you mean a Transformer as in https://github.com/unifiedjs/unified#function-transformernode-file-next?
That works on an entire tree and if a return is given, it expects an entire tree in return.

Or do you mean Visitor https://github.com/syntax-tree/unist-util-visit-parents#next--visitornode-ancestors?
Visitor does not accept a node as a return, it accepts an action.
A node can be mutated in-place to change it, but returning the node will not have the expected effect.

[horacioh]

I mean a Transformer! specifically something similar to hast-util-to-mdast!

so I do need to return an entire tree right?

[ChristianMurphy]

right, Transformer needs an entire tree

[horacioh]

thanks @ChristianMurphy !