Help integrating rehype-starry-night with MDX

[oliver-lister]

Hi mdx community,

I'm currently trying to integrate a syntax highlighting rehype plugin - 'rehype-starry-night' with my current MDX setup.

I'm following the process in this MDX guide, and I'm trying to get the following code to work in a server component, in my Next.JS project (App Router). This MDXContent.tsx is a dumbed down version of what I'm trying to get up and running.

// MDXContent.tsx

import { compile } from "@mdx-js/mdx";
import rehypeStarryNight from "rehype-starry-night";

const MDXContent = async ({ slug }: { slug: string }) => {
    const code = `~~~js
        console.log(1)
        ~~~`;

    const compiledCode = await compile(code, {
      rehypePlugins: [rehypeStarryNight],
    });

    console.log(String(compiledCode));
      return <div>test</div>;
};

export default MDXContent;

Unfortunately, I'm getting the following error:

Build Error
Failed to compile
Next.js (14.2.9)
./node_modules/@wooorm/starry-night/lang/source.mdx.js
source.mdx.js:16:13: Could not parse expression with acorn
This error occurred during the build process and can only be dismissed by fixing the error.

When I remove the rehype plugin from the compile function, the component runs as expected and a compiled version of the code is returned in the terminal.

Strangely enough, I can get this process to work on codesandbox, but not on my own app. My repo is here.
Any ideas on how I can get this to work with the rehype plugin?

In the best world, I would be fetching the mdx's content using 'fs.readFileSync', compiling it with the rehype-starry-night plugin and then finding a way to output this content using MDXRemote. I've tried to create this process here. (I believe I probably have the output wrong as well given I'm sending MDXRemote an already compiled source)

Any ideas would be greatly appreciated, thank you in advance!
@wooorm @y-nk

My versions are:
"@mdx-js/mdx": "^3.0.1",
"next": "^14.2.9",
"next-mdx-remote": "^5.0.0",
"rehype-starry-night": "^2.1.0",

[wooorm]

Hi Oliver.

This seems like a webpack or Next or next-mdx-remote bug.
Some tool thinks that source.mdx.js is an MDX file. It is not.
Try removing tools. To figure out where the bug is.

[oliver-lister]

You are amazing, thank you so much @wooorm.
I changed my extension property in my withMDX function in next.config.js from /\.(md|mdx)/ to /\.mdx?$/, and this sorted it.

[wooorm]

Right, sounds like that would do it!
Where did you find /\.(md|mdx)/? Was that on our website? If that regex was in some docs, I am sure the maintainers of those docs would love to hear about this bug!

[oliver-lister]

I was following a tutorial initially and had copied their boilerplate. Not entirely sure where they found the source, but have left a comment.

[oliver-lister]

Update

Thanks to @wooorm, I managed to get my app to correctly compile/serialise mdx content with the rehype-starry-night plugin and I could console.log the output string. But I then struggled to find a way to output this "compiled" / "serialised" code in a Next JS server component. I'm probably missing something quite simple.

I decided to use react-syntax-highlighter in the meantime and customised the built-in 'code' tag in mdx-components.tsx to use a SyntaxHighlight component. @beeinger has a great implementation of this that I followed here.

Of course - I'm open to any other thoughts of how I could've made rehype-starry-night work, but this workaround seems like it'll do the trick for now.

[wooorm]

Oliver, I am sorry, but I am not customer service. I will not talk you through how to make your app. You don’t have to ping me.
Please, read our docs. Feel free to ask actionable, small, questions. When you have done some research.