Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run memory comparison script between Prism and Sorbet #243

Open
wants to merge 2 commits into
base: prism
Choose a base branch
from

Conversation

egiurleo
Copy link

Motivation

Now that we can run the entire Sorbet pipeline on the RBI gem, we can include it in our benchmarks. Here, I've updated the memory comparison script to run on the RBI gem, and then committed the results of running the script.

Results

The results here have held consistent. When just comparing the memory usage of the two parsers, Prism uses about 65% as much memory as the Sorbet parser. When running the entire pipeline, the results are less dramatic because we run the translation layer, which will allocate the same Sorbet nodes as the Sorbet parser, thus evening out the results.

**Note: ** There might also be another stage in the Sorbet pipeline that allocates more memory than the parser, in which case the results of the latter two benchmarks will return peak memory in that stage. However, even if that's true, the fact that these numbers are similar is still promising, because it means that even with the translation layer, Prism isn't causing Sorbet to use more memory than it normally does at its peak.

#### Memory Check 1: yjit-bench, parser only ####
-> Peak Memory Usage with prism parser: 132.691 MB
-> Peak Memory Usage with sorbet parser: 200.477 MB

#### Memory Check 2: shopify, parser only ####
-> Peak Memory Usage with prism parser: 856.734 MB
-> Peak Memory Usage with sorbet parser: 1317.7 MB

#### Memory Check 3: prism regression tests, whole pipeline ####
-> Peak Memory Usage with prism parser: 68.1836 MB
-> Peak Memory Usage with sorbet parser: 68.9102 MB

#### Memory Check 4: RBI gem, whole pipeline ####
-> Peak Memory Usage with prism parser: 122.797 MB
-> Peak Memory Usage with sorbet parser: 131.012 MB

@egiurleo egiurleo self-assigned this Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants