[Feature] Discussion around release strategy of Yarn v2 #766
Comments
|
Yes, we are discussing this at the moment. I can see the problem and definitely don't want to break workflows (hence why we hold off on updating the The rational behind reusing the name is mostly to make clear that no evolutions will happen on the 1.x trunk (I mean, maybe someone will step up and take over, but I would find this highly unlikely). That being said, maybe a different release name doesn't necessarily preclude from this. What I think could be a good way to go forward would be to:
But to release the CLI binary under the Once the ecosystem is ready (for example we can put a reconductible hard reevaluation date on Jan 24 2021) we would then consider to take back the Would that answer your concerns? |
|
What about using |
|
I don't think that that would be sufficient. It would still technically live under the yarn branding with all the packages and also the domain being Instead, I think the right move would be to entirely embrace the name |
|
Yarn was primarily built to solve problems at large companies who had trouble managing their dependencies. They have most likely made significant investments into the integration of Yarn. Now, these companies do not have a viable migration path and will be stuck with Yarn v1 for years to come given Berry's change in dependency management behavior. I believe deprecating Yarn v1 is premature and I am confident that various people and companies will keep supporting it if necessary. There is no need to turn the existing community away given that they are quite happy with the current version of Yarn. I am also in favor of releasing Berry as a separate project. Let's reconsider replacing Yarn with Berry once it gained significant community adoption, and is known to support all parts of the JavaScript ecosystem just like Yarn v1. At that point in time we'll all be able to confidently agree that Yarn v1 is obsolete. |
|
If the governance believes that |
|
I think your proposal will largely solve my concerns @arcanis, with a few changes:
|
|
@brentvatne +1 |
|
Have to agree strongly with everyone here. At Twitter, we're going to be stuck on Yarn v1 or else need to move back to npm and hope that the improvements they've put in over the last couple of years will be enough that our tooling built on top of yarn can be ported over. |
|
Thinking about this more, I'm not a fan of changing the name and adding another package manager to the JS ecosystem. Let's not get JS fatigue trending again. Is there a technical reason for not supporting the I'm sure a lot of people and companies would love to use the new yarn v2 features without PnP until it is more proven and widely supported. There should be a transition period, especially considering this is completely rewritten. I suggest
This allows everyone to benefit from v2 improvements while keeping breaking changes and migration very minimal. |
|
From the original author of yarn:
|
|
I mentioned this already on Twitter to you @arcanis but I'd like to repeat it here: I think what you did with Berry is amazing and if this was a green field and Node and the ecosystem was in it's infancy this would definitely be a much easier transition. But with 1.2 million packages currently published on the registry and so many projects using Node, npm and yarn (v1) as their backbone, this is waaaay too much of a drastic sweeping change for a single major release while also immediately deprecating the "old" version. Especially considering that it's currently absolutely non trivial to migrate from v1 to v2. Just ask yourself this: Would you expect React v17 to "simply work" after upgrading from v16? Or with a minimal amount of easy fixes? I would. |
It's already the case, as pointed out in the blog post. We are investing in both areas. Some improvements are on their way in this regard, and we fully intend to support |
|
@arcanis In this case does the migration plan I suggest makes sense. PnP looks promising but sadly years of infra work depends on the |
|
I like @janicduplessis's migration plan proposal as well. If people can update to the latest yarn version and continue using it the same as they are today then this will get pnp just a few keystrokes away from users and they will be much more likely to try it out. |
|
@janicduplessis @brentvatne Given the complete rewrite of all algorithms in Berry (rather than an incremental major version bump with some breaking changes, bug fixes and new features), what framework could we use to ensure that it meets the bar in terms of behavior and functionality that will enable >95% of users to upgrade without any work involved? It seems that Brent's initial proposal is healthier for the community in the short term. |
That would be the first point of the migration strategy. Ideally we'd get to the point where If we can get a couple large projects to test v2 to provide feedback on the node_modules mode that would be great and give enough confidence to be able to replace yarn v1 with it. Do you think facebook would be interested in testing yarn v2 with |
|
We can definitely commit to testing it, but anything that's not a drop-in-replacement at this time will probably not work for us. Another approach could be to focus on improving Yarn 1's PnP mode and gaining significant adoption first (See: https://yarnpkg.com/lang/en/docs/pnp/ ). However, Given that PnP was released more than one year ago and hasn't gained significant traction makes me worried that the approach may not be viable to become a default at any time in the future and that we'll be stuck with materialized |
|
@cpojer The |
|
To be crystal clear, PnP will remain as the default on this repository and the 2.x releases, this isn't something that's going to change. That being said - I'm ok with dropping the idea of adding a "deprecated" notice on the The website however should stay as it is. We all have to make compromises, I think asking you to click on an extra link to access the confusing documentation that isn't maintained about the version that isn't maintained is reasonable. Everything is already redirected, so referenced links will keep working and search engines will pick it up. |
|
@arcanis I highly suggest you reconsider @brentvatne’s suggested solution. My beliefs are that Yarn 2 is a different package manager at the moment since it has no upgrade path for a significant amount of the userbase. Big players such as FB and Twitter have publicly stated that they cannot use Yarn 2 because of PnP. Like it or not, they have large sway when it comes to JS standardization. My worry with your proposed plan is that you will damage the reputation of Yarn and kill the thing you have been working so hard on as well. That would be a shame, since what you have built is awesome. However, maintainers including myself, will immediately remove yarn from all docs so as to avoid confusing people around installation. The website needs to be Yarn 1, with a big banner stating that berry is the blessed successor and linking to its docs. |
|
Why not just call it |
|
One current problem is testing Yarn 2. I would love to try it in some of my projects, but doing It would be nice to have a separate (temporary) package with the (This could be done with some bash hacks for now, but a new package would help adoption of v2 until it is "ready" for prime time) |
I'm not sure about yarn's governance and its relationship to Facebook (since I believe it was started as a Facebook project), but, having done some and just a bit of work on v1 and v2, I would be interested in maintaining yarn v1. Happy to chat more elsewhere about this. Not aiming to be controversial by any means, but to highlight that Yarn gained community adoption where many other JS package managers did not ( and many, like pnpm, are quite good!). I think it's worth noting the stability and momentum v1 has while having virtually no work done on it since last year. When yarn v1 released, people happily migrated because At the moment, I think the ergonomics and benefits of berry are yet to be realized by most people (including myself) so if there's no governing body to decide how / when to rollout, I would say allowing the community to decide overtime would be the safest option that would not risk fracturing the current user base. |
@Cretezy FWIW you can specify Yarn version per-project by using |
This will definitely be the case for the non-npm installation methods (Debian/Ubuntu package, Windows installer, Chocolatey and Homebrew), that part of the release infra just hasn't been finalized yet. It might be a good idea for the npm package too! |
|
I've been in a plane for a few hours and got some time to think more in depth about this issue. First, regarding using a different binary name: I rescind what I said. Changing the package name would be a bad idea for the project health. The reason for that is that many tools are hardcoding Yarn in their logic - for example VSCode, but also WebStorm, Husky, project generators, etc. A different binary name would bring a lot of additional maintenance hurdle not only to us but to the global ecosystem we all care about. But there's another option: stop releasing Yarn as a global binary. With 2.x we are in a state where
With this plan things will stay basically the same as they are right now. Existing applications will keep working, people following instructions on third-party website will keep using the 1.x workflow. The website will stay on the 2.x however, this isn't something I'm willing to change considering how likely it is that I'll be doing most of the support, if this year is a good indication.
Yes, this is what I have in mind. Together with for example |
|
I like the idea of keeping the global binary on Yarn 1. This will allow users and projects to opt-in to Yarn 2 as they are ready. This creates an opportunity to encourage projects to "pin" their desired version of yarn in the repo using I'm discovering that there are still a lot of packages that are not ready for PnP. Now that Yarn 2 is stable, most of these packages will start resolving these issues and make the migration smoother. |
The global ecosystem is currently built around yarn (v1), npm and ... node_modules. The one thing that will be bad for the project's health, imho, will be the loss of trust in yarn if all these integrations that you mentioned stop working or simply aren't compatible with the documentation and installation guides found on the website and branded as Piggybacking on the success and wide spread adoption of yarn v1 to force such a drastic change for the entire ecosystem without any sort of upfront warning, long deprecation phase and, most importantly, a bearable upgrade path is wrong. And it's still wrong even if, technically, the underlying changes are sound. By branding berry as a direct successor of yarn and, even worse, labeling yarn v1 as deprecated while also hijacking the main website (that pretty much everyone who advertises yarn in their own packages' docs, blog posts, etc. links to) is simply creating a level of confusion that would be hard to recover from. The level of friction this creates will simply be too much for the vast majority of the community and will eventually drive them away. You are righteously imposing a monumental amount of work on the entire ecosystem with such a sweeping change. I appreciate the effort to fix the brokeness of the current status quo but you can't force this over night. |
|
I’d like to throw in a historical anecdote from the React side. Not that it’s the same situation, but might be helpful to consider. We knew in 2015 that we wanted incremental reconciliation. This is something Jordan prototyped and it seems like a fundamental shift that challenges many assumptions about how React works. We started on a rewrite around that time. However, we knew we can’t change how React apps work over a short period of time. That would be a large blow to people’s confidence in React. If we create a bad impression, no matter how superior the solution is, the lost goodwill is a major hurdle for adoption. Which would set us backwards rather than forwards. So instead, we rewrote React to make incremental reconciliation possible. But we kept the old behavior. We even “reimplemented” some of the old bugs to make the transition more seamless. That was the React 16 release: https://engineering.fb.com/web/react-16-a-look-inside-an-api-compatible-rewrite-of-our-frontend-ui-library/ Soon after React 16 release, we added a “strict mode”. It allowed people to opt into a stricter set of warnings that tell you about patterns that won’t work with incremental reconciliation. Judging by the issues we see in open source libraries, Strict Mode is widely adopted. People push component libraries to pass Strict Mode with no warnings. On the other hand, using something incompatible isn’t insurmountable. Some libraries are unmaintained, and it takes months of community effort to get them fixed or migrate away. If we warned without an opt in, everything would be so noisy that people would give up. So this has been a really effective strategy in getting people upgraded. Now we have an opt-in to incremental reconciliation as well. It’s still experimental but people are trying it and the ecosystem is so much more ready for it than years ago. We also released a set of APIs (Hooks) that make it easier to write code that’s friendly to incremental reconciliation. I doubt we would be in this place if we simply turned out incremental reconciliation in React 16 by default. I’d like to add that I deeply sympathise with the idea that sometimes, you just need to move forward. And that requires dedication and, sometimes, cutting losses. Still, it’s important to remember that technical innovation can only succeed when there is a climate for it to happen. There is a real cost to scaring the ecosystem, and that cost is that people won’t consider the technical aspects of the solution altogether. Some smart solutions from 1960s still aren’t in use today because the adoption strategy hasn’t been considered as closely as technical brilliance. Hope this helps. |
|
Currently Yarn 2 (at least with PnP) doesn't work for most Gatsby websites. For example themes don't work with pnp, some plugins that write to node_modules don't work with pnp (eg gatsby-plugin-typography), there are many peer dependency issues. We also rely on node_modules being availaible internally. At current stage we wouldn't be recommending any of our users to use Yarn 2 (in fact all our documentation is using npm). Gatsby project by itself, however, uses Yarn (because of workspaces). We are also interested in leveraging new features in Yarn 2, like releases. For us @cpojer solution would be preferable, as it would allow us to continue supporting Yarn with Gatsby projects and at the same time use new features inside our monorepo. However, I'm worried that a minor release after a full rewrite will cause too many problems. Releasing Yarn version 2 without PnP as default could be a safer alternative. |
|
I disagree with the proposals mentioned here. I think the Yarn maintainers had the right idea with the original launch. It might be painful at first as libraries and tools become compatible, but it’s the only way to ensure the adoption of PnP over the long term. Shipping with PnP off by default will result in PnP never gaining adoption. We’ve seen this already with Yarn 1, which has had PnP support for over a year. So, the question is, if it is turned off again, how long do we wait to turn it on by default? Another year? Two years? Three? Yarn 2 is a major version bump, so it won’t break anything unless you upgrade. If you try it out and it doesn’t work for you, you can always downgrade! But, the bug reports that are sent to libraries and tools with incompatibilities are very important to PnP’s adoption, and won’t happen without this process. I think we need to suck up the short term pain for the long term gain. This is the way. 😉 |
|
I think one thing to note why pnp might not be gaining mainstream adoption on v1 is because there are no warnings to fix what should be fixed for pnp to work, rather than it being an option |
|
I saw the announcement post and was knocked over by how much amazing stuff is coming. Several parts of it solve real problems my team are fighting today. I’m also conceptually a fan of PnP and have been looking forwards to its future progress since it came out. Unfortunately, about 1/5 of my apps’ direct dependencies need patches making them incompatible with PnP. We use React-Native, and TypeScript, so we won’t be able to upgrade until those projects are fixed. This feels like a large hurdle that will keep these great tools away from us, and the nature of this migration could fracture the community. I think many people dislike using a “legacy” product, and many people can’t fix some of the stumbling blocks under the planned rollout. I’d personally love @cpojer or similar proposals of delaying the hard PnP switch until more community investments iron out the rough spots. This lets the community come to know and love all of Berry’s other awesome features, and ideally even better understand the effort of switching to PnP. I still like @brentvatne‘s proposal as a second fallback plan, it recognizes that community pain may have been underestimated, and might be easier (on a technical level) if slower for the maintainers to advance. I have enjoyed yarn’s existence and have advocated for it at several shops, and feel it has produced a better npm community by its very existence, not to mention that npm literally adopted some of yarn’s learnings. I hope that that continues rather than making “giving up on yarn” the more attractive road for the majority of users; this pain is real, and if PnP is the future, then we can’t force it on people when the ecosystem isn’t ready or yarn itself doesn’t have enough tools to bridge us there. |
That makes sense for libraries. But does it make sense for global commands? If everyone installing Yarn today gets something very different from what people who added
Do you find the opposite of this statement believable? That turning it on by default for users who have no technical expertise to fix PnP issues will lead to PnP gaining adoption? I'm somewhat skeptical of this, although I'd love to be proven wrong. |
|
I don’t think installation instructions in readmes have ever been that specific about which package manager to use. Many document npm install but that doesn’t stop people from using yarn or pnpm. pnpm in particular already works pretty differently, which causes some bugs for library/tool authors already, so there is some historical precedent here. |
|
When you read But if the readme already reads Personally, in all my packages' READMEs I put both |
Yeah. My concern is that if people's first experiences with Yarn V2 look like #843, open source maintainers will likely start removing the Yarn-specific instructions to avoid confusing their users. Personally, I don't have investment in what happens either way. But it would be a shame if this set the project back even if the solution is technically superior and the pain is supposed to be short term. |
It’s possible I guess, but that’s a pretty pessimistic viewpoint. We can’t move the status quo forward without trying. Also remember: it’s opt in, so users are choosing to turn it on by upgrading. They also have a way out by downgrading. Users don’t need to “fix” the issues themselves, they can report them, and downgrade if needed. |
|
I don't quite get why this has to be forced on end users, if PnP is a superior solution it will eventually find its way to widespread adoption. Why the rush? I think the famous Linus quote about "never ever breaking userspace" is quite fitting for Forcing breaking changes to end users that mostly only care about stability and have actual work to do is wrong in my opinion, moreover, I'm afraid it would irremediably damage this project. Please ship |
|
I second the idea of having The thing is that we get so used to work with these tools daily that we forget that every day new developers come to the ecosystem, meaning they have to go through the whole JS-fatigue process. How are we going to explain to them that I don’t think it’s fair to only worry about the current community and avoid thinking about the ones to come. Let’s learn from other communities that went through similar hurdles (e.g. python ended up choosing In short, I would love to try As a closing note, in particular, to @arcanis, take all these comments with a grain of salt. The JS community is extremely grateful for the work individuals like you and the @yarnpkg maintainers (and companies like @DataDog that sponsor this!), that it’s easy to get lost in the many “Well, I think that...” comments when a good chunk of developers sit on the side just looking to try the new toys. Projects like Babel got their fair share of hate on an upgrade (can’t remember which), that got their maintainers to feel pretty bad about work they did for (guess what) free. Do not let impostor syndrome kick in: you are doing awesome work here. |
|
There is so much FUD being spread in this thread.
Nothing is being forced. You need to explicitly upgrade. That's why it's a major version release.
PnP has been available for over a year already with only a small amount of adoption. Enabling it by default is the next step. If not now, when?
How is this any different? Yarn 2 is already opt-in, just like a separate project would be. |
|
Of all the proposed solutions, releasing We need less fragmentation in our ecosystem, not more. |
|
@devongovett I think the issue lies in the all or nothing approach of v2. It comes with many improvements that no one would find an issue with, yet a large portion of the community will have to downgrade because node_modules support is not ready yet in v2. A compromise would be to release v2 without PnP enabled by default, so as to not force people to stay on v1 and miss out on other improvements. I don't know if the v2 improvements are possible without PnP, if not then that's another problem |
|
@devongovett - a comparable example that comes to mind for me here is Node / Deno — what if once Deno has partial compatibility with Node ("It's unlikely we can make Node code work 100% of the time") it were to ship as the next major version of Node? I think this would create a massive split in the community between pre-Deno Node, like Python2/3. Deno is, in my mind, a technically superior solution — but that doesn't mean it should outright replace Node. There is room for both. @jorgegonzalez - regardless of the outcome there will be friction here, the difference is whether it will be more clear for users to differentiate between |
|
You don't have to upgrade on day one. You can choose to wait for stability in the ecosystem. Nothing is forcing you. |
A simple
I'm fine with it, @jjperezaguinaga said it quite well, the JS ecosystem is built around package diversification, with forks and new projects popping up every other day, we're used to it. Keep the same name and I bet you'll see people forking away yarn v1 as a new package manager. We will end up in the same state but the damage will be done. |
The difference there is that Node is actively maintained whereas Yarn 1 will not be.
@arcanis already addressed this in a previous comment: #766 (comment). Yarn 1 will stay as the latest package on package managers like |
Someone in this thread who has contributed to yarn in the past has already stepped up and offered to maintain it. I believe that others will get involved too. |
|
I tend to agree with @devongovett 's thinking, hence my suggestion earlier in the discussion to adopt a "next" metaphor to represent the state of Berry instead of Yarn 2. It is the direction where the tool is headed but important components are still being discussed and reasoned about. |
This comment contributes to spreading fears.
This is exactly what @devongovett talks about. This thread has too many fears and too few rational thoughts. There are plenty of options if pnp doesn't work for you from day one.
|
|
Please correct me if I'm wrong, but the node_modules plugin should provide the compatibility layer to whom is not able to upgrade to PnP, right? If so, IMHO, all we need to do is wait (and help, if able) until it is stable, then change the latest tag to v2, so we can have an easy migration path. Library maintainers can also document this on their README, pointing that if you are using Yarn V2 you need this plugin. |
|
|
Thanks everyone, I think all arguments have been made. I'll lock this issue for tonight, and will come back tomorrow to share a longer retrospective and detail the plans our team discussed. |
|
Edit by @arcanis: When I lock a thread as "too heated", I expect discussions to stop and people to go home - people with repository write-access included. One-sided arguments do nothing to "stop the flamewar". Thanks for understanding.
If we do this long term, then PnP will still not get significant adoption. PnP has been an opt-in for Yarn 1 for the past 18 months and we've seen that people do not opt-in to stricter guarantees. An opt-in to Berry and PnP is effectively the same except the opportunity cost is higher for a marginal benefit to use additional features. If we only do this short term as the announcement indicates, then the concerns above are valid and people are justifiably worried about being forced to migrate. This is a real concern that can't be brushed off with "don't use it". Many people here would love to just not use it, and they want guarantees that they won't be forced to today or in the future, or be able to use a drop-in replacement by running a single upgrade command.
Many of the people in this issue like @brentvatne and @janicduplessis have been involved since the very beginning of Yarn. The React Native community embraced Yarn on day one because of the huge install time wins. I have been involved with Yarn since before it was open sourced, funded the work on PnP at Facebook, monitored its deployment, helped fix problems and eventually removed it. I have seen the Berry codebase evolve from a prototype almost three years ago to what is in this repo today. What we all presented above was entirely facts based from our heavy involvement with Yarn and early adoption of PnP. I do not believe you can claim that we are spreading misinformation or invoking fear to disadvantage this project. We are here because we care about the future of this community. |
|
Hi everyone. I didn't want to answer this thread immediatly because the release is still young and, quite frankly, I didn't want to take decisions over the course of a week-end. @brentvatne was right to start the discussion, but that didn't mean we needed to act in a hurry. The release itself took weeks to plan (the blog post itself got reviewed by at least four people from the community, plus our own team), follow ups should be thought through with similar care. This out of the way, our team (@yarnpkg/berry) discussed and we agreed on the following points. I believe most of you will be satisfied, but just as importantly we are satisfied with it and we think it's the best way for the project to move forward. Regardless of how you feel about it, I'd ask you to show us the respect and empathy we deserve. Yarn 2.x and forward - our goalsJavascript has been a wild ride for the past twenty-or-so years. From browsers to servers, from desktops to smartphones, from a handful of developers building cute stuff to word-class applications with billions of users. I myself remember my first open-source experiences - they were in ... 2006? I was writing tutorials for a phpBB forum hosting platform, writing small scripts to mark topics as closed and stuff. Distribution was funny: we had to tell users which files to modify, and where to put the code. It often didn't go very well... Anyway, years later appeared npm, followed by Yarn, and the ecosystem took a very different turn. I was a bit skeptical when I first learned about Yarn, btw - speed was nice, but it failed to install the app I was working on at the time (it eventually did - go figure, it’s almost like projects improve given time 😉). Then in 2017 I started working on it, and something clicked. I was still seeing broken installs that we couldn't control. My coworkers still had to All this is what we are set to change with Yarn 2.x. We want installs to be fast (a typical install of our repository is 5s). We want installs to be light (a stock CRA app is now 60MB, vs 230MB before). We want installs to be fully deterministic (flat cache with full integrity checks, no chance for different behaviours) and stable (boundary checks, no more delta between prod and dev). Today, what we need the most isn't code. It isn't users. And it most definitely isn't drama or trolls or jokes about design decisions. It's support. We've been working incredibly hard to give the JS ecosystem a chance to get rid of problems that plagued it for a decade. Our work is literally worth millions in computer resources and developer efficiency, and for that we'll get little to no rewards (guess where is Yarn on this chart). So we now need you to care enough to make a change by yourself. If you don't, fine - we'll spend our resources elsewhere helping those who do. About the initial release strategyI did a mistake by not anticipating how our words would be interpreted. From my very pragmatical viewpoint, I didn't expect much to change due to the release itself: the This reasoning unfortunately didn't account for the weight that words like “legacy“ or “deprecated“ would bear on some of our users. What we meant to express was that the 1.x branch was from our perspective considered “feature-complete”, but some instead interpreted it as an “upgrade now“ sign surrounded by flashing lights. Words have weight, and maybe choosing our words more carefully would have helped. Is React Native abandoned by Yarn? /
|
Haroenv
changed the title
and others
Describe the user story
As a library author, I want to recommend the best and most reliable way to use my libraries, so I recommend users use yarn. With yarn v2 I can no longer do this because it is not compatible with React Native. I would like to continue to recommend yarn.
As a JS developer, I want to use one package manager everywhere. I use yarn in all of my projects and I would like to continue to do this but I won't be able to because yarn v2 will not work in many of them.
Describe the solution you'd like
I think that until yarn pnp is ready to be used in all major projects in the existing ecosystem, it should be released under another name. In environments where yarn pnp works well, people can adopt this easily and enjoy the benefits. Once people experience it, library authors will start to make sure their libraries work with it. When it has become very popular and widely supported, release it under a new major version for yarn or keep the name if the branding is working.
Describe the drawbacks of your solution
It is going to be unpleasant and tedious work to rename everything and create some different branding and domains. Probably there are more things I haven't considered.
Describe alternatives you've considered
The main alternative is to continue to release yarn v2 as-is. This would be less work but it runs a serious risk of hurting the trust that the community has gained in yarn. People will no longer be able to depend on yarn working on all of their projects, and library authors will move away from recommending it. There will always be a concern about large breaking changes potentially disrupting workflows and users will build around other tools instead.
The text was updated successfully, but these errors were encountered: