Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixFlakes: can't figure out nix-instantiate equivalent #3908

Open
colemickens opened this issue Aug 6, 2020 · 20 comments · May be fixed by #11506
Open

nixFlakes: can't figure out nix-instantiate equivalent #3908

colemickens opened this issue Aug 6, 2020 · 20 comments · May be fixed by #11506
Labels

Comments

@colemickens
Copy link
Member

colemickens commented Aug 6, 2020

Describe the bug

I can't figure out the equivalent for many nix-instantiate commands. Another bug #???? describes how nix eval and nix eval --{raw|json} behave differently. And the latter that works for me, returns the out path, rather than the derivation (this feels similar to how nix copy aggressively builds instead of just copying drvs #3696).

How do I get the drv path for a flake output? This is what I have now, I don't like it:

function instantiate() {
  machine="${1}"; shift
  # do it
  true
  #drv="$(nix eval --pure-eval ".#nixosConfigurations.${machine}.config.system.build.toplevel")" # TODO: why????
  set -e
  drv="$(set -eu; nix --experimental-features 'nix-command flakes' --pure-eval \
    eval \
    --raw ".#machines.${machine}")"
  drv="$(set -euo pipefail; nix --experimental-features 'nix-command flakes' --pure-eval \
    show-derivation "${drv}" | jq -r 'to_entries[].key')"
  echo -e "${drv}"
}
nix (Nix) 2.4pre20200721_ff314f1

EDIT (workaround)

You can do something like this instead:

thing=".#packages.x86_64-linux.testPackage" # pkg example
thing=".#nixosConfigurations.yourHostName.config.system.build.toplevel" # nixos cfg example
nix build \
  --experimental-features 'nix-command flakes' \
  "${thing}.drvPath"

This gives you a /nix/store/abc123..thing.drv path that you can then nix copy --derivation or nix-copy-closure to a remote machine to build, etc.

@colemickens colemickens added the bug label Aug 6, 2020
@colemickens colemickens changed the title nixFlakes: can't figure out nix-isntantiate equivalent nixFlakes: can't figure out nix-instantiate equivalent Aug 6, 2020
@zimbatm
Copy link
Member

zimbatm commented Aug 7, 2020

I think it's on purpose, Eelco wanted to hide the .drv implementation detail. With the idea that it will allow us to change the format down the line.

@colemickens
Copy link
Member Author

This blocks "the good way" of doing large remote nix builds. I sympathize with the desire to iterate on the format, but can that be done in a backward compatible way that still allows all of the places where I've been using drvs?

I know I'm not the only one that nix-instantiates, copies drvs, and then uses cachix to move binaries.

@xbreak
Copy link
Contributor

xbreak commented Oct 16, 2020

I know I'm not the only one that nix-instantiates, copies drvs, and then uses cachix to move binaries.

We do the same but on-premise cache.

@colemickens
Copy link
Member Author

I remembered that --derivation works for nix copy and was hoping that it might apply here, but it appears not. While --derivation appears in the help for nix eval, it still outputs the build output path, rather than the path to derivation. This is another thing that is forcing me to keep extra tooling around to work with flakes.

@thufschmitt
Copy link
Member

It should always possible to get the drv path with something like nix eval --raw foo#bar.drvPath. But indeed as @zimbatm pointed out, this might not be a viable long-term solution and there should be a cleaner way of doing that

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/to-flake-or-not-to-flake/10047/4

@stale
Copy link

stale bot commented Jun 2, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the stale label Jun 2, 2021
@dhess
Copy link

dhess commented Dec 28, 2021

Please un-stale.

@stale stale bot removed the stale label Dec 28, 2021
@virchau13
Copy link

It should always possible to get the drv path with something like nix eval --raw foo#bar.drvPath.

It's worth saying that this is the current approach taken by nixos-rebuild when --build-host is specified. So it's got precedent, at least.

@disassembler
Copy link
Member

disassembler commented Jun 16, 2022

My primary use case is for nix show-derivation. In non-flakes I used to do nix-instantiate foo.nix -A bar which gives me the drv without building and I could then run nix show-derivation on that file to inspect it. Now with flakes, I've been doing nix build .#bar, building the entire thing, then nix-store -q --deriver result/ and then I can show-derivation that. I'm assuming there has to be a better way that doesn't build the derivation. It also makes it is hard to automate if an error occurs in the build, as you have to go through the logs to find the text output of the drv file.

@thufschmitt
Copy link
Member

I'm assuming there has to be a better way that doesn't build the derivation

nix show-derivation .#bar? 😛

More generally, most of the new-style commands that act on a derivation or a store path indifferently accept a litteral store path or a Nix expression/flake ref as input (and will transparently build it or not depending on whether it needs to)

@thufschmitt
Copy link
Member

Going back to the original motivation for the issue (fast remote builds by instantiating locally, copying the drv and building remotely), it's worth pointing out that this is possible with:

$ nix build --eval-store auto --store ${remoteMachine} .#foo

@AleXoundOS
Copy link

I think it's on purpose, Eelco wanted to hide the .drv implementation detail.

.drv files is quite an easy concept (way more easier than flakes itself, imho). I doubt that hiding .drv from an end user is worth...

@plietar
Copy link

plietar commented Jul 17, 2022

I've found out nix path-info --derivation .#foo does exactly this.

@hraban
Copy link
Member

hraban commented May 9, 2023

I've found out nix path-info --derivation .#foo does exactly this.

This is brilliant, thank you @plietar , this effectively answers the question and closes the ticket. Solid!

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tool-for-evaluating-stateversion-changes/31043/6

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/webkit-2-36-3-keeps-trying-to-build-from-source-nixos-22-05/19874/12

@Yeshey
Copy link

Yeshey commented Oct 22, 2023

I've found out nix path-info --derivation .#foo does exactly this.

How would this be used in a flake with only nixosConfigurations? like this:

git+file:///home/yeshey/.setup
└───nixosConfigurations
    ├───arm-oracle: NixOS configuration
    ├───laptop: NixOS configuration
    ├───surface: NixOS configuration
    └───vm: NixOS configuration

it complains that it does not provide an atribute:

> nix path-info --derivation /home/yeshey/.setup#laptop                    
warning: Git tree '/home/yeshey/.setup' is dirty
warning: input 'nur' has an override for a non-existent input 'nixpkgs'
error: flake 'git+file:///home/yeshey/.setup' does not provide attribute 'packages.x86_64-linux.laptop', 'legacyPackages.x86_64-linux.laptop' or 'laptop'
> nix path-info --derivation /home/yeshey/.setup#nixosConfigurations.laptop
warning: Git tree '/home/yeshey/.setup' is dirty
warning: input 'nur' has an override for a non-existent input 'nixpkgs'
error: 'nixosConfigurations.laptop.type' is not a string but a set

@Patryk27
Copy link
Member

I think you're looking for:

nix path-info --derivation .#nixosConfigurations.systemName.config.system.build.toplevel

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/webkit-2-36-3-keeps-trying-to-build-from-source-nixos-22-05/19874/13

@layus layus linked a pull request Sep 15, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.