-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sharing and distributing log files/build metadata? #441
Comments
Actually, if |
Yeah, I've been really happy with the entire suite of
Yes, there is. You can see some stuff related to log uploading here: buck2/app/buck2_client_ctx/src/manifold.rs Line 131 in 062f014
This has been incredibly useful for understanding and providing support for user builds. I think it would be great if we could figure out the appropriate API for supporting this for open source users as well. I think it probably should be considered together with #226. The answer there may be that it's actually best to have separate apis for them (you'll note that internally we are doing them basically entirely separately). One important thing we've found is that we want to upload the log incrementally as the build progresses to ensure that we get logs even on failed things (particularly our CI may aggressively kill off timing out or ooming things that makes it hard to reliably capture on failures). |
buck2 log
is pretty cool, since it lets you look at what build commands a user ran and what their output is, and you can do things likelog cmd
orlog replay
to watch the build. Is it possible or recommended to 'share' these log files? What would some infrastructure for doing that look like? Maybe something like:buck2 build ...
a bunchbuck2 log cmd --trace-id $BUILD_ID
to find out what happened.There could be other useful things to derive from this possibly, assuming you had these logs. For example, you could use these logs as a direct source of build analytic information from users to derive information about build times, etc.
Is there any kind of "thing" like this inside Meta? Does this seem interesting? To be truly useful it would need some support in the core executable, I think. It seems like something kind of like what
buck2 rage
does, right? Except I'd want it in all cases, not just failures.I imagine this could look like the following for OSS users:
.buckconfig
key likebuck2_logs.upload_address
is set tohttps://buck2-logs.aseipp.dev/
.pb.zst
files are POST'd to$UPLOAD_ADDRESS/upload
with their trace ID as a primary key, asynchronously, by the daemonbuck2 log cmd --trace-id $TRACE_ID
, then:$UPLOAD_ADDRESS/trace/$TRACE_ID
.pb.zst
fileThe text was updated successfully, but these errors were encountered: