Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate away from the kubernetes-release bucket #3729

Open
puerco opened this issue Aug 27, 2024 · 6 comments
Open

Migrate away from the kubernetes-release bucket #3729

puerco opened this issue Aug 27, 2024 · 6 comments
Labels
kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.

Comments

@puerco
Copy link
Member

puerco commented Aug 27, 2024

Context

Release artifact have historically been published to a GCS bucket called kubernetes-release, this bucket is not under community control. As of the week of Aug 19th the release artifacts are being served from a CDN backed by a community bucket.

Currently, the contents of kubernetes-release are mirrored to the community bucket for serving every two hours.

/cc @kubernetes/release-engineering

TODO

Identify processes that need to be moved

We need to comprehensively search our processes to find those relying on data from kubernetes-release. Once we have an idea of which ones are reading and/or writing to the google owned bucket, let's expand the lists below with those that need to be migrated.

Migrating to the community bucket involves two groups of tasks, let's expand these as we find them:

Kubernetes Release Process

  • TBD

Internal Processes and Tests

@k8s-ci-robot k8s-ci-robot added needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority labels Aug 27, 2024
@ameukam
Copy link
Member

ameukam commented Aug 28, 2024

We had many conversations about this in kubernetes/k8s.io#2396.

@BenTheElder
Copy link
Member

We should also confirm the GCB project being used.

We need to comprehensively search our processes to find those relying on data from kubernetes-release. Once we have an idea of which ones are reading and/or writing to the google owned bucket, let's expand the lists below with those that need to be migrated.

I believe expected writes are only krel? There's a constant in krel for the bucket.

For reads, we've already made a big push to point things at dl.k8s.io instead, if any more crop up we can fix them later as worst case they won't have new releases until they switch, and the new bucket is intentionally not public read (only through the CDN).

I think it should be:

  • make sure krel GCB service account has write to the new bucket (should be done already but double check)
  • swap krel to write to the new bucket
  • spin down sync job
  • confirm next release goes smoothly
  • googler (me): drop remaining write permissions to legacy google-containers project

@ameukam
Copy link
Member

ameukam commented Sep 3, 2024

make sure krel GCB service account has write to the new bucket (should be done already but double check)

It's not done yet. krel leverage the GCB service agent of the kubernetes-release-test GCP project to cut releases.

@BenTheElder
Copy link
Member

We should also migrate out of kubernetes-release-test which is in google.com to a project in kubernetes.io, but we could do that in two phases.

@ameukam
Copy link
Member

ameukam commented Sep 3, 2024

We should also migrate out of kubernetes-release-test which is in google.com to a project in kubernetes.io, but we could do that in two phases.

See: #3425

@ameukam
Copy link
Member

ameukam commented Sep 9, 2024

/kind feature
/priority important-soon

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority labels Sep 9, 2024
ameukam added a commit to ameukam/release that referenced this issue Sep 9, 2024
Part of:
 - kubernetes#3729

Use a community-bucket for Kubernetes releases
ameukam added a commit to ameukam/k8s.io that referenced this issue Sep 9, 2024
Ref:
  - kubernetes/release#3729

Temporary allow the Service Agent for the GCB Service from project
`kubernetes-release-test`. This will enable artifacts release for
Kubernetes to a community-owned bucket.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Development

No branches or pull requests

4 participants