Skip to content

Releases: GoogleCloudPlatform/cluster-toolkit

V1.20.0 Filestore for GKE, Improved Windows support, & git-hosted Packer Modules

10 Jul 19:00
@mr0re1 mr0re1
v1.20.0
252694a
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
V1.20.0 Filestore for GKE, Improved Windows support, & git-hosted Packer Modules

Key New Features

  • Native GKE support for Filestore: storage-gke example.
  • Improved support for Windows - Packer and windows-startup-script;
  • Packer "packages" - treat remote (git-hosted) Packer modules as packages when using Terraform's "//" notation;
  • Automate DAOS server/client images

New Modules

Module Improvements

  • vm_instance:

    • do not swap boot disk (and VM) each time a new disk image is available;

  • vpc:

    • enabling TCP tunneling to the WinRM port used by PowerShell;
    • add firewall rule for SSH from arbitrary IP ranges;

  • gke-node-pool:

    • add option for static node count;
    • add option to enable gcfs;

  • gke-cluster:

    • expose the option to not create a system node pool;
    • add option to create and update timeouts;
    • update service account variable to separate email and scopes;

  • gke-job-template: add templating for persistent volume claims

  • custom_image:

    • add disk_type support;
    • add Powershell script support;
    • improved support for Windows;
    • treat remote (git-hosted) Packer modules as packages when using Terraform's "//" notation

  • htcondor-install:

    • add support for fixed version of HTCondor;
    • improve resilience;

  • schedmd-slurm-gcp-v5-controller: allow providing short references for image project

  • batch-job-template: use Batch HPC CentOS images as default image

Version updates

  • Update to slurm-gcp 5.7.4
  • Update google-cloud-daos from v0.4.0 to v0.4.1

What's Changed

Read more

Contributors

mr0re1, issacg, and 8 other contributors
Assets 2
Loading
0 Join discussion

v1.19.1 Fix panic on null fields in terraform outputs

20 Jun 14:36
@tpdownes tpdownes
v1.19.1
d6a3ef4
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.19.1 Fix panic on null fields in terraform outputs

What's Changed

  • Hotfix: eliminate panic on null value in terraform outputs by @tpdownes in #1471

Full Changelog: v1.19.0...v1.19.1

Contributors

tpdownes
Assets 2
Loading
0 Join discussion

v1.19.0: ghpc destroy command, automatic ssh configuration, and Ramble integration

15 Jun 17:46
@rohitramu rohitramu
v1.19.0
5b01711
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.19.0: ghpc destroy command, automatic ssh configuration, and Ramble integration

Key New Features

  • New destroy command that automates deletion of all infrastructure from a deployment
  • New ramble-execute module. Example blueprint: ramble.yaml.
  • Automated SSH configuration using startup-script module with configure_ssh_host_patterns setting.

Module Improvements

Improvements

Version updates

  • Intel DAOS from 0.3.0 to 0.4.0:
  • Upgraded Terraform provider from 4.63.1 to 4.65.2
  • Upgraded Spack default version from 0.19.0 to 0.20.0
  • Update to slurm-gcp 5.7.3
    • Allow metadata key slurmd_feature to initiate dynamic node setup.
    • Disable TreeWidth when dynamic nodes are configured.
    • Fix NVIDIA driver install after kernel upgrade for rocky-linux-8.

What's Changed

Read more

Contributors

wkharold, douglasjacobsen, and 9 other contributors
Assets 2
Loading
0 Join discussion

v1.18.1: Update Package Requirements for Open Front End

08 Jun 20:01
@nick-stroud nick-stroud
v1.18.1
96c09b3
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.18.1: Update Package Requirements for Open Front End

What's Changed

  • Bump cryptography from 40.0.2 to 41.0.0 in /community/front-end/ofe by @dependabot in #1418

Full Changelog: v1.18.0...v1.18.1

Contributors

dependabot
Assets 2
Loading

v1.18.0: ghpc deploy, new examples, better examples names, slurm-gcp 5.7.2

19 May 04:04
@cboneti cboneti
v1.18.0
f1e2ec1
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.18.0: ghpc deploy, new examples, better examples names, slurm-gcp 5.7.2

Key New Features

  • ghpc deploy is now the recommended way of deploying your environments
  • multigroup blueprints may now use module outputs from one group to another
    • e.g., a. network may be dynamically created in group 1 and its name will be available directly in group 2
  • New hpc-enterprise blueprint with various high performance options
  • New ML blueprints: ml-slurm.yaml and ml-gke.yaml
  • Blueprints renamed for more clarity
  • Ability to communicate variables across deployment groups with ghpc deploy or ghpc export-outputs and ghpc import-inputs
  • Slurm on GCP V4.x is now deprecated, all core examples are moved to V5.7.2

Examples

  • htc-slurm.yaml: shows how to provision a cluster with configuration tuned for many short-duration, loosely coupled jobs.
  • client-google-cloud-storage.yaml: demonstrates different ways to use Google Cloud Storage (GCS) buckets in the HPC Toolkit.

New Modules

  • gke-job-template: Creates a Kubernetes job templated file that can be used to submit jobs.
  • kubernetes-operations: Performs pre-defined operations on Kubernetes resources that would otherwise be executed using kubectl.

Module Improvements

  • gke-cluster: Added GPU support and automated installation of Nvidia drivers.

Deprecations

Version updates

  • schedmd-slurm-gcp-v5-controller: update SchedMD modules to 5.7.2
  • Min required Terraform version bumped 1.0 -> 1.2
  • Min required Packer version bumped 1.6 -> 1.7.9

What's Changed

Read more

Contributors

douglasjacobsen, mr0re1, and 6 other contributors
Assets 2
Loading
0 Join discussion

v1.17.0: Initial Support for GKE, Slurm v5.6.3

04 May 19:05
@mr0re1 mr0re1
v1.17.0
f7abfbe
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.17.0: Initial Support for GKE, Slurm v5.6.3

Key New Features

  • Initial Support for Kubernetes with GKE (example).
  • Enable specification of all fields of module outputs
  • Instructions to run the toolkit from Cloud Workstations

New Modules

Module Improvements

Improvements

  • Added support for OFE deployment from a configuration file

Version updates

What's Changed

  • Replace startup-srcipt examples with bool inputs by @mr0re1 in #1100
  • Copy all embedded modules into deployment, use unique source for locals by @mr0re1 in #1086
  • Close copy file descriptor in EmbeddedSourceReader by @mr0re1 in #1114
  • Improve error match in embedded_test by @mr0re1 in #1115
  • Adds a gke-cluster module to community by @nick-stroud in #1113
  • DAOS docs update by @cboneti in #1116
  • Simplify and relax type constraints for variables.tf by @mr0re1 in #1111
  • Make every integration test into individual build config by @mr0re1 in #1112
  • Fix validator test_deployment_variable_not_used by @mr0re1 in #1120
  • Add basic documentation for gke-cluster module and example by @nick-stroud in #1117
  • Updating packer documentation to make usage easier to find by @cboneti in #1118
  • Add image_storage_locations input to modules/packer/custom-image by @mr0re1 in #1123
  • Add TF definition for DAILY-test-X,PR-test-X, and PR-validation by @mr0re1 in #1119
  • Add "babysit_tests" tool to automatically approve PR tests by @mr0re1 in #1106
  • Solve state/world discrepancies in TF dev infra. by @mr0re1 in #1126
  • Move SlurmV5 tests affected by stockouts to us-west4-c by @mr0re1 in #1124
  • Improve variable references by @tpdownes in #1127
  • Remove test groups, update documentation by @mr0re1 in #1128
  • Fix bug in check for mixing module kinds within a group by @mr0re1 in #1130
  • Update GitHub bug report template by @mr0re1 in #1131
  • Remove deprecated pod_security_policy by @nick-stroud in #1133
  • Add test selectors to babysit tool by @mr0re1 in #1136
  • Add TF for legacy PR tests. To be removed after release by @mr0re1 in #1135
  • Add SPACK_CACHE secret to spack-gromacs test by @mr0re1 in #1132
  • Add instructions for connecting to the gke-cluster by @nick-stroud in #1138
  • Address need for SystemD override in HTCondor module by @tpdownes in #1139
  • Update TFLint and rules plugin for Google Cloud Platform by @tpdownes in #1146
  • Add double quotes on variables: SC2086 – ShellCheck by @nick-stroud in #1148
  • Add support for sensitive output values by @tpdownes in #1129
  • Represent TerraformBackend.Config with cty.Value by @mr0re1 in #1141
  • Bump github.com/otiai10/copy from 1.9.0 to 1.10.0 by @dependabot in #1143
  • Bump github.com/spf13/cobra from 1.6.1 to 1.7.0 by @dependabot in #1145
  • Truncate short sha length to 7 chars when filtering from cloud build by @nick-stroud in #1151
  • Bump google.golang.org/api from 0.114.0 to 0.117.0 by @dependabot in #1150
  • Bring develop up to date with release of v1.16.0 by @nick-stroud in #1153
  • Pin google terraform provider to latest version by @nick-stroud in #1154
  • Add selectors for batch and spack tests to babysit_tests tool by @nick-stroud in #1155
  • Reduce the number of execution hosts in pbs test to reduce the change… by @nick-stroud in #1149
  • Ensure that PBS test config explicitly uses network module by @tpdownes in #1159
  • Align internal use of Toolkit GitHub refs by @tpdownes in #1160
  • Move Ubuntu test and example to reduce chance of stockout by @nick-stroud in #1163
  • Fix HTCondor central manager configuration by @tpdownes in #1162
  • Add specialized tokenizer to handle ((HCL literals)) by @mr0re1 in #1167
  • Move Slurm v5 high io test to reduce stockouts by @nick-stroud in #1168
  • Gke node pool by @nick-stroud in #1140
  • Make babysit_tests compatible with Python3.7 (VertexAI) by @mr0re1 in #1173
  • Instructions to run the toolkit from Cloud Workstations by @cboneti in #1170
  • Write group metadata to deployment folder by @tpdownes in #1169
  • Update quantum example with new build instructions by @tpdownes in #1176
  • Add TransformSimpleToHcl for cty.Value by @mr0re1 in #1165
  • Developer setup on login is causing workstation to crash on startup by @nick-stroud in #1177
  • Add conditions on Slurm partition enable_placement, exclusive, Oversu… by @mr0re1 in #1174
  • Move tests to avoid stockouts by @nick-stroud in #1179
  • Use a unique Packer SSH username to avoid clashes with previous Packer builds by @nick-stroud in #1184
  • Bump google.golang.org/api from 0.117.0 to 0.118.0 by @dependabot in #1183
  • Bump cloud.google.com/go/compute from 1.19.0 to 1.19.1 by @dependabot in #1182
  • Update SchedMD modules to 5.6.3 (from 5.6.2) by @SkylerMalinowski in #1171
  • Updated chrome rem...
Read more

Contributors

mr0re1, tpdownes, and 6 other contributors
Assets 2
Loading
0 Join discussion

v1.16.0: New Lustre Example, Slurm v5.6.2, & HTCondor Improvements

11 Apr 00:53
@nick-stroud nick-stroud
v1.16.0
8bac6ec
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.16.0: New Lustre Example, Slurm v5.6.2, & HTCondor Improvements

Improvements

  • New simple Lustre example. (blueprint, documentation)
  • htcondor-execute-point: Added option to HTCondor autoscaler for minimum number of idle VMs to decrease job startup time.
  • htcondor-execute-point: Added option to set boot disk size.
  • New validator reports unused deployment variables. (documentation)
  • Expanded options for skipping individual validators. (documentation)
  • terraform.tfvars file generated in the deployment folder is written in stable order, making it easier to track in version control.
  • Test and documentation updates.

Version updates

  • Slurm V5 modules minor update v5.6.0 > v5.6.2 (full changelog)
    • Now compatible with Terraform 1.4.0.
    • Resume failures now notify srun of the error.
    • setup.log now discoverable in GCP Cloud Logger.
    • Fix slurm and slurm-gcp logs not showing up in Cloud Logging.

What's Changed

  • Bump googleapis-common-protos from 1.54.0 to 1.58.0 in /community/front-end/ofe by @dependabot in #1014
  • Bump httplib2 from 0.20.4 to 0.21.0 in /community/front-end/ofe by @dependabot in #1015
  • Bump wrapt from 1.13.3 to 1.15.0 in /community/front-end/ofe by @dependabot in #1016
  • Bump django-extensions from 3.1.5 to 3.2.1 in /community/front-end/ofe by @dependabot in #1018
  • Remove ModuleToGroup from DeploymentConfig by @mr0re1 in #1022
  • Add enforcement of minimum # of idle VMs to HTCondor autoscaler by @tpdownes in #983
  • Bump grpcio-status from 1.43.0 to 1.51.3 in /community/front-end/ofe by @dependabot in #1017
  • Fix broken markdown link by @mr0re1 in #1024
  • Update year of Terraform deployment license by @tpdownes in #1026
  • Fix startup-options test by @mr0re1 in #1029
  • Fix broken test config use-resources.yaml by @mr0re1 in #1030
  • Add an integration test for chrome-remote-desktop module by @nick-stroud in #1027
  • Fix: dependabot proposed incompatible requirements by @nick-stroud in #1028
  • Merge v1.14.1 back to develop by @nick-stroud in #1033
  • Add retries to apt tasks in chrome-remote-desktop to account for lock contention with unattended-upgrades by @nick-stroud in #1025
  • Bring all provider_meta versions up to current version by @nick-stroud in #1034
  • Update pre-commit hook repos by @tpdownes in #1035
  • Fix dtype of slurm node-group.preemtible to bool by @mr0re1 in #1036
  • Set label dtype = map(string) for community/modules by @mr0re1 in #1038
  • Fix labels dtype in DDN-EXAScaler by @mr0re1 in #1039
  • Make readers usable outside of package modulereader by @mr0re1 in #1040
  • Fix slurm gcp v5 validation message in partition by @cboneti in #1043
  • Add a test to enforce contracts on modules interfaces by @mr0re1 in #1041
  • Bump google.golang.org/api from 0.112.0 to 0.114.0 by @dependabot in #1044
  • Bump github.com/googleapis/gax-go/v2 from 2.7.1 to 2.8.0 by @dependabot in #1045
  • Bump cloud.google.com/go/serviceusage from 1.5.0 to 1.6.0 by @dependabot in #1046
  • Bump github.com/zclconf/go-cty from 1.13.0 to 1.13.1 by @dependabot in #1047
  • Bump github.com/go-git/go-git/v5 from 5.6.0 to 5.6.1 by @dependabot in #1048
  • Add integration test coverage for add_deployment_name_before_prefix by @nick-stroud in #1037
  • Minor refactoring of code around expressions by @mr0re1 in #1042
  • Upgrade pip in builder image by @tpdownes in #1056
  • Add simple sanity test installing the OFE virtual environment by @tpdownes in #1057
  • Order settings alphabettically and format main.tf by @mr0re1 in #1059
  • Add links to related material on YouTube by @nick-stroud in #1061
  • Resolve anticipated merge conflict between develop and main by @tpdownes in #1066
  • Merge v1.15.0 release into develop by @tpdownes in #1070
  • Bump github.com/hashicorp/go-getter from 1.7.0 to 1.7.1 by @dependabot in #1067
  • Use stable order while writing variables and backend configs by @mr0re1 in #1071
  • Increase version of Terraform google providers by @tpdownes in #1072
  • Implement exponential backoff in startup-script by @tpdownes in #1073
  • Update to slurm-gcp 5.6.2 (from 5.6.0) by @SkylerMalinowski in #1074
  • Update slurm images by @tpdownes in #1075
  • Add deployment_name to HTCondor example VMs by @tpdownes in #1076
  • Remove labels-specific logic from tfwriter. by @mr0re1 in #1064
  • Add additional documentation on enable_reconfigure troubleshooting by @nick-stroud in #1077
  • Use stable order while writing terraform.tfvars by @mr0re1 in #1080
  • Refactoring to support edge-tracking in graph by @tpdownes in #1078
  • Refactor moduleConnections as a map tracking source module of connection by @tpdownes in #1079
  • Remove field Module.ModuleName add DeploymentSource instead by @mr0re1 in #1081
  • Improve version constraint on batch-job-template module by @tpdownes in #1084
  • Lustre example by @cboneti in #1082
  • Resolve accidental destruction of startup-scripts by @tpdownes in #1085
  • Updating image in the slurm-gcp-v5 Ubuntu example by @cboneti in #1087
  • Show core examples first by @cboneti in #1088
  • Use latest update to startup-script module by @tpdownes in #1090
  • Bump cloud.google.com/go/compute from 1.18.0 to 1.19.0 by @dependabot in #1089
  • Check validations during make tests; Add flag to skip validators by @mr0re1 in #1032
  • Changing CRD test to us-central1-f by @cboneti in #1093
  • Changing the GPU type of quantum-circuit-simulator to T4. by @cboneti in #1091
  • Unify network_storage variables type by @mr0re1 in #1092
  • Several minor fixes by @tpdownes in #1095
  • Bump google-api-python-client from 2.37.0 to 2.82.0 in /community/front-end/ofe by @dependabot in #1068
  • Bump platformdirs from 2.5.0 to 3.1.1 in /community/front-end/ofe by @dependabot in #1053
  • Bump pyparsing fro...
Read more

Contributors

mr0re1, tpdownes, and 6 other contributors
Assets 2
Loading
0 Join discussion

v1.15.0: Improvements to Slurm and HTCondor solutions

21 Mar 16:13
@tpdownes tpdownes
v1.15.0
4787cde
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.15.0: Improvements to Slurm and HTCondor solutions

Key New Features

  • Support for HTCondor pools with both On-demand and Spot VMs
  • Slurm solution updated to 5.6.0
    • Support for custom machine types
    • Label exclusive nodes with job ID for cost-tracking
    • New zone_target_shape parameter corresponding to bulkInsert targetShape parameter
    • FIX: lustre mounting regression introduced in 5.5.0

Improvements

  • [filestore] module added supported for Shared VPCs via use keyword and pre-existing-vpc module
  • HTCondor modules now use minimally-scoped authentication for each daemon
  • HTCondor execute points disable benchmarks to decrease time to join pool
  • Improved type alignment across modules. e.g. var.labels aligned to map(string)

What's Changed

  • Rename filestore network_name to network_id to enable shared VPC via use by @nick-stroud in #962
  • Improve attribute tracking in HTCondor scheduler by @tpdownes in #965
  • Update fluent tutorial to use pre-existing-vpc module and other minor syntax updates by @nick-stroud in #963
  • Revert "Rename filestore network_name to network_id to enable shared VPC via use" by @nick-stroud in #967
  • Mask sleep/suspend targets on chrome-remote-desktop to prevent shutdown by @nick-stroud in #968
  • Update image building example to use Slurm V5 by @mr0re1 in #964
  • Improve HTCondor job matchmaking speed by @tpdownes in #971
  • Roll-forward:"Rename filestore network_name to network_id to enable shared VPC via use" by @nick-stroud in #969
  • Increase reliability of blueprints using DDN Exascaler by @tpdownes in #972
  • Further increase speed at which HTCondor daemons update their ClassAds by @tpdownes in #974
  • Initial support for Spot VMs within HTCondor pools by @tpdownes in #973
  • Convert HTCondor autoscaler to SystemD timer by @tpdownes in #975
  • Add validation to prevent usage of variables in backend block. by @mr0re1 in #970
  • Making OFE deploy.sh MacOS compatible. Fixes #978 by @ek-nag in #979
  • Improve Slurm log capturing by @tpdownes in #980
  • Support Spot VMs in HTCondor pools by @tpdownes in #981
  • Add utils for parising and normalizing HCL dtype by @mr0re1 in #977
  • Enable depth-first filling of HTCondor pools by @tpdownes in #982
  • Escalate to root priveleges to fetch Slurm logs by @mr0re1 in #987
  • Bump google.golang.org/api from 0.110.0 to 0.111.0 by @dependabot in #984
  • Bump github.com/spf13/afero from 1.9.4 to 1.9.5 by @dependabot in #985
  • Bump github.com/go-git/go-git/v5 from 5.4.2 to 5.6.0 by @dependabot in #986
  • Bump dill from 0.3.4 to 0.3.6 in /community/front-end/ofe by @dependabot in #990
  • Bump google-cloud-core from 2.2.2 to 2.3.2 in /community/front-end/ofe by @dependabot in #991
  • Bump astroid from 2.9.3 to 2.15.0 in /community/front-end/ofe by @dependabot in #992
  • Bump proto-plus from 1.20.1 to 1.22.2 in /community/front-end/ofe by @dependabot in #993
  • Bump isort from 5.10.1 to 5.12.0 in /community/front-end/ofe by @dependabot in #994
  • Merge main into develop after release v1.14.0 by @mr0re1 in #997
  • Bump terraform providers version 4.53.1 -> 4.56.0 by @mr0re1 in #998
  • Clean up Filestore regardless of instances presence by @mr0re1 in #999
  • Upgrade to slurm-gcp 5.6.0 by @SkylerMalinowski in #995
  • Fix nfs-server example to use local_mounts instead of local_mount by @nick-stroud in #1001
  • Add missing description for gcs_bucket_path by @nick-stroud in #1002
  • Doc fix by @issacg in #1010
  • Add mounting of cloud-storage-bucket to Slurm v5 test by @nick-stroud in #1007
  • Use DeploymentName getter instead of looking up Vars by @mr0re1 in #1005
  • Specify strict type for labels = map(string) by @mr0re1 in #1000
  • Pass empty string instead of null to avoid mounting failure in Slurm by @nick-stroud in #1003
  • Remove ghpc_role setting from nfs-server example by @nick-stroud in #1008
  • Actually check mount instead of just checking dir exists by @nick-stroud in #1004
  • Remove hostname test as it is not providing incremental value by @nick-stroud in #1006
  • Double length of time for HTCondor integration test to detect job queue by @tpdownes in #1020
  • Bump github.com/googleapis/gax-go/v2 from 2.7.0 to 2.7.1 by @dependabot in #1011
  • Bump github.com/hashicorp/hcl/v2 from 2.16.1 to 2.16.2 by @dependabot in #1012
  • Update slurm v5 readme about local-exec dependencies by @mr0re1 in #1023
  • Bump google.golang.org/api from 0.111.0 to 0.112.0 by @dependabot in #1013
  • Update OFE Dependabot configuration by @tpdownes in #1055
  • Release v1.15.0 by @tpdownes in #1065

New Contributors

Full Changelog: v1.14.1...v1.15.0

Contributors

mr0re1, issacg, and 5 other contributors
Assets 2
Loading
0 Join discussion

v1.14.1: Fix vm-instance naming

16 Mar 00:44
@nick-stroud nick-stroud
v1.14.1
1b2d029
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.14.1: Fix vm-instance naming

What's Changed

  • Hotfix: incorrect syntax for terraform interpolation in string by @nick-stroud in #1031

Full Changelog: v1.14.0...v1.14.1

Contributors

nick-stroud
Assets 2
Loading

v1.14.0: HTCondor highly available, HCLS blueprint

07 Mar 21:40
@mr0re1 mr0re1
v1.14.0
e7c0c24
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.14.0: HTCondor highly available, HCLS blueprint

Key New Features

  • HCLS blueprint supports running GROMACS on GPUs and has added several tutorials.
  • Support for highly available HTCondor pools
    • Job queue (SchedD) high availability remains experimental see README

Module Improvements

  • vpc: new option to enable firewall rule that allows tunneling of Windows Remote Desktop connections
  • schedmd-slurm-gcp-v5-partition: all deprecated variables have been removed; these have migrated to schedmd-slurm-gcp-v5-node-group
  • htcondor-configure:
    • job history will now include VM instance ID, zone and machine type
    • VMs are now provisioned with minimally-permissioned IDTOKENs for their respective daemons (e.g. ADVERTISE_STARTD)
  • startup-script: installation script for Cloud Ops Agent on Debian platforms will retry when other processes are blocking apt operations
  • htcondor-execute-point: add a simple health check of port 9618 on any machine within the execute point
  • vm-instance: vm-instance can be named using both a prefix and the deployment name

Improvements

  • Improved error message when YAML blueprint has syntax errors preventing it from being loaded
  • Regular updates to Go and Python dependencies to address potential security vulnerabilities
  • Fixed Open Front End (OFE) issue with static content (icons) not displaying properly

What's Changed

  • Add Windows Remote Desktop IAP firewall rule by @tpdownes in #885
  • Bump oauthlib from 3.2.1 to 3.2.2 in /community/front-end/ofe by @dependabot in #886
  • Fix: having the same share name and local mount caused slurm failure by @nick-stroud in #887
  • Implicitly add outputs to modules when they are being used across deployment groups by @tpdownes in #878
  • Reorder validator list to test blueprint correctness first by @heyealex in #889
  • Bump cryptography from 36.0.1 to 39.0.1 in /community/front-end/ofe by @dependabot in #890
  • Adding example sbatch and config for Factor Xa protein by @nick-stroud in #888
  • Update HCLS blueprint examples to run Gromacs w/ GPUs by @nick-stroud in #891
  • Add support for highly available HTCondor Central Managers by @tpdownes in #892
  • Remove deprecations from slurm-gcp v5 partition by @heyealex in #893
  • Bump cryptography from 36.0.1 to 39.0.1 in /community/front-end/ofe by @tpdownes in #896
  • Bump cryptography from 36.0.1 to 39.0.1 in /community/front-end/ofe by @dependabot in #894
  • Ensure Open Front End dependabot updates target develop by @tpdownes in #897
  • Fix: Always generate Batch instance template to avoid known at apply time error by @nick-stroud in #898
  • Update sbatch to copy results to output bucket & minor tweaks by @nick-stroud in #895
  • Add troubleshooting documentation for filestore share name exportfs bug by @nick-stroud in #899
  • Bump github.com/hashicorp/hcl/v2 from 2.16.0 to 2.16.1 by @dependabot in #900
  • Fix addlicense check in weekly image building by @tpdownes in #901
  • Update HTCondor modules by @tpdownes in #902
  • Bump django from 3.2.16 to 3.2.17 in /community/front-end/ofe by @dependabot in #905
  • Bump cryptography from 36.0.1 to 39.0.1 in /community/front-end/ofe by @dependabot in #906
  • Bump oauthlib from 3.2.1 to 3.2.2 in /community/front-end/ofe by @dependabot in #907
  • Bump TFLint version in image to latest by @tpdownes in #903
  • Add instructions to ssh to VM created by vm-instance by @nick-stroud in #880
  • Address terraform_unused_required_providers errors by @tpdownes in #908
  • Manage HTCondor yum repo configuration directly by @tpdownes in #904
  • Bump github.com/aws/aws-sdk-go from 1.33.0 to 1.34.0 by @dependabot in #911
  • Fail integration tests on validation warnings. by @mr0re1 in #910
  • Set minimal scopes for HTCondor IDTOKENs by @tpdownes in #919
  • Fix cmd/root_test.go test runs from linked Git worktrees. by @mr0re1 in #918
  • OFE update 14/02/2023. by @ek-nag in #913
  • Improve error message for yaml parsing failures by @heyealex in #923
  • HTCondor job track machine information by @tpdownes in #924
  • Update develop with release v1.13.0 by @nick-stroud in #928
  • Add explicit output dependencies to HTCondor by @tpdownes in #925
  • Pin terraform google provider to v4.53.1 by @nick-stroud in #929
  • Fix root_test failure on MacOS by @mr0re1 in #932
  • Update htcondor-configure README example snippet by @tpdownes in #935
  • Bump github.com/hashicorp/go-getter from 1.6.2 to 1.7.0 by @dependabot in #930
  • Add retries to cloud ops install by @heyealex in #933
  • Bump django from 3.2.17 to 3.2.18 in /community/front-end/ofe by @dependabot in #922
  • Remove unused error message by @tpdownes in #939
  • Bump google.golang.org/api from 0.109.0 to 0.110.0 by @dependabot in #937
  • Fix typo in hcls instructions command by @nick-stroud in #940
  • Update hcls example to use lysozyme protein instead of factor xa by @nick-stroud in #942
  • HTCondor Job Queue High Availability by @tpdownes in #934
  • Refactor useModule by @tpdownes in #941
  • Bugfix to HTCondor autoscaler script by @tpdownes in #945
  • Update hcls spack builder to use c2 machine by @nick-stroud in #948
  • Enable OS Login by default in HTCondor execute points by @tpdownes in #944
  • Remove dependency on unused module by @mr0re1 in #947
  • Add health check for HTCondor VMs by @tpdownes in #946
  • HCLS tutorial update by @nick-stroud in #950
  • Bump github.com/spf13/afero from 1.9.3 to 1.9.4 by @dependabot in #955
  • Bump github.com/zclconf/go-cty from 1.12.1 to 1.13.0 by @dependabot in #954
  • Allow VM instance name to include prefix and deployment name by @nick-stroud in #949
  • Use HTCondor Python bindings in autoscaler by @tpdownes in #951
  • Update hcls Lysozyme example to include visualization instructions by @nick-stroud in #958
  • Unify validator...
Read more

Contributors

mr0re1, tpdownes, and 5 other contributors
Assets 2
Loading
0 Join discussion