Add Docker-based Tofu and Ansible operations #12

floatingman · 2025-11-20T15:55:59Z

Introduce shared library functions for Tofu and Ansible operations, enabling execution within Docker containers. Enhance AWS credentials handling and SSH key management for improved security and functionality. Implement infrastructure helpers for configuration management and workspace handling.

- Add tofu.groovy: Functions for backend init, workspace management, apply/destroy - Add ansible.groovy: Functions for running playbooks and managing inventory - Add infrastructure.groovy: Helper functions for config, SSH keys, and workspace naming

- Update tofu.groovy to execute all commands in Docker - Update ansible.groovy runPlaybook to use Docker - Pass AWS credentials as environment variables to container - Mount workspace and SSH keys into container - Fixes 'tofu: not found' error on Jenkins agents

- Remove invalid steps.env access that caused MissingPropertyException - AWS credentials now inherited from withCredentials block via -e flags - Docker run automatically picks up AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from environment - Fixes 'No such property: env for class: org.jenkinsci.plugins.workflow.cps.DSL' error

slickwarren

not very well versed in this but I don't see anything standing out as an issue. Would like the main contributor's review just in case we miss something.

slickwarren · 2025-11-20T23:33:08Z

vars/tofu.groovy

+    }
+
+    // Run the init-backend.sh script which generates backend.tf and runs tofu init
+    def initCommand = "cd ${config.dir} && ./scripts/init-backend.sh ${scriptArgs.join(' ')}"


not a fan of this being hard-coded, do we do this anywhere else in the shared lib?

what about something like this instead?

qa-jenkins-library/vars/container.groovy

Lines 73 to 82 in 61a5552

if (params?.configureScript) {

// by doing this we accept all build and configure going to run from dir, 'docker build .', currently target is in validation dir so this is necessary

def statusConfigure = steps.sh(script: "./${params.dir}/${params?.configureScript}", returnStatus: true)

if (statusConfigure != 0) {

error "Build script failed with ${statusBuild}"

}

}

def statusBuild = steps.sh(script: "./${params.dir}/${params.buildScript}", returnStatus: true)

hamistao

Looks great and looks like will be very helpful. Still I have a couple comments/questions.

vars/ansible.groovy

hamistao · 2025-11-27T16:48:26Z

vars/ansible.groovy

+// Run ansible-playbook in a container
+// [ container: string, dir: string, inventory: string, playbook: string, extraVars?: Map, tags?: string, envVars?: Map ]
+def runPlaybookInContainer(Map config) {
+    if (!(config.container && config.dir && config.inventory && config.playbook)) {


Another nit, if config.container is just the container name, maybe we could generate one and return it or just name it config.container_name. Looks good anyway if you don't want to change this.

I'm not understanding how this makes it better.

maybe it doesn't, I guess this well within the subjective realm so this is only a suggestion.

My thought was that you could prefer an alternative that has less arguments.

hamistao · 2025-11-27T16:49:25Z

vars/ansible.groovy

+        "-w /workspace",
+        config.container,
+        "-c",
+        "\"${ansibleArgs.join(' ')}\""


I don't think it is, but just to make sure, is it possible that ansibleArgs to contain double quotes?

Add a check for that

hamistao · 2025-11-27T16:54:41Z

vars/infrastructure.groovy

+        error 'SSH key content and name must be provided.'
+    }
+
+    def sshDir = config.dir ?: '.ssh'


Could we do a resilient approach like ${HOME}/ssh or something similar that is less dependant on the current directory?

hamistao · 2025-11-27T17:02:50Z

vars/infrastructure.groovy

+        config.envVars.each { key, value ->
+            // Replace both ${VAR} and $VAR patterns
+            processedContent = processedContent.replaceAll(/\$\{${key}\}/, value.toString())
+            processedContent = processedContent.replaceAll(/\$${key}(?![a-zA-Z0-9_])/, value.toString())
+        }


Do you think we should process both patterns in write_config as well?

hamistao · 2025-11-27T17:09:04Z

vars/infrastructure.groovy

+
+// Archive workspace name for later use
+// [ workspaceName: string, fileName?: string ]
+def archiveWorkspaceName(Map config) {


Just a curiosity, is there a reason why you are passing all the parameters as a Map config?

Just asking because it seems less straightforward in some cases but perhaps it is a pattern we use that I am not aware of.

I think this is a limitation of jenkins / shared libraries IIRC. Something about groovy-java conversions?

For this example, it can be two string parameters, and the second one can have a default value.

For general Map config:

I haven't tried it fully, but I think the other approach is to know all the types that are being passed to a function -- if that type is needed.

This may introduce a significant curve to writing these functions (especially if our function is calling a DSL or Plugin function), as the types can change with Jenkins and/or plugin versions.

Personally, I don't have a strong opinion on any approach since I used them interchangeably. Trade-off between velocity and static types.

More of a custom function that can be typed vs a function that uses a generic Map instead of the plugin's type.

If this turns out to be a problem, we can refactor it later.

rancher-max

LGTM! Few comments from others -- I commented on a couple from my understanding.

vars/ansible.groovy

rancher-max · 2025-12-02T01:09:49Z

vars/infrastructure.groovy

+
+// Archive workspace name for later use
+// [ workspaceName: string, fileName?: string ]
+def archiveWorkspaceName(Map config) {


I think this is a limitation of jenkins / shared libraries IIRC. Something about groovy-java conversions?

…eters

…guration

…ent MissingMethodException

floatingman · 2025-12-09T04:02:38Z

rancher/qa-tasks#1925

…tion vulnerabilities

…dling and directory management; enhance generateWorkspaceName to support suffix sanitization

floatingman added 7 commits November 19, 2025 17:05

Add platform specification to Docker run commands

afc29c0

Use init-backend.sh script for proper S3 backend configuration

be1a487

Generate SSH public key from private key for Ansible playbooks

87e160c

Fix SSH public key generation to strip extension before adding .pub

61a5552

floatingman requested review from caliskanugur, lscalabrini01, rancher-max and slickwarren November 20, 2025 16:11

floatingman mentioned this pull request Nov 20, 2025

Add airgap ansible for rancher into qa-infra-automation rancher/qa-tasks#1857

Open

slickwarren reviewed Nov 20, 2025

View reviewed changes

hamistao reviewed Nov 27, 2025

View reviewed changes

rancher-max approved these changes Dec 2, 2025

View reviewed changes

floatingman added 2 commits December 3, 2025 14:49

Add centralized configuration management for Docker and testing param…

b3c3e4a

…eters

Refactor Tofu commands to use dynamic Docker image and platform confi…

28ca749

…guration

rancher-max mentioned this pull request Dec 8, 2025

Add Elemental with Havester automation into qa-infra-automation rancher/tests#351

Merged

Coerce non-string values to String in useWithFolderProperties to prev…

6d8cd9f

…ent MissingMethodException

Escape Ansible arguments and inventory paths to prevent command injec…

a0c17cd

…tion vulnerabilities

floatingman self-assigned this Dec 9, 2025

floatingman requested review from hamistao and slickwarren December 10, 2025 03:52

lscalabrini01 approved these changes Dec 10, 2025

View reviewed changes

Refactor writeConfig and writeSshKey functions for improved error han…

9f69bef

…dling and directory management; enhance generateWorkspaceName to support suffix sanitization

rancher-max approved these changes Dec 10, 2025

View reviewed changes

floatingman merged commit 9f69d50 into main Dec 11, 2025

floatingman deleted the feature/ansible-tofu-jenkins-pipelines branch December 11, 2025 16:52

	if (params?.configureScript) {
	// by doing this we accept all build and configure going to run from dir, 'docker build .', currently target is in validation dir so this is necessary
	def statusConfigure = steps.sh(script: "./${params.dir}/${params?.configureScript}", returnStatus: true)

	if (statusConfigure != 0) {
	error "Build script failed with ${statusBuild}"
	}
	}

	def statusBuild = steps.sh(script: "./${params.dir}/${params.buildScript}", returnStatus: true)

Add Docker-based Tofu and Ansible operations #12

Add Docker-based Tofu and Ansible operations #12

Uh oh!

Conversation

floatingman commented Nov 20, 2025

Uh oh!

slickwarren left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hamistao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

caliskanugur Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rancher-max left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

floatingman commented Dec 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

caliskanugur Dec 9, 2025 •

edited

Loading