Skip to content

Conversation

@dhimanAbhi
Copy link

@dhimanAbhi dhimanAbhi commented Dec 4, 2025

Pull request description

This PR introduces support for MinIO/S3-based test sources in Testworkflows. It enables fetching test assets directly from MinIO or any S3-compatible object storage as an alternative to Git-based sources. This addition provides a reliable way to distribute test content in environments with no Git access , such as restricted, air-gapped, or production Kubernetes clusters where cloning repositories is not allowed. This PR solves #4270 and #4465

Testworkflow Definition

Users can now define a MinIO/S3-based test source by specifying:

  • endpoint - MinIO/S3 storage URL
  • bucket - name of the bucket to fetch from
  • path - path within the bucket to fetch
  • region - region of the storage (optional)
  • accessKey/secretKey - for private storages (optional)

Template

apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
  name: minio-example
spec:
  content:
    minio:
      endpoint: "<minio_or_s3_endpoint>"
      bucket: "<bucket_name>"
      path: "<folder_inside_bucket>"      // defaults to ""
      accessKey: "<access_key>"           // optional
      secretKey: "<secret_key>"           // optional
      region: "<region_or_empty_string>"  // optional
  steps:
    - shell: "tree /data/minio"

Example

apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
  name: minio-example
spec:
  content:
    minio:
      endpoint: "host.docker.internal:9000"
      bucket: "cypress-12"
      path: ""                     
      accessKey: "youraccesskey"
      secretKey: "yoursecretkey"
      region: ""                    

  steps:
    - shell: |
        echo "Listing downloaded files:"
        tree /data/repo

Proof Manifests

These logs are from a TestWorkflow executed locally. I configured a local MinIO instance and uploaded the Cypress test suite located at testkube/test/cypress/cypress-12 from the Testkube repository. This TestWorkflow successfully fetches the test content from the MinIO bucket in the same way Testkube currently fetches content from Git-based sources.

$ testkube watch twe 6931548d567308bb30700032

Context:  (999.0.0-e004297f0)   Namespace: testkube
---------------------------------------------------
parsing server version 'dev': Invalid Semantic Version
Test Workflow Execution:
Name:                 miniotestworkflow
Execution ID:         6931548d567308bb30700032
Execution name:       miniotestworkflow-1
Execution namespace:
Execution number:     1
Requested at:         2025-12-04 09:29:49.211 +0000 UTC
Disabled webhooks:    false
Status:               running
Queued at:            2025-12-04 09:29:49.211 +0000 UTC
Started at:           2025-12-04 09:29:49.211 +0000 UTC

Getting logs from test workflow job 6931548d567308bb30700032
(SuccessfulCreate) Created pod: 6931548d567308bb30700032-2697c
(Scheduled) Successfully assigned testkube/6931548d567308bb30700032-2697c to kind-control-plane
(Pulled) Container image "docker.io/testworkflow-toolkit:minio" already present on machine
Creating state... done
Initializing state... done
Configuring init process... skipped
Configuring toolkit... done
Configuring shell... skipped

• passed in 4.528s

• (1/2) Fetch MinIO Bucket
📦 Downloading from MinIO bucket from endpoint host.docker.internal:9000, bucket cypress-12, path ...
{"type":"line","content":"📂 Getting the contents of bucket folders []","time":"2025-12-04T09:29:54.027084324Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:175","msg":"listing files","bucket":"cypress-12","bucketFolder":""} 
{"type":"event","content":"📂 Downloading file .gitignore []","time":"2025-12-04T09:29:54.232418625Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:556","msg":"Getting file {.gitignore %!s(int32=55)  }"}
{"type":"event","content":"✅ File .gitignore successfully downloaded into /data/minio []","time":"2025-12-04T09:29:54.245938969Z"}      
{"type":"event","content":"📂 Downloading file cypress.config.js []","time":"2025-12-04T09:29:54.245977877Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:556","msg":"Getting file {cypress.config.js %!s(int32=164)  }"}     
{"type":"event","content":"✅ File cypress.config.js successfully downloaded into /data/minio []","time":"2025-12-04T09:29:54.253892685Z"}
{"type":"event","content":"📂 Downloading file cypress/e2e/smoke.cy.js []","time":"2025-12-04T09:29:54.253931998Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:556","msg":"Getting file {cypress/e2e/smoke.cy.js %!s(int32=494)  }"}
{"type":"event","content":"✅ File cypress/e2e/smoke.cy.js successfully downloaded into /data/minio []","time":"2025-12-04T09:29:54.262120785Z"}
{"type":"event","content":"📂 Downloading file cypress/e2e/smoke2.cy.js []","time":"2025-12-04T09:29:54.262162429Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:556","msg":"Getting file {cypress/e2e/smoke2.cy.js %!s(int32=103)  }"}
{"type":"event","content":"✅ File cypress/e2e/smoke2.cy.js successfully downloaded into /data/minio []","time":"2025-12-04T09:29:54.269288228Z"}
{"type":"event","content":"📂 Downloading file cypress/fixtures/.gitkeep []","time":"2025-12-04T09:29:54.269327238Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:556","msg":"Getting file {cypress/fixtures/.gitkeep %!s(int32=3)  }"}
{"type":"event","content":"✅ File cypress/fixtures/.gitkeep successfully downloaded into /data/minio []","time":"2025-12-04T09:29:54.27715673Z"}
{"type":"event","content":"📂 Downloading file cypress/support/.gitkeep []","time":"2025-12-04T09:29:54.277197969Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:556","msg":"Getting file {cypress/support/.gitkeep %!s(int32=3)  }"}
{"type":"event","content":"✅ File cypress/support/.gitkeep successfully downloaded into /data/minio []","time":"2025-12-04T09:29:54.288798632Z"}
{"type":"event","content":"📂 Downloading file cypress/support/e2e.js []","time":"2025-12-04T09:29:54.288835717Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:556","msg":"Getting file {cypress/support/e2e.js %!s(int32=0)  }"}  
{"type":"event","content":"✅ File cypress/support/e2e.js successfully downloaded into /data/minio []","time":"2025-12-04T09:29:54.29630632Z"}
{"type":"event","content":"📂 Downloading file package-lock.json []","time":"2025-12-04T09:29:54.296350396Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:556","msg":"Getting file {package-lock.json %!s(int32=70457)  }"}   
{"type":"event","content":"✅ File package-lock.json successfully downloaded into /data/minio []","time":"2025-12-04T09:29:54.310519516Z"}
{"type":"event","content":"📂 Downloading file package.json []","time":"2025-12-04T09:29:54.310562782Z"}
{"level":"info","ts":"2025-12-04T09:29:54Z","caller":"minio/minio.go:556","msg":"Getting file {package.json %!s(int32=61)  }"}
{"type":"event","content":"✅ File package.json successfully downloaded into /data/minio []","time":"2025-12-04T09:29:54.319666117Z"}    
📥 Adjusting access permissions...
🔎 Destination folder contains following files ...
/data/minio
/data/minio/.gitignore
/data/minio/cypress
/data/minio/cypress/e2e
/data/minio/cypress/e2e/smoke.cy.js
/data/minio/cypress/e2e/smoke2.cy.js
/data/minio/cypress/fixtures
/data/minio/cypress/fixtures/.gitkeep
/data/minio/cypress/support
/data/minio/cypress/support/.gitkeep
/data/minio/cypress/support/e2e.js
/data/minio/cypress.config.js
/data/minio/package-lock.json
/data/minio/package.json
✅ Successfully downloaded MinIO content to /data/minio

• passed in 600ms
(Pulled) Container image "docker.io/testworkflow-toolkit:minio" already present on machine

• (2/2) Run shell command

Checklist (choose whats happened)

  • breaking change! (describe)
  • tested locally
  • tested on cluster
  • added new dependencies
  • updated the docs
  • added a test

Breaking changes

Changes

Fixes

Signed-off-by: Abhishek Dhiman <[email protected]>
@dhimanAbhi dhimanAbhi requested a review from a team as a code owner December 4, 2025 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant