Skip to content

Minio Argo Workflow Tenant

This tenant provides buckets for Argo Workflows artifacts.

  1. minio storage: http://storage.cluster.home/
  2. minio console: http://storage.cluster.home/console/

Artifact Bucket

Access Keys

A key-secret pair is required for Argo Workflows to log in to the minio tenant and place artifacts in the artifact bucket.

NGINX Reverse-Proxy

The NGINX reverse-proxy on the Gateway Device has to upgrade the connection to support websockets. The client_max_body_size is also set to 0 (unlimited) since we may want to upload/download large artifacts. Similarly, the max-body-size for the NGINX Ingress must also be large enough to support large artifacts.

server {
    server_name storage.cluster.home;
    client_max_body_size 0;
    resolver 127.0.0.1;

   location / {
        proxy_pass http://storage.cluster.home;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # This allows WebSocket connections
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
   }
}

NGINX Ingress

The ingress is created by the Argo Workflows chart. It is important to use working version of Minio and NGINX Ingress Controller.

kubectl -n minio-tenant-argo-workflows describe ingress
Name:             argo-workflows
Labels:           app.kubernetes.io/managed-by=Helm
Namespace:        minio-tenant-argo-workflows
Address:          192.168.57.200
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host                  Path  Backends
  ----                  ----  --------
  storage.cluster.home  
                        /   minio:http-minio (10.244.107.253:9000,10.244.122.117:9000,10.244.169.179:9000 + 1 more...)
Annotations:            meta.helm.sh/release-name: tenant
                        meta.helm.sh/release-namespace: minio-tenant-argo-workflows
Events:                 <none>

Name:             argo-workflows-console
Labels:           app.kubernetes.io/managed-by=Helm
Namespace:        minio-tenant-argo-workflows
Address:          192.168.57.200
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host                  Path  Backends
  ----                  ----  --------
  storage.cluster.home  
                        /console/(.*)   argo-workflows-console:http-console (10.244.107.253:9090,10.244.122.117:9090,10.244.169.179:9090 + 1 more...)
Annotations:            meta.helm.sh/release-name: tenant
                        meta.helm.sh/release-namespace: minio-tenant-argo-workflows
                        nginx.ingress.kubernetes.io/rewrite-target: /$1
                        nginx.ingress.kubernetes.io/use-regex: true
Events:                 <none>
  1. RELEASE.2023–03–24T21–41–23Z last known working release from https://github.com/minio/minio/tags. This tag is specified in the Minio Tenant values.yaml file.
  2. Newer release do not work until https://github.com/minio/console/releases/tag/v0.28.0 is included in minio operator release. (Issue 2774, PR2818).

Note, PR2818 is really well written and has a lot of great information on how sub-path support in minio operator can be tested.

RELEASE.2023–03–24T21–41–23Z Configuration changes to minio tenant deployment values.yaml file.

      ingress:
        api:
          enabled: true
          ingressClassName: "nginx"
          labels: { }
          annotations: { }
          tls: [ ]
          host: http://storage.cluster.home/
          path: /
          pathType: ImplementationSpecific
        console:
          enabled: true
          ingressClassName: "nginx"
          labels: { }
          annotations:
            #kubernetes.io/ingress.class: nginx    
            nginx.ingress.kubernetes.io/rewrite-target: /$1
            nginx.ingress.kubernetes.io/use-regex: "true"
          tls: []
          host: http://storage.cluster.home/
          path: /console/(.*)
          pathType: ImplementationSpecific

It is also important to tell Minio Tenant what is the console and api URLs. This used by done via ENV but seems it can be done in the features subsection of the Minio Tenant chart.

        features:
          domains:
            console: http://storage.cluster.home/console/
            minio:
              - http://storage.cluster.home