Skip to content

Gateway Device

Any low-end PC to act as a Gateway Device. It will not be running any heavy compute tasks and will not be part of the kubernetes cluster.

The RaspberryPi v4 or higher is a good choice and has sufficient processing power to act as a ecosystem gateway device. A celeron-based Intel NUC (or clone) would be a good choice if network speed between cluster and NAS is important.

Do you need a dedicated Gateway Device?

Short answer: You don’t need one but it helps.

Longer answer: A Gateway Device provides good seperation of concerns and a predictable ecosystem for the Cluster nodes which are not “normal” devices attached to your Home Network. This will cut down on potential network troubleshooting time.

RaspberryPi Installation

In this project, the RaspberryPi is a good low-power gateway device that can be left on 24/7. Depending on the RaspberryPi model it has reasonable network and USB throughput although it does get bogged down under continous strain.

Provisioning takes quite a long time as ISO images are downloaded and unpacked but this is done once at initial setup. (The Ansible playbooks skip these steps in future runs.)

Installation

High level steps:

  1. Manual installation flashing SDHC card with Raspbian.
    • Enable SSH
    • Disable the Desktop environment
    • Keep “simple” ethernet names (ie: eth0, eth1 etc) if possible
  2. Install all necessary ecosystem using the Ansible Gateway-Server playbook.
    • Connect to the Gateway Device using the Home Network IP Address.

Fixed or Wifi Network?

You can use the built-in RaspberryPi Wifi or add a USB NIC.

Notice

The inbuilt NIC is used to connect to the Cluster and will be provisioned with a static IP cluster subnet address.
Do not use the inbuilt NIC to connect to the Home Network.

Ansible Playbook - Gateway-Server

The following sections describe the steps that are automated in the Ansible Gateway-Server Playbook.

Gateway Router Function

Services / DNS is described in more detail and is provided on the Cluster Network interface of the Gateway Device.

Gateway Device - Cluster IP Address

The gateway has a fixed IP address, outside of the DHCP ranges.

gateway_ip_addr: 192.168.57.30

Node IP Subnet Address Space

The subnet space for kubnernetes nodes is predefined so the gateway-install Ansible Playbook can assign meaningful hostnames.

k8s_worker_nodes_dhcp_subnet: 192.168.57.255
k8s_worker_nodes_dhcp_start: 192.168.57.31
k8s_worker_nodes_dhcp_end: 192.168.57.41

The gateway-install playbook will provision known nodes using a list of known nodes defined in gateway-install/defaults/main.yaml

# Known nodes on cluster network
nodes:
- name: "k8s-control-plane-node"
  mac: "7c:83:34:b7:74:65"
  static_ip: "192.168.57.31"
  model: "intel nuc i5"
- name: "k8s-node2"
  mac: "f4:4d:30:65:88:c0"
  static_ip: "192.168.57.32"
  model: "intel nuc i5"
- name: "k8s-node3"
...

See Services / Cluster / Add Node for more detailed information on adding a new node to the cluster.

Network Interfaces

The gateway-install Ansible Playbook assumes two NIC, one WWW and one Cluster LAN interface.

sequenceDiagram
  Home Network->Gateway Device: gateway_www_interface
  Cluster Node->Gateway Device: gateway_dnsmasq_interface

You can change these to suit whatever the Gateway Device hardware you wish to use, including Wifi

gateway_www_interface: eth1
gateway_dnsmasq_interface: eth0

PXE Ubuntu Installer

This topic is outside of the scope of this documentation. It is not essential as cluster nodes can be manually built using Ubuntu Server installation via Bootable USB stick.

The PXE ISO image is defined in the gateway-install Ansible Playbook at gateway-install/defaults/main.yaml

# PXE ISO Image
pxe_menu_label: PXE Ubuntu Server 22.04.2 Auto Install
linux_iso_filename: ubuntu-22.04.2-live-server-amd64.iso
linux_iso_download_url: https://releases.ubuntu.com/jammy/ubuntu-22.04.2-live-server-amd64.iso
linux_iso_sha256: 5e38b55d57d94ff029719342357325ed3bda38fa80054f9330dc789cd2d43931

The ISO SHA256 is useful to ensure it is correctly downloaded otherwise a corrupt ISO can lead to some unobvious installation problems.

Note, if ths ISO is changed, it is important that all the supporting PXE Ubuntu Linux files are also verified to be compatible with PXE Ubuntu Installer TFTP method.

(more details)

NFS Server

By default, this project installs the NFS Server on the Gateway Device. This may not be ideal but provides a starting point. (Serving files from RaspberryPi SD card is not ideal and will impact the lifespan of the memory card.)

(more details)

NGINX Proxy

The Gateway Device acts as a reverse-proxy using NGINX to forward some FQDN on the cluster.home into the Cluster Network.

Because the Flamenco Manager doesn’t support path-prefix configuration, it is necessary to do subdomain routing and contact the Flamenco Manager on flamenco.cluster.home. Otherwise, all traffic will be redirected to the mkdocs documentation site.

Both the Flamenco Manager and Grafana need websockets and the NGINX Proxy is configured to “upgrade” to HTTP1.1 to support them.

The Flamenco Add-On communicates with the Flamenco Manager via this NGINX reverse-proxy and will post Blender projects through it. It is necessary to change the default 1MB HTTP BODY Post size to something larger. It can be configured in gateway-install Role and specifically in the nginx.sites.available.proxy.http.j2 template file.

(more details)