Input parameters for the provision tool

Fill in all required parameters in input/provision_config.yml, provision_config_credentials.yml, input/software_config.json.

Caution

Do not remove or comment any lines in the input/provision_config.yml file.

provision_config.yml

Parameter

Details

public_nic

string

Required

The nic/ethernet card that is connected to the public internet.

Default values: eno2

iso_file_path

string

Required

Path where user has placed the iso image that needs to be provisioned on target nodes. Accepted files are Rocky8-DVD or RHEL-8.x-DVD (full OS). iso_file_path should contain the provision_os and provision_os_version values in the filename.

Default values: "/home/RHEL-8.6.0-20220420.3-x86_64-dvd1.iso"

node_name

string

Required

  • Prefix for target node names, if dynamically allocated.

  • Hostname = node_name + ‘0000x’ + domain_name

  • Hostname <= 65 characters

  • Example: servernode00001.Omnia.test , where node_name =servernode, domain_name =Omnia.test , 00001 used by Omnia.

Default values: node

domain_name

string

Required

  • Domain name the user intends to configure on the cluster.

  • Hostname = node_name + ‘0000x’ + domain_name

  • Hostname <= 65 characters

  • Please provide a valid domain name according to the domain name standards.

  • Example: servernode00001.Omnia.test , where node_name=servernode, domain_name=Omnia.test , 00001 used by Omnia.

pxe_mapping_file_path

string

Optional

  • This variable is required to discover nodes using a mapping file.

  • The mapping file consists of the Service tag, Admin MAC,Hostname and its respective admin IP address and/or BMC IP.

  • Ensure that the admin IP addresses provided are within the admin_static_ranges.

  • A sample file is provided here: examples/pxe_mapping_file.csv.

  • The headers of the CSV are SERVICE_TAG,ADMIN_MAC,HOSTNAME,ADMIN_IP,BMC_IP.

enable_switch_based

boolean [1]

Required

  • Variable indicates whether switch based discovery should be enabled to discover the nodes

  • To enable switch based discovery, set enable_switch_based to true.

  • If enable_switch_based is set to true,the following inputs should be provided:

    • switch_based_details should be provided in provision_config.yml

    • switch_snmp3_username and switch_snmp3_password should be provided in provision_config_credentials.yml

    • bmc_network details with nic_name, netmask_bits, static_range, and dynamic_range should be provided in network_spec.yml

Caution

If enable_switch_based is set to true, bmc discovery will be skipped.

  • For enabling bmc discovery, set enable_switch_based to false and provide bmc_network details in network_spec.yml

  • Accepted values:

    • true

    • false <- Default

switch_based_details

JSON List

Optional

  • This variable is required when nodes are to be discovered via switch details (that is enable_switch_based is set to true).

  • JSON list of switches to query for target nodes.

  • Split port ranges are not accepted here. (Ex: 10:5-10:10 will not be valid).

  • Example:

    - { ip: 172.96.28.12, ports: '1-48,49:3,50' }

  • Example with 2 switches:

    - { ip: 172.96.28.12, ports: '1-48,49:3,50' }
- { ip: 172.96.28.14, ports: '1,2,3,5' }

disk_partition

JSON list

Optional

  • User defined disk partition applied to remote servers.

  • The disk partition desired_capacity has to be provided in MB.

  • Valid mount_point values accepted for disk partition are /home, /var, /tmp, /usr, swap.

  • Default partition size provided for RHEL/Rocky is /boot: 1024MB, /boot/efi: 256MB and remaining space to / partition.

  • Default partition size provided for Ubuntu is /boot: 2148MB, /boot/efi: 1124MB and remaining space to / partition.

  • Values are accepted in the form of JSON list such as: , - { mount_point: “/home”, desired_capacity: “102400” }

Default values: - { mount_point: "", desired_capacity: "" }

timezone

string

Required

Timezone to be used during OS provisioning. Available timezones are provided here.

Choices:

  • GMT <- default

  • EST

  • CET

  • MST

  • CST6CDT

  • PST8PDT

language

string

Required

Language to be used during OS provisioning.

Default values: en-US

default_lease_time

integer

Required

Default lease time for IPs assigned by DHCP. Range: 21600-86400

Default values: 86400

ubuntu_kernel_flavor

string

Optional

  • This is a user-configured variable that is used to specify the kernel flavor on Ubuntu 22.04.

  • This variable is only supported when the cluster runs Ubuntu 22.04.

  • To specify this variable, ensure to append the string ubuntu_kernel_flavor: generic or ubuntu_kernel_flavor: hwe to the end of the input/provision_config.yml file.

  • Accepted values: generic, hwe.
provision_config_credentials.yml

Parameter

Details

provision_password

string

Required

  • Password set for the root account of target nodes during provisioning.

  • Length >= 8 characters

  • Password must not contain -,, ‘,”

postgresdb_password

string

Required

  • Password set for the postgresDB on target nodes during provisioning.

  • Length >= 8 characters

  • Password must not contain -,, ‘,”

bmc_username

string

Required

  • The username set on target iDRACs.

  • Username must not contain -,, ‘,”

bmc_password

string

Required

  • The password set on target iDRACs.

  • The username must not contain -,, ‘,”

switch_snmp3_username

string

Optional

  • This variable is required when discovering nodes via switch details.

  • Non-admin SNMPv3 credentials of the PXE switch.

  • If multiple switches are provided, ensure the credentials are same across switches.

  • Username must not contain -,, ‘,”

switch_snmp3_password

string

Optional

  • This variable is required when discovering nodes via switch details.

  • Non-admin SNMPv3 credentials of the PXE switch.

  • If multiple switches are provided, ensure the credentials are same across switches.

  • Password must not contain -,, ‘,”

docker_username

string

Optional

  • Username for Dockerhub account used for Docker logins.

  • A kubernetes secret will be created and patched to the service account in default namespace.

  • This kubernetes secret can be used to pull images from private repositories.

docker_password

string

Optional

  • Password for Dockerhub account used for Docker logins.

  • This value is mandatory if docker_username is provided.
software_config.json

Parameter

Details

cluster_os_type

string

Required

  • The operating system running on the cluster (rhel, rocky, and ubuntu).

Default value: ubuntu.

cluster_os_version

string

Required

  • The OS Version that will be provisoned on compute nodes.

  • For RHEL, the accepted values are 8.6, 8.7, and 8.8.

  • For Rocky, the accepted values are 8.6, 8.7, and 8.8.

  • For Ubuntu, the accepted values are 20.04, 22.04.

  • Default value: 22.04

repo_config

string

Required

  • The type of offline configuration user needs.

  • When the value is set to always, Omnia creates a local repository/registry on the Control plane hosting all the packages/images required for the cluster.

  • When the value is set to partial, Omnia creates a local repository/registry on the Control plane hosting all the packages/images except those listed in the user_registry in input/local_repo_config.yml.

  • When the value is set to never, Omnia does not create a local repository/registry. All the packages/images are directly downloaded on the cluster.

Note

  • After local_repo.yml has run, the value of repo_config in input/software_config.json cannot be updated without running the control_plane_cleanup.yml script first.

  • Irrespective of the value of repo_config, all local repositories that are not available as images, debian packages, or RPMs will be downloaded and configured locally on the control plane. Additionally, AMD GPU drivers, ROCm drivers, CUDA, and OFED are downloaded by default.

  • Accepted values:

    • always

    • partial <- Default

    • never

softwares

JSON list

Required

  • A JSON list of required software and (optionally) the software revision.

  • The following software should be listed with a version in the list: BeeGFS, AMD GPU, Kubernetes, CUDA, OFED, BCM RoCE, UCX, and ROCm.

  • A minimum of one software should be provided in the list for local_repo.yml to execute correctly.

Note

The accepted names for software is taken from input/config/<cluster_os_type>/<cluster_os_version>/.

Update the input/network_spec.yml file for all networks available for use by the control plane.

  • The following admin_nic details are mandatory:

    • nic_name: The name of the NIC on which the administrative network is accessible to the control plane.

    • netmask_bits: The 32-bit “mask” used to divide an IP address into subnets and specify the network’s available hosts.

    • static_range: The static range of IPs to be provisioned on target nodes.

    • dynamic_range: The dynamic range of IPs to be provisioned on target nodes.

    • correlation_to_admin: Boolean value used to indicate whether all other networks specified in the file (eg: bmc_network) should be correlated to the admin network. For eg: if a target node is assigned the IP xx.yy.0.5 on the admin network, it will be assigned the IP aa.bb.0.5 on the BMC network. This value is irrelevant when discovering nodes using a mapping file.

    • admin_uncorrelated_node_start_ip: If correlation_to_admin is set to true but correlated IPs are not available on non-admin networks, provide an IP within the static_range of the admin network that can be used to assign admin static IPs to uncorrelated nodes. If this is empty, then the first IP in the static_range of the admin network is taken by default. This value is irrelevant when discovering nodes using a mapping file.

    • CIDR: Classless or Classless Inter-Domain Routing (CIDR) addresses use variable length subnet masking (VLSM) to alter the ratio between the network and host address bits in an IP address.

    • MTU: Maximum transmission unit (MTU) is a measurement in bytes of the largest data packets that an Internet-connected device can accept.

    • DNS: A DNS server is a computer equipped with a database that stores the public IP addresses linked to the domain names of websites, enabling users to reach websites using their IP addresses.

    • VLAN: A 12-bit field that identifies a virtual LAN (VLAN) and specifies the VLAN that an Ethernet frame belongs to. This value is not supported on admin and bmc networks.

  • If the nic_name is the same on both the admin_network and the bmc_network, a LOM setup is assumed.

  • BMC network details are not required when target nodes are discovered using a mapping file.

  • If bmc_network properties are provided, target nodes will be discovered using the BMC method in addition to the methods whose details are explicitly provided in provision_config.yml.

Caution

  • Do not assign the subnet 10.4.0.0/24 to any interfaces in the network as nerdctl uses it by default.

  • If a DNS server is available on the network, ensure that the ranges provided in the input/network_spec.yml file do not include the IP ranges of the DNS server.

  • All provided network ranges and nic IP addresses should be distinct with no overlap in the input/network_spec.yml.

A sample is provided below:

---
    Networks:
    - admin_network:
        nic_name: "eno1"
        netmask_bits: "16"
        static_range: "10.5.0.1-10.5.0.200"
        dynamic_range: "10.5.1.1-10.5.1.200"
        correlation_to_admin: true
        admin_uncorrelated_node_start_ip: "10.5.0.50"
        network_gateway: ""
        DNS: ""
        MTU: "1500"

    - bmc_network:
        nic_name: ""
        netmask_bits: ""
        static_range: ""
        dynamic_range: ""
        reassignment_to_static: true
        discover_ranges: ""
        network_gateway: ""
        MTU: "1500"

Note

  • The input/provision_config_credentials.yml file is encrypted on the first run of the provision tool:

    To view the encrypted parameters:

    ansible-vault view provision_config_credentials.yml --vault-password-file .provision_vault_key

    To edit the encrypted parameters:

    ansible-vault edit provision_config_credentials.yml --vault-password-file .provision_vault_key

  • The strings admin_network and bmc_network in the input/network_spec.yml file should not be edited. Also, the properties nic_name, static_range, and dynamic_range cannot be edited on subsequent runs of the provision tool.

  • Netmask bits are mandatory and should be same for both the admin_network and bmc_network (ie between 1 and 32; 1 and 32 are acceptable values).

  • Ensure that the CIDR is aligned with the netmask_bits provided.

  • The discover_ranges property of the bmc_network can accept multiple comma-separated ranges.

  • The VLAN property is optional but should be between 0 and 4095 (0 and 4095 are not acceptable values).

If you have any feedback about Omnia documentation, please reach out at omnia.readme@dell.com.