This page explains how to setup Gluster network filesystems that can be mounted by multiple virtual machines, in particular, MPI(or GNU/parallel) cluster nodes.
If you have questions and suggestions about this tutorial page and related problems, please, contact Min-Su Shin.
Gluster filesystem can be prepared with a single server and its disk space to serve multiple clients. In this case A, we assume that a single Gluster server is made to be used by multiple VMs (virtual machines) included in a cluster.
Following KASI Science Cloud : VM instances#Step3.CreateaVMinstance, create a single VM with a VM flavor which fits to your requirement such as the required size of the network filesystem. The created single VM will work as a server serving the Gluster filesystem. Since the required memory for the Gluster server is not huge, the recommended flavor is large-CPU + small-memory for a given size of disk.
On the created VMs, we install the Gluster server program and configure a Gluster filesystem. For your convenience, you can find a shell script that should be executed by root account on the master node. See https://github.com/astromsshin/cloud_ex and the script https://github.com/astromsshin/cloud_ex/blob/main/tool_setup_glusterfs_single_server.sh which is presented below:
#!/bin/bash
# Directory serving the Gluster filesystem
TARGETDIR="/glusterfs/vol"
# Gluster filesystem volume available for clients
TARGETVOL="gvol"
# Name of the Gluster server
VMNAME="test-gl-vm"
# Install the server
apt install glusterfs-server -y
systemctl enable --now glusterd
# Information
# (optional)
#gluster --version
#systemctl status glusterd
# Prepare the directory
mkdir -p ${TARGETDIR}
# Produce the Gluster filesystem volume with the above directory
gluster volume create ${TARGETVOL} ${VMNAME}:${TARGETDIR} force
# Make the volume available
gluster volume start ${TARGETVOL}
# Information
# (optional)
#gluster volume info
#gluster volume status |
The created network filesystem can be mounted on client VMs by using mount command with required options. A simple script is available as https://github.com/astromsshin/cloud_ex/blob/main/tool_setup_glusterfs_client_all_nodes-single_server.sh in the github repository https://github.com/astromsshin/cloud_ex. The script, which needs to be executed on the client cluster master node, is given below:
#!/bin/bash
# Name of the cluster which defines the clients' names
CLUSTERNAME="ml-image"
# Last integer index of the minions in the cluster
MINIONLASTIND="8"
# IP of the Gluster server
GLUSTERSERVERIP="10.0.100.150"
# Name of the Gluster server
GLUSTERSERVERNAME="test-gl-vm"
# Directory name which is a mount point on clients
TARGETDIR="/mnt/gluster"
RUNCMD="apt -y install glusterfs-client; echo ${GLUSTERSERVERIP} ${GLUSTERSERVERNAME} >> /etc/hosts;mkdir ${TARGETDIR}; mount -t glusterfs ${GLUSTERSERVERNAME}:/gvol ${TARGETDIR}; chmod -R 777 ${TARGETDIR}"
echo "... setuping on ${CLUSTERNAME}-master"
echo $RUNCMD | bash
for ind in $(seq 0 ${MINIONLASTIND})
do
echo "... setuping on ${CLUSTERNAME}-minion-${ind}"
ssh ${CLUSTERNAME}-minion-${ind} "${RUNCMD}"
done
|
In the above script, we assume that the filesystem is mounted on multiple VMs created as explained in Deploy Message Passing Interface (MPI) (and/or GNU parallel) cluster.
We explain how to setup the Gluster filesystem in the form of distributed volumes by using multiple Gluster servers. As depicted in the following figure and explained in https://docs.gluster.org/en/latest/Administrator-Guide/Setting-Up-Volumes/,
there is a type of distrbiuted volume where files are distributed in multiple gluster servers.
This step is the same as the step 1 of the Case A as explained above.
The configuration of the Gluster VMs can be done with the provided script https://github.com/astromsshin/cloud_ex/blob/main/tool_setup_glusterfs_multiple_servers.sh on the github repository https://github.com/astromsshin/cloud_ex. The script is given below:
The configuration of the Gluster firesystem affects how the client VMs mount and access the Gluster filesystem. The following script, which is available on the github repository https://github.com/astromsshin/cloud_ex as https://raw.githubusercontent.com/astromsshin/cloud_ex/main/tool_setup_glusterfs_client_all_nodes-multiple_servers.sh.