...
| Code Block |
|---|
|
#!/bin/bash
CLUSTERNAME="mycluster"
TMPDIR="/tmp"
CONDAENV="xclass"
CONDAURL="https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh"
# additional apt packages
apt install zip
# installation of miniconda
cd ${TMPDIR}
wget "${CONDAURL}" -O ./miniconda.sh
bash ./miniconda.sh -b -p ${HOME}/miniconda
eval "$(${HOME}/miniconda/bin/conda shell.bash hook)"
conda init
conda update -y -n base -c defaults conda
# creating the environment
conda create -y -n ${CONDAENV} python=2.7
# adding new conda packages
conda install -y -n ${CONDAENV} numpy
conda install -y -n ${CONDAENV} scipy
conda install -y -n ${CONDAENV} matplotlib
conda install -y -n ${CONDAENV} astropy
conda install -y -n ${CONDAENV} sqlite
# adding pip packages
conda activate ${CONDAENV}
pip install pyfits
echo "Do the following things to use the environment ${CONDAENV}"
echo "1) source ~/.bashrc"
echo "2) conda activate ${CONDAENV}" |
Downloading/uploading files from/to the external storage
The following script show how to download files from the external storage, which is described in KASI Science Cloud : VM instances#Step4.ConfiguretheVMinstanceforremotedesktop&externaldatastore, in the the network-shared volume. A typical usage includes downloading compiled binary codes, related scripts, configuration files, and data from the external storage. The script is available at https://github.com/astromsshin/cloud_ex/blob/main/tool_download_from_external_storage.sh.
| Code Block |
|---|
|
#!/bin/bash
# edit the folloiwing variables
TARGETDIR="/mnt/mpi"
# assuming webdav accesses
WEBDAVIP="xxxx"
WEBDAVID="xxxx"
WEBDAVPW="xxxx"
# array of filenames that will be downloaded and saved
SRCFNARR=("XCLASS.zip" "ins_custom.sh")
DESTFNARR=("XCLASS.zip" "ins_custom.sh")
cd ${TARGETDIR}
CNT=0
for SRCFN in ${SRCFNARR[@]}
do
DESTFN=${DESTFNARR[$CNT]}
wget -O ${DESTFN} --no-check-certificate -r -c --user ${WEBDAVID} --password ${WEBDAVPW} https://${WEBDAVIP}/home/${SRCFN}
CNT=$((CNT+1))
done |
You can also upload files from the cluster to the external storage as shown in the following example script (https://github.com/astromsshin/cloud_ex/blob/main/tool_upload_to_external_storage.sh).
| Code Block |
|---|
|
#!/bin/bash
# edit the folloiwing variables
# assuming webdav accesses
WEBDAVIP="xxxx"
WEBDAVID="xxxx"
WEBDAVPW="xxxx"
# array of filenames that will be downloaded and saved
SRCFNARR=("/root/.ssh/id_rsa.pub")
DESTFNARR=("cluster_master_id_rsa.pub")
CNT=0
for SRCFN in ${SRCFNARR[@]}
do
DESTFN=${DESTFNARR[$CNT]}
curl --insecure -u ${WEBDAVID}:${WEBDAVPW} -T ${SRCFN} https://${WEBDAVIP}/home/${DESTFN}
CNT=$((CNT+1))
done |