...
| Code Block | ||
|---|---|---|
| ||
#!/bin/bash
# edit the folloiwing variables
# assuming webdav accesses
WEBDAVIP="xxxx"
WEBDAVID="xxxx"
WEBDAVPW="xxxx"
# array of filenames that will be downloaded and saved
SRCFNARR=("/root/.ssh/id_rsa.pub")
DESTFNARR=("cluster_master_id_rsa.pub")
CNT=0
for SRCFN in ${SRCFNARR[@]}
do
DESTFN=${DESTFNARR[$CNT]}
curl --insecure -u ${WEBDAVID}:${WEBDAVPW} -T ${SRCFN} https://${WEBDAVIP}/home/${DESTFN}
CNT=$((CNT+1))
done |
Running your own task queue
Without Slurm queue, you can still have your own queue when you use the environment for the GNU parallel, i.e., password-less ssh logins among the cluster nodes. It is recommended to use a simple task queue introduced in https://github.com/guo-yong-zhi/DistributedTaskQueue, which requires only passoword-less ssh connection and Python3 in the cluster master and worker nodes.