The MISTIS Cluster

home  
Monitoring jobs: condor_q -- Displays which jobs are currently running, and which are waiting condor_q -r -- Displays which jobs are currently running, and on which machines condor_status -- Displays what the machines in the cluster are currently doing condor_status -available -- Displays the machines in the cluster that are currently available Managing jobs: condor_hold [cluster number] -- Places a job in the queue on hold condor_release [cluster number] -- Releases a job in the queue that is on hold Submitting jobs: condor_submit [submit file] -- Submits a jobs to the Condor pool Deleting jobs: condor_rm [cluster number] -- Removes a cluster of jobs from the queue condor_rm [cluster number].[process number] -- Removes a specific job from the queue baudelaire2 eric calvino jonathon connelly stephane conrad florence greene darren yeats senan ionesco eugen joyce alexandre toole laurent tremblay julie to do: flaubert marie-jose paul-valery jb xiaolong alex kailash * 2 vasil vargas matthieu maupassant lamiae We have built a Condor cluster from the pool of MISTIS machines. The cluster combines the distributed processing power of 30+ cores, and can be used to run many types of jobs such as Matlab, R and C/C++. Users can submit jobs to the cluster, which in turn sends these jobs out to idle computers when they become available.
  To add your machine to the cluster, just copy and paste the code below into a terminal. You have to be root.  
  Mount the MISTIS network directory, if it is not already mounted: 
cat <<-END >>/etc/fstab
ral-nas2:/vol/ral_projets/mistis /commun/mistis nfs rw,defaults
END
mkdir -p /commun/mistis
mount /commun/mistis
Set environment variables: 
cat <<-END >>/root/.bashrc
CONDOR_CONFIG=/commun/mistis/cluster/condor_config
export CONDOR_CONFIG
END

cat <<-END >>/etc/profile
CONDOR_CONFIG=/commun/mistis/cluster/condor_config
export CONDOR_CONFIG
END

CONDOR_CONFIG=/commun/mistis/cluster/condor_config
export CONDOR_CONFIG
Create local directory structure: 
mkdir -p /usr/local/cluster
/bin/cp /commun/mistis/cluster/condor_config.local /usr/local/cluster/
Install (*)
yum install -y condor
mv /etc/condor/condor_config /etc/condor/condor_config.old
condor_init
Add to boot sequence:
/bin/cp /commun/mistis/cluster/condor.init /etc/init.d/condor 
/sbin/chkconfig --add condor
/sbin/chkconfig --level 235 condor on
Launch cluster processes: 
condor_master
and check their status with 
ps -ef | egrep condor_
which should produce an output similar to 
condor    8475     1  0 Jun17 ?        00:01:23 condor_master -f
condor   15894     1  0 Jun17 ?        00:00:00 condor_collector
condor   26624  8475  0 Jun18 ?        00:00:07 condor_collector -f
condor   26625  8475  0 Jun18 ?        00:00:17 condor_negotiator -f
condor   26626  8475  0 Jun18 ?        00:00:00 condor_schedd -f
condor   26627  8475  0 Jun18 ?        00:00:49 condor_startd -f
Finally, check that your machine has been added to the cluster by typing: 
condor_status
You should see something similar to the following: 
Name               OpSys      Arch   State     Activity LoadAv Mem   ActvtyTime

slot1@conrad.inria LINUX      INTEL  Unclaimed Idle     0.000  1012  0+00:45:04
slot2@conrad.inria LINUX      INTEL  Unclaimed Idle     0.000  1012  0+00:45:05
slot1@yeats.inrial LINUX      INTEL  Unclaimed Idle     0.040  1011  0+01:30:04
slot2@yeats.inrial LINUX      INTEL  Unclaimed Idle     0.000  1011  0+01:30:05
slot1@toole.inrial LINUX      X86_64 Owner     Idle     0.070   987  0+00:10:08
slot2@toole.inrial LINUX      X86_64 Owner     Idle     0.000   987  0+00:10:09
slot3@toole.inrial LINUX      X86_64 Owner     Idle     0.000   987  0+00:10:10
slot4@toole.inrial LINUX      X86_64 Owner     Idle     0.000   987  0+00:10:11

                     Total Owner Claimed Unclaimed Matched Preempting Backfill

         INTEL/LINUX     4     0       0         4       0          0        0
        X86_64/LINUX     4     4       0         0       0          0        0

               Total     8     4       0         4       0          0        0
(*) Older versions of fedora do not ship with condor, and so you need to do the following to install: 
cd /etc/yum.repos.d
wget http://www.cs.wisc.edu/condor/yum/repo.d/condor-stable-rhel5.repo
yum install -y condor
mv /etc/condor/condor_config /etc/condor/condor_config.old
export PATH=/usr/sbin:$PATH
condor_init
 
  For more information, see the condor website and this document