Running on Bigben

From ScorecWiki

Jump to: navigation, search

Contents

Overview

This page describes some of the steps needed to checkout PHASTA and run it on Bigben. Bigben is a Cray XT3 MPP system with 2068 compute nodes linked by a custom-designed interconnect.

Setting your account for BASH

  1. Login into Bigben by ssh username@tg-login.bigben.psc.teragrid.org
  2. Copy bigben_scripts.tgz into your home directory. This file contains the following files:
    • bash_profile
    • bashrc
    • svn_phasta.sh
    • make_phasta.sh
    • make_phasta_dbg.sh
    • pgi7.1_evn.sh
    • bigben-phSolverIC_rerun.sh
    • bigben-phSolverIC.sh
  3. Decompress bigben_scripts.tgz by tar xzvf bigben_scripts.tgz
  4. Copy bashrc from bigben_scripts directory into your home directory and change the name to .bashrc
  5. Copy bash_profile from bigben_scripts directory into your home directory and change the name to .bash_profile
  6. source the bash files.

Installing PHASTA

  1. Create directory develop.
  2. Move to develop.
  3. Copy pgi7.1_evn.sh from bigben_scripts and run it. This script sets up the environment for pgi.
  4. Copy svn_phasta.sh from bigben_scripts and run it. You will need to enter your gforge username and password the first time you use it. This script will checkout PHASTA from the top of the trunk.
  5. If you are running the incompressible code, before compiling PHASTA you will need to get the LIBLES directory into this directory, which contains the ACUSIM libraries. If you are running the compressible code, you can continue to the next step.
  6. Copy make_phasta_dbg.sh or/and make_phasta.sh from bigben_scripts and run it. The first script will compile PHASTA for debugging and the second script for production runs. Once the script is finish, you should have an executable build.
    • NOTE: You might see warning information at the end of compiling, like In function `Partition_Problem__FiPcT2T1': warning: warning: system is not implemented and will always fail. These warning message are typical as catamount on Cray XT3 is a reduced kernel. In general runs these would not cause problems.
    • NOTE: These files are set default for the incompressible code. For the compressible code you will need to remove the comment from
export COMPRESSIBLE=1

Running PHASTA

  1. create a directory Runs in your home directory (only for small tests). /usr/users/n/username is your home directory, which has a 5-Gbyte quota. $SCRATCH is bigben's scratch area to be used as a working space for your running jobs. There are no quotas for $SCRATCH, so you must police your own disk usage. However, $SCRATCH files might be deleted if more space are needed to keep jobs running.
  2. go to directory Runs. Copy files to run phasta (solver.inp, input.config, #_procs-case) into Runs. Also copy script file bigben-phSolverIC_rerun.sh or/and bigben-phSolverIC.sh to "Runs"
  3. submit job by qsub bigben-phSolverIC_rerun.sh
  4. to check the job status qstat
  5. to delete job qdel jobid

Useful links

bigben user's guide

Personal tools