Last updated on 6th December 2022.Â
At present, this is how we can set up our environment and use the runs efficiently
Param Pravega's webpage is a good place to start : https://www.serc.iisc.ac.in/supercomputer/for-traditional-hpc-simulations-param-pravega/
If you want to create your own environment, the following steps might be useful.
Module load the spack environmentÂ
The spack environment in PARAM (use which spack to know your current SPACK_ROOT environment) needs to be loaded.
module load spack/0.17
Download spack from git  Â
Move to a scratch space where you want to set up your spack environment and clone from spack's git repository. Use your username at PARAM in place of <username> in the following commands.
cd /scratch/<username>/
git clone https://github.com/spack/spack.git
Chain the spack installation
You can point your Spack installation to another installation, in order to use any packages that are installed there. To register the other Spack instance, you can add it as an entry to upstreams.yaml (located in spack/etc/spack/).
vi /scratch/<username>/spack/etc/spack/defaults/upstreams.yaml
Add the following text in the .yaml file.
upstreams:
  spack-instance-1:
    install_tree: /home-ext/apps/spack/opt/spack/
    modules:
          tcl: /home-ext/apps/spack/share/spack/modules
Provide the compiler configuration in compilers.yaml file
Find out the path where the parent spack installation exists.
which spack
$ /home-ext/apps/spack/bin/spack
      Copy the compilers.yaml file to the spack/etc/spack folder.Â
cp /home-ext/apps/spack/etc/spack/compilers.yaml /scratch/<username>/spack/etc/spack/
Add the location of your spack installation to bashrc
export SPACK_ROOT=/scratch/<username>/spack
. $SPACK_ROOT/share/spack/setup-env.sh
(NOTE: Don't forget the . in the beginning of above line)
Check if the SPACK_ROOT was set correctly
Source your bashrc file or re-login.
Then, check if the path of your installation has been set as SPACK_ROOT using:
$ /scratch/<username>/spack/bin/spack
Load necessary packages
Here's a list of spack loads one can use for running PLUTO.
spack load intel-oneapi-compilers@2021.4.0
spack load hwloc@2.7.0%intel@2021.4.0
spack load zlib@1.2.11%intel@2021.4.0
spack load gsl@2.7%intel@2021.4.0
spack load gmp@6.2.1%intel@2021.4.0
spack load fftw@3.3.10%intel@2021.4.0/4s24afk
spack load hdf5@1.10.8%intel@2021.4.0/ssgjscn
spack load cmake@3.22.1/ozzmsd4
(NOTE: You can add these to your bashrc too, or do it on every login)
Check the LD_LIBRARY_PATH
With recent updates on develop branch of spack, the paths with spack load are not added automatically to LD_LIBRARY_PATHS.
You will not see anything by echo $LD_LIBRARY_PATH if you spack loaded the packages.
But, during compiling, one would need the spack loaded package paths in LIBRARY_PATH. To do this use these commands once (you can unset the behavior using spack config remove later, if required).
spack config add modules:prefix_inspections:lib64:[LIBRARY_PATH]
spack config add modules:prefix_inspections:lib:[LIBRARY_PATH]
 Now, you would see the LD_LIBRARY_PATH to be set correctly. Add these to your ~/.bashrc now:
export LIBRARY_PATH=$LD_LIBRARY_PATH
export CPATH=/home-ext/apps/spack/opt/spack/linux-centos7-cascadelake/intel-2021.4.0/hdf5-1.10.8-ssgjscncg3xcw7ogodzjg3whjdmvx3bg/include:$CPATH
       (NOTE: the include directory is confirmed in the location given by which h5ls for its bin and then added. It's only important for people working       with  hdf5 data dumps)
Look where you are running!
Make sure that the file I/O of PLUTO is done in the scratch space (/scratch/<username>). Otherwise parallel I/O takes too much time!
    Voila! You are ready for the run!
Quick run to check you have reasonable run times!
Click Here
In-situ visualisation with ParaView Catalyst on your PLUTO runs at Param Pravega
BACK TO BLOGS