4.2 Mathematical libraries

Very short notes for mathematical libraries available in Sisu:

4.2.1 Cray LibSci

module:cray-libsci   (module is loaded by default)

Available for PrgEnv-cray, PrgEnv-gnu and PrgEnv-intel (module is loaded by default).

Man page: man intro_libsci


The general components of Cray LibSci are:

  • BLAS
  • BLACS
  • LAPACK
  • ScaLAPACK
  • FFT
  • FFTW
  • CRAFFT
  • CASE
  • IRT

Compiling and linking

statically linked executable:

ftn -o my_exe my_code.f95

dynamically linked executable:

ftn -o my_exe my_code.f95 -dynamic

When you run an MPI/OpenMP (threaded) application there are environment variables for threading control. Please read carefully chapter 4.4 Shared memory and hybrid parallelization ( PrgEnv-intel examples will show how to use Intel specific thread control environment variables).

4.2.2 fftw3

module: module load fftw

Available for PrgEnv-crayPrgEnv-gnu and PrgEnv-intel

Man page: man intro_fftw3
 

Compiling and linking

Statically linked executable:

ftn -o my_exe fftw3_code.f90
cc -o my_code fftw3_code.c
dynamically linked executable:
ftn -o my_exe fftw3_code.f90 -dynamic
cc -o my_code fftw3_code.c -dynamic

 


4.2.3 fftw2

module: module load  fftw/2.1.5.9

Available for PrgEnv-crayPrgEnv-gnu and PrgEnv-intel

Man page: man intro_fftw2
 

Compiling and linking

Statically linked executable:

ftn -o my_exe fftw2_code.f90 

(User must specify which libraries to link against, see man intro_fftw2)

cc -o my_code fftw2_code.c

(User must specify which libraries to link against, see man intro_fftw2)

Dynamically linked executable:

ftn -o my_exe fftw2_code.f90 -dynamic  

(User must specify which libraries to link against, see man intro_fftw2)

cc -o my_code fftw2_code.c  -dynamic 

(User must specify which libraries to link against, see man intro_fftw2)


4.2.4 PETSc

module (for real data): module load cray-petsc  ( loads also cray-tpsl, that has libraries: MUMPS, SuperLU, SuperLU_dist, ParMETIS, HYPRE, SUNDIALS and Scotch ). Please load also module cray-libsci if not loaded.

module (for complex data): module load cray-petsc-complex. Please load also module cray-libsci if not loaded.

Available for PrgEnv-crayPrgEnv-gnu and PrgEnv-intel

Man page: man intro_petsc
 

Compiling and linking

Statically linked executable:

ftn -o my_exe petsc_code.F
cc -o my_exe  petsc_code.c
dynamically linked executable:
ftn -o my_exe petsc_code.F -dynamic
cc -o my_exe  petsc_code.c -dynamic

 

4.2.5 Trilinos


module: module load cray-trilinos  ( loads also cray-tpsl, that has libraries: MUMPS, SuperLU, SuperLU_dist, ParMETIS, HYPRE, SUNDIALS and Scotch ). Please load also module cray-libsci if not loaded.

Available for PrgEnv-cray, PrgEnv-gnu and PrgEnv-intel.

man page: man intro_trilinos

Compiling and linking

statically linked executable:

CC -o my_exe trilinos_code.cpp

 

4.2.6 Intel MKL

Intel MKL library is not fully supported on Sisu (perhaps Cray will provide it later).

There is not any MKL module. Easiest way to use it is to load PrgEnv-intel and unload cray-libsci

User's Guide and Reference manual

Intel MKL is a mathematical library collection that is optimized for Intel processors. On Sisu, MKL can be used mainly with Intel compiler for Fortran and C/C++ programming. Please note that cluster libraries (BLACS, ScaLAPACK and Cluster FFT functions) are not compiled with Cray MPI libraries (but Cray MPI and IntelMPI both are MPICH2 libraries. IntelMPI is not available on Sisu).

MKL includes the following groups of routines:

  • BLAS (Basic Linear Algebra Subprograms)
  • Sparse BLAS
  • LAPACK (Linear Algebra PACKage)
  • PBLAS (Parallel Basic Linear Algebra Subprograms)
  • BLACS (Basic Linear Algebra Communication Subprograms)
  • ScaLAPACK (Scalable LAPACK)
  • Sparse Solver routines (direct sparse solver PARDISO, direct sparse solver DSS, iterative sparse solvers RCI, preconditioners for iterative solution process)
  • Vector Mathematical Functions (VML, arithmetic, power, trigonometric, exponential, hyperbolic, special, and rounding)
  • Vector Statistical Functions (VSL, random numbers, convolution and correlation, statistical estimates)
  • General Fast Fourier Transform (FFT) Functions
  • Cluster FFT functions
  • Partial Differential Equations (PDE) support tools (Trigonometric Transform routines, Poisson routines)
  • Nonlinear least squares problem solver routines
  • Data Fitting functions (spline-based)
  • Support Functions (timing, thread control, memory management, error handling, numerical reproducibility)


MKL library has two integer interfaces 32-bit (they call it: LP64) and 64-bit integer (ILP64) interfaces. So if you work with data arrays that have more than 2^31-1 elements ILP64 interface is for you.

MKL supports sequential and threaded programming modes. Intel MKL is based on the OpenMP threading. See: Threaded MKL Functions and Problems for more information.
 

Compiling and linking

For quick linking Intel compiler supports variants of the -mkl compiler option. The compiler links your application using the LP64 interface and it does not use Intel MKL Fortran module 95 interfaces.

-mkl or -mkl=parallel to link with threaded MKL
-mkl=sequential to link with sequential MKL
-mkl=cluster to link with cluster libraries that use IntelMPI (MPICH2, there is no guarantee that this cluster option will work on Sisu. BUT it can work very well)


dynamically linked executables (if cray-libsci is loaded please unload cray-libsci before compiling and linking):

cc -o my_exe mkl_code.c -mkl=sequential -dynamic
ftn -o my_exe mkl_code.95 -mkl=sequential -dynamic
cc -o my_exe mkl_code.c -mkl=parallel -dynamic
ftn -o my_exe mkl_code.95 -mkl=parallel -dynamic
Statically linked executable (if cray-libsci is loaded please unload cray-libsci before compiling and linking):

Because MKL applications are not designed to to be fully static you might get compiler/linker warnings with following examples.
cc -o my_exe mkl_code.c -mkl=sequential
ftn -o my_exe mkl_code.95 -mkl=sequential
cc -o my_exe mkl_code.c -mkl=parallel
ftn -o my_exe mkl_code.95 -mkl=parallel
Note that the -mkl option NEEDS TO APPEAR AT THE END OF THE LINE.
Because MKL includes many libraries and programming interfaces a link line can be a long list. To find a correct line for a case, specify your choices using Intel Math Kernel Library (MKL) Link Line Advisor tool:

http://software.intel.com/sites/products/mkl/

Copy the results (link line and compiler option results) into a Makefile. In command line compiling and linking case remove all brackets that the advisor gives (for example if there is a variable (MKLROOT) in brackets then remove the brackets.

 

When you run an MPI/OpenMP (threaded) application there are environment variables for threading control. Please read carefully chapter 4.4 Shared memory and hybrid parallelization ( PrgEnv-intel examples will show how to use Intel specific thread control environment variables)

    Previous chapter     One level up     Next chapter