Install Lumen on Leonardo
From Lumen wiki
Below a list of configure scripts to compile on Leonardo with different compilers and versions.
Known issues.
- For the GPU ported slepc 3.22 is the latest version compiling. See also here: https://gitlab.com/lumen-code/lumen/-/work_items/275
Leonardo (GPU version)
nvfortran 23.11
Warning, with slepc 3.20 gpu porting is not good
module load nvhpc/23.11 module load openmpi/4.1.6--nvhpc--23.11 module load libxc/5.2.3--nvhpc--23.11-bwqd5we module load fftw/3.3.10--openmpi--4.1.6--nvhpc--23.11 module load hdf5/1.14.3--openmpi--4.1.6--nvhpc--23.11 module load netcdf-c/4.9.2--openmpi--4.1.6--nvhpc--23.11 module load netcdf-fortran/4.6.1--openmpi--4.1.6--nvhpc--23.11 module load openblas/0.3.24--nvhpc--23.11 module load netlib-scalapack/2.2.0--openmpi--4.1.6--nvhpc--23.11 module load petsc/3.20.1--openmpi--4.1.6--nvhpc--23.11-complex-6q7sfss module load slepc/3.20.0--openmpi--4.1.6--nvhpc--23.11-lqtaegm export FC=nvfortran export F77=nvfortran export CC=nvc export CPP="cpp -E" export FPP="nvfortran -Mpreprocess -E" export F90SUFFIX=".f90" ./configure \ --with-cuda-cc=80, --with-cuda-runtime=12.3 \ --enable-mpi --enable-open-mp \ --enable-msgs-comps --enable-time-profile --enable-memory-profile \ --enable-par-linalg \ --with-blas-libs="-L$OPENBLAS_LIB -lopenblas" \ --with-lapack-libs="-L$OPENBLAS_LIB -lopenblas" \ --with-hdf5-path=$HDF5_HOME \ --with-netcdf-path=$NETCDF_C_HOME \ --with-netcdff-path=$NETCDF_FORTRAN_HOME \ --with-libxc-path=$LIBXC_HOME \ --with-fft-path=$FFTW_HOME \ --with-scalapack-libs=$NETLIB_SCALAPACK_LIB/libscalapack.so \ --with-blacs-libs=$NETLIB_SCALAPACK_LIB/libscalapack.so \ --enable-slepc-linalg \ --with-petsc-path=$PETSC_HOME \ --with-slepc-path=$SLEPC_HOME \ --enable-cuda-fortran --with-elpa-libs="yes" \ --with-gpu-incs="-I$NVHPC_ROOT/cuda/include" \ --with-elpa-libs="-L$ELPA_LIB -lelpa" \ --with-elpa-includedir="$ELPA_INC/elpa-2024.03.001" \
nvfortran 24.9
Slepc 3.22 gpu porting is ok. Also, this is the latest version which can be compiled with nvfortran This receipt was created using the modules by Nicola Spallanzani. It might require to enable access these modules. Explicit elpa linking command might need to be fixed.
module load /leonardo_work/Max3_devel_2/nspallan/opt/modulefiles/nvhpc-nompi/24.9 module use /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/openmpi/4.1.5--nvhpc--24.9-qdinnmz module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/devicexlib/develop--nvhpc--24.9-cuda-11.8-cuf-h5kxqe6 module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/libxc/5.2.3--nvhpc--24.9-3rkgqwj module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/fftw/3.3.10--openmpi--4.1.5--nvhpc--24.9-fwl5ne2 module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/hdf5/1.14.3--openmpi--4.1.5--nvhpc--24.9-dm3eloh module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/netcdf-c/4.9.2--openmpi--4.1.5--nvhpc--24.9-k5dclkn module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/netcdf-fortran/4.6.1--openmpi--4.1.5--nvhpc--24.9-4n4inng module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/openblas/0.3.24--nvhpc--24.9-2ewnkxs module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/netlib-scalapack/2.2.0--openmpi--4.1.5--nvhpc--24.9-uuvfbc5 module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/elpa/2024.03.001--openmpi--4.1.5--nvhpc--24.9-cuda-11.8 module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/petsc/3.22.1--openmpi--4.1.5--nvhpc--24.9-cuda-11.8-complex module load /leonardo/pub/userexternal/nspallan/spack-0.21.0-5.2/modules/slepc/3.22.1--openmpi--4.1.5--nvhpc--24.9-cuda-11.8 export FC=nvfortran export F77=nvfortran export CC=nvcc export CPP="cpp -E" export FPP="nvfortran -Mpreprocess -E" export F90SUFFIX=".f90" ./configure \ --with-cuda-cc=80 --with-cuda-runtime=12.3 \ #11.8?? --enable-mpi --enable-open-mp \ --enable-msgs-comps --enable-time-profile --enable-memory-profile \ --enable-par-linalg \ --with-blas-libs="-L$OPENBLAS_LIB -lopenblas" \ --with-lapack-libs="-L$OPENBLAS_LIB -lopenblas" \ --with-hdf5-path=$HDF5_HOME \ --with-netcdf-path=$NETCDF_C_HOME \ --with-netcdff-path=$NETCDF_FORTRAN_HOME \ --with-libxc-path=$LIBXC_HOME \ --with-fft-path=$FFTW_HOME \ --with-scalapack-libs=$NETLIB_SCALAPACK_LIB/libscalapack.so \ --with-blacs-libs=$NETLIB_SCALAPACK_LIB/libscalapack.so \ --enable-slepc-linalg \ --with-petsc-path=$PETSC_HOME \ --with-slepc-path=$SLEPC_HOME \ --enable-cuda-fortran \ --with-elpa-libs="yes" \ --with-gpu-incs="-I$NVHPC_ROOT/cuda/include" \ --with-elpa-libs="-L$ELPA_LIB -lelpa" \ --with-elpa-includedir="$ELPA_INC/elpa-2024.03.001"
Leonardo (CPU version)
gcc 12.2
#!/bin/bash module load gcc/12.2.0-binut2.41 module load openmpi/4.1.6--gcc--12.2.0 module load fftw/3.3.10--openmpi--4.1.6--gcc--12.2.0 module load hdf5/1.14.3--openmpi--4.1.6--gcc--12.2.0 module load netcdf-fortran/4.6.1--openmpi--4.1.6--gcc--12.2.0 module load netcdf-c/4.9.2--openmpi--4.1.6--gcc--12.2.0 module load openblas/0.3.24--gcc--12.2.0 module load netlib-scalapack/2.2.0--openmpi--4.1.6--gcc--12.2.0 export FC=gfortran export F77=gfortran export CC=gcc export CPP="cpp -E" export F90SUFFIX=".f90" YAMBO_LIBS="/leonardo/home/userexternal/username/external_libs" FCFLAGS="-O3 -g -mtune=native -fallow-argument-mismatch" ./configure \ --enable-mpi --enable-open-mp \ --enable-keep-src \ --enable-msgs-comps --enable-time-profile --enable-memory-profile \ --enable-par-linalg \ --enable-slepc-linalg \ --with-blas-libs="-L$OPENBLAS_LIB -lopenblas" \ --with-lapack-libs="-L$OPENBLAS_LIB -lopenblas" \ --with-fft-path=$FFTW_HOME \ --with-scalapack-libs=$NETLIB_SCALAPACK_LIB/libscalapack.so \ --with-blacs-libs=$NETLIB_SCALAPACK_LIB/libscalapack.so \ --with-hdf5-path=$HDF5_HOME \ --with-netcdf-path=$NETCDF_C_HOME \ --with-netcdff-path=$NETCDF_FORTRAN_HOME \ --with-extlibs-path="$YAMBO_LIBS"
