Oculus-Software-VASP

Aus PC2 Doc
Dies ist die bestätigte sowie die neueste Version dieser Seite.
Zur Navigation springen Zur Suche springen

News[Bearbeiten]

We have recently noticed a memory leak in hybrid functional calculations (e.g. HSE) with VASP 5.4.4 when using Intel 19.0.3. The memory leak seems to occur in MPI-broadcast messages. The problem doesn't show up when compiled with Intel 19.0.4 or Intel 20.0.0.

Please Note[Bearbeiten]

Please note, that due to the licensing model of VASP, we can't provide a compiled version to every user. Similarly, we can't test VASP ourselves on our cluster but rely on user input. We will try to provide here a guide on how to compile VASP on our clusters. Instructions for Noctua can be found in the article Noctua-Software-VASP.

VASP 5.4.4 on Oculus[Bearbeiten]

  • module purge
  • module add default-environment
  • module add ps_xe_2019_04_compilers
  • cd VASP544
  • edit makefile.include
# Precompiler options
CPP_OPTIONS= -DHOST=\"Oculus\"\
             -DMPI -DMPI_BLOCK=8000 \
             -DscaLAPACK \
             -Davoidalloc \
             -Duse_bse_te \
             -Dtbdyn \
             -Duse_shmem \
             -DnoSTOPCAR

CPP        = fpp -f_com=no -free -w0  $*$(FUFFIX) $*$(SUFFIX) $(CPP_OPTIONS)

FC         = mpiifort
FCL        = mpiifort -mkl

FREE       = -free -names lowercase

FFLAGS     = -assume byterecl -heap-arrays -diag-disable:8291
OFLAG      = -O2 -xHOST -g
OFLAG_IN   = $(OFLAG)
DEBUG      = -O0

MKL_PATH   = $(MKLROOT)/lib/intel64
BLAS       =
LAPACK     =
BLACS      = -lmkl_blacs_intelmpi_lp64
SCALAPACK  = $(MKL_PATH)/libmkl_scalapack_lp64.a $(BLACS)


OBJECTS    = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o $(MKLROOT)/interfaces/fftw3xf/libfftw3xf_intel.a

INCS       =-I$(MKLROOT)/include/fftw

LLIBS      = $(SCALAPACK) $(LAPACK) $(BLAS)

OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o

# For what used to be vasp.5.lib
CPP_LIB    = $(CPP)
FC_LIB     = $(FC)
CC_LIB     = icc
CFLAGS_LIB = -O
FFLAGS_LIB = -O1
FREE_LIB   = $(FREE)

OBJECTS_LIB= linpack_double.o getshmem.o

# For the parser library
CXX_PARS   = icpc

LIBS       += parser
LLIBS      += -Lparser -lparser -lstdc++

# Normally no need to change this
SRCDIR     = ../../src
BINDIR     = ../../bin

  • rm -rf build; mkdir build
  • make
  • you should now have vasp_gam, vasp_ncl, and vasp_std in the directory bin/
  • you can run it with the template jobscript
#! /bin/bash
#CCS --stdout=vasp.out
#CCS --stderr=vasp.err
#CCS --name VASP_Job
#CCS --res=rset=1:ncpus=16:mpiprocs=16:ompthreads=1 
#CCS -t 1h

module purge
module add default-environment
module add intel/19.0.4_compilers

VASPDIR=!!SET DIRECTORY NAME HERE!!
export I_MPI_DEBUG=4
mpirun -genvall -machinefile $CCS_NODEFILE $VASPDIR/bin/vasp_std


  • you have to modify the VASPDIR, adapt the time limit (-t), job name, resource specification (--res)
  • then make the script file executable with chmod +x FILENAME and submit with ccsalloc FILENAME


VASP 5.4.4 with wannier90v2 on Oculus[Bearbeiten]

  • module purge
  • module add default-environment
  • module add ps_xe_2019_04_compilers
  • wget http://www.wannier.org/code/wannier90-2.1.0.tar.gz
  • tar -xf wannier90-2.1.0.tar.gz
  • cd wannier90-2.1.0/
  • replace in config/make.inc.ifort
    • LIBDIR = /opt/intel/mkl/lib/intel64
    • with LIBDIR = /cm/shared/apps/pc2/INTEL/ps_xe_2019_u4/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64
  • cp ./config/make.inc.ifort ./make.inc
  • make
  • make lib
  • export WANNIERDIR=`pwd`
  • cd ..
  • cd VASP544
  • edit makefile.include
# Precompiler options
CPP_OPTIONS= -DHOST=\"Oculus\"\
             -DMPI -DMPI_BLOCK=8000 \
             -DscaLAPACK \
             -Davoidalloc \
             -Duse_bse_te \
             -Dtbdyn \
             -Duse_shmem \
             -DnoSTOPCAR \
             -DVASP2WANNIER90v2


CPP        = fpp -f_com=no -free -w0  $*$(FUFFIX) $*$(SUFFIX) $(CPP_OPTIONS)

FC         = mpiifort
FCL        = mpiifort -mkl

FREE       = -free -names lowercase

FFLAGS     = -assume byterecl -heap-arrays -diag-disable:8291
OFLAG      = -O2 -xHOST -g
OFLAG_IN   = $(OFLAG)
DEBUG      = -O0

MKL_PATH   = $(MKLROOT)/lib/intel64
BLAS       =
LAPACK     =
BLACS      = -lmkl_blacs_intelmpi_lp64
SCALAPACK  = $(MKL_PATH)/libmkl_scalapack_lp64.a $(BLACS)


OBJECTS    = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o $(MKLROOT)/interfaces/fftw3xf/libfftw3xf_intel.a

INCS       =-I$(MKLROOT)/include/fftw

LLIBS      = $(SCALAPACK) $(LAPACK) $(BLAS)
LLIBS     += $(WANNIERDIR)/libwannier.a

OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o

# For what used to be vasp.5.lib
CPP_LIB    = $(CPP)
FC_LIB     = $(FC)
CC_LIB     = icc
CFLAGS_LIB = -O
FFLAGS_LIB = -O1
FREE_LIB   = $(FREE)

OBJECTS_LIB= linpack_double.o getshmem.o

# For the parser library
CXX_PARS   = icpc

LIBS       += parser
LLIBS      += -Lparser -lparser -lstdc++

# Normally no need to change this
SRCDIR     = ../../src
BINDIR     = ../../bin

  • rm -rf build; mkdir build
  • make
  • you should now have vasp_gam, vasp_ncl and vasp_std in the directory bin/
  • you can run it with the template jobscript
#! /bin/bash
#CCS --stdout=vasp.out
#CCS --stderr=vasp.err
#CCS --name VASP_Job
#CCS --res=rset=1:ncpus=16:mpiprocs=16:ompthreads=1 
#CCS -t 1h

module purge
module add default-environment
module add intel/19.0.4_compilers

VASPDIR=!!SET DIRECTORY NAME HERE!!
export I_MPI_DEBUG=4
mpirun -genvall -machinefile $CCS_NODEFILE $VASPDIR/bin/vasp_std


  • you have to modify the VASPDIR, adapt the time limit (-t), job name, resource specification (--res)
  • then make the script file executable with chmod +x FILENAME and submit with ccsalloc FILENAME