Strange DFT problem

Dear Developers

I have been running into a problem with calculating
DFT energies that I am unable to resolve. I am using
MRCC version of 2016-07-15, compiled using Intel Fortran
V 17 on 2016-12-29. (omp version, 1 thread)

On the cluster in question we have a mounted GPFS file system
(/scicore/....) as well as a local file system. Programs and
MKL libraries reside on the former, while for performance
reasons I wish to use the latter for the calculation.

Everything seems fine as long as I run on the GPFS file
system, I get an energy (-235.5953025) reasonably close to
a reference (Turbomole, -235.5952516), the small difference
likely due differences in technical detail (integration
grid etc).

If I use the local file system I get an energy too low by
about 197 hartrees (!, -432.1584561), and an extremely low
orbital energy for the first orbital. I cannot see anything
in the output that gives me a hint on what is going wrong.

This error is reproducible. If I run on the local filesystem,
I always get the same wrong energy, if I run on the other, I
always get the same correct energy. I have seen the same
problem with unreasonably large energies also for other
molecules, even of different composition, and the "energy error"
always seems to be around 197 hartrees.

I have assembled a test script that demonstrates the error.
The calculation is done 5 times:

Calculation MRCC code MKL libraries Result
1) GPFS GPFS GFPS OK
2) local GPFS GFPS error
3) local local GFPS error
4) local local local error
5) GPFS GPFS GPFS OK

Calculation 5) is essentially a repetition of 1), just to show
reproducibility.

Do you have any idea of what might be going wrong or how
I might proceed to spot and correct the error?

Thanks a lot,
Dirk Bakowies

Dear Dirk,
It is very strange, we have never experienced such problems, and we have no idea what is going on.
You should try to use statically compiled binaries. You can download them from the homepage or, if you need to compile the program for some reason, you can create your own static binaries by commenting in the corresponding comment lines setting the compileropts variable in the file build.mrcc.config.

Dear Mihaly,

thanks a lot for the very fast and helpful reply! Using statically linked
binaries indeed helps. I have tried it with your precompiled versions
of 2016 (should be the same version that I compiled myself, but
using shared-object MKL libraries) and of 2017,
ran a few tests, and did not observe the error anymore. Another run
with my dynamically-linked version reproduced the error, though.
I guess we can conclude that the ultra-low energy (and orbital
energy) that I obtained when using the dynamically linked version
on a local file system was due to some problem with the dynamically
linked linear algebra librarires (MKL, Intel), although it still does not
really make sense that the error did not appear when the calculation
was run on the GPFS filesystem.

Thanks and best regards,
Dirk

Dear Dirk,

although it still does not really make sense that the error did not appear when the calculation was run on the GPFS filesystem.

It is not unusual for things to work on GPFS that fail on NFS,
it would be surprising if it were the other way around.

Why not run all your calculations on the GPFS system? It will be more robust, and the I/O should be faster.

With best wishes!!
Nike

Dear Nike,

the file system that shows the problems is a locally mounted one
(/dev/sd....), not an NFS file system. I use it for scratch to minimize
network traffic.

Thanks for all your input,
best wishes,

Dirk