Dear Mihaly,
I think I have to be more specific. So what I did, I compiled twice mrcc, once sequential, and once mpi (in different directories). Then I moved the mpi version of mrcc to the directory of sequential version with a name mrcci.mpi
Then I made a "mrcc" script in the sequential version as following:
mrcc:
Code:
NP=`grep -c nic /tmp/hajgato/hosts`
for i in `tail -n +2 /tmp/hajgato/hosts`
do
scp -r /tmp/hajgato/mrcc $i:/tmp/hajgato
done
mpirun -hostfile /tmp/hajgato/hosts -wdir /tmp/hajgato/mrcc -np $NP mrcc.mpi
The script to run my cacluation was:
submit:
Code:
cd /tmp/hajgato
cat <<END>/tmp/hajgato/hosts
nic151
nic152
END
NP=`grep -c nic /tmp/hajgato/hosts`
mkdir mrcc
cd mrcc
cp /u/hajgato/calc/freija/Q_dz/$1 MINP
cp /u/hajgato/src/mrcc.14.07.10.mpi/GENBAS .
dmrcc > /u/hajgato/calc/freija/Q_dz/${1%minp}mrccout
cd /tmp/hajgato
mpirun -hostfile /tmp/hajgato/hosts rm -rf /tmp/hajgato/mrcc
(The /tmp/ directory is the local disk of the node.)
I have checked that only one copy on both node nic151 and node nic152 was running.
Then the relevant part of the output is
output:
Code:
**********************************************************************
MPI parallel version is running.
Number of CPUs: 2
**********************************************************************
CCSDT(Q) calculation
Allocation of 668.1 Mbytes of memory...
Number of spinorbitals: 114
Number of alpha electrons: 13
Number of beta electrons: 13
Spin multiplicity: 1
z-component of spin: 0.0
Spatial symmetry: 1
Convergence criterion: 1.0E-06
Construction of occupation graphs...
**********************************************************************
CCSDT(Q) calculation
Allocation of 668.1 Mbytes of memory...
Number of spinorbitals: 114
Number of alpha electrons: 13
Number of beta electrons: 13
Spin multiplicity: 1
z-component of spin: 0.0
Spatial symmetry: 1
Convergence criterion: 1.0E-06
Construction of occupation graphs...
Number of 0-fold excitations: 1
Number of 1-fold excitations: 602
Number of 2-fold excitations: 237774
Number of 3-fold excitations: 45993540
Total number of determinants: 46231917
Calculation of coupling coefficients...
Number of 0-fold excitations: 1
Number of 1-fold excitations: 602
Number of 2-fold excitations: 237774
Number of 3-fold excitations: 45993540
Total number of determinants: 46231917
Calculation of coupling coefficients...
Initial cluster amplitudes are generated.
Initial cluster amplitudes are generated.
Length of intermediate file (Mbytes): 817.6
Length of intermediate file (Mbytes): 817.6
Reading integral list from unit 55...
Reading integral list from unit 55...
Sorting integrals...
Sorting integrals...
Energy of reference determinant [au]: -336.788506858264
Calculation of MP denominators...
Energy of reference determinant [au]: -336.788506858264
Calculation of MP denominators...
Starting CC iteration...
======================================================================
Starting CC iteration...
======================================================================
Norm of residual vector: 1.96621997
CPU time [min]: 3.183 Wall time [min]: 3.159
Iteration 1 CC energy: -337.47958067 Energy decrease: 0.69107381
======================================================================
Norm of residual vector: 1.96621997
CPU time [min]: 3.190 Wall time [min]: 3.164
Iteration 1 CC energy: -337.47958067 Energy decrease: 0.69107381
======================================================================
2 total processes killed (some possibly by mpirun during cleanup)
(i was killed the job)
It looks like that after wrining the number of CPUs, the mpi processes do not communicate with each other anymore.
Sincerely,
Balazs
