Run BigDFT and manipulate its output files

After the presentation of the BigDFT input files, it now time run actual BigDFT calculations with MyBigDFT. This is made possible via the Job class.

The result of a BigDFT calculation is stored in a logfile using the YAML format (just like the input file). The Logfile class is meant to ease the manipulation of this output data.

The Logfile class

As a first example, you can read an already existing logfile via the from_file method. It only requires the path to an already existing logfile:

from mybigdft import Logfile
log = Logfile.from_file("../../../tests/log.yaml")

You can get the input parameters used to run the calculation via the inputparams attribute:

inp = log.inputparams
print(inp)

{}

Here, default parameters were used.

You can also get the initial positions used via the posinp attribute:

pos = log.posinp
print(pos)

2   angstroem
free
N   2.97630782434901e-23   6.87220595204354e-23   0.0107161998748779
N  -1.10434491945017e-23  -4.87342174483075e-23   1.10427379608154

The same Posinp instance is actually stored under the posinp attribute of the input parameters:

assert inp.posinp == log.posinp

The Job class

To run a calculation, use the Job class. It requires the input geometry (given by the input parameters or by the posinp) and generally some input parameters. You may also provide a name to the calculation, so as to set the name for the input and output files. You can also define where to run the calculation by specifying the run_dir argument.

Finally, the most important thing to know about the Job class is that it must be used with a context manager. This ensures that the files are created in the correct location and that the BigDFT calculation is run in the same location.

from mybigdft import Job
import os

# No need to provide a posinp explicitly, since it is stored in inp
# under the `posinp` attribute.
with Job(inputparams=inp, name="test") as job:
    # The calculation must be run in the context manager.
    # Here, it uses 6 processors and 3 OpenMP threads.
    job.run(nmpi=6, nomp=3, dry_run=True)
    # Setting dry_run to True runs bigdft-tool instead of
    # bigdft and creates the input files. This is a good way
    # of testing the input parameters and positions before
    # running the actual BigDFT calculation.
    assert os.path.isfile("test.yaml")
    assert os.path.isfile("test.xyz")
    # A logfile is also created, containing the output of the
    # bigdft-tool run. You can check it to get an estimation
    # of the memory requirement of the calculation.
    assert os.path.isfile("log-test.yaml")
    # The input and output files can also be removed from the
    # disk. This must be performed in the context manager as
    # well, by running the clean method. However, you generally
    # do not want to erase your output files after running the
    # calculation.
    job.clean()
    assert not os.path.isfile("log-test.yaml")
    assert not os.path.isfile("test.yaml")
    assert not os.path.isfile("test.xyz")

/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks
/Users/maximemoriniere/Documents/bigdft/bigdft/build-1997/install/bin/bigdft-tool --name test -n 6 ...

Note that you can still use job after exiting the context manager. It is for instance possible to look for some attributes of the output of the bigdft-tool executable, and compare them to those of the base logfile used to initialize the input parameters and geometry:

for attr in ["energy", "dipole", "inputparams", "posinp"]:
    print(f"{attr}")
    print(f"   job: {getattr(job.logfile, attr)}")
    print(f"   log: {getattr(log, attr)}")

energy
   job: None
   log: -19.884659235401838
dipole
   job: None
   log: [-0.00051199, -0.00051199, -0.00055711]
inputparams
   job: {}
   log: {}
posinp
   job: 2   angstroem
free
N   2.97630782434901e-23   6.87220595204354e-23   0.0107161998748779
N  -1.10434491945017e-23  -4.87342174483075e-23   1.10427379608154

   log: 2   angstroem
free
N   2.97630782434901e-23   6.87220595204354e-23   0.0107161998748779
N  -1.10434491945017e-23  -4.87342174483075e-23   1.10427379608154

Note that the output of the bigdft-tool do not give the energy nor the dipole of the system: to run the actual bigdft calculation, you must not set dry_run to True (see below). However, the input parameters and the initial positions are exactly the same as those of the base logfile, as expected!

To run the bigdft executable, you must therefore run the following code:

with Job(inputparams=inp, name="test") as job:
    assert not job.is_completed
    job.run(nmpi=6, nomp=3)
    assert job.is_completed

/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks
mpirun -np 6 /Users/maximemoriniere/Documents/bigdft/bigdft/build-1997/install/bin/bigdft test ...
 <BigDFT> log of the run will be written in logfile: ./log-test.yaml

Now, the output of the calculation is the same as the base logfile, up to some numerical noise:

for attr in ["energy", "dipole"]:
    print(f"{attr}")
    print(f"   job: {getattr(job.logfile, attr)}")
    print(f"   log: {getattr(log, attr)}")

energy
   job: -19.884659235399404
   log: -19.884659235401838
dipole
   job: [-0.00051199, -0.00051199, -0.00055711]
   log: [-0.00051199, -0.00051199, -0.00055711]

Force the calculation to run

The default behaviour of the run method is to read an already existing logfile instead of actually running the calculation again:

with Job(inputparams=inp, name="test") as job:
    job.run(nmpi=6, nomp=3)

/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks
Logfile log-test.yaml already exists!

You can force to run the calculation by setting the force_run argument of the run method to True. This forces to run the calculation even though a logfile already exists.

with Job(inputparams=inp, name="test") as job:
    # The calculation is forced to run: the previous logfile
    # is moved to the logfiles directory (created for the occasion).
    job.run(nmpi=6, nomp=3, force_run=True)
    # You can remove the logfiles directory as well
    job.clean(logfiles_dir=True)
    assert not os.path.isdir("logfiles")

/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks
mpirun -np 6 /Users/maximemoriniere/Documents/bigdft/bigdft/build-1997/install/bin/bigdft test ...
 <BigDFT> log of the run will be written in logfile: ./log-test.yaml
 <BigDFT> Logfile existing, renamed into: ./logfiles/log-test.17:04:17.065.yaml

What if the run fails?

The run or dry_run should both fail if invalid input parameters are caught by the initialization procedure. Here, we take the example of a case where the user wanted to use surface boundary conditions but placed the “.inf” as the cell size along the \(z\) direction (instead of the \(y\) direction). Note that the error message prints the actual error message received when running the bigdft-tool or bigdft executable (depending on the value of dry_run).

from mybigdft import InputParams

inp = InputParams({"posinp": {
    "units": "angstroem",
    "cell": [40, 40, ".inf"],
    "positions": [
        {'N': [2.97630782434901e-23, 6.87220595204354e-23, 0.0107161998748779]},
        {'N': [-1.10434491945017e-23, -4.87342174483075e-23, 1.10427379608154]},
    ]
}})

with Job(inputparams=inp, name="test") as job:
    job.run(nmpi=6, nomp=3, dry_run=True)
    job.clean()

/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks
/Users/maximemoriniere/Documents/bigdft/bigdft/build-1997/install/bin/bigdft-tool --name test -n 6 ...

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-11-111fe3f096f8> in <module>()
     11
     12 with Job(inputparams=inp, name="test") as job:
---> 13     job.run(nmpi=6, nomp=3, dry_run=True)
     14     job.clean()

~/Documents/Python/MyBigDFT/mybigdft/job.py in run(self, nmpi, nomp, force_run, dry_run)
    453             self.write_input_files()
    454             command = self._get_command(nmpi, dry_run)
--> 455             output_msg = self._launch_calculation(command)
    456             if dry_run:
    457                 self._write_bigdft_tool_output(output_msg)

~/Documents/Python/MyBigDFT/mybigdft/job.py in _launch_calculation(command)
    644             raise RuntimeError(
    645                 "The calculation ended with the following error message:{}"
--> 646                 .format(error_msg))
    647         return out
    648

RuntimeError: The calculation ended with the following error message:
 abi_symdet: ERROR -
  Abs(determinant) for symmetry number    1 is         0 .
  For a legitimate symmetry, abs(determinant) must be 1.
  Action : check your symmetry operations (symrel) in input file.
Note: The following floating-point exceptions are signalling: IEEE_INVALID_FLAG
STOP MPIFAKE: mpi_error_string

# Let us clean the present directory again:
with job as job:
    job.clean()

/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks

It might happen that the error messages are not properly sent or received, depending on your platform (for instance with Bash on Ubuntu on Windows, under Windows 10). This might break this particulat behaviour, but you should still get an error. The error will however differ if running in dry mode or not.