{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run BigDFT and manipulate its output files\n",
    "\n",
    "After the [presentation of the BigDFT input files](https://mmoriniere.gitlab.io/MyBigDFT/notebooks/Input_files.html), it now time run actual BigDFT calculations with MyBigDFT. This is made possible via the [Job](https://mmoriniere.gitlab.io/MyBigDFT/job.html) class.\n",
    "\n",
    "The result of a BigDFT calculation is stored in a logfile using the YAML format (just like the input file). The [Logfile](https://mmoriniere.gitlab.io/MyBigDFT/logfile.html#logfile) class is meant to ease the manipulation of this output data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The [Logfile](https://mmoriniere.gitlab.io/MyBigDFT/logfile.html#logfile) class\n",
    "\n",
    "As a first example, you can read an already existing logfile via the `from_file` method. It only requires the path to an already existing logfile:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mybigdft import Logfile\n",
    "log = Logfile.from_file(\"../../../tests/log.yaml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can get the input parameters used to run the calculation via the `inputparams` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{}\n"
     ]
    }
   ],
   "source": [
    "inp = log.inputparams\n",
    "print(inp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, default parameters were used.\n",
    "\n",
    "You can also get the initial positions used via the `posinp` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2   angstroem\n",
      "free\n",
      "N   2.97630782434901e-23   6.87220595204354e-23   0.0107161998748779\n",
      "N  -1.10434491945017e-23  -4.87342174483075e-23   1.10427379608154\n",
      "\n"
     ]
    }
   ],
   "source": [
    "pos = log.posinp\n",
    "print(pos)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same `Posinp` instance is actually stored under the `posinp` attribute of the input parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert inp.posinp == log.posinp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The [Job](https://mmoriniere.gitlab.io/MyBigDFT/job.html) class\n",
    "\n",
    "To run a calculation, use the `Job` class. It requires the input geometry (given by the input parameters or by the posinp) and generally some input parameters. You may also provide a name to the calculation, so as to set the name for the input and output files. You can also define where to run the calculation by specifying the `run_dir` argument.\n",
    "\n",
    "Finally, the most important thing to know about the `Job` class is that it must be used with a context manager. This ensures that the files are created in the correct location and that the BigDFT calculation is run in the same location."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks\n",
      "/Users/maximemoriniere/Documents/bigdft/bigdft/build-1997/install/bin/bigdft-tool --name test -n 6 ...\n"
     ]
    }
   ],
   "source": [
    "from mybigdft import Job\n",
    "import os\n",
    "\n",
    "# No need to provide a posinp explicitly, since it is stored in inp\n",
    "# under the `posinp` attribute.\n",
    "with Job(inputparams=inp, name=\"test\") as job:\n",
    "    # The calculation must be run in the context manager.\n",
    "    # Here, it uses 6 processors and 3 OpenMP threads.\n",
    "    job.run(nmpi=6, nomp=3, dry_run=True)\n",
    "    # Setting dry_run to True runs bigdft-tool instead of\n",
    "    # bigdft and creates the input files. This is a good way\n",
    "    # of testing the input parameters and positions before\n",
    "    # running the actual BigDFT calculation.\n",
    "    assert os.path.isfile(\"test.yaml\")\n",
    "    assert os.path.isfile(\"test.xyz\")\n",
    "    # A logfile is also created, containing the output of the\n",
    "    # bigdft-tool run. You can check it to get an estimation\n",
    "    # of the memory requirement of the calculation.\n",
    "    assert os.path.isfile(\"log-test.yaml\")\n",
    "    # The input and output files can also be removed from the\n",
    "    # disk. This must be performed in the context manager as\n",
    "    # well, by running the clean method. However, you generally \n",
    "    # do not want to erase your output files after running the\n",
    "    # calculation.\n",
    "    job.clean()\n",
    "    assert not os.path.isfile(\"log-test.yaml\")\n",
    "    assert not os.path.isfile(\"test.yaml\")\n",
    "    assert not os.path.isfile(\"test.xyz\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that you can still use `job` after exiting the context manager. It is for instance possible to look for some attributes of the output of the bigdft-tool executable, and compare them to those of the base logfile used to initialize the input parameters and geometry:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "energy\n",
      "   job: None\n",
      "   log: -19.884659235401838\n",
      "dipole\n",
      "   job: None\n",
      "   log: [-0.00051199, -0.00051199, -0.00055711]\n",
      "inputparams\n",
      "   job: {}\n",
      "   log: {}\n",
      "posinp\n",
      "   job: 2   angstroem\n",
      "free\n",
      "N   2.97630782434901e-23   6.87220595204354e-23   0.0107161998748779\n",
      "N  -1.10434491945017e-23  -4.87342174483075e-23   1.10427379608154\n",
      "\n",
      "   log: 2   angstroem\n",
      "free\n",
      "N   2.97630782434901e-23   6.87220595204354e-23   0.0107161998748779\n",
      "N  -1.10434491945017e-23  -4.87342174483075e-23   1.10427379608154\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for attr in [\"energy\", \"dipole\", \"inputparams\", \"posinp\"]:\n",
    "    print(f\"{attr}\")\n",
    "    print(f\"   job: {getattr(job.logfile, attr)}\")\n",
    "    print(f\"   log: {getattr(log, attr)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the output of the bigdft-tool do not give the energy nor the dipole of the system: to run the actual bigdft calculation, you must not set dry_run to `True` (see below).\n",
    "However, the input parameters and the initial positions are exactly the same as those of the base logfile, as expected!\n",
    "\n",
    "To run the bigdft executable, you must therefore run the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks\n",
      "mpirun -np 6 /Users/maximemoriniere/Documents/bigdft/bigdft/build-1997/install/bin/bigdft test ...\n",
      " <BigDFT> log of the run will be written in logfile: ./log-test.yaml\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with Job(inputparams=inp, name=\"test\") as job:\n",
    "    assert not job.is_completed\n",
    "    job.run(nmpi=6, nomp=3)\n",
    "    assert job.is_completed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, the output of the calculation is the same as the base logfile, up to some numerical noise:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "energy\n",
      "   job: -19.884659235399404\n",
      "   log: -19.884659235401838\n",
      "dipole\n",
      "   job: [-0.00051199, -0.00051199, -0.00055711]\n",
      "   log: [-0.00051199, -0.00051199, -0.00055711]\n"
     ]
    }
   ],
   "source": [
    "for attr in [\"energy\", \"dipole\"]:\n",
    "    print(f\"{attr}\")\n",
    "    print(f\"   job: {getattr(job.logfile, attr)}\")\n",
    "    print(f\"   log: {getattr(log, attr)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Force the calculation to run\n",
    "\n",
    "The default behaviour of the `run` method is to read an already existing logfile instead of actually running the calculation again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks\n",
      "Logfile log-test.yaml already exists!\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with Job(inputparams=inp, name=\"test\") as job:\n",
    "    job.run(nmpi=6, nomp=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can force to run the calculation by setting the `force_run` argument of the `run` method to `True`. This forces to run the calculation even though a logfile already exists. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks\n",
      "mpirun -np 6 /Users/maximemoriniere/Documents/bigdft/bigdft/build-1997/install/bin/bigdft test ...\n",
      " <BigDFT> log of the run will be written in logfile: ./log-test.yaml\n",
      " <BigDFT> Logfile existing, renamed into: ./logfiles/log-test.17:04:17.065.yaml\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with Job(inputparams=inp, name=\"test\") as job:\n",
    "    # The calculation is forced to run: the previous logfile\n",
    "    # is moved to the logfiles directory (created for the occasion).\n",
    "    job.run(nmpi=6, nomp=3, force_run=True)\n",
    "    # You can remove the logfiles directory as well\n",
    "    job.clean(logfiles_dir=True)\n",
    "    assert not os.path.isdir(\"logfiles\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What if the run fails?\n",
    "\n",
    "The run or dry_run should both fail if invalid input parameters are caught by the initialization procedure. Here, we take the example of a case where the user wanted to use surface boundary conditions but placed the \".inf\" as the cell size along the $z$ direction (instead of the $y$ direction). Note that the error message prints the actual error message received when running the bigdft-tool or bigdft executable (depending on the value of `dry_run`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks\n",
      "/Users/maximemoriniere/Documents/bigdft/bigdft/build-1997/install/bin/bigdft-tool --name test -n 6 ...\n"
     ]
    },
    {
     "ename": "RuntimeError",
     "evalue": "The calculation ended with the following error message:\n abi_symdet: ERROR -\n  Abs(determinant) for symmetry number    1 is         0 .\n  For a legitimate symmetry, abs(determinant) must be 1.\n  Action : check your symmetry operations (symrel) in input file.\nNote: The following floating-point exceptions are signalling: IEEE_INVALID_FLAG\nSTOP MPIFAKE: mpi_error_string\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-11-111fe3f096f8>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     12\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mJob\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputparams\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"test\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mjob\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m     \u001b[0mjob\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnmpi\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m6\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnomp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdry_run\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     14\u001b[0m     \u001b[0mjob\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclean\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Documents/Python/MyBigDFT/mybigdft/job.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, nmpi, nomp, force_run, dry_run)\u001b[0m\n\u001b[1;32m    453\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwrite_input_files\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    454\u001b[0m             \u001b[0mcommand\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_command\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnmpi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdry_run\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 455\u001b[0;31m             \u001b[0moutput_msg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_launch_calculation\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcommand\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    456\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mdry_run\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    457\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_write_bigdft_tool_output\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutput_msg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Documents/Python/MyBigDFT/mybigdft/job.py\u001b[0m in \u001b[0;36m_launch_calculation\u001b[0;34m(command)\u001b[0m\n\u001b[1;32m    644\u001b[0m             raise RuntimeError(\n\u001b[1;32m    645\u001b[0m                 \u001b[0;34m\"The calculation ended with the following error message:{}\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 646\u001b[0;31m                 .format(error_msg))\n\u001b[0m\u001b[1;32m    647\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mout\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    648\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mRuntimeError\u001b[0m: The calculation ended with the following error message:\n abi_symdet: ERROR -\n  Abs(determinant) for symmetry number    1 is         0 .\n  For a legitimate symmetry, abs(determinant) must be 1.\n  Action : check your symmetry operations (symrel) in input file.\nNote: The following floating-point exceptions are signalling: IEEE_INVALID_FLAG\nSTOP MPIFAKE: mpi_error_string\n"
     ]
    }
   ],
   "source": [
    "from mybigdft import InputParams\n",
    "\n",
    "inp = InputParams({\"posinp\": {\n",
    "    \"units\": \"angstroem\",\n",
    "    \"cell\": [40, 40, \".inf\"],\n",
    "    \"positions\": [\n",
    "        {'N': [2.97630782434901e-23, 6.87220595204354e-23, 0.0107161998748779]},\n",
    "        {'N': [-1.10434491945017e-23, -4.87342174483075e-23, 1.10427379608154]},\n",
    "    ]\n",
    "}})\n",
    "\n",
    "with Job(inputparams=inp, name=\"test\") as job:\n",
    "    job.run(nmpi=6, nomp=3, dry_run=True)\n",
    "    job.clean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/maximemoriniere/Documents/Python/MyBigDFT/doc/source/notebooks\n"
     ]
    }
   ],
   "source": [
    "# Let us clean the present directory again:\n",
    "with job as job:\n",
    "    job.clean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It might happen that the error messages are not properly sent or received, depending on your platform (for instance with Bash on Ubuntu on Windows, under Windows 10). This might break this particulat behaviour, but you should still get an error. The error will however differ if running in dry mode or not."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}