pymatgen.io.abinit.launcher module

Tools for the submission of Tasks.

class ScriptEditor[source]

Bases: object

Simple editor that simplifies the writing of shell scripts

add_comment(comment)[source]

Add a comment

add_emptyline()[source]

Add an empty line.

add_line(line)[source]
add_lines(lines)[source]
declare_var(key, val)[source]

Declare a env variable. If val is None the variable is unset.

declare_vars(d)[source]

Declare the variables defined in the dictionary d.

export_envar(key, val)[source]

Export an environment variable.

export_envars(env)[source]

Export the environment variables contained in the dict env.

get_script_str(reset=True)[source]

Returns a string with the script and reset the editor if reset is True

load_module(module)[source]
load_modules(modules)[source]

Load the list of specified modules.

reset()[source]

Reset the editor.

shebang()[source]

Adds the shebang line.

shell
class PyLauncher(flow, **kwargs)[source]

Bases: object

This object handle the submission of the tasks contained in a Flow

Initialize the object

Parameters:
  • flowFlow object
  • max_njobs_inqueue – The launcher will stop submitting jobs when the number of jobs in the queue is >= Max number of jobs
Error

alias of PyLauncherError

fetch_tasks_to_run()[source]

Return the list of tasks that can be submitted. Empty list if no task has been found.

rapidfire(max_nlaunch=-1, max_loops=1, sleep_time=5)[source]

Keeps submitting Tasks until we are out of jobs or no job is ready to run.

Parameters:
  • max_nlaunch – Maximum number of launches. default: no limit.
  • max_loops – Maximum number of loops
  • sleep_time – seconds to sleep between rapidfire loop iterations
Returns:

The number of tasks launched.

single_shot()[source]

Run the first Task than is ready for execution.

Returns:Number of jobs launched.
class PyFlowScheduler(**kwargs)[source]

Bases: object

This object schedules the submission of the tasks in a Flow. There are two types of errors that might occur during the execution of the jobs:

  1. Python exceptions
  2. Errors in the ab-initio code

Python exceptions are easy to detect and are usually due to a bug in the python code or random errors such as IOError. The set of errors in the ab-initio is much much broader. It includes wrong input data, segmentation faults, problems with the resource manager, etc. The flow tries to handle the most common cases but there’s still a lot of room for improvement. Note, in particular, that PyFlowScheduler will shutdown automatically in the following cases:

  1. The number of python exceptions is > max_num_pyexcs
  2. The number of task errors (i.e. the number of tasks whose status is S_ERROR) is > max_num_abierrs
  3. The number of jobs launched becomes greater than (safety_ratio * total_number_of_tasks).
  4. The scheduler will send an email to the user (specified by mailto) every remindme_s seconds. If the mail cannot be sent, the scheduler will shutdown automatically. This check prevents the scheduler from being trapped in an infinite loop.
Parameters:
  • weeks – number of weeks to wait (DEFAULT: 0).
  • days – number of days to wait (DEFAULT: 0).
  • hours – number of hours to wait (DEFAULT: 0).
  • minutes – number of minutes to wait (DEFAULT: 0).
  • seconds – number of seconds to wait (DEFAULT: 0).
  • mailto – The scheduler will send an email to mailto every remindme_s seconds. (DEFAULT: None i.e. not used).
  • verbose – (int) verbosity level. (DEFAULT: 0)
  • use_dynamic_manager – “yes” if the TaskManager must be re-initialized from file before launching the jobs. (DEFAULT: “no”)
  • max_njobs_inqueue – Limit on the number of jobs that can be present in the queue. (DEFAULT: 200)
  • remindme_s – The scheduler will send an email to the user specified by mailto every remindme_s seconds. (int, DEFAULT: 1 day).
  • max_num_pyexcs – The scheduler will exit if the number of python exceptions is > max_num_pyexcs (int, DEFAULT: 0)
  • max_num_abierrs – The scheduler will exit if the number of errored tasks is > max_num_abierrs (int, DEFAULT: 0)
  • safety_ratio – The scheduler will exits if the number of jobs launched becomes greater than safety_ratio * total_number_of_tasks_in_flow. (int, DEFAULT: 5)
  • max_nlaunches – Maximum number of tasks launched in a single iteration of the scheduler. (DEFAULT: -1 i.e. no limit)
  • debug – Debug level. Use 0 for production (int, DEFAULT: 0)
  • fix_qcritical – “yes” if the launcher should try to fix QCritical Errors (DEFAULT: “yes”)
  • rmflow – If “yes”, the scheduler will remove the flow directory if the calculation completed successfully. (DEFAULT: “no”)
  • killjobs_if_errors – “yes” if the scheduler should try to kill all the runnnig jobs before exiting due to an error. (DEFAULT: “yes”)
Error

alias of PyFlowSchedulerError

USER_CONFIG_DIR = '/Users/shyuep/.abinit/abipy'
YAML_FILE = 'scheduler.yml'
add_flow(flow)[source]

Add an Flow flow to the scheduler.

classmethod autodoc()[source]
callback()[source]

The function that will be executed by the scheduler.

cleanup()[source]

Cleanup routine: remove the pid file and save the pickle database

flow

Flow.

classmethod from_file(filepath)[source]

Read the configuration parameters from a Yaml file.

classmethod from_string(s)[source]

Create an istance from string s containing a YAML dictionary.

classmethod from_user_config()[source]

Initialize the PyFlowScheduler from the YAML file ‘scheduler.yml’. Search first in the working directory and then in the configuration directory of abipy.

Raises:RuntimeError if file is not found.
get_delta_etime()[source]

Returns a timedelta object representing with the elapsed time.

num_excs

Number of exceptions raised so far.

pid

The pid of the process associated to the scheduler.

pid_file

Absolute path of the file with the pid. The file is located in the workdir of the flow

send_email(msg, tag=None)[source]

Send an e-mail before completing the shutdown. Returns 0 if success.

shutdown(msg)[source]

Shutdown the scheduler.

start()[source]

Starts the scheduler in a new thread. Returns 0 if success. In standalone mode, this method will block until there are no more scheduled jobs.