patroni.config module

Facilities related to Patroni configuration.

class patroni.config.Config(configfile: str, validator: ~typing.Callable[[~typing.Dict[str, ~typing.Any]], ~typing.List[str]] | None = <function default_validator>)

Bases: object

Handle Patroni configuration.

This class is responsible for:

  1. Building and giving access to effective_configuration from:

    • Config.__DEFAULT_CONFIG – some sane default values;

    • dynamic_configuration – configuration stored in DCS;

    • local_configuration – configuration from config.yml or environment.

  2. Saving and loading dynamic_configuration into ‘patroni.dynamic.json’ file located in local_configuration[‘postgresql’][‘data_dir’] directory. This is necessary to be able to restore dynamic_configuration if DCS was accidentally wiped.

  3. Loading of configuration file in the old format and converting it into new format.

  4. Mimicking some dict interfaces to make it possible to work with it as with the old config object.

Variables:

  • PATRONI_CONFIG_VARIABLE – name of the environment variable that can be used to load Patroni configuration from.

  • __CACHE_FILENAME – name of the file used to cache dynamic configuration under Postgres data directory.

  • __DEFAULT_CONFIG – default configuration values for some Patroni settings.

PATRONI_CONFIG_VARIABLE = 'PATRONI_CONFIGURATION'
__CACHE_FILENAME = 'patroni.dynamic.json'
__DEFAULT_CONFIG: Dict[str, Any] = {'loop_wait': 10, 'postgresql': {'parameters': <CaseInsensitiveDict{'wal_level': 'hot_standby', 'hot_standby': 'on', 'max_connections': 100, 'max_wal_senders': 10, 'max_prepared_transactions': 0, 'max_locks_per_transaction': 64, 'track_commit_timestamp': 'off', 'max_replication_slots': 10, 'max_worker_processes': 8, 'wal_log_hints': 'on'} at 7f47e734bd10>, 'use_slots': True}, 'retry_timeout': 10, 'standby_cluster': {'archive_cleanup_command': '', 'create_replica_methods': '', 'host': '', 'port': '', 'primary_slot_name': '', 'recovery_min_apply_delay': '', 'restore_command': ''}, 'ttl': 30}
__get_and_maybe_adjust_int_value(config: Dict[str, Any], param: str, min_value: int) → int

Get, validate and maybe adjust a param integer value from the config dict.

Parameters:

  • config – dict object with new global configuration.

  • param – name of the configuration parameter we want to read/validate/adjust.

  • min_value – the minimum possible value that a given param could have.

Returns:

an integer value which corresponds to a provided param.

__init__(configfile: str, validator: ~typing.Callable[[~typing.Dict[str, ~typing.Any]], ~typing.List[str]] | None = <function default_validator>) → None

Create a new instance of Config and validate the loaded configuration using validator.

Note

Patroni will read configuration from these locations in this order:

  • file or directory path passed as command-line argument (configfile), if it exists and the file or files found in the directory can be parsed (see _load_config_path()), otherwise

  • YAML file passed via the environment variable (see PATRONI_CONFIG_VARIABLE), if the referenced file exists and can be parsed, otherwise

  • from configuration values defined as environment variables, see _build_environment_configuration().

Parameters:

  • configfile – path to Patroni configuration file.

  • validator – function used to validate Patroni configuration. It should receive a dictionary which represents Patroni configuration, and return a list of zero or more error messages based on validation.

Raises:

ConfigParseError: if any issue is reported by validator.

_build_effective_configuration(dynamic_configuration: Dict[str, Any], local_configuration: Dict[str, Dict[str, Any] | Any]) → Dict[str, Any]

Build effective configuration by merging dynamic_configuration and local_configuration.

Note

local_configuration takes precedence over dynamic_configuration if a setting is defined in both.

Parameters:

  • dynamic_configuration – Patroni dynamic configuration.

  • local_configuration – Patroni local configuration.

Returns:

_description_

static _build_environment_configuration() → Dict[str, Any]

Get local configuration settings that were specified through environment variables.

Returns:

dictionary containing the found environment variables and their values, respecting the expected structure of Patroni configuration.

_load_cache() → None

Load dynamic configuration from patroni.dynamic.json.

_load_config_file() → Dict[str, Any]

Load configuration file(s) from filesystem and apply values which were set via environment variables.

Returns:

final configuration after merging configuration file(s) and environment variables.

_load_config_path(path: str) → Dict[str, Any]

Load Patroni configuration file(s) from path.

If path is a file, load the yml file pointed to by path. If path is a directory, load all yml files in that directory in alphabetical order.

Parameters:

path – path to either an YAML configuration file, or to a folder containing YAML configuration files.

Returns:

configuration after reading the configuration file(s) from path.

Raises:

ConfigParseError: if path is invalid.

static _process_postgresql_parameters(parameters: Dict[str, Any], is_local: bool = False) → Dict[str, Any]

Process Postgres parameters.

Note

If is_local configuration discard any setting from parameters that is listed under CMDLINE_OPTIONS as those are supposed to be set only through dynamic configuration.

When setting parameters from CMDLINE_OPTIONS through dynamic configuration their value will be validated as per the validator defined in that very same attribute entry. If the given value cannot be validated, a warning will be logged and the default value of the GUC will be used instead.

Some parameters from CMDLINE_OPTIONS cannot be set even if not is_local configuration:

  • listen_addresses: inferred from postgresql.listen local configuration or from

    PATRONI_POSTGRESQL_LISTEN environment variable;

  • port: inferred from postgresql.listen local configuration or from

    PATRONI_POSTGRESQL_LISTEN environment variable;

  • cluster_name: set through scope local configuration or through PATRONI_SCOPE environment

    variable;

  • hot_standby: always enabled;

  • wal_log_hints: always enabled.

Parameters:

  • parameters – Postgres parameters to be processed. Should be the parsed YAML value of postgresql.parameters configuration, either from local or from dynamic configuration.

  • is_local – should be True if parameters refers to local configuration, or False if parameters refers to dynamic configuration.

Returns:

new value for postgresql.parameters after processing and validating parameters.

_safe_copy_dynamic_configuration(dynamic_configuration: Dict[str, Any]) → Dict[str, Any]

Create a copy of dynamic_configuration.

Merge dynamic_configuration with __DEFAULT_CONFIG (dynamic_configuration takes precedence), and process postgresql.parameters from dynamic_configuration through _process_postgresql_parameters(), if present.

Note

The following settings are not allowed in postgresql section as they are intended to be local configuration, and are removed if present:

  • connect_address;

  • proxy_address;

  • listen;

  • config_dir;

  • data_dir;

  • pgpass;

  • authentication;

Besides that any setting present in dynamic_configuration but absent from __DEFAULT_CONFIG is discarded.

Parameters:

dynamic_configuration – Patroni dynamic configuration.

Returns:

copy of dynamic_configuration, merged with default dynamic configuration and with some sanity checks performed over it.

_validate_and_adjust_timeouts(config: Dict[str, Any]) → None

Validate and adjust loop_wait, retry_timeout, and ttl values if necessary.

Minimum values:

  • loop_wait: 1 second;

  • retry_timeout: 3 seconds.

  • ttl: 20 seconds;

Maximum values: In case if values don’t fulfill the following rule, retry_timeout and loop_wait are reduced so that the rule is fulfilled:

loop_wait + 2 * retry_timeout <= ttl
Parameters:

config – dict object with new global configuration.

_validate_failover_tags() → None

Check nofailover/failover_priority config and warn user if it’s contradictory.

Note

To preserve sanity (and backwards compatibility) the nofailover tag will still exist. A contradictory configuration is one where nofailover is True but failover_priority > 0, or where nofailover is False, but failover_priority <= 0. Essentially, nofailover and failover_priority are communicating different things. This checks for this edge case (which is a misconfiguration on the part of the user) and warns them. The behaviour is as if failover_priority were not provided (i.e nofailover is the bedrock source of truth)

property config_file: str | None

Path to Patroni configuration file, if any, else None.

copy() → Dict[str, Any]

Get a deep copy of effective Patroni configuration.

Returns:

a deep copy of the Patroni configuration.

property dynamic_configuration: Dict[str, Any]

Deep copy of cached Patroni dynamic configuration.

get(key: str, default: Any | None = None) → Any

Get effective value of key setting from Patroni configuration root.

Designed to work the same way as dict.get().

Parameters:

  • key – name of the setting.

  • default – default value if key is not present in the effective configuration.

Returns:

value of key, if present in the effective configuration, otherwise default.

classmethod get_default_config() → Dict[str, Any]

Deep copy default configuration.

Returns:

copy of __DEFAULT_CONFIG

get_global_config(cluster: Cluster | None) → GlobalConfig

Instantiate GlobalConfig based on input.

Use the configuration from provided cluster (the most up-to-date) or from the local cache if cluster.config is not initialized or doesn’t have a valid config.

Parameters:

cluster – the currently known cluster state from DCS.

Returns:

GlobalConfig object.

property local_configuration: Dict[str, Any]

Deep copy of cached Patroni local configuration.

Returns:

copy of _local_configuration

reload_local_configuration() → bool | None

Reload configuration values from the configuration file(s).

Note

Designed to be used when user applies changes to configuration file(s), so Patroni can use the new values with a reload instead of a restart.

Returns:

True if changes have been detected between current local configuration

save_cache() → None

Save dynamic configuration to patroni.dynamic.json under Postgres data directory.

Note

patroni.dynamic.jsonXXXXXX is created as a temporary file and than renamed to patroni.dynamic.json, where XXXXXX is a random suffix.

set_dynamic_configuration(configuration: ClusterConfig | Dict[str, Any]) → bool

Set dynamic configuration values with given configuration.

Parameters:

configuration – new dynamic configuration values. Supports dict for backward compatibility.

Returns:

True if changes have been detected between current dynamic configuration and the new dynamic configuration, False otherwise.

class patroni.config.GlobalConfig(config: Dict[str, Any])

Bases: object

A class that wraps global configuration and provides convenient methods to access/check values.

It is instantiated either by calling get_global_config() or Config.get_global_config(), which picks either a configuration from provided Cluster object (the most up-to-date) or from the local cache if ClusterConfig is not initialized or doesn’t have a valid config.

__init__(config: Dict[str, Any]) → None

Initialize GlobalConfig object with given config.

Parameters:

config – current configuration either from ClusterConfig or from Config.dynamic_configuration().

check_mode(mode: str) → bool

Checks whether the certain parameter is enabled.

Parameters:

mode – parameter name, e.g. synchronous_mode, failsafe_mode, pause, check_timeline, and so on.

Returns:

True if parameter mode is enabled in the global configuration.

get(name: str) → Any

Gets global configuration value by name.

Parameters:

name – parameter name.

Returns:

configuration value or None if it is missing.

get_int(name: str, default: int = 0) → int

Gets current value of name from the global configuration and try to return it as int.

Parameters:

  • name – name of the parameter.

  • default – default value if name is not in the configuration or invalid.

Returns:

currently configured value of name from the global configuration or default if it is not set or invalid.

get_standby_cluster_config() → Dict[str, Any] | Any

Get standby_cluster configuration.

Returns:

a copy of standby_cluster configuration.

property is_paused: bool

True if cluster is in maintenance mode.

property is_standby_cluster: bool

True if global configuration has a valid standby_cluster section.

property is_synchronous_mode: bool

True if synchronous replication is requested and it is not a standby cluster config.

property is_synchronous_mode_strict: bool

True if at least one synchronous node is required.

property maximum_lag_on_failover: int

Currently configured value of maximum_lag_on_failover from the global configuration.

Assume 1048576 if it is not set or invalid.

property maximum_lag_on_syncnode: int

Currently configured value of maximum_lag_on_syncnode from the global configuration.

Assume -1 if it is not set or invalid.

property min_synchronous_nodes: int

The minimal number of synchronous nodes based on whether synchronous_mode_strict is enabled or not.

property primary_start_timeout: int

Currently configured value of primary_start_timeout from the global configuration.

Assume 300 if it is not set or invalid.

Note

master_start_timeout is still supported to keep backward compatibility.

property primary_stop_timeout: int

Currently configured value of primary_stop_timeout from the global configuration.

Assume 0 if it is not set or invalid.

Note

master_stop_timeout is still supported to keep backward compatibility.

property synchronous_node_count: int

Currently configured value of synchronous_node_count from the global configuration.

Assume 1 if it is not set or invalid.

patroni.config.default_validator(conf: Dict[str, Any]) → List[str]

Ensure conf is not empty.

Designed to be used as default validator for Config objects, if no specific validator is provided.

Parameters:

conf – configuration to be validated.

Returns:

an empty list – Config expects the validator to return a list of 0 or more issues found while validating the configuration.

Raises:

ConfigParseError: if conf is empty.

patroni.config.get_global_config(cluster: Cluster | None, default: Dict[str, Any] | None = None) → GlobalConfig

Instantiates GlobalConfig based on the input.

Parameters:

  • cluster – the currently known cluster state from DCS.

  • default – default configuration, which will be used if there is no valid cluster.config.

Returns:

GlobalConfig object.