Using configuration files

Most of the TreeCorr classes can take a config parameter in lieu of a set of keyword arguments. This is not necessarily incredibly useful when driving the code from Python; however, it enables running the code from some executable scripts, described below.

Specifically, the parameters defined in the configuration file are loaded into a Python dict, which is passed to each of the classes as needed. The advantage of this is that TreeCorr will only use the parameters it actually needs when initializing each object. Any additional parameters (e.g. those that are relevant to a different class) are ignored.

The corr2 and corr3 executables

Along with the installed Python library, TreeCorr also includes two executable scripts, called corr2 and corr3. The scripts takes one required command-line argument, which is the name of a configuration file:

corr2 config.yaml
corr3 config.yaml

A sample configuration file is provided, called sample_config.yaml.

For the complete documentation about the allowed parameters, see:

• Configuration Parameters
  • Parameters about the input file(s)
  • Parameters about the binned correlation function to be calculated
  • Parameters about the output file(s)
  • Derived output quantities
  • Miscellaneous parameters

YAML is the recommended format for the configuration file, but we also allow JSON files if you prefer, or a legacy format, which is like an .ini file, but without the section headings, consisting of key = value lines. The three formats are normally distinguished by their extensions (.yaml, .json, or .params respectively), but you can also give the file type explicitly with the -f option. E.g.:

corr2 my_config_file.txt -f params

would specify that the configuration file my_config_file.txt uses the legacy “params” format.

You can also specify parameters on the command line after the name of the configuration file. e.g.:

corr2 config.yaml file_name=file1.dat gg_file_name=file1.out
corr2 config.yaml file_name=file2.dat gg_file_name=file2.out
...

This can be useful when running the program from a script for lots of input files.

The corr2 function from python

The same functionality that you have from the corr2 executable is available in python via the corr2 function:

import treecorr
config = treecorr.read_config(config_file)
config['file_name'] = 'catalog.dat'
config['gg_file_name'] = 'gg.out'
treecorr.corr2(config)

treecorr.corr2(config, logger=None)

Run the full two-point correlation function code based on the parameters in the given config dict.

The function print_corr2_params will output information about the valid parameters that are expected to be in the config dict.

Optionally a logger parameter maybe given, in which case it is used for logging. If not given, the logging will be based on the verbose and log_file parameters.

Parameters:

• config – The configuration dict which defines what to do.

• logger – If desired, a logger object for logging. (default: None, in which case one will be built according to the config dict’s verbose level.)

The corr3 function from python

treecorr.corr3(config, logger=None)

Run the full three-point correlation function code based on the parameters in the given config dict.

The function print_corr3_params will output information about the valid parameters that are expected to be in the config dict.

Optionally a logger parameter maybe given, in which case it is used for logging. If not given, the logging will be based on the verbose and log_file parameters.

Parameters:

• config – The configuration dict which defines what to do.

• logger – If desired, a logger object for logging. (default: None, in which case one will be built according to the config dict’s verbose level.)

Other utilities related to corr2 and corr3

treecorr.exec_corr2.print_corr2_params()

Print information about the valid parameters that may be given to the corr2 function.

treecorr.exec_corr3.print_corr3_params()

Print information about the valid parameters that may be given to the corr3 function.

Utilities related to the configuration dict

treecorr.config.check_config(config, params, aliases=None, logger=None)

Check (and update) a config dict to conform to the given parameter rules. The params dict has an entry for each valid config parameter whose value is a tuple with the following items:

• type

• can be a list?

• default value

• valid values

• description (Multiple entries here are allowed for longer strings)

The file corr2.py has a list of parameters for the corr2 program.

Parameters:

• config – The config dict to check.

• params – A dict of valid parameters with information about each one.

• aliases – A dict of deprecated parameters that are still aliases for new names. (default: None)

• logger – If desired, a logger object for logging any warnings here. (default: None)

Returns:

The updated config dict.

treecorr.config.convert(value, value_type, key)

Convert the given value to the given type.

The key helps determine what kind of conversion should be performed. Specifically if ‘unit’ is in the key value, then a unit conversion is done. Otherwise, it just parses the value according to the value_type.

Parameters:

• value – The input value to be converted. Usually a string.

• value_type – The type to convert to.

• key – The key for this value. Only used to see if it includes ‘unit’.

Returns:

The converted value.

treecorr.config.get(config, key, value_type=<class 'str'>, default=None)

A helper function to get a key from config converting to a particular type

Parameters:

• config – The configuration dict from which to get the key value.

• key – Which key to get from config.

• value_type – Which type should the value be converted to. (default: str)

• default – What value should be used if the key is not in the config dict, or the value corresponding to the key is None. (default: None)

Returns:

The specified value, converted as needed.

treecorr.config.get_from_list(config, key, num, value_type=<class 'str'>, default=None)

A helper function to get a key from config that is allowed to be a list

Some of the config values are allowed to be lists of values, in which case we take the num item from the list. If they are not a list, then the given value is used for all values of num.

Parameters:

• config – The configuration dict from which to get the key value.

• key – What key to get from config.

• num – Which number element to use if the item is a list.

• value_type – What type should the value be converted to. (default: str)

• default – What value should be used if the key is not in the config dict, or the value corresponding to the key is None. (default: None)

Returns:

The specified value, converted as needed.

treecorr.config.make_minimal_config(config, valid_params)

Make a minimal version of a config dict, excluding any values that are the default.

Parameters:

• config (dict) – The source config (will not be modified)

• valid_params (dict) – A dict of valid parameters that are allowed for this usage.

Returns:

minimal_config The dict without any default values.

treecorr.config.merge_config(config, kwargs, valid_params, aliases=None)

Merge in the values from kwargs into config.

If either of these is None, then the other one is returned. If they are both dicts, then the values in kwargs take precedence over ones in config if there are any keys that are in both. Also, the kwargs dict will be modified in this case.

Parameters:

• config – The root config (will not be modified)

• kwargs – A second dict with more or updated values

• valid_params – A dict of valid parameters that are allowed for this usage. The config dict is allowed to have extra items, but kwargs is not.

• aliases – An optional dict of aliases. (default: None)

Returns:

The merged dict, including only items that are in valid_params.

treecorr.config.parse(value, value_type, name)

Parse the input value as the given type.

Parameters:

• value – The value to parse.

• value_type – The type expected for this.

• name – The name of this value. Only used for error reporting.

Returns:

value

treecorr.config.parse_bool(value)

Parse a value as a boolean.

Valid string values for True are: ‘true’, ‘yes’, ‘t’, ‘y’ Valid string values for False are: ‘false’, ‘no’, ‘f’, ‘n’, ‘none’ Capitalization is ignored.

If value is a number, it is converted to a bool in the usual way.

Parameters:

value – The value to parse.

Returns:

The value converted to a bool.

treecorr.config.parse_unit(value)

Parse the input value as a string that should be one of the valid angle units in coord.AngleUnit.valid_names.

The value is allowed to merely start with one of the unit names. So ‘deg’, ‘degree’, ‘degrees’ all convert to ‘deg’ which is the name in coord.AngleUnit.valid_names. The return value in this case would be coord.AngleUnit.from_name(‘deg’).value, which has the value pi/180.

Parameters:

value – The unit as a string value to parse.

Returns:

The given unit in radians.

treecorr.config.parse_variable(config, v)

Parse a configuration variable from a string that should look like ‘key = value’ and write that value to config[key].

Parameters:

• config – The configuration dict to wich to write the key,value pair

• v – A string of the form ‘key = value’

treecorr.config.print_params(params)

Print the information about the valid parameters, given by the given params dict. See check_config for the structure of the params dict.

Parameters:

params – A dict of valid parameters with information about each one.

treecorr.config.read_config(file_name, file_type='auto')

Read a configuration dict from a file.

Parameters:

• file_name – The file name from which the configuration dict should be read.

• file_type – The type of config file. Options are ‘auto’, ‘yaml’, ‘json’, ‘params’. (default: ‘auto’, which tries to determine the type from the extension)

Returns:

A config dict built from the configuration file.

treecorr.config.setup_logger(verbose, log_file=None, name=None)

Parse the integer verbosity level from the command line args into a logging_level string

Parameters:

• verbose – An integer indicating what verbosity level to use.

• log_file – If given, a file name to which to write the logging output. If omitted or None, then output to stdout.

Returns:

The logging.Logger object to use.

File Writers

class treecorr.writer.FitsWriter(file_name, *, logger=None)

Writer interface for FITS files.

write(col_names, columns, *, params=None, ext=None)

Write some columns to an output ASCII file with the given column names.

If name is not None, then it is used as the name of the extension for these data.

Parameters:

• col_names – A list of columns names for the given columns.

• columns – A list of numpy arrays with the data to write.

• params – A dict of extra parameters to write at the top of the output file.

• ext – Optional ext name for these data. (default: None)

write_array(data, *, ext=None)

Write a (typically 2-d) numpy array to the output file.

Parameters:

• data – The array to write.

• ext – Optional ext name for these data. (default: None)

class treecorr.writer.HdfWriter(file_name, *, logger=None)

Writer interface for HDF5 files. Uses h5py to read columns, etc.

write(col_names, columns, *, params=None, ext=None)

Write some columns to an output ASCII file with the given column names.

If name is not None, then it is used as the name of the extension for these data.

Parameters:

• col_names – A list of columns names for the given columns.

• columns – A list of numpy arrays with the data to write.

• params – A dict of extra parameters to write at the top of the output file.

• ext – Optional group name for these data. (default: None)

write_array(data, *, ext=None)

Write a (typically 2-d) numpy array to the output file.

Parameters:

• data – The array to write.

• ext – Optional ext name for these data. (default: None)

class treecorr.writer.AsciiWriter(file_name, *, precision=4, logger=None)

Write data to an ASCII (text) file.

write(col_names, columns, *, params=None, ext=None)

Write some columns to an output ASCII file with the given column names.

Parameters:

• col_names – A list of columns names for the given columns. These will be written in a header comment line at the top of the output file.

• columns – A list of numpy arrays with the data to write.

• params – A dict of extra parameters to write at the top of the output file.

• ext – Optional ext name for these data. (default: None)

write_array(data, *, ext=None)

Write a (typically 2-d) numpy array to the output file.

Parameters:

• data – The array to write.

• ext – Optional ext name for these data. (default: None)