Utility functions

qp.utils.array: Array utility functions

Utility functions for array handling in the the qp package

qp.utils.array.edge_to_center(edges: ArrayLike) → ndarray

Return the centers of a set of bins given the edges

qp.utils.array.bin_widths(edges: ArrayLike) → ndarray

Return the widths of a set of bins given the edges

qp.utils.array.get_bin_indices(bins: ArrayLike, x: ArrayLike) → ndarray[int]

Return the bin indexes for a set of values

If the bins are equal width this will use arithmetic, If the bins are not equal width this will use a binary search

qp.utils.array.get_eval_case(x: ArrayLike, row: ArrayLike) → tuple[int, ndarray, ndarray]

Figure out which of the various input formats scipy.stats has passed us

Parameters:

xArrayLike

Pdf x-vals

rowArrayLike

Pdf row indices

Returns:

caseint

The case code

xxnp.ndarray

The x-values properly shaped

rrnp.ndarrray

The y-values, properly shaped

Notes

The cases are:

CASE_FLAT : x, row have shapes (n), (n) and do not factor CASE_FACTOR : x, row have shapes (n), (n) but can be factored to shapes (1, nx) and (npdf, 1)

(i.e., they were flattened by scipy)

CASE_PRODUCT : x, row have shapes (1, nx) and (npdf, 1) CASE_2D : x, row have shapes (npdf, nx) and (npdf, nx)

qp.utils.array.profile(x_data: ArrayLike, y_data: ArrayLike, x_bins: ArrayLike, std: bool = True) → tuple[ndarray[float], ndarray[float]]

Make a ‘profile’ plot

Parameters:

x_dataArrayLike, length n

The x-values

y_dataArrayLike, length n

The y-values

x_binsArrayLike, length nbins+1

The values of the bin edges

stdbool, optional

If true, return the standard deviations, if false return the errors on the means, default True.

Returns:

valsnp.ndarray[float], length nbins

The means

errsnp.ndarray[float], length nbins

The standard deviations or errors on the means

qp.utils.array.reshape_to_pdf_size(vals: ndarray, split_dim: int) → ndarray

Reshape an array to match the number of PDFs in a distribution

Parameters:

valsnp.ndarray

The input array

split_dimint

The dimension at which to split between pdf indices and per_pdf indices

Returns:

outnp.ndarray

The reshaped array

qp.utils.array.reshape_to_pdf_shape(vals: np.ndarray, pdf_shape: int, per_pdf: int | ArrayLike) → np.ndarray

Reshape an array to match the shape of PDFs in a distribution

Parameters:

valsnp.ndarray

The input array

pdf_shapeint

The shape for the pdfs

per_pdfint | ArrayLike

The shape per pdf

Returns:

outnp.ndarray

The reshaped array

qp.utils.array.encode_strings(data: Mapping[str, ndarray]) → Mapping[str, ndarray]

Encodes any dictionary values that are Unicode strings (or just strings if not numpy arrays). Other data types are not affected.

Parameters:

dataMapping[str, np.ndarray]

Dictionary of data to encode.

Returns:

Mapping[str, np.ndarray]

Dictionary of data with strings encoded.

qp.utils.array.decode_strings(data: Mapping[str, ndarray]) → Mapping[str, ndarray]

Decodes dictionary values that have been encoded (dtype = bytes). Other data types are not affected.

Parameters:

dataMapping[str, np.ndarray]

The dictionary of data to be decoded.

Returns:

Mapping[str, np.ndarray]

The dictionary of data with any strings decoded.

qp.utils.array.reduce_dimensions(arr: ndarray, x: ArrayLike) → float | ndarray

If the given array has dimensionality greater than x, reduces its dimensionality to match x, if this will not result in a loss of data.

Parameters:

arrnp.ndarray

Array to reduce dimensionality

xArrayLike

Object to match dimensionality to

Returns:

Union[float, np.ndarray]

The array with dimension reduced (if possible)

qp.utils.conversion: Conversion utility functions

This module implements functions to convert distributions between various representations These functions should then be registered with the qp.ConversionDict using qp_add_mapping. That will allow the automated conversion mechanisms to work.

qp.utils.conversion.extract_xy_vals(in_dist: Ensemble, xvals: np.ndarray) → dict[str, np.ndarray]

Convert using a set of x and y values.

Parameters:

in_distEnsemble

Input distributions

xvalsnp.ndarray

Locations at which the pdf is evaluated

Returns:

datadict[str, np.ndarray]

The extracted data

qp.utils.conversion.extract_fit(in_dist, **kwargs)

Convert to a functional distribution by fitting it to a set of x and y values

Parameters:

in_distEnsemble

Input distributions

Returns:

datadict

The extracted data

Other Parameters:

xvalsnp.array

Locations at which the pdf is evaluated

qp.utils.conversion.extract_voigt_mixmod(in_dist, **kwargs)

Convert to a voigt mixture model starting with a gaussian mixture model, trivially by setting gammas to 0

Parameters:

in_distEnsemble

Input distributions

Returns:

datadict

The extracted data

qp.utils.conversion.extract_voigt_xy(in_dist, **kwargs)

Build a voigt function basis and run a match-pursuit algorithm to fit gridded data

Parameters:

in_distEnsemble

Input distributions

Returns:

datadict

The extracted data as sparse indices, basis, and metadata to rebuild the basis

qp.utils.conversion.extract_voigt_xy_sparse(in_dist, **kwargs)

Build a voigt function basis and run a match-pursuit algorithm to fit gridded data

Parameters:

in_distEnsemble

Input distributions

Returns:

datadict

The extracted data as shaped parameters means, stds, weights, gammas

qp.utils.dictionary: Multi-level dictionary manipulation

This module implements tools to handle dictionaries

qp.utils.dictionary.get_val_or_default(in_dict: dict, key: str) → Any | None

Helper functions to return either an item in a dictionary or the default value of the dictionary

Parameters:

in_dictdict

input dictionary

keystr

key to search for

Returns:

outAny | None

The requested item

Notes

This will first try to return:

in_dict[key] : i.e., the requested item.

If that fails it will try

in_dict[None] : i.e., the default for that dictionary.

If that fails it will return

None

qp.utils.dictionary.set_val_or_default(in_dict: dict, key: str, val: Any)

Helper functions to either get and item from or add an item to a dictionary and return that item

Parameters:

in_dictdict

input dictionary

keystr

key to search for

valAny

item to add to the dictionary

Returns:

outAny

If the key already existed, return the current value. Otherwise, return val.

Notes

This will first try to return:

in_dict[key] : i.e., the requested item.

If that fails it will return

val

qp.utils.dictionary.pretty_print(in_dict: dict, prefixes: list, idx=0, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>) → None

Print a level of the converstion dictionary in a human-readable format

Parameters:

in_dictdict

input dictionary

prefixslist

The prefixs to use at each level of the printing

idxint

The level of the input dictionary we are currently printing

streamstream

The stream to print to

qp.utils.dictionary.print_dict_shape(in_dict: dict) → None

Print the shape of arrays in a dictionary. This is useful for debugging table creation.

Parameters:

in_dictdict

The dictionary to print

qp.utils.dictionary.slice_dict(in_dict: dict, subslice: int | slice) → dict

Create a new dict by taking a slice of of every array in a dict

Parameters:

in_dictdict

The dictionary to conver

subsliceint or slice

Used to slice the arrays

Returns:

out_dictdict

The converted dictionary

qp.utils.dictionary.check_keys(in_dicts: list[dict]) → None

Check that the keys in all the in_dicts match

Raises KeyError if one does not match.

qp.utils.dictionary.concatenate_dicts(in_dicts: list[dict], add_axis: int = 0) → dict

Create a new dict by concatenate each array in in_dicts

Parameters:

in_dictslist[dict]

The dictionaries to stack

add_axis: int

The axis to add when ensuring an array is 2D

Returns:

out_dictdict

The stacked dicionary

qp.utils.dictionary.check_array_shapes(in_dict: dict, npdf: int) → None

Check that all the arrays in in_dict match the number of pdfs

Raises ValueError if one does not match.

qp.utils.dictionary.compare_two_dicts(d1: dict, d2: dict) → bool

Check that all the items in d1 and d2 match

Returns:

matchbool

True if they all match, False otherwise

qp.utils.dictionary.compare_dicts(in_dicts: list[dict]) → bool

Check that all the dicts in in_dicts match

Returns:

matchbool

True if they all match, False otherwise

qp.utils.dictionary.reduce_arrays_to_1d(in_dict: Mapping) → Mapping

Checks if any arrays in the dictionary have ndim greater than 1, and if the first dimension is equal to 1 it reshapes the array to remove that dimension.

Parameters:

in_dictMapping

A dictionary of array-like objects.

Returns:

Mapping

The updated dictionary.

qp.utils.dictionary.make_len_equal(in_dict: Mapping[str, ndarray], l_arr: int = 1) → Mapping

Ensures that all arrays in the dictionary have shape[0] of at least l_arr.

This essentially assures that a dictionary of numpy arrays is a numpyDict Table-like object according to tables_io to allow for writing.

Parameters:

in_dictMapping

A dictionary of array-like objects.

l_arrint, optional

The value of shape[0] to ensure, by default 1

Returns:

Mapping

The updated dictionary.

qp.utils.dictionary.expand_dimensions(in_dict: Mapping[str, ndarray], npdf: int, nvals: int) → Mapping

qp.utils.dictionary.ensure_2d_array(arr: ndarray, add_axis: int = 0) → ndarray

Makes sure that the input array is at least 2 dimensions, by adding a new axis to 1D arrays. By default, the new axis is added as axis=0, so the new array will have shape (1, len(arr)).

Parameters:

arrnp.ndarray

The input array

add_axisint

Where to add the new axis.

Returns:

np.ndarray

Returns the input array if it’s 2D, or an array with an extra dimension if given a 1D array.

qp.utils.interpolation: PDF evaluation and construction utility functions

qp.utils.interpolation.interpolate_multi_x_y(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, shape (npdf, n)

X values to interpolate at

rowArrayLike, shape (npdf, 1)

Which rows to interpolate at

xvalsArrayLike, shape (npdf, npts)

X-values used for the interpolation

yvalsArrayLike, shape (npdf)

Y-values used for the interpolation

Returns:

valsnp.ndarray

The interpolated values

qp.utils.interpolation.interpolate_multi_x_y_product(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, length n

X values to interpolate at

rowArrayLike, shape (npdf, 1)

Which rows to interpolate at

xvalsArrayLike, shape (npdf, npts)

X-values used for the interpolation

yvalsArrayLike, length npdf

Y-values used for the interpolation

Returns:

valsnp.ndarray, shape (npdf, n)

The interpolated values

qp.utils.interpolation.interpolate_multi_x_y_2d(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, shape (npdf, n)

X values to interpolate at

rowArrayLike, shape (npdf, 1)

Which rows to interpolate at

xvalsArrayLike, shape (npdf, npts)

X-values used for the interpolation

yvalsArrayLike, length npdf

Y-values used for the interpolation

Returns:

valsnp.ndarray, shape (npdf, n)

The interpolated values

qp.utils.interpolation.interpolate_multi_x_y_flat(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, length n

X values to interpolate at

rowArrayLike, length n

Which rows to interpolate at

xvalsArrayLike, shape (npdf, npts)

X-values used for the interpolation

yvalsArrayLike, length npdf

Y-values used for the interpolation

Returns:

valsnp.ndarray, shape (npdf, n)

The interpolated values

qp.utils.interpolation.interpolate_x_multi_y_product(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, length n

X values to interpolate at

rowArrayLike, shape (npdf, 1)

Which rows to interpolate at

xvalsArrayLike, length npts

X-values used for the interpolation

yvalsArrayLike, shape (npdf, npts)

Y-values used for the interpolation

Returns:

valsnp.ndarray, shape (npdf, n)

The interpolated values

qp.utils.interpolation.interpolate_x_multi_y(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, shape (npdf, n)

X values to interpolate at

rowArrayLike, shape (npdf, 1)

Which rows to interpolate at

xvalsArrayLike, length npts

X-values used for the interpolation

yvalsArrayLike, shape (npdf, npts)

Y-values used for the interpolation

Returns:

valsnp.ndarray

The interpolated values

qp.utils.interpolation.interpolate_x_multi_y_2d(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, shape (npdf, n)

X values to interpolate at

rowArrayLike, shape (npdf, 1)

Which rows to interpolate at

xvalsArrayLike, length npts

X-values used for the interpolation

yvalsArrayLike, shape (npdf, npts)

Y-values used for the interpolation

Returns:

valsnp.ndarray, shape (npdf, n)

The interpolated values

qp.utils.interpolation.interpolate_x_multi_y_flat(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, length n

X values to interpolate at

rowArrayLike, length n

Which rows to interpolate at

xvalsArrayLike, length npts

X-values used for the interpolation

yvalsArrayLike, shape (npdf, npts)

Y-values used for the interpolation

Returns:

valsnp.ndarray, shape (npdf, n)

The interpolated values

qp.utils.interpolation.interpolate_multi_x_multi_y_flat(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, length n

X values to interpolate at

rowArrayLike, length n

Which rows to interpolate at

xvalsArrayLike, shape (npdf, npts)

X-values used for the interpolation

yvalsArrayLike, shape (npdf, npts)

Y-values used for the interpolation

Returns:

valsnp.ndarray, shape (npdf, n)

The interpolated values

qp.utils.interpolation.interpolate_multi_x_multi_y_product(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, length n

X values to interpolate at

rowArrayLike, shape (npdf, 1)

Which rows to interpolate at

xvalsArrayLike, shape (npdf, npts)

X-values used for the interpolation

yvalsArrayLike, shape (npdf, npts)

Y-values used for the interpolation

Returns:

valsnp.ndarray, shape (npdf, n)

The interpolated values

qp.utils.interpolation.interpolate_multi_x_multi_y_2d(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, shape (npdf, n)

X values to interpolate at

rowArrayLike, shape (npdf, 1)

Which rows to interpolate at

xvalsArrayLike, shape (npdf, npts)

X-values used for the interpolation

yvalsArrayLike, shape (npdf, npts)

Y-values used for the interpolation

Returns:

valsnp.ndarray, shape (npdf, n)

The interpolated values

qp.utils.interpolation.interpolate_multi_x_multi_y(x: ArrayLike, row: ArrayLike, xvals: ArrayLike, yvals: ArrayLike, **kwargs) → ndarray

Interpolate a set of values

Parameters:

xArrayLike, shape (npdf, n)

X values to interpolate at

rowArrayLike, shape (npdf, 1)

Which rows to interpolate at

xvalsArrayLike, shape (npdf, npts)

X-values used for the interpolation

yvalsArrayLike, shape (npdf, npts)

Y-values used for the interpolation

Returns:

valsnp.ndarray

The interpolated values