dymoval.dataset module

Module containing everything related to measurements datasets. Here are defined special datatypes, classes and auxiliary functions.

class Dataset(name, signal_list, u_names, y_names, target_sampling_period=None, tin=None, tout=None, full_time_interval=False, overlap=False, verbosity=0)

Bases: object

The Dataset class stores measurements datasets and it provides methods for analyzing and manipulating them.

A Signal list shall be passed to the initializer along with:

  1. the list of signal names that shall be considered as input

  2. the list of signal names that shall be considered as output

The initializer will attempt to resample all the signals to have the same sampling period. Signals that cannot be resampled will be excluded from the Dataset and will be stored in the excluded_signals attribute.

Furthermore, all the signals will be trimmed to have the same length.

If none of tin, tout and full_time_interval arguments are passed, then the Dataset time-interval selection is done graphically.

Example

>>> # How to create a Dataset object from a list of Signals
>>>
>>> import numpy as np
>>> import dymoval as dmv
>>>
>>> signal_names = [
>>>     "SpeedRequest",
>>>     "AccelPedalPos",
>>>     "OilTemp",
>>>     "ActualSpeed",
>>>     "SteeringAngleRequest",
>>> ]
>>> signal_values = np.random.rand(5, 100)
>>> signal_units = ["m/s", "%", "°C", "m/s", "deg"]
>>> sampling_periods = [0.1, 0.1, 0.1, 0.1, 0.1]
>>> # Create dymoval signals
>>> signals = []
>>> for ii, val in enumerate(signal_names):
>>>     tmp: dmv.Signal = {
>>>         "name": val,
>>>         "samples": signal_values[ii],
>>>         "signal_unit": signal_units[ii],
>>>         "sampling_period": sampling_periods[ii],
>>>         "time_unit": "s",
>>>     }
>>>     signals.append(tmp)
>>> # Validate signals
>>> dmv.validate_signals(*signals)
>>> # Specify which signals are inputs and which signals are output
>>> input_labels = ["SpeedRequest", "AccelPedalPos", "SteeringAngleRequest"]
>>> output_labels = ["ActualSpeed", "OilTemp"]
>>> # Create dymoval Dataset objects
>>> ds = dmv.Dataset("my_dataset", signals, input_labels, output_labels)
>>>
>>> # At this point you can plot, trim, manipulate, analyze, etc you dataset
>>> # through the Dataset class methods.

You can also create Dataset objects from pandas DataFrames if the DataFrames have a certain structure. Look at validate_dataframe() for more information about this option.

Example

>>> # How to create a Dataset object from a pandas DataFrame
>>>
>>> import dymoval as dmv
>>> import pandas as pd
>>>
>>> # Signals names, units and values
>>> signal_names = [
>>>     "SpeedRequest",
>>>     "AccelPedalPos",
>>>     "OilTemp",
>>>     "ActualSpeed",
>>>     "SteeringAngleRequest",
>>> ]
>>> signal_units = ["m/s", "%", "°C", "m/s", "deg"]
>>> signal_values = np.random.rand(100, 5)
>>>
>>> # time axis
>>> sampling_period = 0.1
>>> timestamps = np.arange(0, 10, sampling_period)
>>>
>>> # Build a candidate DataFrame
>>> cols = list(zip(signal_names, signal_units))
>>> index_name = ("Time", "s")
>>>
>>> # Create dymoval signals
>>> df = pd.DataFrame(data=signal_values, columns=cols)
>>> df.index = pd.Index(data=timestamps, name=index_name)
>>>
>>> # Check if the dataframe is suitable for a dymoval Dataset
>>> dmv.validate_dataframe(df)
>>>
>>> # Specify input and output signals
>>> input_labels = ["SpeedRequest", "AccelPedalPos", "SteeringAngleRequest"]
>>> output_labels = ["ActualSpeed", "OilTemp"]
>>>
>>> # Create dymoval Dataset objects
>>> ds = dmv.Dataset("my_dataset", df, input_labels, output_labels)
Parameters:

  • name (str) – Dataset name.

  • signal_list (list[Signal] | DataFrame) – Signals to be included in the Dataset object.

  • u_names (str | list[str]) – List of input signal names. Each signal name must be unique and must be contained in the signal_list argument.

  • y_names (str | list[str]) – List of input signal names. Each signal name must be unique and must be contained in the signal_list argument.

  • target_sampling_period (float | None) – The passed signals will be re-sampled at this sampling period. If some signal could not be resampled, then its name will be added in the excluded_signals attribute.

  • tin (float | None) – Initial time instant of the Dataset.

  • tout (float | None) – Final time instant of the Dataset.

  • full_time_interval (bool) – If True, the Dataset time interval will be equal to the longest time interval among all of the signals included in the signal_list argument. This is overriden if the arguments tin and tout are passed.

  • overlap (bool) – If True it will overlap the input and output signals plots in the Dataset time interval graphical selection. The units of the outputs are displayed on the secondary y-axis.

  • verbosity (int) – Display information depending on its level. Higher numbers correspond to higher verbosity.

add_input(*signals)

Add input signals to the Dataset object.

Signals will be trimmed to the length of the Dataset. Shorter signals will be padded with NaN:s.

Parameters:

*signals (Signal) – Input signals to be added.

Return type:

Self

add_output(*signals)

Add output signals to the Dataset object.

Signals will be trimmed to the length of the Dataset. Shorter signals will be padded with NaN:s.

Parameters:

*signals (Signal) – Output signals to be added.

Return type:

Self

apply(*signal_function, **kwargs)

Apply a function to specified signals and change their unit.

Note

If you need to heavily manipulate your signals, it is suggested to dump the Dataset into Signals through dump_to_signals(), manipulate them, and then create a brand new Dataset object.

Warning

This function may be removed in the future.

Parameters:

  • signal_function (tuple[str, Any, str]) – Signals where to apply a function. This argument shall have the form (name, func, new_unit).

  • **kwargs (Any) – Additional keyword arguments to pass as keywords arguments to the underlying pandas DataFrame apply method.

Return type:

Self

coverage: DataFrame

Measurements dataset coverage.

dataset: DataFrame

Measurements dataset values.

dataset_values()

Return the dataset values as a tuple (t,u,y) of numpy ndarrays.

Return type:

tuple[ndarray, ndarray, ndarray]

Returns:

  • t – The dataset time interval.

  • u – The values of the input signal.

  • y – The values of the output signal.

detrend(*signals)

Linearly detrend the specified signals.

Parameters:

*signals (str) – Linearly detrend the specified signals. If not specified, then the mean value is removed to all the input signals in the dataset.

Return type:

Self

dump_to_signals()

Dump a Dataset object into a list of Signals objects.

Return type:

dict[Literal['INPUT', 'OUTPUT'], list[Signal]]

Warning

Additional information contained in the Dataset, such as NaNs intervals, coverage region, etc. are lost.

excluded_signals: list[str]

Excluded signals during the re-sampling process.

export_to_mat(filename)

Write the dataset in a .mat file.

Parameters:

filename (str) – Target filename.

Return type:

None

fft(*signals)

Return the FFT of the dataset as pandas DataFrame.

It only works with real-valued signals.

Parameters:

signals (str) – The FFT is computed for these signals.

Return type:

DataFrame

low_pass_filter(*signals_cutoffs)

Low-pass filter a list of specified signals.

The low-pass filter is first-order IIR filter.

Parameters:

*signals_cutoffs (tuple[str, float]) – Tuples of the form (signal_name, cutoff_frequency). The values of signal_name are low-pass filtered with a first-order low-pass filter with cutoff frequency cutoff_frequency

Return type:

Self

name: str

Measurements dataset name.

plot(*signals, overlap=False, linecolor_input='blue', linestyle_fg='-', alpha_fg=1.0, linecolor_output='green', linestyle_bg='--', alpha_bg=1.0, _grid=None, layout='tight', ax_height=1.8, ax_width=4.445)

Plot the measurements Dataset.

If two signals are passed as a tuple, then they will be placed in the same subplot. For example, if ds is a Dataset object with signals s1, s2, … sn, then ds.plot(("s1", "s2"), "s3", "s4") will plot s1 and s2 on the same subplot and it will plot s3 and s4 on separate subplots, thus displaying the total of three subplots.

Possible values for the parameters describing the line used in the plot (e.g. linecolor_input , alpha_output. etc). are the same for the corresponding plot function in matplotlib.

Note

It is possible to overlap at most two signals (this to avoid adding too many y-axes in the same subplot).

Example

>>> fig = ds.plot() # ds is a dymoval Dataset
>>> fig = ds.plot(("u1","y3"),"y1") # Signals u1 and y3 will be placed
# in the same subplot whereas y1 will be placed in another subplot
# The following are methods of the class `matplotlib.figure.Figure`
>>> fig.set_size_inches(10,5)
>>> fig.set_layout_engine("constrained")
>>> fig.savefig("my_plot.svg")
Parameters:

  • *signals (str | tuple[str, str] | None) – Signals to be plotted.

  • overlap (bool) – If True overlap input the output signals plots pairwise. Eventual signals passed as argument will be discarded. The units of the outputs are displayed on the secondary y-axis.

  • linecolor_input (Tuple[float, float, float, float] | str) – Line color of the input signals.

  • linestyle_fg (str) – Line style of the first signal of the tuple if two signals are passed as a tuple.

  • alpha_fg (float) – Transparency value of the first signal of the tuple if two signals are passed as a tuple.

  • linecolor_output (Tuple[float, float, float, float] | str) – Line color of the output signals.

  • linestyle_bg (str) – Line style of the second signal of the tuple if two signals are passed as a tuple.

  • alpha_bg (float) – Transparency value of the second signal of the tuple if two signals are passed as a tuple.

  • _grid (GridSpec | None) – Grid where the spectrum ploat will be placed (Used only internally.)

  • layout (Literal['constrained', 'compressed', 'tight', 'none']) – Figure layout.

  • ax_height (float) – Approximative height (inches) of each subplot.

  • ax_width (float) – Approximative width (inches) of each subplot.

Return type:

Figure

plot_coverage(*signals, nbins=100, linecolor_input='b', linecolor_output='g', alpha=1.0, histtype='bar', _grid=None, layout='tight', ax_height=1.8, ax_width=4.445)

Plot the dataset Dataset coverage in histograms.

Example

>>> fig = ds.plot_coverage() # ds is a dymoval Dataset
>>> fig = ds.plot_coverage("u1","y3","y1")
# The following are methods of the class `matplotlib.figure.Figure`
>>> fig.set_size_inches(10,5)
>>> fig.set_layout_engine("constrained")
>>> fig.savefig("my_plot.svg")
Parameters:

  • *signals (str) – The coverage of these signals will be plotted.

  • nbins (int) – The number of bins in the x-axis.

  • linecolor_input (Tuple[float, float, float, float] | str) – Line color for the input signals.

  • linecolor_output (Tuple[float, float, float, float] | str) – Line color for the output signals.

  • alpha (float) – Transparency value for the plots.

  • histtype (Literal['bar', 'barstacked', 'step', 'stepfilled']) – Histogram aesthetic.

  • _grid (GridSpec | None) – Grid where the spectrum ploat will be placed (Used only internally.)

  • layout (Literal['constrained', 'compressed', 'tight', 'none']) – Figure layout.

  • ax_height (float) – Approximative height (inches) of each subplot.

  • ax_width (float) – Approximative width (inches) of each subplot.

Return type:

Figure

plot_spectrum(*signals, kind='power', overlap=False, linecolor_input='blue', linestyle_fg='-', alpha_fg=1.0, linecolor_output='green', linestyle_bg='--', alpha_bg=1.0, _grid=None, layout='tight', ax_height=1.8, ax_width=4.445)

Plot the spectrum of the specified signals in the dataset in different format.

If some signals have NaN values, then the FFT cannot be computed and an error is raised.

Example

>>> fig = ds.plot_spectrum() # ds is a dymoval Dataset
>>> fig = ds.plot_spectrum(("u1","y3"),"u2", kind ="amplitude")
# The following are methods of the class `matplotlib.figure.Figure`
>>> fig.set_size_inches(10,5)
>>> fig.set_layout_engine("constrained")
>>> fig.savefig("my_plot.svg")
Parameters:

  • *signals (str | tuple[str, str] | None) – The spectrum of these signals will be plotted.

  • kind (Literal['amplitude', 'power', 'psd']) –

    • amplitude plot both the amplitude and phase spectrum. If the signal has unit V, then the amplitude has unit V. Angle is in degrees.

    • power plot the auto-power spectrum. If the signal has unit V, then the amplitude has unit V^2.

    • psd plot the power density spectrum. If the signal has unit V and the time is s, then the amplitude has unit V^2/Hz.

  • overlap (bool) – If True it overlaps the input and the output signals plots. The units of the outputs are displayed on the secondary y-axis.

  • linecolor_input (Tuple[float, float, float, float] | str) – Line color of the input signals.

  • linestyle_fg (str) – Line style of the foreground signal in case of overlapping plots.

  • alpha_fg (float) – Transparency value of the foreground signal in case of overlapping plots.

  • linecolor_output (Tuple[float, float, float, float] | str) – Line color for the output signals.

  • linestyle_bg (str) – Line style for the background signal in case of overlapping plots.

  • alpha_bg (float) – Transparency value of background signal in case of overlapping plots.

  • _grid (GridSpec | None) – Grid where the spectrum plot will be placed (Used only internally.)

  • layout (Literal['constrained', 'compressed', 'tight', 'none']) – Figure layout.

  • ax_height (float) – Approximative height (inches) of each subplot.

  • ax_width (float) – Approximative width (inches) of each subplot.

Return type:

Figure

Example

>>> fig = ds.plot() # ds is a dymoval Dataset
# The following are methods of the class `matplotlib.figure.Figure`
>>> fig.set_size_inches(10,5)
>>> fig.set_layout_engine("constrained")
>>> fig.savefig("my_plot.svg")
plotxy(*signal_pairs, layout='tight', ax_height=1.8, ax_width=4.445)

Plot a signal against another signal in a plane (XY-plot).

The signal_pairs shall be passed as tuples. If no signal_pairs is passed then the function will zip the input and output signals.

Example

>>> fig = ds.plotxy() # ds is a dymoval Dataset
>>> fig = ds.plotxy(("u1","y3"),("u2","y1"))
# The following are methods of the class `matplotlib.figure.Figure`
>>> fig.set_size_inches(10,5)
>>> fig.set_layout_engine("constrained")
>>> fig.savefig("my_plot.svg")
Parameters:

  • signals_pairs – Pairs of signals to plot in a XY-diagram.

  • layout (Literal['constrained', 'compressed', 'tight', 'none']) – Figure layout.

  • ax_height (float) – Approximative height (inches) of each subplot.

  • ax_width (float) – Approximative width (inches) of each subplot.

Return type:

Figure

remove_NaNs(**kwargs)

Replace NaN:s values in the Dataset.

It uses pandas DataFrame.interpolate() method, so that the **kwargs* are directly routed to such a method.

Parameters:

**kwargs (Any) – Keyword arguments to pass on to the interpolating function.

Return type:

Self

remove_means(*signals)

Remove the mean value to the specified signals.

Parameters:

*signals (str) – Remove means to the specified signals. If not specified, then the mean value is removed to all the input signals in the dataset.

Return type:

Self

remove_offset(*signals_values)

Remove specified offsets to the specified signals.

For each target signal a tuple of the form (name,value) shall be passed. The value specified in the offset parameter is removed from the signal with name name.

Example

>>> fig = ds.remove_offset(("u1",0.4),("u2",-1.5)) # ds is a Dataset
Parameters:

*signals – Tuples of the form (name, offset). The name parameter must match the name of a signal stored in the Dataset. The offset parameter is the value to remove to the name signal.

Return type:

Self

remove_signals(*signals)

Remove signals from the Dataset.

Parameters:

signals (str) – Signal name to be removed.

Return type:

Self

sampling_period: float

Measurements dataset sampling period.

signal_list()

Return the list of signals in form ([“INPUT” | “OUTPUT”], name, unit)

Return type:

list[tuple[str, str, str]]

trim(*signals, tin=None, tout=None, verbosity=0, **kwargs)

Trim the Dataset Dataset object.

If not tin or tout are passed, then the selection is made graphically.

Parameters:

  • *signals (str | tuple[str, str] | None) – Signals to be plotted in case of trimming from a plot.

  • tin (float | None) – Initial time of the desired time interval

  • tout (float | None) – Final time of the desired time interval.

  • verbosity (int) – Depending on its level, more or less info is displayed. The higher the value, the higher is the verbosity.

  • **kwargs (Any) – kwargs to be passed to the Dataset.plot() method.

Return type:

Self

class Signal

Bases: TypedDict

Signals are used to represent real-world measurements.

They are used to instantiate Dataset objects. Before instantiating a Dataset object, it is good practice to validate Signals through the validate_signals() function.

Although Signals have compulsory attribtues, there is freedom to append additional ones.

Example

>>> # How to create a simple dymoval Signal
>>> import dymoval as dmv
>>>
>>> my_signal: dmv.Signal = {
"name": "speed",
"samples": np.random.rand(100),
"signal_unit": "mps",
"sampling_period": 0.1,
"time_unit": "s",
}
>>> dmv.plot_signals(my_signal)
name: str

Signal name.

samples: ndarray

Signal samples.

sampling_period: float

Signal sampling period.

signal_unit: str

Signal samples unit.

time_unit: str

Signal time unit.

change_axes_layout(fig, nrows, ncols)

Change Axes layout of an existing Matplotlib Figure.

Parameters:

  • fig (Figure) – Reference figure.

  • nrwos – New number or rows.

  • ncols (int) – New number of columns

Return type:

tuple[Figure, list[Axes]]

compare_datasets(*datasets, kind='time', layout='tight', ax_height=1.8, ax_width=4.445)

Compare different measurements Datasets graphically by overlapping them.

Example

>>> import dymoval as dmv
>>> fig = dmv.compare_datasets(ds,ds1,ds2, kind="coverage")
# The following are methods of the class `matplotlib.figure.Figure`
>>> fig.set_size_inches(10,5)
>>> fig.set_layout_engine("constrained")
>>> fig.savefig("my_plot.svg")
Parameters:

  • *datasets (Dataset) – Datasets to be compared.

  • kind (Literal['time', 'coverage'] | Literal['amplitude', 'power', 'psd']) – Kind of graph to be plotted.

  • layout (Literal['constrained', 'compressed', 'tight', 'none']) – Figure layout.

  • ax_height (float) – Approximative height (inches) of each subplot.

  • ax_width (float) – Approximative width (inches) of each subplot.

Return type:

Figure

plot_signals(*signals)

Plot Signals.

Example

>>> fig = dmv.plot_signals(s1,s2,s3) # s1, s2 and s3 are dymoval Signal objects.
# The following are methods of the class `matplotlib.figure.Figure`
>>> fig.set_size_inches(10,5)
>>> fig.set_layout_engine("constrained")
>>> fig.savefig("my_plot.svg")
Parameters:

*signals (Signal) – Signals to be plotted.

Return type:

Figure

validate_dataframe(df)

Check if a pandas DataFrame is suitable for instantiating a Dataset object.

The index of the DataFrame shall represent the common timeline for all the signals, whereas the j-th column values shall represents the realizations of the j-th signal.

The column names are tuples of strings of the form (signal_name, signal_unit).

It must be specified which signal(s) are the input through the u_names and which signal(s) is the output through the y_names argument.

The candidate DataFrame shall meet the following requirements

  • Columns names shall be unique,

  • Columns name shall be a tuple of str of the form (name, unit),

  • The index shall represent the common timeline for all and its name shall be ‘(Time, time_unit)’, where time_unit is a string.

  • Each signal must have at least two samples (i.e. the DataFrame has at least two rows),

  • Only one index and columns levels are allowed (no MultiIndex),

  • There shall be at least two signals representing one input and one output,

  • Both the index values and the column values must be float and the index values must be a a 1D vector of monotonically, equi-spaced, increasing floats.

Example

>>> # How to create a Dataset object from a pandas DataFrame
>>>
>>> import dymoval as dmv
>>> import pandas as pd
>>>
>>> # Signals names, units and values
>>> signal_names = [
>>>     "SpeedRequest",
>>>     "AccelPedalPos",
>>>     "OilTemp",
>>>     "ActualSpeed",
>>>     "SteeringAngleRequest",
>>> ]
>>> signal_units = ["m/s", "%", "°C", "m/s", "deg"]
>>> signal_values = np.random.rand(100, 5)
>>>
>>> # time axis
>>> sampling_period = 0.1
>>> timestamps = np.arange(0, 10, sampling_period)
>>>
>>> # Build a candidate DataFrame
>>> cols = list(zip(signal_names, signal_units))
>>> index_name = ("Time", "s")
>>>
>>> # Create dymoval signals
>>> df = pd.DataFrame(data=signal_values, columns=cols)
>>> df.index = pd.Index(data=timestamps, name=index_name)
>>>
>>> # Check if the dataframe is suitable for a dymoval Dataset
>>> dmv.validate_dataframe(df)
>>>
>>> # Specify input and output signals
>>> input_labels = ["SpeedRequest", "AccelPedalPos", "SteeringAngleRequest"]
>>> output_labels = ["ActualSpeed", "OilTemp"]
>>>
>>> # Create dymoval Dataset objects
>>> ds = dmv.Dataset("my_dataset", df, input_labels, output_labels)
Parameters:

df (DataFrame) – DataFrame to be validated.

Return type:

None

validate_signals(*signals)

Perform a number of checks to verify that the passed list of Signals can be used to create a Dataset.

Every Signal in signals must have all the attributes adequately set.

A Signal is a TypedDict with the following keys

  1. name: str

  2. samples: 1D np.ndarray

  3. signal_unit: str

  4. sampling_period: float

  5. time_unit: str

Example

>>> # How to create a Dataset object from a list of Signals
>>>
>>> import numpy as np
>>> import dymoval as dmv
>>>
>>> signal_names = [
>>>     "SpeedRequest",
>>>     "AccelPedalPos",
>>>     "OilTemp",
>>>     "ActualSpeed",
>>>     "SteeringAngleRequest",
>>> ]
>>> signal_values = np.random.rand(5, 100)
>>> signal_units = ["m/s", "%", "°C", "m/s", "deg"]
>>> sampling_periods = [0.1, 0.1, 0.1, 0.1, 0.1]
>>> # Create dymoval signals
>>> signals = []
>>> for ii, val in enumerate(signal_names):
>>>     tmp: dmv.Signal = {
>>>         "name": val,
>>>         "samples": signal_values[ii],
>>>         "signal_unit": signal_units[ii],
>>>         "sampling_period": sampling_periods[ii],
>>>         "time_unit": "s",
>>>     }
>>>     signals.append(tmp)
>>> # Validate signals
>>> dmv.validate_signals(*signals)
>>> # Specify which signals are inputs and which signals are output
>>> input_labels = ["SpeedRequest", "AccelPedalPos", "SteeringAngleRequest"]
>>> output_labels = ["ActualSpeed", "OilTemp"]
>>> # Create dymoval Dataset objects
>>> ds = dmv.Dataset("my_dataset", signals, input_labels, output_labels)
>>>
>>> # At this point you can plot, trim, manipulate, analyze, etc you dataset
>>> # through the Dataset class methods.
Parameters:

*signals (Signal) – Signal to be validated.

Return type:

None

dymoval.validation module

Module containing everything related to validation.

class ValidationSession(name, validation_dataset, U_bandwidths=None, Y_bandwidths=None, validation_thresholds=None, ignore_input=False, r2_statistic='min', Ruu_nlags=None, Ruu_local_statistic_type='abs_mean', Ruu_global_statistic_type='max', Ruu_local_weights=None, Ruu_global_weights=None, Ree_nlags=None, Ree_local_statistic_type='abs_mean', Ree_global_statistic_type='max', Ree_local_weights=None, Ree_global_weights=None, Rue_nlags=None, Rue_local_statistic_type='abs_mean', Rue_global_statistic_type='max', Rue_local_weights=None, Rue_global_weights=None)

Bases: object

The ValidationSession class is used to validate models against a given dataset.

A ValidationSession object is instantiated from a Dataset class object. A validation session name shall be also provided.

Multiple simulation results can be appended to the same ValidationSession instance, but for each ValidationSession instance only a Dataset class object is considered.

Parameters:

  • name (str) – The ValidationSession object name.

  • validation_dataset (Dataset) – The Dataset object to be used for validation.

  • U_bandwidths (ndarray | float | None) – 1-D array representing the bandwidths of each signal in the input U. U_bandwidths[i] corresponds to the bandwidth of signal U[i].

  • Y_bandwidths (ndarray | float | None) – 1-D array representing the bandwidths of each signal in the output Y. Y_bandwidths[i] corresponds to the bandwidth of signal Y[i].

  • validation_thresolds –

    Threshold used for validation. The dict keys shall be:

    • ”Ruu_whiteness””

    • ”r2”

    • ”Ree_whiteness””

    • ”Rue_whiteness””

  • ignore_input (bool) – If True input auto-correlation is not considered in the validation.

  • r2_statistic (Literal['min', 'mean']) – Statistic to be used for computing the global \(R^2\) in case of multiple output signals.

  • Ruu_nlags (ndarray | None) – Number of lags for the input auto-correlation array Ruu.

  • Ruu_local_statistic_type (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic used for estimating the whiteness of each element of the validation,XCorrelation object associated to the input signal.

  • Ruu_global_statistic_type (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic used for estimating the overall whiteness of the resulting \(p\times p\) matrix after the whiteness of each element of Ruu has been computed.

  • Ruu_local_weights (ndarray | None) – Weights associated to each element of Ruu. It must be a \(p\times p\) array where each element is a 1-D array.

  • Ruu_local_weights – Weights associated to the resulting matrix after the local statistics for each element or Ruu have been computed. It must be a \(p\times p\) array.

  • Ree_nlags (ndarray | None) – Number of lags for the residuals auto-correlation array Ree.

  • Ree_local_statistic_type (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic used for estimating the whiteness of each element of the validation,XCorrelation object associated to the residuals auto-correlation.

  • Ree_global_statistic_type (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic used for estimating the overall whiteness of the resulting \(p\times p\) matrix after the whiteness of each element of Ree has been computed.

  • Ree_local_weights (ndarray | None) – Weights associated to each element of Ree. It must be a \(q\times q\) array where each element is a 1-D array.

  • Ree_local_weights – Weights associated to the resulting matrix after the local statistics for each element or Ree have been computed. It must be a \(q\times q\) array.

  • Rue_nlags (ndarray | None) – Number of lags for the input-residuals cross-correlation array Rue.

  • Rue_local_statistic_type (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic used for estimating the whiteness of each element of the validation,XCorrelation object associated to the input-residuals cross-correlation.

  • Rue_global_statistic_type (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic used for estimating the overall whiteness of the resulting \(p\times q\) matrix after the whiteness of each element of Rue has been computed.

  • Rue_local_weights (ndarray | None) – Weights associated to each element of Rue. It must be a \(p\times q\) array where each element is a 1-D array.

  • Rue_local_weights – Weights associated to the resulting matrix after the local statistics for each element or Rue have been computed. It must be a \(p\times q\) array.

property Ree: dict[str, XCorrelation]

Residuals auto-correlation arrays.

property Rue: dict[str, XCorrelation]

Input-residuals cross-correlation arrays.

property Ruu: XCorrelation

Input auto-correlation arrays.

append_simulation(sim_name, y_names, y_data)

Append simulation results.

Parameters:

  • sim_name (str) – Simulation name.

  • y_label – Simulation output signal names.

  • y_data (ndarray) – Simulated out expressed as \(N\times q\) array with N observations of q signals.

Return type:

Self

clear()

Remove all the stored simulation results in the current ValidationSession object.

Return type:

Self

property dataset: Dataset

The reference Dataset class object.

drop_simulations(*sims)

Drop simulation results from the validation session object.

Parameters:

*sims (str) – Name of the simulations to be dropped.

Return type:

Self

name: str

ValidationSession object name.

property outcome: dict[str, str]

Validation outcome.

For each simulation return validation outcome.

plot_residuals(list_sims=None, *, plot_input=True, layout='tight', ax_height=1.8, ax_width=4.445)

Plot the residuals auto- and cross-correlation functions.

Parameters:

  • list_sims (str | list[str] | None) – List of simulations. If empty, all the simulations are plotted.

  • layout (Literal['constrained', 'compressed', 'tight', 'none']) – Figures layout.

  • ax_height (float) – Approximative height (inches) of each subplot.

  • ax_width (float) – Approximative width (inches) of each subplot.

Return type:

tuple[Figure, Figure, Figure]

You are free to manipulate the returned figure as you want by using any method of the class matplotlib.figure.Figure.

Please, refer to matplotlib docs for more info.

Example

>>> fig = vs.plot_residuals() # vs is a dymoval ValidationSession
object
# The following are methods of the class `matplotlib.figure.Figure`
>>> fig.set_size_inches(10,5)
>>> fig.set_layout_engine("constrained")
>>> fig.savefig("my_plot.svg")
plot_simulations(list_sims=None, dataset=None, layout='tight', ax_height=1.8, ax_width=4.445)

Plot the stored simulation results.

Possible values of the parameters describing the plot aesthetics, such as the linecolor_input or the alpha_output, are the same for the corresponding matplotlib.axes.Axes.plot.

You are free to manipulate the returned figure as you want by using any method of the class matplotlib.figure.Figure.

Please, refer to matplotlib docs for more info.

Example

>>> fig = vs.plot_simulations() # ds is a dymoval ValidationSession
object
# The following are methods of the class `matplotlib.figure.Figure`
>>> fig.set_size_inches(10,5)
>>> fig.set_layout_engine("constrained")
>>> fig.savefig("my_plot.svg")
Parameters:

  • list_sims (str | list[str] | None) – List of simulation names.

  • dataset (Literal['in', 'out', 'both'] | None) –

    Specify whether the dataset shall be plotted.

    • ”in”: dataset only the input signals of the dataset.

    • ”out”: dataset only the output signals of the dataset.

    • ”both”: dataset both the input and the output signals of the dataset.

  • layout (Literal['constrained', 'compressed', 'tight', 'none']) – Figure layout.

  • ax_height (float) – Approximative height (inches) of each subplot.

  • ax_width (float) – Approximative width (inches) of each subplot.

Return type:

Figure

simulation_signals_list(sim_name)

Return the signal name list of a given simulation.

Parameters:

sim_name (str | list[str]) – Simulation name.

Return type:

list[str]

property simulations_names: list[str]

Names of the stored simulations.

property simulations_values: DataFrame

Simulated out values.

trim(tin=None, tout=None, verbosity=0, **kwargs)

Trim the Validation session ValidationSession object.

If not tin or tout are passed, then the selection is made graphically.

Parameters:

  • *signals – Signals to be plotted in case of trimming from a plot.

  • tin (float | None) – Initial time of the desired time interval

  • tout (float | None) – Final time of the desired time interval.

  • verbosity (int) – Depending on its level, more or less info is displayed. The higher the value, the higher is the verbosity.

  • **kwargs (Any) – kwargs to be passed to the plot_simulations() method.

Return type:

Self

property validation_statistics: Any

Return the computed statistics for each simulation.

property validation_thresholds: dict[str, float]

Input-residuals cross-correlation arrays.

class XCorrelation(name, X, Y, nlags=None, X_bandwidths=None, Y_bandwidths=None, sampling_period=None)

Bases: object

Cross-correlation of two signals X and Y.

The signals can be MIMO and shall have dimension \(N\times p\) and \(N\times q\), respectively.

If X = Y then it return the normalized auto-correlation of X. If additional arguments are passed, then either X_Bandwidth, Y_Bandwidth and sampling_period are passed or none of them.

The cross-correlation functions are stored in the attribute R which is an array where the (i, j)-th element is the cross-correlation function between the i-th signal of X and the j-th signal of Y. The cross-correlation functions are NamedTuple s with attributes values and lags.

Parameters:

  • name (str) – The XCorrelation object name.

  • X (ndarray) – MIMO signal realizations expressed as \(N\times p\) array of N observations of p signals.

  • Y (ndarray) – MIMO signal realizations expressed as \(N\times q\) array of N observations of q signals.

  • nlags (ndarray | None) – \(p \times q\) array where the (i, j)-th element represents the number of lags of the cross-correlation function associated to the i-th signal of X with the j-th signal of Y.

  • X_bandwidths (ndarray | float | None) – 1-D array representing the bandwidths of each signal in X. X_bandwidths[i] corresponds to the bandwidth of signal X[i].

  • Y_bandwidths (ndarray | float | None) – 1-D array representing the bandwidths of each signal in Y. Y_bandwidths[i] corresponds to the bandwidth of signal Y[i].

  • sampling_period (float | None) – Sampling period of the signals X and Y.

Example

>>> import dymoval as dmv
>>> import numpy as np
>>> rng = np.random.default_rng()
>>> X = rng.uniform(low=-1, high=1, size=(10,3))
>>> Y = rng.normal(size=(10,4))
>>> lags = np.array([[10,8,20,12],[8 ,6 ,2 ,10],[20, 12, 8, 8]])
>>> Rxy = dmv.XCorrelation("foo", X, Y, nlags=lags)
# Cross-correlation between the first element of X (1D time-series) and
# the third element of Y (1D time-series).
>>> Rxy.R[0,2].lags
    array([-9, -8, -7, -6, -5, -4, -3, -2, -1,  0,  1,  2,  3,  4,  5,  6,
    7,
    8,  9])
>>> Rxy.R[0,2].values
    array([-0.06377225, -0.0083634 ,  0.14850791,  0.06379516,
    -0.16405862,
           -0.24074438,  0.14147755,  0.06538316, -0.26679362,
           0.14813509,
            0.64887265,  0.22247482, -0.4785613 , -0.30908332,
            0.12834458,
           -0.08259541, -0.27451256,  0.25320947,  0.06828447])
property R: ndarray

Auto- or cross-correlation array.

It is a \(p \times q\) array where the \((i, j)\)-th element represent the auto- or cross-correlation function of the \(i\)-th component of the argument X and the \(j\)-th component of the argument Y.

Each element of such an array is a NamedTuple object with attributes values and lags.

estimate_whiteness(local_statistic='abs_mean', local_weights=None, global_statistic='max', global_weights=None)

Return the whiteness estimate based on the selected statistics.

Parameters:

  • local_statistic (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic type for each (i,j) cross-correlation function of R array.

  • local_weights (ndarray | None) – Weights associated with the value of each (i, j) element of R. It must have the same shape of R.

  • global_statistic (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic used to estimate the whiteness of the flattened \(p \times q\) array after the whiteness of each element of R is estimated.

  • global_weights (ndarray | None) – Weights associated with each element of the resulting \(p \times q\) array. It shall be a \(p \times q\) array.

Return type:

tuple[float, ndarray]

Returns:

  • whiteness_estimate – The overall whiteness estimate.

  • whiteness_matrix – A \(p \times q\) array where the (i, j)-th element is the statistic computed for the (i, j)-th cross-correlation function of R.

Example

>>> #  Assume that RXY is a XCorrelation instance
>>> local_weights = np.empty(RXY.R.shape, dtype=np.ndarray)
>>> local_weights[0, 0] = np.ones(11)
>>> local_weights[0, 1] = np.ones(3)
>>> local_weights[1, 0] = np.ones(13)
>>> local_weights[1, 1] = np.ones(6)
>>> w, W = RXY.estimate_whiteness(local_weights=local_weights)
property kind: str

Kind of the XCorrelation object.

It can be “auto-correlation” or “cross-correlation”.

name: str

XCorrelation object name.

plot()

Plot the \(p imes q\) cross-correlation functions contained in R.

Return type:

Figure

compute_statistic(data, statistic='mean', weights=None)

Compute the statistic of a sequence of numbers.

The elements of data can be weighted through the weights array.

If data.shape dimension is greater than 1 then data will be flatten to a 1-D array. The return values are normalized such that the function always return values between 0 and 1, with the exclusion of the statistic quadratic that may return values greater than 1.0.

The statistic S is computed as it follows. Let \(w_i\) is the i-th element of weights and \(x_i\) is the i-th element of data.

mean This is the classic weighted mean value, computed as:

\[S = \frac{\sum_{i=1}^N w_ix_i}{\sum_{i=1}^N w_i}\]

abs_mean Mean of absolute values, computed as:

\[S = \frac{\sum_{i=1}^N w_i|x_i|}{\sum_{i=1}^N w_i}\]

max Max of absolute values, computed as:

\[S = \max_i\{|x_i|\}\]

std Standard deviation, computed as:

\[S =\sqrt{\sum_{i=1}^N w_i(x_i - \bar x)^2}\]

where \(\bar x\) is the weighted mean value computed above.

quadratic This is a generic quadratic form of the form:

\[S = \frac{1}{N \|W\|_{\infty}}x^TWx = \frac{\sum_{i=1}^N w_i|x_i|}{N\max_i \{|w_i|\}}\]

This is particular useful since many famous statistics, such as Ljung-Box, Box-Pierce, Lagrange Multiplier, etc., can be rewritten in the above form through an appropriate choice of the weights.

Parameters:

  • data (ndarray) – Array containing values for which the statistic shall be computed.

  • statistic (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Kind of statistic to be computed.

  • weights (ndarray | None) – An array of weights associated with the values in data. More precisely, weights[i] correspond to data[i].

Return type:

float

rsquared(x, y)

Return the \(R^2\) value of two signals.

Signals can be MIMO.

Parameters:

  • x (ndarray) – First input signal. It must have shape \(N\times p\), where \(N\) is the number of observation and \(p\) is the signal dimension.

  • y (ndarray) – Second input signal. It must have shape \(N\times p\), where \(N\) is the number of observation and \(p\) is the signal dimension.

Return type:

ndarray

validate_models(measured_in, measured_out, simulated_out, sampling_period=None, **kwargs)

Validate models based on measured and simulated data.

Parameters:

  • measured_in (ndarray | Sequence[Signal]) – Real-world measurements data related to the input. If dtype is np.ndarray, then the shape must be \(N\times p\), where N is the number of observations and p is the number of inputs.

  • measured_out (ndarray | list[Signal]) – Real-world measurements data related to the output. If dtype is np.ndarray, then the shape must be \(N\times q\), where N is the number of observations and q is the number of outputs.

  • simulated_out (ndarray | list[ndarray]) – Simulated output. The shape of the np.ndarray must be \(N\times q\), where N is the number of observations and q is the number of outputs.

  • sampling_period (float | None) – Signals sampling period.

  • **kwargs (Any) – Keyword arguments passed to ValidationSession constructor.

Return type:

ValidationSession

whiteness_level(data, data_bandwidths=None, sampling_period=None, nlags=None, local_statistic='abs_mean', local_weights=None, global_statistic='max', global_weights=None)

Estimate the whiteness of the signal data.

If data is a multivariate signal of shape \(p \times p\), then the whiteness is computed in two steps:

  1. The cross-correlation function for each \((i, j)\) pair of signal in data is computed, and their whiteness of is computed and arranged in a \(p \times p\) array.

  2. The resulting \(p \times p\) array is flattened and the overall signal whiteness is estimated.

It returns the values computed in points 1. and 2.

The whiteness is computed through compute_statistic().

Parameters:

  • data (ndarray) – Signal samples.

  • data_bandwidths (ndarray | float | None) – Signal bandwidth. If the signal is multivariate, then this specify the bandwidth of each of its component.

  • sampling_period (float | None) – Signal sampling period.

  • nlags (ndarray | None) – Number of lags to be considered for the whiteness estimate computation. If the signal is multivariate with p components, then this must be a \(p\times p\) array.

  • local_statistic (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic to be used for estimate the whiteness of each (i, j) cross-correlation function.

  • local_weights (ndarray | None) – Weights to be used for the whiteness estimation of each (i, j) cross-correlation function. It shall have the same size of R.

  • global_statistic (Literal['mean', 'quadratic', 'std', 'max', 'abs_mean']) – Statistic to be used for estimate the whiteness of the resulting \(p \times q\) array.

  • global_weights (ndarray | None) – Weight of each element of the resulting \(p \times q\) array for estimating the overall signal whiteness.

Return type:

tuple[float, ndarray]

dymoval.utils module

Module containing some useful functions.

difference_lists_of_str(A, B)

Return the strings contained in the list A but not in the list B.

In set formalism, this function returns a list representing the set difference \(A \backslash (A \cap B)\). Note that the operation is not commutative.

Parameters:

  • A (str | list[str]) – First list of strings.

  • B (str | list[str]) – Second list of strings.

Returns:

The set difference of A and B.

Return type:

list[str]

factorize(n)

Find the smallest and closest integers (a,b) such that \(n \le ab\).

Return type:

tuple[int, int]

is_interactive_shell()
Return type:

bool

obj2list(x)

Convert an object obj into list[obj].

If obj is already a list, then it return it as-is.

Parameters:

x (TypeVar(T) | list[TypeVar(T)]) – Input object.

Return type:

list[TypeVar(T)]

open_tutorial()

Create a dymoval_tutorial folder containing all the files needed to run the tutorial in your home folder. All you have to do is to run Jupyter notebook named dymoval_tutorial.ipynb. You need an app for opening .ipynb files.

The content of the dymoval_tutorial folder will be overwritten every time this function is called.

Return type:

Any