Description of all functions and classes

Description of all functions and classes#

Top-Level#

`version`()	Version of `spey` package
`about`()	Prints the information regarding spey installation
`check_updates`()	Check Spey Updates.
`ExpectationType`(value)	Expectation type has been used to determine the nature of the statistical model through out the package.
`AvailableBackends`()	Returns a list of available backends.
`get_backend`(name)	Statistical model backend retreiver.
`get_backend_metadata`(name)	Retreive metadata about the backend.
`get_backend_bibtex`(name)	Retreive BibTex entry for backend plug-in if available.
`cite`()	Retreive BibTex information for Spey
`reset_backend_entries`()	Scan the system for backends and reset the entries
`statistical_model_wrapper`(func)	Backend wrapper for `StatisticalModel`.
`helper_functions.correlation_to_covariance`(...)	Convert correlation matrix into covariance matrix
`helper_functions.covariance_to_correlation`(...)	Convert covariance matrix into correlation matrix.
`helper_functions.merge_correlated_bins`(...)	Merge correlated bins in a histogram/cutflow.
`optimizer.core.fit`(func, model_configuration)	Operating the fitting of the given function.
`set_log_level`(level)	Set log level for spey

Main Classes#

class spey.StatisticalModel(backend: BackendBase, analysis: str, xsection: float = nan, ntoys: int = 1000)[source]#

Bases: HypothesisTestingBase

Statistical model base. This class wraps around the various statistical model backends available through spey’s plugin system. Each backend has to inherit BackendBase which sets certain requirements on the available functionality to be used for hypothesis testing. These requirements are such as accessibility to log-likelihood, \(\log\mathcal{L}\), it’s derivative with respect to \(\mu\) and nuisance parameters, \(\partial_\theta\log\mathcal{L}\), its Hessian and Assimov data generation. Depending on availablility StatisticalModel will take propriate action to perform requested computation. The goal of this class is to collect all different backends under same roof in order to perform combination of different likelihood recipies.

Parameters:

backend (BackendBase) – Statistical model backend
analysis (Text) – Unique identifier of the statistical model. This attribue will be used for book keeping purposes.
xsection (float, default np.nan) – cross section, unit is determined by the user. Cross section value is only used for computing upper limit on excluded cross-section value.
ntoys (int, default 1000) – Number of toy samples for hypothesis testing. (Only used for toy-based hypothesis testing)

Raises:

AssertionError – If the given backend does not inherit BackendBase

Returns:

General statistical model object that wraps around different likelihood prescriptions.

Return type:

StatisticalModel

analysis: str#: Unique identifier as analysis name

asimov_likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, test_statistics: str = 'qtilde', init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → float[source]#

Compute likelihood of the statistical model generated with the Asimov data.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
test_statistics (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
The choice of test_statistics will effect the generation of the Asimov data where the fit is performed via \(\mu=1\) if test_statistics="q0" and \(\mu=0\) for others. Note that this \(\mu\) does not correspond to the poi_test input of this function but it determines how Asimov data is generated.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

likelihood computed for asimov data

Return type:

float

property available_calculators: List[str]#: Retruns available calculator names i.e. 'toy', 'asymptotic' and 'chi_square'.

property backend: BackendBase#: Accessor to the backend

property backend_type: str#: Return type of the backend

chi2(poi_test: float = 1.0, poi_test_denominator: float | None = None, expected: ExpectationType = observed, allow_negative_signal: bool = False, **kwargs) → float#

If poi_test_denominator=None computes

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\hat\mu,\hat\theta)}\right)\]

else

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\mu_{\rm denom},\theta_{\mu_{\rm denom}})}\right)\]

where \(\mu_{\rm denom}\) is poi_test_denominator which is typically zero to compare signal model with the background only model.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
poi_test_denominator (float, default None) – parameter of interest for the denominator, \(\mu\). If None maximum likelihood will be computed.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative. Only valid when poi_test_denominator=None.
kwargs – keyword arguments for the optimiser.

Returns:

value of the \(\chi^2\).

Return type:

float

chi2_test(expected: ExpectationType = observed, confidence_level: float = 0.95, limit_type: Literal['right', 'left', 'two-sided'] = 'two-sided', allow_negative_signal: bool = None) → List[float]#

Determine the parameter of interest (POI) value(s) that constrain the \(\chi^2\) distribution at a specified confidence level.

Added in version 0.2.0.

Attention

The degrees of freedom are set to one, referring to the POI. Currently, spey does not support multiple POIs, but this feature is planned for future releases.

Parameters:

expected (ExpectationType) –
Specifies the type of expectation for the fitting algorithm and p-value computation.
- observed: Computes p-values using post-fit prescription, assuming experimental data as the truth.
- apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.
confidence_level (float, default 0.95) – The confidence level for the upper limit. Must be between 0 and 1. This refers to the total inner area under the bell curve. Noted as \(CL\) below.
limit_type ('right', 'left' or 'two-sided', default "two-sided") – Specifies which side of the \(\chi^2\) distribution should be constrained. For two-sided limits, the inner area of the \(\chi^2\) distribution is set to confidence_level, making the threshold \(\alpha=(1-CL)/2\), where CL is the confidence_level. For left or right limits alone, \(\alpha=1-CL\). The \(\chi^2\)-threshold is calculated using inverse survival function at \(\alpha\).
allow_negative_signal (bool, default None) – Controls whether the signal can be negative. If None, it will be set to True for two-sided and left limits, and False for right limits. Otherwise, user can control this behaviour.

Returns:

POI value(s) that constrain the \(\chi^2\) distribution at the given threshold.

Return type:

List[float]

combine(other, **kwargs)[source]#

Combination routine between two statistical models.

Note

This function’s availability is backend dependent.

Parameters:

other (StatisticalModel) – Statistical model to be combined with this model
kwargs – backend specific arguments.

Raises:

CombinerNotAvailable – If this statistical model does not have a combination routine implementation.
AssertionError – If the combination routine in the backend does not return a BackendBase object.

Returns:

Returns a new combined statistical model.

Return type:

StatisticalModel

excluded_cross_section(expected: ExpectationType = observed) → float[source]#

Compute excluded cross section value at 95% CL

Parameters:

expected (ExpectationType) –

Sets which values the fitting algorithm should focus and p-values to be computed.

observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

Raises:

UnknownCrossSection – If the cross-section is nan.

Returns:

Returns the upper limit at 95% CL on cross section value where the unit is defined by the user.

Return type:

float

exclusion_confidence_level(poi_test: float = 1.0, expected: ExpectationType = observed, allow_negative_signal: bool = False, calculator: Literal['asymptotic', 'toy', 'chi_square'] = 'asymptotic', **kwargs) → List[float]#

Compute exclusion confidence level (\(CL_s\)) at a given POI, \(\mu\).

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed. If expected is set to "all" it will return both p-values and expected p-values where the likelihood will be fitted to observations.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
  
  Note
  
  In case of observed, function will return one value which has been fit to the observed data.
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
  
  Note
  
  In case of aposteriori, function will return five value for expected p-values which has been fit to the observed data. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The order of the output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  In case of apriori, function will return five value for expected p-values which has been fit to the SM background. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
allow_negative_signal (bool, default False) – If True \(\hat\mu\) value will be allowed to be negative.
calculator (Literal["asymptotic", "toy", "chi_square"], default "asymptotic") –
Chooses the computation basis for hypothesis testing
- "asymptotic": Uses asymptotic hypothesis testing to compute p-values.
- "toy": Uses generated toy samples to compute p-values.
- "chi_square": Computes p-values via chi-square; \(\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}\).
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Raises:

CalculatorNotAvailable – If calculator is not available.

Returns:

Exclusion confidence level i.e. \(CL_s\).

Return type:

List[float]

fixed_poi_sampler(poi_test: float, size: int | None = None, expected: ExpectationType = observed, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → ndarray | Callable[[int], ndarray][source]#

Sample data from the statistical model with fixed parameter of interest.

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
size (int, default None) – sample size. If None a callable function will be returned which takes sample size as input.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have sampler implementation.

Returns:

Sampled data with shape of (size, number of bins) or callable function to sample from directly.

Return type:

Union[np.ndarray, Callable[[int], np.ndarray]]

generate_asimov_data(expected: ExpectationType = observed, test_statistic: str = 'qtilde', init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → List[float][source]#

Generate Asimov data for the statistical model. This function generates a set of parameters (nuisance and poi i.e. \(\theta\) and \(\mu\)) with respect to test_statistic input which determines the value of \(\mu\) i.e. if test_statistic="q0" \(\mu=1\) and 0 for anything else. The objective function is used to optimize the statistical model to find the fit parameters for fixed poi optimisation. Then fit parameters are used to retreive the expected data through expected_data() function.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistic (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

Asimov data

Return type:

List[float]

property is_alive: bool#: Returns True if at least one bin has non-zero signal yield.

property is_asymptotic_calculator_available: bool#: Check if Asymptotic calculator is available for the backend

property is_chi_square_calculator_available: bool#: Check if chi-square calculator is available for the backend

property is_toy_calculator_available: bool#: Check if Toy calculator is available for the backend

likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, data: List[float] | ndarray | None = None, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → float[source]#

Compute the likelihood of the statistical model at a fixed parameter of interest

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

Likelihood of the statistical model at a fixed signal strength.

Return type:

float

maximize_asimov_likelihood(return_nll: bool = True, expected: ExpectationType = observed, test_statistics: str = 'qtilde', init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → Tuple[float, float][source]#

Find the maximum of the likelihood which computed with respect to Asimov data.

Parameters:

return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default "qtilde") –
test statistic.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

Find the maximum of the likelihood.

Parameters:

return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

ntoys#: Number of toy samples for sample generator during hypothesis testing

poi_upper_limit(expected: ExpectationType = observed, confidence_level: float = 0.95, low_init: float = 1.0, hig_init: float = 1.0, expected_pvalue: Literal['nominal', '1sigma', '2sigma'] = 'nominal', maxiter: int = 10000, optimiser_arguments: Dict[str, Any] | None = None) → float | List[float]#

Compute the upper limit for the parameter of interest (POI), denoted as \(\mu\).

Parameters:

expected (ExpectationType, default observed) –
Specifies the type of expectation for the fitting algorithm and p-value computation.
- observed: Computes p-values using post-fit prescription, assuming experimental data as the truth (default).
- aposteriori: Computes expected p-values using post-fit prescription, assuming experimental data as the truth.
- apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.
confidence_level (float, default 0.95) – Confidence level for the upper limit, representing \(1 - CL_s\). Must be between 0 and 1. Default is 0.95.
low_init (Optional[float], default 1.0) –
Initial lower limit for the search algorithm. If None, it is determined by \(\hat\mu + 1.5\sigma_{\hat\mu}\). Default is 1.0.

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
hig_init (Optional[float], default 1.0) –
Initial upper limit for the search algorithm. If None, it is determined by \(\hat\mu + 2.5\sigma_{\hat\mu}\). Default is 1.0.

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
expected_pvalue (Literal["nominal", "1sigma", "2sigma"], default "nominal") –
In case of aposteriori and apriori expectation, specifies the type of expected p-value for upper limit calculation.
- "nominal": Computes the upper limit for the central p-value. Returns a single value.
- "1sigma": Computes the upper limit for the central p-value and \(1\sigma\) fluctuation from background. Returns 3 values.
- "2sigma": Computes the upper limit for the central p-value and \(1\sigma\) and \(2\sigma\) fluctuation from background. Returns 5 values.
Note

For expected=spey.ExpectationType.observed, expected_pvalue argument will be overwritten to "nominal".
allow_negative_signal (bool, default True) – Allows for negative signal values, changing the computation of the test statistic. Default is False.
maxiter (int, default 10000) – Maximum number of iterations for the optimiser. Default is 10000.
optimiser_arguments (Dict, default None) – Additional arguments for the optimiser used to compute the likelihood and its maximum. Default is None.

Returns:

A single value representing the upper limit for the nominal case.
A list of values representing the upper limits for the central value and statistical deviations (for “1sigma” and “2sigma” cases). The order is: \(-2\sigma\), \(-1\sigma\), central value, \(1\sigma\), \(2\sigma\).

Return type:

Union[float, List[float]]

Raises:

AssertionError – If the confidence level is not between 0 and 1.

prepare_for_fit(data: List[float] | ndarray | None = None, expected: ExpectationType = observed, allow_negative_signal: bool | None = True, **kwargs) → Dict[source]#

Prepare backend for the optimiser.

Parameters:

data (Union[List[float], np.ndarray], default None) – input data that to fit
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.

Returns:

Dictionary of necessary toolset for the fit. objective function, "func", use gradient boolean, "do_grad" and function to compute negative log-likelihood with given fit parameters, "nll".

Return type:

Dict

property s95exp: float#

Compute excluded cross section value at 95% CL with apriori expectation. See excluded_cross_section() for reference.

Raises:: UnknownCrossSection – If the cross-section is nan.

property s95obs: float#

Compute excluded cross section value at 95% CL with observed expectation. See excluded_cross_section() for reference.

Raises:: UnknownCrossSection – If the cross-section is nan.

sigma_mu(poi_test: float, expected: ExpectationType = observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float#

If available, \(\sigma_\mu\) will be computed through Hessian of negative log-likelihood see spey.StatisticalModel.sigma_mu_from_hessian() for details. However, if not available it will be estimated via \(q_{\mu,A}\)

\[\sigma^2_A = \frac{(\mu - \mu^\prime)^2}{q_{\mu,A}}\quad , \quad q_{\mu,A} = -2\ln\lambda_A(\mu)\]

see eq. (31) in [arXiv:1007.1727]

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

value of the variance on \(\mu\).

Return type:

float

sigma_mu_from_hessian(poi_test: float, expected: ExpectationType = observed, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → float[source]#

Compute variance of \(\mu\) from inverse Hessian. See eq. (27-28) in [arXiv:1007.1727].

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have Hessian implementation.

Returns:

variance on parameter of interest.

Return type:

float

significance(expected: ExpectationType = observed, **kwargs) → Tuple[float, float, List[float], List[float]]#

Compute the discovery of a positive signal. See [arXiv:1007.1727] eq. (53). and sec. 5.1.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  Since aposteriori and observed are both represent post-fit computation the result will be the same. The only difference can be seen via prefit, apriori, computation.
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

(\(\sqrt{q_{0,A}}\), \(\sqrt{q_0}\), p-values and expected p-values)

Return type:

Tuple[float, float, List[float], List[float]]

xsection: float#: Value of the cross section, unit is defined by the user.

Base Classes#

class spey.BackendBase[source]#

An abstract class construction to enforce certain behaviour on statistical model backend. In order to perform certain computations, spey needs to have access to specific function constructions such as precsription to form likelihood. Hence, each backend is required to inherit BackendBase.

asimov_negative_loglikelihood(poi_test: float = 1.0, expected: ExpectationType = observed, test_statistics: str = 'qtilde', **kwargs) → Tuple[float, ndarray][source]#

Compute negative log-likelihood at fixed \(\mu\) for Asimov data.

Note

Interface first calls backend specific methods to compute likelihood. If they are not implemented, it optimizes objective function through spey interface. Either prescription to optimizing the likelihood or objective function must be available for a backend to be sucessfully integrated to the spey interface.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood at POI of interest and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

combine(other, **kwargs)[source]#

A routine to combine to statistical models.

Note

This function is only available if the backend has a specific routine for combination between same or other backends.

Parameters:: other (BackendBase) – Statistical model object to be combined.
Raises:: NotImplementedError – If the backend does not have a combination scheme.
Returns:: Create a new statistical model from combination of this and other one.
Return type:: BackendBase

abstractmethod config(allow_negative_signal: bool = True, poi_upper_bound: float = 10.0) → ModelConfig[source]#

Model configuration.

Parameters:

allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
poi_upper_bound (float, default 10.0) – upper bound for parameter of interest, \(\mu\).

Returns:

Model configuration. Information regarding the position of POI in parameter list, suggested input and bounds.

Return type:

ModelConfig

expected_data(pars: List[float]) → List[float][source]#

Compute the expected value of the statistical model. This function is mainly used to generate Asimov data within the package, see generate_asimov_data().

Parameters:: pars (List[float]) – nuisance, \(\theta\) and parameter of interest, \(\mu\).
Returns:: Expected data of the statistical model
Return type:: List[float]

get_hessian_logpdf_func(expected: ExpectationType = observed, data: List[float] | ndarray | None = None) → Callable[[ndarray], float][source]#

Currently Hessian of \(\log\mathcal{L}(\mu, \theta)\) is only used to compute variance on \(\mu\). This method returns a callable function which takes fit parameters (\(\mu\) and \(\theta\)) and returns Hessian.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
data (Union[List[float], np.ndarray], default None) – input data that to fit

Raises:

NotImplementedError – If the Hessian of the backend has not been implemented.

Returns:

Function that takes fit parameters (\(\mu\) and \(\theta\)) and returns Hessian of \(\log\mathcal{L}(\mu, \theta)\).

Return type:

Callable[[np.ndarray], float]

abstractmethod get_logpdf_func(expected: ExpectationType = observed, data: List[float] | ndarray | None = None) → Callable[[ndarray], float][source]#

Generate function to compute \(\log\mathcal{L}(\mu, \theta)\) where \(\mu\) is the parameter of interest and \(\theta\) are nuisance parameters.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
data (Union[List[float], np.ndarray], default None) – input data that to fit

Returns:

Function that takes fit parameters (\(\mu\) and \(\theta\)) and computes \(\log\mathcal{L}(\mu, \theta)\).

Return type:

Callable[[np.ndarray], float]

get_objective_function(expected: ExpectationType = observed, data: List[float] | ndarray | None = None, do_grad: bool = True) → Callable[[ndarray], float | Tuple[float, ndarray]][source]#

Objective function is the function to perform the optimisation on. This function is expected to be negative log-likelihood, \(-\log\mathcal{L}(\mu, \theta)\). Additionally, if available it canbe bundled with the gradient of negative log-likelihood.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
data (Union[List[float], np.ndarray], default None) – input data that to fit
do_grad (bool, default True) – If True return objective and its gradient as tuple (subject to availablility) if False only returns objective function.

Returns:

Function which takes fit parameters (\(\mu\) and \(\theta\)) and returns either objective or objective and its gradient.

Return type:

Callable[[np.ndarray], Union[float, Tuple[float, np.ndarray]]]

get_sampler(pars: ndarray) → Callable[[int], ndarray][source]#

Retreives the function to sample from.

Parameters:: pars (np.ndarray) – fit parameters (\(\mu\) and \(\theta\))
Raises:: NotImplementedError – If the sampler for the backend has not been implemented.
Returns:: Function that takes number_of_samples as input and draws as many samples from the statistical model.
Return type:: Callable[[int], np.ndarray]

property is_alive: bool#: Returns True if at least one bin has non-zero signal yield.

minimize_asimov_negative_loglikelihood(expected: ExpectationType = observed, test_statistics: str = 'qtilde', **kwargs) → Tuple[float, ndarray][source]#

A backend specific method to minimize negative log-likelihood for Asimov data.

Note

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

minimize_negative_loglikelihood(expected: ExpectationType = observed, allow_negative_signal: bool = True, **kwargs) → Tuple[float, ndarray][source]#

A backend specific method to minimize negative log-likelihood.

Note

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

negative_loglikelihood(poi_test: float = 1.0, expected: ExpectationType = observed, **kwargs) → Tuple[float, ndarray][source]#

Backend specific method to compute negative log-likelihood for a parameter of interest \(\mu\).

Note

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood at POI of interest and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

class spey.base.hypotest_base.HypothesisTestingBase(ntoys: int = 1000)[source]#

Abstract class that ensures classes that are performing hypothesis teststing includes certain set of function to perform necessary computations. This class gives the ability to compute exclusion limits and upper limits for the class inherits it.

Parameters:: ntoys (int, default 1000) – Number of toy samples for hypothesis testing. (Only used for toy-based hypothesis testing)

abstractmethod asimov_likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float[source]#

Compute likelihood at fixed \(\mu\) for Asimov data

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs – keyword arguments for the optimiser.

Returns:

value of the likelihood.

Return type:

float

chi2(poi_test: float = 1.0, poi_test_denominator: float | None = None, expected: ExpectationType = observed, allow_negative_signal: bool = False, **kwargs) → float[source]#

If poi_test_denominator=None computes

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\hat\mu,\hat\theta)}\right)\]

else

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\mu_{\rm denom},\theta_{\mu_{\rm denom}})}\right)\]

where \(\mu_{\rm denom}\) is poi_test_denominator which is typically zero to compare signal model with the background only model.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
poi_test_denominator (float, default None) – parameter of interest for the denominator, \(\mu\). If None maximum likelihood will be computed.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative. Only valid when poi_test_denominator=None.
kwargs – keyword arguments for the optimiser.

Returns:

value of the \(\chi^2\).

Return type:

float

Determine the parameter of interest (POI) value(s) that constrain the \(\chi^2\) distribution at a specified confidence level.

Added in version 0.2.0.

Attention

The degrees of freedom are set to one, referring to the POI. Currently, spey does not support multiple POIs, but this feature is planned for future releases.

Parameters:

expected (ExpectationType) –
Specifies the type of expectation for the fitting algorithm and p-value computation.
- observed: Computes p-values using post-fit prescription, assuming experimental data as the truth.
- apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.
confidence_level (float, default 0.95) – The confidence level for the upper limit. Must be between 0 and 1. This refers to the total inner area under the bell curve. Noted as \(CL\) below.
limit_type ('right', 'left' or 'two-sided', default "two-sided") – Specifies which side of the \(\chi^2\) distribution should be constrained. For two-sided limits, the inner area of the \(\chi^2\) distribution is set to confidence_level, making the threshold \(\alpha=(1-CL)/2\), where CL is the confidence_level. For left or right limits alone, \(\alpha=1-CL\). The \(\chi^2\)-threshold is calculated using inverse survival function at \(\alpha\).
allow_negative_signal (bool, default None) – Controls whether the signal can be negative. If None, it will be set to True for two-sided and left limits, and False for right limits. Otherwise, user can control this behaviour.

Returns:

POI value(s) that constrain the \(\chi^2\) distribution at the given threshold.

Return type:

List[float]

Compute exclusion confidence level (\(CL_s\)) at a given POI, \(\mu\).

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed. If expected is set to "all" it will return both p-values and expected p-values where the likelihood will be fitted to observations.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
  
  Note
  
  In case of observed, function will return one value which has been fit to the observed data.
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
  
  Note
  
  In case of aposteriori, function will return five value for expected p-values which has been fit to the observed data. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The order of the output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  In case of apriori, function will return five value for expected p-values which has been fit to the SM background. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
allow_negative_signal (bool, default False) – If True \(\hat\mu\) value will be allowed to be negative.
calculator (Literal["asymptotic", "toy", "chi_square"], default "asymptotic") –
Chooses the computation basis for hypothesis testing
- "asymptotic": Uses asymptotic hypothesis testing to compute p-values.
- "toy": Uses generated toy samples to compute p-values.
- "chi_square": Computes p-values via chi-square; \(\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}\).
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Raises:

CalculatorNotAvailable – If calculator is not available.

Returns:

Exclusion confidence level i.e. \(CL_s\).

Return type:

List[float]

Sample data from the statistical model with fixed parameter of interest.

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
size (int, default None) – sample size. If None a callable function will be returned which takes sample size as input.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have sampler implementation.

Returns:

Sampled data with shape of (size, number of bins) or callable function to sample from directly.

Return type:

Union[np.ndarray, Callable[[int], np.ndarray]]

abstract property is_alive: bool#: Returns True if at least one bin has non-zero signal yield.

abstract property is_asymptotic_calculator_available: bool#: Check if Asymptotic calculator is available for the backend

abstract property is_chi_square_calculator_available: bool#: Check if chi-square calculator is available for the backend

abstract property is_toy_calculator_available: bool#: Check if Toy calculator is available for the backend

abstractmethod likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, data: List[float] | ndarray | None = None, **kwargs) → float[source]#

Compute likelihood of the statistical model

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.
data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.
kwargs – keyword arguments for the optimiser.

Returns:

value of likelihood at fixed \(\mu\).

Return type:

float

abstractmethod maximize_asimov_likelihood(return_nll: bool = True, expected: ExpectationType = observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → Tuple[float, float][source]#

Compute maximum of the likelihood for Asimov data.

Parameters:

return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs – keyword arguments for the optimiser.

Returns:

value of \(\hat\mu\) and maximum likelihood.

Return type:

Tuple[float, float]

abstractmethod maximize_likelihood(return_nll: bool = True, expected: ExpectationType = observed, allow_negative_signal: bool = True, data: List[float] | ndarray | None = None, **kwargs) → Tuple[float, float][source]#

Compute maximum of the likelihood.

Parameters:

return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.
kwargs – keyword arguments for the optimiser.

Returns:

value of \(\hat\mu\) and maximum likelihood.

Return type:

Tuple[float, float]

ntoys#: Number of toy samples for sample generator during hypothesis testing

Compute the upper limit for the parameter of interest (POI), denoted as \(\mu\).

Parameters:

expected (ExpectationType, default observed) –
Specifies the type of expectation for the fitting algorithm and p-value computation.
- observed: Computes p-values using post-fit prescription, assuming experimental data as the truth (default).
- aposteriori: Computes expected p-values using post-fit prescription, assuming experimental data as the truth.
- apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.
confidence_level (float, default 0.95) – Confidence level for the upper limit, representing \(1 - CL_s\). Must be between 0 and 1. Default is 0.95.
low_init (Optional[float], default 1.0) –
Initial lower limit for the search algorithm. If None, it is determined by \(\hat\mu + 1.5\sigma_{\hat\mu}\). Default is 1.0.

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
hig_init (Optional[float], default 1.0) –
Initial upper limit for the search algorithm. If None, it is determined by \(\hat\mu + 2.5\sigma_{\hat\mu}\). Default is 1.0.

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
expected_pvalue (Literal["nominal", "1sigma", "2sigma"], default "nominal") –
In case of aposteriori and apriori expectation, specifies the type of expected p-value for upper limit calculation.
- "nominal": Computes the upper limit for the central p-value. Returns a single value.
- "1sigma": Computes the upper limit for the central p-value and \(1\sigma\) fluctuation from background. Returns 3 values.
- "2sigma": Computes the upper limit for the central p-value and \(1\sigma\) and \(2\sigma\) fluctuation from background. Returns 5 values.
Note

For expected=spey.ExpectationType.observed, expected_pvalue argument will be overwritten to "nominal".
allow_negative_signal (bool, default True) – Allows for negative signal values, changing the computation of the test statistic. Default is False.
maxiter (int, default 10000) – Maximum number of iterations for the optimiser. Default is 10000.
optimiser_arguments (Dict, default None) – Additional arguments for the optimiser used to compute the likelihood and its maximum. Default is None.

Returns:

A single value representing the upper limit for the nominal case.
A list of values representing the upper limits for the central value and statistical deviations (for “1sigma” and “2sigma” cases). The order is: \(-2\sigma\), \(-1\sigma\), central value, \(1\sigma\), \(2\sigma\).

Return type:

Union[float, List[float]]

Raises:

AssertionError – If the confidence level is not between 0 and 1.

sigma_mu(poi_test: float, expected: ExpectationType = observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float[source]#

\[\sigma^2_A = \frac{(\mu - \mu^\prime)^2}{q_{\mu,A}}\quad , \quad q_{\mu,A} = -2\ln\lambda_A(\mu)\]

see eq. (31) in [arXiv:1007.1727]

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

value of the variance on \(\mu\).

Return type:

float

significance(expected: ExpectationType = observed, **kwargs) → Tuple[float, float, List[float], List[float]][source]#

Compute the discovery of a positive signal. See [arXiv:1007.1727] eq. (53). and sec. 5.1.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  Since aposteriori and observed are both represent post-fit computation the result will be the same. The only difference can be seen via prefit, apriori, computation.
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

(\(\sqrt{q_{0,A}}\), \(\sqrt{q_0}\), p-values and expected p-values)

Return type:

Tuple[float, float, List[float], List[float]]

class spey.base.model_config.ModelConfig(poi_index: int, minimum_poi: float, suggested_init: List[float], suggested_bounds: List[Tuple[float, float]], parameter_names: List[str] | None = None, suggested_fixed: List[bool] | None = None)[source]#

Container to hold certain properties of the backend and statistical model. This will ensure the consistency of the computation through out the package.

Parameters:

poi_index (int) – index of the parameter of interest within the parameter list.
minimum_poi (float) –
minimum value parameter of interest can take to ensure \(N^{\rm bkg}+\mu N^{\rm sig}\geq0\). This value can be set to \(-\infty\) but note that optimiser will take it as a lower bound so such low value might effect the convergence of the optimisation algorithm especially for relatively flat objective surfaceses. Hence we suggest setting the minimum value to

\[\mu_{\rm min} = - \min\left( \frac{N^{\rm bkg}_i}{N^{\rm sig}_i} \right)\ ,\ i\in {\rm bins}\]
suggested_init (List[float]) – suggested initial parameters for the optimiser.
suggested_bounds (List[Tuple[float, float]]) – suggested parameter bounds for the optimiser.

Returns:

obj:~spey.base.model_config.ModelConfig: Model configuration container for optimiser.

fixed_poi_bounds(poi_value: float | None = None) → List[Tuple[float, float]][source]#

Adjust the bounds for the parameter of interest for fixed POI fit.

Parameters:: poi_value (Optional[float], default None) – parameter of interest, \(\mu\).
Returns:: Updated bounds.
Return type:: List[Tuple[float, float]]

minimum_poi: float#: minimum value parameter of interest can take to ensure \(N^{\rm bkg}+\mu N^{\rm sig}\geq0\). This value can be set to \(-\infty\) but note that optimiser will take it as a lower bound so such low value might effect the convergence of the optimisation algorithm especially for relatively flat objective surfaceses. Hence we suggest setting the minimum value to

\[\mu_{\rm min} = - \min\left( \frac{N^{\rm bkg}_i}{N^{\rm sig}_i} \right)\ ,\ i\in {\rm bins}\]

property npar: int#: Number of parameters

parameter_names: List[str] | None = None#: Names of the parameters

poi_index: int#: Index of the parameter of interest wihin the parameter list

rescale_poi_bounds(allow_negative_signal: bool = True, poi_upper_bound: float | None = None) → List[Tuple[float, float]][source]#

Rescale bounds for POI.

Parameters:

allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
poi_upper_bound (float, default None) – Maximum value POI can take during optimisation.

Returns:

Updated bounds.

Return type:

List[Tuple[float, float]]

suggested_bounds: List[Tuple[float, float]]#: Suggested upper and lower bounds for parameters

suggested_fixed: List[bool] | None = None#: Suggested fixed values

suggested_init: List[float]#: Suggested initialisation for parameters

Statistical Model Combiner#

class spey.UnCorrStatisticsCombiner(*args, ntoys: int = 1000)[source]#

Bases: HypothesisTestingBase

Module to combine uncorrelated statistical models. It takes serries of StatisticalModel object as input. These statistical models does not need to have same backend. However, this class assumes that all the input statistical model’s are completely independent from each other.

Warning

UnCorrStatisticsCombiner assumes that all input are uncorrelated and non of the statistical models posesses the same set of nuisance parameters. Thus each statistical model is optimised independently.

Parameters:

arguments (input) – Uncorrelated statistical models
ntoys (int, default 1000) – Number of toy samples for hypothesis testing. (Only used for toy-based hypothesis testing)

Raises:

AnalysisQueryError – If multiple StatisticalModel has the same analysis attribute.
TypeError – If the input type is not StatisticalModel.

property analyses: List[str]#

List of the unique identifiers of the statistical models within the stack.

Returns:: List of analysis names.
Return type:: List[Text]

append(statistical_model: StatisticalModel) → None[source]#

Append new independent StatisticalModel to the stack.

Parameters:

statistical_model (StatisticalModel) – new statistical model to be added to the stack.

Raises:

AnalysisQueryError – If multiple StatisticalModel has the same analysis attribute.
TypeError – If the input type is not StatisticalModel.

asimov_likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, test_statistics: str = 'qtilde', statistical_model_options: Dict[str, Dict] | None = None, **kwargs) → float[source]#

Compute likelihood of the statistical model stack generated with the Asimov data.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
test_statistics (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
statistical_model_options (Dict[Text, Dict], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

likelihood computed for asimov data

Return type:

float

chi2(poi_test: float = 1.0, poi_test_denominator: float | None = None, expected: ExpectationType = observed, allow_negative_signal: bool = False, **kwargs) → float#

If poi_test_denominator=None computes

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\hat\mu,\hat\theta)}\right)\]

else

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\mu_{\rm denom},\theta_{\mu_{\rm denom}})}\right)\]

where \(\mu_{\rm denom}\) is poi_test_denominator which is typically zero to compare signal model with the background only model.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
poi_test_denominator (float, default None) – parameter of interest for the denominator, \(\mu\). If None maximum likelihood will be computed.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative. Only valid when poi_test_denominator=None.
kwargs – keyword arguments for the optimiser.

Returns:

value of the \(\chi^2\).

Return type:

float

Determine the parameter of interest (POI) value(s) that constrain the \(\chi^2\) distribution at a specified confidence level.

Added in version 0.2.0.

Attention

The degrees of freedom are set to one, referring to the POI. Currently, spey does not support multiple POIs, but this feature is planned for future releases.

Parameters:

expected (ExpectationType) –
Specifies the type of expectation for the fitting algorithm and p-value computation.
- observed: Computes p-values using post-fit prescription, assuming experimental data as the truth.
- apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.
confidence_level (float, default 0.95) – The confidence level for the upper limit. Must be between 0 and 1. This refers to the total inner area under the bell curve. Noted as \(CL\) below.
limit_type ('right', 'left' or 'two-sided', default "two-sided") – Specifies which side of the \(\chi^2\) distribution should be constrained. For two-sided limits, the inner area of the \(\chi^2\) distribution is set to confidence_level, making the threshold \(\alpha=(1-CL)/2\), where CL is the confidence_level. For left or right limits alone, \(\alpha=1-CL\). The \(\chi^2\)-threshold is calculated using inverse survival function at \(\alpha\).
allow_negative_signal (bool, default None) – Controls whether the signal can be negative. If None, it will be set to True for two-sided and left limits, and False for right limits. Otherwise, user can control this behaviour.

Returns:

POI value(s) that constrain the \(\chi^2\) distribution at the given threshold.

Return type:

List[float]

Compute exclusion confidence level (\(CL_s\)) at a given POI, \(\mu\).

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed. If expected is set to "all" it will return both p-values and expected p-values where the likelihood will be fitted to observations.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
  
  Note
  
  In case of observed, function will return one value which has been fit to the observed data.
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
  
  Note
  
  In case of aposteriori, function will return five value for expected p-values which has been fit to the observed data. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The order of the output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  In case of apriori, function will return five value for expected p-values which has been fit to the SM background. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
allow_negative_signal (bool, default False) – If True \(\hat\mu\) value will be allowed to be negative.
calculator (Literal["asymptotic", "toy", "chi_square"], default "asymptotic") –
Chooses the computation basis for hypothesis testing
- "asymptotic": Uses asymptotic hypothesis testing to compute p-values.
- "toy": Uses generated toy samples to compute p-values.
- "chi_square": Computes p-values via chi-square; \(\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}\).
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Raises:

CalculatorNotAvailable – If calculator is not available.

Returns:

Exclusion confidence level i.e. \(CL_s\).

Return type:

List[float]

find_most_sensitive() → StatisticalModel[source]#

If the cross sections are defined, finds the statistical model with minimum prefit excluded cross section value. See s95exp.

Returns:: Statistical model with minimum prefit excluded cross section value.
Return type:: StatisticalModel

Sample data from the statistical model with fixed parameter of interest.

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
size (int, default None) – sample size. If None a callable function will be returned which takes sample size as input.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have sampler implementation.

Returns:

Sampled data with shape of (size, number of bins) or callable function to sample from directly.

Return type:

Union[np.ndarray, Callable[[int], np.ndarray]]

generate_asimov_data(expected: ExpectationType = observed, test_statistic: str = 'qtilde', statistical_model_options: Dict[str, Dict] | None = None, **kwargs) → Dict[str, List[float]][source]#

Generate Asimov data for the statistical model. This function calls generate_asimov_data() function for each statistical model with appropriate statistical_model_options and generates Asimov data for each statistical model independently.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistic (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
statistical_model_options (Dict[Text, Dict], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

Returns a dictionary for data specific to each analysis. keywords will be analysis names and the items are data.

Return type:

Dict[Text, List[float]]

property is_alive: bool#

Returns true if there is at least one statistical model with at least one non zero bin with signal yield.

Returns:: bool

property is_asymptotic_calculator_available: bool#: Check if Asymptotic calculator is available for the backend

property is_chi_square_calculator_available: bool#: Check if \(\chi^2\) calculator is available for the backend

property is_toy_calculator_available: bool#: Check if Toy calculator is available for the backend

items() → Iterator[Tuple[str, StatisticalModel]][source]#

Returns a generator that returns analysis name and StatisticalModel every iteration.

Returns:: Iterator[Tuple[Text, StatisticalModel]]

likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, data: Dict[str, List[float]] | None = None, statistical_model_options: Dict[str, Dict] | None = None, **kwargs) → float[source]#

Compute the likelihood of the statistical model stack at a fixed parameter of interest

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
data (Dict[Text, List[float]], default None) – Input data to be used for fit. It needs to be given as a dictionary where the key argument is the name of the analysis and item is the statistical model specific data.
statistical_model_options (Optional[Dict[Text, Dict]], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

value of the likelihood of the stacked statistical model.

Return type:

float

maximize_asimov_likelihood(return_nll: bool = True, expected: ExpectationType = observed, test_statistics: str = 'qtilde', initial_muhat_value: float | None = None, par_bounds: List[Tuple[float, float]] | None = None, statistical_model_options: Dict[str, Dict] | None = None, **optimiser_options) → Tuple[float, float][source]#

Find the maximum of the likelihood which computed with respect to Asimov data.

Parameters:

return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default "qtilde") –
test statistic.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
initial_muhat_value (float, default None) – Initial value for \(\hat\mu\).
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
statistical_model_options (Dict[Text, Dict], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

maximize_likelihood(return_nll: bool = True, expected: ExpectationType = observed, allow_negative_signal: bool = True, data: Dict[str, List[float]] | None = None, initial_muhat_value: float | None = None, par_bounds: List[Tuple[float, float]] | None = None, statistical_model_options: Dict[str, Dict] | None = None, **optimiser_options) → Tuple[float, float][source]#

Find the maximum of the likelihood.

Parameters:

return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
data (Dict[Text, List[float]], default None) – Input data to be used for fit. It needs to be given as a dictionary where the key argument is the name of the analysis and item is the statistical model specific data.
initial_muhat_value (float, default None) –
Initialisation for the \(\hat\mu\) for the optimiser. If None the initial value will be estimated by weighted combination of \(\hat\mu_i\) where \(i\) stands for the statistical models within the stack.

\[\mu_{\rm init} = \frac{1}{\sum_i \sigma_{\hat\mu, i}^2} \sum_i \frac{\hat\mu_i}{\sigma_{\hat\mu, i}^2}\]

where the value of \(\sigma_{\hat\mu}\) has been estimated by sigma_mu() function.
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
statistical_model_options (Dict[Text, Dict], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

property minimum_poi: float#

Find the maximum minimum poi, \(\mu\) value within the analysis stack.

Returns:: minimum poi value that the combined stack can take.
Return type:: float

ntoys#: Number of toy samples for sample generator during hypothesis testing

Compute the upper limit for the parameter of interest (POI), denoted as \(\mu\).

Parameters:

expected (ExpectationType, default observed) –
Specifies the type of expectation for the fitting algorithm and p-value computation.
- observed: Computes p-values using post-fit prescription, assuming experimental data as the truth (default).
- aposteriori: Computes expected p-values using post-fit prescription, assuming experimental data as the truth.
- apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.
confidence_level (float, default 0.95) – Confidence level for the upper limit, representing \(1 - CL_s\). Must be between 0 and 1. Default is 0.95.
low_init (Optional[float], default 1.0) –
Initial lower limit for the search algorithm. If None, it is determined by \(\hat\mu + 1.5\sigma_{\hat\mu}\). Default is 1.0.

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
hig_init (Optional[float], default 1.0) –
Initial upper limit for the search algorithm. If None, it is determined by \(\hat\mu + 2.5\sigma_{\hat\mu}\). Default is 1.0.

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
expected_pvalue (Literal["nominal", "1sigma", "2sigma"], default "nominal") –
In case of aposteriori and apriori expectation, specifies the type of expected p-value for upper limit calculation.
- "nominal": Computes the upper limit for the central p-value. Returns a single value.
- "1sigma": Computes the upper limit for the central p-value and \(1\sigma\) fluctuation from background. Returns 3 values.
- "2sigma": Computes the upper limit for the central p-value and \(1\sigma\) and \(2\sigma\) fluctuation from background. Returns 5 values.
Note

For expected=spey.ExpectationType.observed, expected_pvalue argument will be overwritten to "nominal".
allow_negative_signal (bool, default True) – Allows for negative signal values, changing the computation of the test statistic. Default is False.
maxiter (int, default 10000) – Maximum number of iterations for the optimiser. Default is 10000.
optimiser_arguments (Dict, default None) – Additional arguments for the optimiser used to compute the likelihood and its maximum. Default is None.

Returns:

A single value representing the upper limit for the nominal case.
A list of values representing the upper limits for the central value and statistical deviations (for “1sigma” and “2sigma” cases). The order is: \(-2\sigma\), \(-1\sigma\), central value, \(1\sigma\), \(2\sigma\).

Return type:

Union[float, List[float]]

Raises:

AssertionError – If the confidence level is not between 0 and 1.

remove(analysis: str) → None[source]#

Remove an analysis from the stack.

Parameters:: analysis (Text) – unique identifier of the analysis to be removed.
Raises:: AnalysisQueryError – If the unique identifier does not match any of the statistical models in the stack.

sigma_mu(poi_test: float, expected: ExpectationType = observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float#

\[\sigma^2_A = \frac{(\mu - \mu^\prime)^2}{q_{\mu,A}}\quad , \quad q_{\mu,A} = -2\ln\lambda_A(\mu)\]

see eq. (31) in [arXiv:1007.1727]

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

value of the variance on \(\mu\).

Return type:

float

significance(expected: ExpectationType = observed, **kwargs) → Tuple[float, float, List[float], List[float]]#

Compute the discovery of a positive signal. See [arXiv:1007.1727] eq. (53). and sec. 5.1.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  Since aposteriori and observed are both represent post-fit computation the result will be the same. The only difference can be seen via prefit, apriori, computation.
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

(\(\sqrt{q_{0,A}}\), \(\sqrt{q_0}\), p-values and expected p-values)

Return type:

Tuple[float, float, List[float], List[float]]

property statistical_models: Tuple[StatisticalModel]#

Accessor to the statistical model stack

Returns:: statistical model stack as an ntuple.
Return type:: Tuple[StatisticalModel]

Hypothesis testing#

`test_statistics.qmu`(mu, muhat, max_logpdf, ...)	Test statistic \(q_{\mu}\), see eq.
`test_statistics.qmu_tilde`(mu, muhat, ...)	Alternative test statistics, \(\tilde{q}_{\mu}\), see eq.
`test_statistics.q0`(mu, muhat, max_logpdf, logpdf)	Discovery test statistics, \(q_{0}\) see eq.
`test_statistics.get_test_statistic`(test_stat)	Retreive the test statistic function
`test_statistics.compute_teststatistics`(mu, ...)	Compute test statistics
`upper_limits.ComputerWrapper`(computer)	Wrapper for the computer function to track inputs and outputs
`upper_limits.find_poi_upper_limit`(...[, ...])	Find upper limit for parameter of interest, \(\mu\)
`upper_limits.find_root_limits`(computer[, ...])	Find upper and lower bracket limits for the root finding algorithm
`utils.pvalues`(delta_test_statistic, ...)	Calculate the p-values for the observed test statistic under the signal + background and background-only model hypotheses.
`utils.expected_pvalues`(...)	Calculate the \(p\) values corresponding to the median significance of variations of the signal strength from the background only hypothesis \(\mu=0\) at \((-2,-1,0,1,2)\sigma\).
`distributions.AsymptoticTestStatisticsDistribution`(shift)	The distribution the test statistic in the asymptotic case.
`distributions.EmpricTestStatisticsDistribution`(samples)	Create emprical distribution.
`asymptotic_calculator.compute_asymptotic_confidence_level`(...)	Compute p values i.e. \(p_{s+b}\), \(p_b\) and \(p_s\).
`toy_calculator.compute_toy_confidence_level`(...)	Compute confidence limits i.e. \(CL_{s+b}\), \(CL_b\) and \(CL_s\).

Gradient Tools#

`value_and_grad`(statistical_model[, ...])	Retreive function to compute negative log-likelihood and its gradient.
`hessian`(statistical_model[, expected, data])	Retreive the function to compute Hessian of negative log-likelihood

Default Backends#

Distributions#

`Poisson`(loc)	Poisson distribution
`Normal`(loc, scale[, weight, domain])	Normal distribution
`MultivariateNormal`(mean, cov[, weight, domain])	Multivariate normal distribution
`MainModel`(loc[, cov, pdf_type])	Main statistical model, modelled as Poisson distribution which has a variable lambda.
`ConstraintModel`(pdf_descriptions)	Constraint term modelled as a Gaussian distribution.

Default PDFs#

`DefaultPDFBase`(signal_yields, ...[, ...])	Default PDF backend base
`UncorrelatedBackground`(signal_yields, ...[, ...])	Interface for uncorrelated background uncertainties.
`CorrelatedBackground`(signal_yields, ...[, ...])	Correlated multi-region statistical model.
`ThirdMomentExpansion`(signal_yields, ...[, ...])	Simplified likelihood interface with third moment expansion.
`EffectiveSigma`(signal_yields, ...[, ...])	Simplified likelihood interface with variable Gaussian.
`third_moment.third_moment_expansion`(...[, ...])	Construct the terms for third moment expansion.
`third_moment.compute_third_moments`(...[, ...])	Assuming that the uncertainties are modelled as Gaussian, it computes third moments using Bifurcated Gaussian with asymmetric uncertainties.

Simple PDFs#

`simple_pdf.Poisson`(signal_yields, ...[, ...])	Poisson distribution without uncertainty implementation.
`simple_pdf.Gaussian`(signal_yields, ...)	Gaussian distribution for uncorrelated likelihoods.
`simple_pdf.MultivariateNormal`(signal_yields, ...)	Multivariate Gaussian distribution.

Exceptions#

`AnalysisQueryError`	Analysis query exception
`MethodNotAvailable`	If the method is not available for a given backend
`NegativeExpectedYields`	Negative expected yields exception
`PluginError`	Invalid plugin exception
`UnknownCrossSection`	Unknown cross-section exception
`UnknownTestStatistics`	Unknown test statistics exception
`CalculatorNotAvailable`	Unavailable calculator Exception
`CanNotFindRoots`	Unable to find roots of the function
`UnknownComputer`	Unknown computation base
`CombinerNotAvailable`	Unavailable combination routine exception

Description of all functions and classes

Contents

Description of all functions and classes#

Top-Level#

Main Classes#

Base Classes#

Statistical Model Combiner#

Hypothesis testing#

Gradient Tools#

Default Backends#

Distributions#

Default PDFs#

Simple PDFs#

Exceptions#