Description of all functions and classes

Top-Level

version()	Version of spey package
about()	Prints the information regarding spey installation
check_updates()	Check Spey Updates.
ExpectationType(value)	Expectation type has been used to determine the nature of the statistical model through out the package.
register_backend(model)	A local backend registry for statistical models.
AvailableBackends()	Returns a list of available backends.
set_optimiser(name)	Set optimiser for fitting interface.
get_backend(name)	Statistical model backend retreiver.
get_backend_metadata(name)	Retreive metadata about the backend.
get_backend_bibtex(name)	Retreive BibTex entry for backend plug-in if available.
cite()	Retreive BibTex information for Spey
reset_backend_entries()	Scan the system for backends and reset the entries
statistical_model_wrapper(func)	Backend wrapper for StatisticalModel.
helper_functions.correlation_to_covariance(...)	Convert correlation matrix into covariance matrix
helper_functions.covariance_to_correlation(...)	Convert covariance matrix into correlation matrix.
helper_functions.merge_correlated_bins(...)	Merge correlated bins in a histogram/cutflow.
optimizer.core.fit(func, model_configuration)	Operating the fitting of the given function.
set_log_level(level)	Set log level for spey

Main Classes

class spey.StatisticalModel(backend: BackendBase, analysis: str, xsection: float = nan, ntoys: int = 1000)

Bases: HypothesisTestingBase

Statistical model base. This class wraps around the various statistical model backends available through spey’s plugin system. Each backend has to inherit BackendBase which sets certain requirements on the available functionality to be used for hypothesis testing. These requirements are such as accessibility to log-likelihood, \(\log\mathcal{L}\), it’s derivative with respect to \(\mu\) and nuisance parameters, \(\partial_\theta\log\mathcal{L}\), its Hessian and Assimov data generation. Depending on availablility StatisticalModel will take propriate action to perform requested computation. The goal of this class is to collect all different backends under same roof in order to perform combination of different likelihood recipies.

Parameters:

• backend (BackendBase) – Statistical model backend

• analysis (Text) – Unique identifier of the statistical model. This attribue will be used for book keeping purposes.

• xsection (float, default np.nan) – cross section, unit is determined by the user. Cross section value is only used for computing upper limit on excluded cross-section value.

• ntoys (int, default 1000) – Number of toy samples for hypothesis testing. (Only used for toy-based hypothesis testing)

Raises:

AssertionError – If the given backend does not inherit BackendBase

Returns:

General statistical model object that wraps around different likelihood prescriptions.

Return type:

StatisticalModel

analysis: str

Unique identifier as analysis name

asimov_likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, test_statistics: str = 'qtilde', init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → float

Compute likelihood of the statistical model generated with the Asimov data.

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.

• test_statistics (Text, default "qtilde") –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

  The choice of test_statistics will effect the generation of the Asimov data where the fit is performed via \(\mu=1\) if test_statistics="q0" and \(\mu=0\) for others. Note that this \(\mu\) does not correspond to the poi_test input of this function but it determines how Asimov data is generated.

• init_pars (List[float], default None) – initial parameters for the optimiser

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• kwargs – keyword arguments for the optimiser.

Returns:

likelihood computed for asimov data

Return type:

float

property available_calculators: List[str]

Retruns available calculator names i.e. 'toy', 'asymptotic' and 'chi_square'.

property backend: BackendBase

Accessor to the backend

property backend_type: str

Return type of the backend

chi2(poi_test: float = 1.0, poi_test_denominator: float | None = None, expected: ExpectationType = observed, allow_negative_signal: bool = False, **kwargs) → float

If poi_test_denominator=None computes

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\hat\mu,\hat\theta)}\right)\]

else

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\mu_{\rm denom},\theta_{\mu_{\rm denom}})}\right)\]

where \(\mu_{\rm denom}\) is poi_test_denominator which is typically zero to compare signal model with the background only model.

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• poi_test_denominator (float, default None) – parameter of interest for the denominator, \(\mu\). If None maximum likelihood will be computed.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative. Only valid when poi_test_denominator=None.

• kwargs – keyword arguments for the optimiser.

Returns:

value of the \(\chi^2\).

Return type:

float

chi2_test(expected: ExpectationType = observed, confidence_level: float = 0.95, limit_type: Literal['right', 'left', 'two-sided'] = 'two-sided', allow_negative_signal: bool = None) → List[float]

Determine the parameter of interest (POI) value(s) that constrain the \(\chi^2\) distribution at a specified confidence level.

Added in version 0.2.0.

Attention

The degrees of freedom are set to one, referring to the POI. Currently, spey does not support multiple POIs, but this feature is planned for future releases.

Parameters:

• expected (ExpectationType) –

  Specifies the type of expectation for the fitting algorithm and p-value computation.

  • observed: Computes p-values using post-fit prescription, assuming experimental data as the truth.

  • apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.

• confidence_level (float, default 0.95) – The confidence level for the upper limit. Must be between 0 and 1. This refers to the total inner area under the bell curve. Noted as \(CL\) below.

• limit_type ('right', 'left' or 'two-sided', default "two-sided") – Specifies which side of the \(\chi^2\) distribution should be constrained. For two-sided limits, the inner area of the \(\chi^2\) distribution is set to confidence_level, making the threshold \(\alpha=(1-CL)/2\), where CL is the confidence_level. For left or right limits alone, \(\alpha=1-CL\). The \(\chi^2\)-threshold is calculated using inverse survival function at \(\alpha\).

• allow_negative_signal (bool, default None) – Controls whether the signal can be negative. If None, it will be set to True for two-sided and left limits, and False for right limits. Otherwise, user can control this behaviour.

Returns:

POI value(s) that constrain the \(\chi^2\) distribution at the given threshold.

Return type:

List[float]

combine(other, **kwargs)

Combination routine between two statistical models.

Note

This function’s availability is backend dependent.

Parameters:

• other (StatisticalModel) – Statistical model to be combined with this model

• kwargs – backend specific arguments.

Raises:

• CombinerNotAvailable – If this statistical model does not have a combination routine implementation.

• AssertionError – If the combination routine in the backend does not return a BackendBase object.

Returns:

Returns a new combined statistical model.

Return type:

StatisticalModel

excluded_cross_section(expected: ExpectationType = observed) → float

Compute excluded cross section value at 95% CL

Parameters:

expected (ExpectationType) –

Sets which values the fitting algorithm should focus and p-values to be computed.

• observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

• aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

• apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

Raises:

UnknownCrossSection – If the cross-section is nan.

Returns:

Returns the upper limit at 95% CL on cross section value where the unit is defined by the user.

Return type:

float

exclusion_confidence_level(poi_test: float = 1.0, expected: ExpectationType = observed, allow_negative_signal: bool = False, calculator: Literal['asymptotic', 'toy', 'chi_square'] = 'asymptotic', **kwargs) → List[float]

Compute exclusion confidence level (\(CL_s\)) at a given POI, \(\mu\).

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed. If expected is set to "all" it will return both p-values and expected p-values where the likelihood will be fitted to observations.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

    Note

    In case of observed, function will return one value which has been fit to the observed data.

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

    Note

    In case of aposteriori, function will return five value for expected p-values which has been fit to the observed data. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The order of the output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

    Note

    In case of apriori, function will return five value for expected p-values which has been fit to the SM background. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.

• allow_negative_signal (bool, default False) – If True \(\hat\mu\) value will be allowed to be negative.

• calculator (Literal["asymptotic", "toy", "chi_square"], default "asymptotic") –

  Chooses the computation basis for hypothesis testing

  • "asymptotic": Uses asymptotic hypothesis testing to compute p-values.

  • "toy": Uses generated toy samples to compute p-values.

  • "chi_square": Computes p-values via chi-square; \(\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}\).

• kwargs –

  keyword arguments for the optimiser.

  • init_pars (List[float], default None): initial parameters for the optimiser

  • par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Raises:

CalculatorNotAvailable – If calculator is not available.

Returns:

Exclusion confidence level i.e. \(CL_s\).

Return type:

List[float]

fixed_poi_sampler(poi_test: float, size: int | None = None, expected: ExpectationType = observed, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → ndarray | Callable[[int], ndarray]

Sample data from the statistical model with fixed parameter of interest.

Parameters:

• poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).

• size (int, default None) – sample size. If None a callable function will be returned which takes sample size as input.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• init_pars (List[float], default None) – initial parameters for the optimiser

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have sampler implementation.

Returns:

Sampled data with shape of (size, number of bins) or callable function to sample from directly.

Return type:

Union[np.ndarray, Callable[[int], np.ndarray]]

generate_asimov_data(expected: ExpectationType = observed, test_statistic: str = 'qtilde', init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → List[float]

Generate Asimov data for the statistical model. This function generates a set of parameters (nuisance and poi i.e. \(\theta\) and \(\mu\)) with respect to test_statistic input which determines the value of \(\mu\) i.e. if test_statistic="q0" \(\mu=1\) and 0 for anything else. The objective function is used to optimize the statistical model to find the fit parameters for fixed poi optimisation. Then fit parameters are used to retreive the expected data through expected_data() function.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistic (Text, default "qtilde") –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• init_pars (List[float], default None) – initial parameters for the optimiser

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• kwargs – keyword arguments for the optimiser.

Returns:

Asimov data

Return type:

List[float]

property is_alive: bool

Returns True if at least one bin has non-zero signal yield.

property is_asymptotic_calculator_available: bool

Check if Asymptotic calculator is available for the backend

property is_chi_square_calculator_available: bool

Check if chi-square calculator is available for the backend

property is_toy_calculator_available: bool

Check if Toy calculator is available for the backend

likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, data: List[float] | ndarray | None = None, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → float

Compute the likelihood of the statistical model at a fixed parameter of interest

Parameters:

• poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.

• data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.

• init_pars (List[float], default None) – initial parameters for the optimiser

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• kwargs – keyword arguments for the optimiser.

Returns:

Likelihood of the statistical model at a fixed signal strength.

Return type:

float

maximize_asimov_likelihood(return_nll: bool = True, expected: ExpectationType = observed, test_statistics: str = 'qtilde', init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → Tuple[float, float]

Find the maximum of the likelihood which computed with respect to Asimov data.

Parameters:

• return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistics (Text, default "qtilde") –

  test statistic.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• init_pars (List[float], default None) – initial parameters for the optimiser

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

maximize_likelihood(return_nll: bool | None = True, expected: ExpectationType | None = observed, allow_negative_signal: bool | None = True, data: List[float] | ndarray | None = None, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → Tuple[float, float]

Find the maximum of the likelihood.

Parameters:

• return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.

• data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.

• init_pars (List[float], default None) – initial parameters for the optimiser

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

ntoys

Number of toy samples for sample generator during hypothesis testing

poi_upper_limit(expected: ExpectationType = observed, confidence_level: float = 0.95, low_init: float = 1.0, hig_init: float = 1.0, expected_pvalue: Literal['nominal', '1sigma', '2sigma'] = 'nominal', maxiter: int = 10000, optimiser_arguments: Dict[str, Any] | None = None) → float | List[float]

Compute the upper limit for the parameter of interest (POI), denoted as \(\mu\).

Parameters:

• expected (ExpectationType, default observed) –

  Specifies the type of expectation for the fitting algorithm and p-value computation.

  • observed: Computes p-values using post-fit prescription, assuming experimental data as the truth (default).

  • aposteriori: Computes expected p-values using post-fit prescription, assuming experimental data as the truth.

  • apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.

• confidence_level (float, default 0.95) – Confidence level for the upper limit, representing \(1 - CL_s\). Must be between 0 and 1. Default is 0.95.

• low_init (Optional[float], default 1.0) –

  Initial lower limit for the search algorithm. If None, it is determined by \(\hat\mu + 1.5\sigma_{\hat\mu}\). Default is 1.0.

  Note

  \(\sigma_{\hat\mu}\) is determined via sigma_mu() function.

• hig_init (Optional[float], default 1.0) –

  Initial upper limit for the search algorithm. If None, it is determined by \(\hat\mu + 2.5\sigma_{\hat\mu}\). Default is 1.0.

  Note

  \(\sigma_{\hat\mu}\) is determined via sigma_mu() function.

• expected_pvalue (Literal["nominal", "1sigma", "2sigma"], default "nominal") –

  In case of aposteriori and apriori expectation, specifies the type of expected p-value for upper limit calculation.

  • "nominal": Computes the upper limit for the central p-value. Returns a single value.

  • "1sigma": Computes the upper limit for the central p-value and \(1\sigma\) fluctuation from background. Returns 3 values.

  • "2sigma": Computes the upper limit for the central p-value and \(1\sigma\) and \(2\sigma\) fluctuation from background. Returns 5 values.

  Note

  For expected=spey.ExpectationType.observed, expected_pvalue argument will be overwritten to "nominal".

• allow_negative_signal (bool, default True) – Allows for negative signal values, changing the computation of the test statistic. Default is False.

• maxiter (int, default 10000) – Maximum number of iterations for the optimiser. Default is 10000.

• optimiser_arguments (Dict, default None) – Additional arguments for the optimiser used to compute the likelihood and its maximum. Default is None.

Returns:

• A single value representing the upper limit for the nominal case.

• A list of values representing the upper limits for the central value and statistical deviations (for “1sigma” and “2sigma” cases). The order is: \(-2\sigma\), \(-1\sigma\), central value, \(1\sigma\), \(2\sigma\).

Return type:

Union[float, List[float]]

Raises:

AssertionError – If the confidence level is not between 0 and 1.

prepare_for_fit(data: List[float] | ndarray | None = None, expected: ExpectationType = observed, allow_negative_signal: bool | None = True, **kwargs) → Dict

Prepare backend for the optimiser.

Parameters:

• data (Union[List[float], np.ndarray], default None) – input data that to fit

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.

Returns:

Dictionary of necessary toolset for the fit. objective function, "func", use gradient boolean, "do_grad" and function to compute negative log-likelihood with given fit parameters, "nll".

Return type:

Dict

property s95exp: float

Compute excluded cross section value at 95% CL with apriori expectation. See excluded_cross_section() for reference.

Raises:

UnknownCrossSection – If the cross-section is nan.

property s95obs: float

Compute excluded cross section value at 95% CL with observed expectation. See excluded_cross_section() for reference.

Raises:

UnknownCrossSection – If the cross-section is nan.

sigma_mu(poi_test: float, expected: ExpectationType = observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float

If available, \(\sigma_\mu\) will be computed through Hessian of negative log-likelihood see spey.StatisticalModel.sigma_mu_from_hessian() for details. However, if not available it will be estimated via \(q_{\mu,A}\)

\[\sigma^2_A = \frac{(\mu - \mu^\prime)^2}{q_{\mu,A}}\quad , \quad q_{\mu,A} = -2\ln\lambda_A(\mu)\]

see eq. (31) in [arXiv:1007.1727]

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistics (Text, default “qtilde”) –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• kwargs –

  keyword arguments for the optimiser.

  • init_pars (List[float], default None): initial parameters for the optimiser

  • par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

value of the variance on \(\mu\).

Return type:

float

sigma_mu_from_hessian(poi_test: float, expected: ExpectationType = observed, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → float

Compute variance of \(\mu\) from inverse Hessian. See eq. (27-28) in [arXiv:1007.1727].

Parameters:

• poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• init_pars (List[float], default None) – initial parameters for the optimiser

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have Hessian implementation.

Returns:

variance on parameter of interest.

Return type:

float

significance(expected: ExpectationType = observed, **kwargs) → Tuple[float, float, List[float], List[float]]

Compute the discovery of a positive signal. See [arXiv:1007.1727] eq. (53). and sec. 5.1.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

    Note

    Since aposteriori and observed are both represent post-fit computation the result will be the same. The only difference can be seen via prefit, apriori, computation.

• kwargs –

  keyword arguments for the optimiser.

  • init_pars (List[float], default None): initial parameters for the optimiser

  • par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

(\(\sqrt{q_{0,A}}\), \(\sqrt{q_0}\), p-values and expected p-values)

Return type:

Tuple[float, float, List[float], List[float]]

xsection: float

Value of the cross section, unit is defined by the user.

Base Classes

Backend Base

class spey.BackendBase

An abstract class construction to enforce certain behaviour on statistical model backend. In order to perform certain computations, spey needs to have access to specific function constructions such as precsription to form likelihood. Hence, each backend is required to inherit BackendBase.

asimov_negative_loglikelihood(poi_test: float = 1.0, expected: ExpectationType = observed, test_statistics: str = 'qtilde', **kwargs) → Tuple[float, ndarray]

Compute negative log-likelihood at fixed \(\mu\) for Asimov data.

Note

Interface first calls backend specific methods to compute likelihood. If they are not implemented, it optimizes objective function through spey interface. Either prescription to optimizing the likelihood or objective function must be available for a backend to be sucessfully integrated to the spey interface.

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistics (Text, default "qtilde") –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood at POI of interest and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

combine(other, **kwargs)

A routine to combine to statistical models.

Note

This function is only available if the backend has a specific routine for combination between same or other backends.

Parameters:

other (BackendBase) – Statistical model object to be combined.

Raises:

NotImplementedError – If the backend does not have a combination scheme.

Returns:

Create a new statistical model from combination of this and other one.

Return type:

BackendBase

abstractmethod config(allow_negative_signal: bool = True, poi_upper_bound: float = 10.0) → ModelConfig

Model configuration.

Parameters:

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.

• poi_upper_bound (float, default 10.0) – upper bound for parameter of interest, \(\mu\).

Returns:

Model configuration. Information regarding the position of POI in parameter list, suggested input and bounds.

Return type:

ModelConfig

expected_data(pars: List[float]) → List[float]

Compute the expected value of the statistical model. This function is mainly used to generate Asimov data within the package, see generate_asimov_data().

Parameters:

pars (List[float]) – nuisance, \(\theta\) and parameter of interest, \(\mu\).

Returns:

Expected data of the statistical model

Return type:

List[float]

get_hessian_logpdf_func(expected: ExpectationType = observed, data: List[float] | ndarray | None = None) → Callable[[ndarray], float]

Currently Hessian of \(\log\mathcal{L}(\mu, \theta)\) is only used to compute variance on \(\mu\). This method returns a callable function which takes fit parameters (\(\mu\) and \(\theta\)) and returns Hessian.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• data (Union[List[float], np.ndarray], default None) – input data that to fit

Raises:

NotImplementedError – If the Hessian of the backend has not been implemented.

Returns:

Function that takes fit parameters (\(\mu\) and \(\theta\)) and returns Hessian of \(\log\mathcal{L}(\mu, \theta)\).

Return type:

Callable[[np.ndarray], float]

abstractmethod get_logpdf_func(expected: ExpectationType = observed, data: List[float] | ndarray | None = None) → Callable[[ndarray], float]

Generate function to compute \(\log\mathcal{L}(\mu, \theta)\) where \(\mu\) is the parameter of interest and \(\theta\) are nuisance parameters.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• data (Union[List[float], np.ndarray], default None) – input data that to fit

Returns:

Function that takes fit parameters (\(\mu\) and \(\theta\)) and computes \(\log\mathcal{L}(\mu, \theta)\).

Return type:

Callable[[np.ndarray], float]

get_objective_function(expected: ExpectationType = observed, data: List[float] | ndarray | None = None, do_grad: bool = True) → Callable[[ndarray], float | Tuple[float, ndarray]]

Objective function is the function to perform the optimisation on. This function is expected to be negative log-likelihood, \(-\log\mathcal{L}(\mu, \theta)\). Additionally, if available it canbe bundled with the gradient of negative log-likelihood.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• data (Union[List[float], np.ndarray], default None) – input data that to fit

• do_grad (bool, default True) – If True return objective and its gradient as tuple (subject to availablility) if False only returns objective function.

Returns:

Function which takes fit parameters (\(\mu\) and \(\theta\)) and returns either objective or objective and its gradient.

Return type:

Callable[[np.ndarray], Union[float, Tuple[float, np.ndarray]]]

get_sampler(pars: ndarray) → Callable[[int], ndarray]

Retreives the function to sample from.

Parameters:

pars (np.ndarray) – fit parameters (\(\mu\) and \(\theta\))

Raises:

NotImplementedError – If the sampler for the backend has not been implemented.

Returns:

Function that takes number_of_samples as input and draws as many samples from the statistical model.

Return type:

Callable[[int], np.ndarray]

property is_alive: bool

Returns True if at least one bin has non-zero signal yield.

minimize_asimov_negative_loglikelihood(expected: ExpectationType = observed, test_statistics: str = 'qtilde', **kwargs) → Tuple[float, ndarray]

A backend specific method to minimize negative log-likelihood for Asimov data.

Note

Interface first calls backend specific methods to compute likelihood. If they are not implemented, it optimizes objective function through spey interface. Either prescription to optimizing the likelihood or objective function must be available for a backend to be sucessfully integrated to the spey interface.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistics (Text, default "qtilde") –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

minimize_negative_loglikelihood(expected: ExpectationType = observed, allow_negative_signal: bool = True, **kwargs) → Tuple[float, ndarray]

A backend specific method to minimize negative log-likelihood.

Note

Interface first calls backend specific methods to compute likelihood. If they are not implemented, it optimizes objective function through spey interface. Either prescription to optimizing the likelihood or objective function must be available for a backend to be sucessfully integrated to the spey interface.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.

• kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

negative_loglikelihood(poi_test: float = 1.0, expected: ExpectationType = observed, **kwargs) → Tuple[float, ndarray]

Backend specific method to compute negative log-likelihood for a parameter of interest \(\mu\).

Note

Interface first calls backend specific methods to compute likelihood. If they are not implemented, it optimizes objective function through spey interface. Either prescription to optimizing the likelihood or objective function must be available for a backend to be sucessfully integrated to the spey interface.

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood at POI of interest and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

Hypothesis Testing Base

class spey.base.hypotest_base.HypothesisTestingBase(ntoys: int = 1000)

Abstract class that ensures classes that are performing hypothesis teststing includes certain set of function to perform necessary computations. This class gives the ability to compute exclusion limits and upper limits for the class inherits it.

Parameters:

ntoys (int, default 1000) – Number of toy samples for hypothesis testing. (Only used for toy-based hypothesis testing)

abstractmethod asimov_likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float

Compute likelihood at fixed \(\mu\) for Asimov data

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.

• test_statistics (Text, default “qtilde”) –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• kwargs – keyword arguments for the optimiser.

Returns:

value of the likelihood.

Return type:

float

chi2(poi_test: float = 1.0, poi_test_denominator: float | None = None, expected: ExpectationType = observed, allow_negative_signal: bool = False, **kwargs) → float

If poi_test_denominator=None computes

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\hat\mu,\hat\theta)}\right)\]

else

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\mu_{\rm denom},\theta_{\mu_{\rm denom}})}\right)\]

where \(\mu_{\rm denom}\) is poi_test_denominator which is typically zero to compare signal model with the background only model.

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• poi_test_denominator (float, default None) – parameter of interest for the denominator, \(\mu\). If None maximum likelihood will be computed.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative. Only valid when poi_test_denominator=None.

• kwargs – keyword arguments for the optimiser.

Returns:

value of the \(\chi^2\).

Return type:

float

chi2_test(expected: ExpectationType = observed, confidence_level: float = 0.95, limit_type: Literal['right', 'left', 'two-sided'] = 'two-sided', allow_negative_signal: bool = None) → List[float]

Determine the parameter of interest (POI) value(s) that constrain the \(\chi^2\) distribution at a specified confidence level.

Added in version 0.2.0.

Attention

The degrees of freedom are set to one, referring to the POI. Currently, spey does not support multiple POIs, but this feature is planned for future releases.

Parameters:

• expected (ExpectationType) –

  Specifies the type of expectation for the fitting algorithm and p-value computation.

  • observed: Computes p-values using post-fit prescription, assuming experimental data as the truth.

  • apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.

• confidence_level (float, default 0.95) – The confidence level for the upper limit. Must be between 0 and 1. This refers to the total inner area under the bell curve. Noted as \(CL\) below.

• limit_type ('right', 'left' or 'two-sided', default "two-sided") – Specifies which side of the \(\chi^2\) distribution should be constrained. For two-sided limits, the inner area of the \(\chi^2\) distribution is set to confidence_level, making the threshold \(\alpha=(1-CL)/2\), where CL is the confidence_level. For left or right limits alone, \(\alpha=1-CL\). The \(\chi^2\)-threshold is calculated using inverse survival function at \(\alpha\).

• allow_negative_signal (bool, default None) – Controls whether the signal can be negative. If None, it will be set to True for two-sided and left limits, and False for right limits. Otherwise, user can control this behaviour.

Returns:

POI value(s) that constrain the \(\chi^2\) distribution at the given threshold.

Return type:

List[float]

exclusion_confidence_level(poi_test: float = 1.0, expected: ExpectationType = observed, allow_negative_signal: bool = False, calculator: Literal['asymptotic', 'toy', 'chi_square'] = 'asymptotic', **kwargs) → List[float]

Compute exclusion confidence level (\(CL_s\)) at a given POI, \(\mu\).

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed. If expected is set to "all" it will return both p-values and expected p-values where the likelihood will be fitted to observations.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

    Note

    In case of observed, function will return one value which has been fit to the observed data.

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

    Note

    In case of aposteriori, function will return five value for expected p-values which has been fit to the observed data. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The order of the output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

    Note

    In case of apriori, function will return five value for expected p-values which has been fit to the SM background. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.

• allow_negative_signal (bool, default False) – If True \(\hat\mu\) value will be allowed to be negative.

• calculator (Literal["asymptotic", "toy", "chi_square"], default "asymptotic") –

  Chooses the computation basis for hypothesis testing

  • "asymptotic": Uses asymptotic hypothesis testing to compute p-values.

  • "toy": Uses generated toy samples to compute p-values.

  • "chi_square": Computes p-values via chi-square; \(\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}\).

• kwargs –

  keyword arguments for the optimiser.

  • init_pars (List[float], default None): initial parameters for the optimiser

  • par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Raises:

CalculatorNotAvailable – If calculator is not available.

Returns:

Exclusion confidence level i.e. \(CL_s\).

Return type:

List[float]

fixed_poi_sampler(poi_test: float, size: int | None = None, expected: ExpectationType = observed, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → ndarray | Callable[[int], ndarray]

Sample data from the statistical model with fixed parameter of interest.

Parameters:

• poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).

• size (int, default None) – sample size. If None a callable function will be returned which takes sample size as input.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• init_pars (List[float], default None) – initial parameters for the optimiser

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have sampler implementation.

Returns:

Sampled data with shape of (size, number of bins) or callable function to sample from directly.

Return type:

Union[np.ndarray, Callable[[int], np.ndarray]]

abstract property is_alive: bool

Returns True if at least one bin has non-zero signal yield.

abstract property is_asymptotic_calculator_available: bool

Check if Asymptotic calculator is available for the backend

abstract property is_chi_square_calculator_available: bool

Check if chi-square calculator is available for the backend

abstract property is_toy_calculator_available: bool

Check if Toy calculator is available for the backend

abstractmethod likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, data: List[float] | ndarray | None = None, **kwargs) → float

Compute likelihood of the statistical model

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.

• data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.

• kwargs – keyword arguments for the optimiser.

Returns:

value of likelihood at fixed \(\mu\).

Return type:

float

abstractmethod maximize_asimov_likelihood(return_nll: bool = True, expected: ExpectationType = observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → Tuple[float, float]

Compute maximum of the likelihood for Asimov data.

Parameters:

• return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistics (Text, default “qtilde”) –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• kwargs – keyword arguments for the optimiser.

Returns:

value of \(\hat\mu\) and maximum likelihood.

Return type:

Tuple[float, float]

abstractmethod maximize_likelihood(return_nll: bool = True, expected: ExpectationType = observed, allow_negative_signal: bool = True, data: List[float] | ndarray | None = None, **kwargs) → Tuple[float, float]

Compute maximum of the likelihood.

Parameters:

• return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.

• data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.

• kwargs – keyword arguments for the optimiser.

Returns:

value of \(\hat\mu\) and maximum likelihood.

Return type:

Tuple[float, float]

ntoys

Number of toy samples for sample generator during hypothesis testing

poi_upper_limit(expected: ExpectationType = observed, confidence_level: float = 0.95, low_init: float = 1.0, hig_init: float = 1.0, expected_pvalue: Literal['nominal', '1sigma', '2sigma'] = 'nominal', maxiter: int = 10000, optimiser_arguments: Dict[str, Any] | None = None) → float | List[float]

Compute the upper limit for the parameter of interest (POI), denoted as \(\mu\).

Parameters:

• expected (ExpectationType, default observed) –

  Specifies the type of expectation for the fitting algorithm and p-value computation.

  • observed: Computes p-values using post-fit prescription, assuming experimental data as the truth (default).

  • aposteriori: Computes expected p-values using post-fit prescription, assuming experimental data as the truth.

  • apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.

• confidence_level (float, default 0.95) – Confidence level for the upper limit, representing \(1 - CL_s\). Must be between 0 and 1. Default is 0.95.

• low_init (Optional[float], default 1.0) –

  Initial lower limit for the search algorithm. If None, it is determined by \(\hat\mu + 1.5\sigma_{\hat\mu}\). Default is 1.0.

  Note

  \(\sigma_{\hat\mu}\) is determined via sigma_mu() function.

• hig_init (Optional[float], default 1.0) –

  Initial upper limit for the search algorithm. If None, it is determined by \(\hat\mu + 2.5\sigma_{\hat\mu}\). Default is 1.0.

  Note

  \(\sigma_{\hat\mu}\) is determined via sigma_mu() function.

• expected_pvalue (Literal["nominal", "1sigma", "2sigma"], default "nominal") –

  In case of aposteriori and apriori expectation, specifies the type of expected p-value for upper limit calculation.

  • "nominal": Computes the upper limit for the central p-value. Returns a single value.

  • "1sigma": Computes the upper limit for the central p-value and \(1\sigma\) fluctuation from background. Returns 3 values.

  • "2sigma": Computes the upper limit for the central p-value and \(1\sigma\) and \(2\sigma\) fluctuation from background. Returns 5 values.

  Note

  For expected=spey.ExpectationType.observed, expected_pvalue argument will be overwritten to "nominal".

• allow_negative_signal (bool, default True) – Allows for negative signal values, changing the computation of the test statistic. Default is False.

• maxiter (int, default 10000) – Maximum number of iterations for the optimiser. Default is 10000.

• optimiser_arguments (Dict, default None) – Additional arguments for the optimiser used to compute the likelihood and its maximum. Default is None.

Returns:

• A single value representing the upper limit for the nominal case.

• A list of values representing the upper limits for the central value and statistical deviations (for “1sigma” and “2sigma” cases). The order is: \(-2\sigma\), \(-1\sigma\), central value, \(1\sigma\), \(2\sigma\).

Return type:

Union[float, List[float]]

Raises:

AssertionError – If the confidence level is not between 0 and 1.

sigma_mu(poi_test: float, expected: ExpectationType = observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float

If available, \(\sigma_\mu\) will be computed through Hessian of negative log-likelihood see spey.StatisticalModel.sigma_mu_from_hessian() for details. However, if not available it will be estimated via \(q_{\mu,A}\)

\[\sigma^2_A = \frac{(\mu - \mu^\prime)^2}{q_{\mu,A}}\quad , \quad q_{\mu,A} = -2\ln\lambda_A(\mu)\]

see eq. (31) in [arXiv:1007.1727]

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistics (Text, default “qtilde”) –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• kwargs –

  keyword arguments for the optimiser.

  • init_pars (List[float], default None): initial parameters for the optimiser

  • par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

value of the variance on \(\mu\).

Return type:

float

significance(expected: ExpectationType = observed, **kwargs) → Tuple[float, float, List[float], List[float]]

Compute the discovery of a positive signal. See [arXiv:1007.1727] eq. (53). and sec. 5.1.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

    Note

    Since aposteriori and observed are both represent post-fit computation the result will be the same. The only difference can be seen via prefit, apriori, computation.

• kwargs –

  keyword arguments for the optimiser.

  • init_pars (List[float], default None): initial parameters for the optimiser

  • par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

(\(\sqrt{q_{0,A}}\), \(\sqrt{q_0}\), p-values and expected p-values)

Return type:

Tuple[float, float, List[float], List[float]]

Model Configuration

class spey.base.model_config.ModelConfig(poi_index: int, minimum_poi: float, suggested_init: List[float], suggested_bounds: List[Tuple[float, float]], parameter_names: List[str] | None = None, suggested_fixed: List[bool] | None = None)

Container to hold certain properties of the backend and statistical model. This will ensure the consistency of the computation through out the package.

Parameters:

• poi_index (int) – index of the parameter of interest within the parameter list.

• minimum_poi (float) –

  minimum value parameter of interest can take to ensure \(N^{\rm bkg}+\mu N^{\rm sig}\geq0\). This value can be set to \(-\infty\) but note that optimiser will take it as a lower bound so such low value might effect the convergence of the optimisation algorithm especially for relatively flat objective surfaceses. Hence we suggest setting the minimum value to

  \[\mu_{\rm min} = - \min\left( \frac{N^{\rm bkg}_i}{N^{\rm sig}_i} \right)\ ,\ i\in {\rm bins}\]

• suggested_init (List[float]) – suggested initial parameters for the optimiser.

• suggested_bounds (List[Tuple[float, float]]) – suggested parameter bounds for the optimiser.

Returns:

obj:~spey.base.model_config.ModelConfig: Model configuration container for optimiser.

fixed_poi_bounds(poi_value: float | None = None) → List[Tuple[float, float]]

Adjust the bounds for the parameter of interest for fixed POI fit.

Parameters:

poi_value (Optional[float], default None) – parameter of interest, \(\mu\).

Returns:

Updated bounds.

Return type:

List[Tuple[float, float]]

minimum_poi: float

minimum value parameter of interest can take to ensure \(N^{\rm bkg}+\mu N^{\rm sig}\geq0\). This value can be set to \(-\infty\) but note that optimiser will take it as a lower bound so such low value might effect the convergence of the optimisation algorithm especially for relatively flat objective surfaceses. Hence we suggest setting the minimum value to

\[\mu_{\rm min} = - \min\left( \frac{N^{\rm bkg}_i}{N^{\rm sig}_i} \right)\ ,\ i\in {\rm bins}\]

property npar: int

Number of parameters

parameter_names: List[str] | None = None

Names of the parameters

poi_index: int

Index of the parameter of interest wihin the parameter list

rescale_poi_bounds(allow_negative_signal: bool = True, poi_upper_bound: float | None = None) → List[Tuple[float, float]]

Rescale bounds for POI.

Parameters:

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.

• poi_upper_bound (float, default None) – Maximum value POI can take during optimisation.

Returns:

Updated bounds.

Return type:

List[Tuple[float, float]]

suggested_bounds: List[Tuple[float, float]]

Suggested upper and lower bounds for parameters

suggested_fixed: List[bool] | None = None

Suggested fixed values

suggested_init: List[float]

Suggested initialisation for parameters

Statistical Model Combiner

class spey.UnCorrStatisticsCombiner(*args, ntoys: int = 1000)

Bases: HypothesisTestingBase

Module to combine uncorrelated statistical models. It takes serries of StatisticalModel object as input. These statistical models does not need to have same backend. However, this class assumes that all the input statistical model’s are completely independent from each other.

Warning

UnCorrStatisticsCombiner assumes that all input are uncorrelated and non of the statistical models posesses the same set of nuisance parameters. Thus each statistical model is optimised independently.

Parameters:

• arguments (input) – Uncorrelated statistical models

• ntoys (int, default 1000) – Number of toy samples for hypothesis testing. (Only used for toy-based hypothesis testing)

Raises:

• AnalysisQueryError – If multiple StatisticalModel has the same analysis attribute.

• TypeError – If the input type is not StatisticalModel.

property analyses: List[str]

List of the unique identifiers of the statistical models within the stack.

Returns:

List of analysis names.

Return type:

List[Text]

append(statistical_model: StatisticalModel) → None

Append new independent StatisticalModel to the stack.

Parameters:

statistical_model (StatisticalModel) – new statistical model to be added to the stack.

Raises:

• AnalysisQueryError – If multiple StatisticalModel has the same analysis attribute.

• TypeError – If the input type is not StatisticalModel.

asimov_likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, test_statistics: str = 'qtilde', statistical_model_options: Dict[str, Dict] | None = None, **kwargs) → float

Compute likelihood of the statistical model stack generated with the Asimov data.

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.

• test_statistics (Text, default "qtilde") –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• statistical_model_options (Dict[Text, Dict], default None) –

  backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.

  >>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
  

• kwargs – keyword arguments for the optimiser.

Returns:

likelihood computed for asimov data

Return type:

float

chi2(poi_test: float = 1.0, poi_test_denominator: float | None = None, expected: ExpectationType = observed, allow_negative_signal: bool = False, **kwargs) → float

If poi_test_denominator=None computes

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\hat\mu,\hat\theta)}\right)\]

else

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\mu_{\rm denom},\theta_{\mu_{\rm denom}})}\right)\]

where \(\mu_{\rm denom}\) is poi_test_denominator which is typically zero to compare signal model with the background only model.

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• poi_test_denominator (float, default None) – parameter of interest for the denominator, \(\mu\). If None maximum likelihood will be computed.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative. Only valid when poi_test_denominator=None.

• kwargs – keyword arguments for the optimiser.

Returns:

value of the \(\chi^2\).

Return type:

float

chi2_test(expected: ExpectationType = observed, confidence_level: float = 0.95, limit_type: Literal['right', 'left', 'two-sided'] = 'two-sided', allow_negative_signal: bool = None) → List[float]

Determine the parameter of interest (POI) value(s) that constrain the \(\chi^2\) distribution at a specified confidence level.

Added in version 0.2.0.

Attention

The degrees of freedom are set to one, referring to the POI. Currently, spey does not support multiple POIs, but this feature is planned for future releases.

Parameters:

• expected (ExpectationType) –

  Specifies the type of expectation for the fitting algorithm and p-value computation.

  • observed: Computes p-values using post-fit prescription, assuming experimental data as the truth.

  • apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.

• confidence_level (float, default 0.95) – The confidence level for the upper limit. Must be between 0 and 1. This refers to the total inner area under the bell curve. Noted as \(CL\) below.

• limit_type ('right', 'left' or 'two-sided', default "two-sided") – Specifies which side of the \(\chi^2\) distribution should be constrained. For two-sided limits, the inner area of the \(\chi^2\) distribution is set to confidence_level, making the threshold \(\alpha=(1-CL)/2\), where CL is the confidence_level. For left or right limits alone, \(\alpha=1-CL\). The \(\chi^2\)-threshold is calculated using inverse survival function at \(\alpha\).

• allow_negative_signal (bool, default None) – Controls whether the signal can be negative. If None, it will be set to True for two-sided and left limits, and False for right limits. Otherwise, user can control this behaviour.

Returns:

POI value(s) that constrain the \(\chi^2\) distribution at the given threshold.

Return type:

List[float]

exclusion_confidence_level(poi_test: float = 1.0, expected: ExpectationType = observed, allow_negative_signal: bool = False, calculator: Literal['asymptotic', 'toy', 'chi_square'] = 'asymptotic', **kwargs) → List[float]

Compute exclusion confidence level (\(CL_s\)) at a given POI, \(\mu\).

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed. If expected is set to "all" it will return both p-values and expected p-values where the likelihood will be fitted to observations.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

    Note

    In case of observed, function will return one value which has been fit to the observed data.

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

    Note

    In case of aposteriori, function will return five value for expected p-values which has been fit to the observed data. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The order of the output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

    Note

    In case of apriori, function will return five value for expected p-values which has been fit to the SM background. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.

• allow_negative_signal (bool, default False) – If True \(\hat\mu\) value will be allowed to be negative.

• calculator (Literal["asymptotic", "toy", "chi_square"], default "asymptotic") –

  Chooses the computation basis for hypothesis testing

  • "asymptotic": Uses asymptotic hypothesis testing to compute p-values.

  • "toy": Uses generated toy samples to compute p-values.

  • "chi_square": Computes p-values via chi-square; \(\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}\).

• kwargs –

  keyword arguments for the optimiser.

  • init_pars (List[float], default None): initial parameters for the optimiser

  • par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Raises:

CalculatorNotAvailable – If calculator is not available.

Returns:

Exclusion confidence level i.e. \(CL_s\).

Return type:

List[float]

find_most_sensitive() → StatisticalModel

If the cross sections are defined, finds the statistical model with minimum prefit excluded cross section value. See s95exp.

Returns:

Statistical model with minimum prefit excluded cross section value.

Return type:

StatisticalModel

fixed_poi_sampler(poi_test: float, size: int | None = None, expected: ExpectationType = observed, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → ndarray | Callable[[int], ndarray]

Sample data from the statistical model with fixed parameter of interest.

Parameters:

• poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).

• size (int, default None) – sample size. If None a callable function will be returned which takes sample size as input.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• init_pars (List[float], default None) – initial parameters for the optimiser

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have sampler implementation.

Returns:

Sampled data with shape of (size, number of bins) or callable function to sample from directly.

Return type:

Union[np.ndarray, Callable[[int], np.ndarray]]

generate_asimov_data(expected: ExpectationType = observed, test_statistic: str = 'qtilde', statistical_model_options: Dict[str, Dict] | None = None, **kwargs) → Dict[str, List[float]]

Generate Asimov data for the statistical model. This function calls generate_asimov_data() function for each statistical model with appropriate statistical_model_options and generates Asimov data for each statistical model independently.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistic (Text, default "qtilde") –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• statistical_model_options (Dict[Text, Dict], default None) –

  backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.

  >>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
  

• kwargs – keyword arguments for the optimiser.

Returns:

Returns a dictionary for data specific to each analysis. keywords will be analysis names and the items are data.

Return type:

Dict[Text, List[float]]

property is_alive: bool

Returns true if there is at least one statistical model with at least one non zero bin with signal yield.

Returns:

bool

property is_asymptotic_calculator_available: bool

Check if Asymptotic calculator is available for the backend

property is_chi_square_calculator_available: bool

Check if \(\chi^2\) calculator is available for the backend

property is_toy_calculator_available: bool

Check if Toy calculator is available for the backend

items() → Iterator[Tuple[str, StatisticalModel]]

Returns a generator that returns analysis name and StatisticalModel every iteration.

Returns:

Iterator[Tuple[Text, StatisticalModel]]

likelihood(poi_test: float = 1.0, expected: ExpectationType = observed, return_nll: bool = True, data: Dict[str, List[float]] | None = None, statistical_model_options: Dict[str, Dict] | None = None, **kwargs) → float

Compute the likelihood of the statistical model stack at a fixed parameter of interest

Parameters:

• poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.

• data (Dict[Text, List[float]], default None) – Input data to be used for fit. It needs to be given as a dictionary where the key argument is the name of the analysis and item is the statistical model specific data.

• statistical_model_options (Optional[Dict[Text, Dict]], default None) –

  backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.

  >>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
  

• kwargs – keyword arguments for the optimiser.

Returns:

value of the likelihood of the stacked statistical model.

Return type:

float

maximize_asimov_likelihood(return_nll: bool = True, expected: ExpectationType = observed, test_statistics: str = 'qtilde', initial_muhat_value: float | None = None, par_bounds: List[Tuple[float, float]] | None = None, statistical_model_options: Dict[str, Dict] | None = None, **optimiser_options) → Tuple[float, float]

Find the maximum of the likelihood which computed with respect to Asimov data.

Parameters:

• return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistics (Text, default "qtilde") –

  test statistic.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• initial_muhat_value (float, default None) – Initial value for \(\hat\mu\).

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• statistical_model_options (Dict[Text, Dict], default None) –

  backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.

  >>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
  

• kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

maximize_likelihood(return_nll: bool = True, expected: ExpectationType = observed, allow_negative_signal: bool = True, data: Dict[str, List[float]] | None = None, initial_muhat_value: float | None = None, par_bounds: List[Tuple[float, float]] | None = None, statistical_model_options: Dict[str, Dict] | None = None, **optimiser_options) → Tuple[float, float]

Find the maximum of the likelihood.

Parameters:

• return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.

• data (Dict[Text, List[float]], default None) – Input data to be used for fit. It needs to be given as a dictionary where the key argument is the name of the analysis and item is the statistical model specific data.

• initial_muhat_value (float, default None) –

  Initialisation for the \(\hat\mu\) for the optimiser. If None the initial value will be estimated by weighted combination of \(\hat\mu_i\) where \(i\) stands for the statistical models within the stack.

  \[\mu_{\rm init} = \frac{1}{\sum_i \sigma_{\hat\mu, i}^2} \sum_i \frac{\hat\mu_i}{\sigma_{\hat\mu, i}^2}\]

  where the value of \(\sigma_{\hat\mu}\) has been estimated by sigma_mu() function.

• par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.

• statistical_model_options (Dict[Text, Dict], default None) –

  backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.

  >>> statistical_model_options = {"default.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
  

• kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

property minimum_poi: float

Find the maximum minimum poi, \(\mu\) value within the analysis stack.

Returns:

minimum poi value that the combined stack can take.

Return type:

float

ntoys

Number of toy samples for sample generator during hypothesis testing

poi_upper_limit(expected: ExpectationType = observed, confidence_level: float = 0.95, low_init: float = 1.0, hig_init: float = 1.0, expected_pvalue: Literal['nominal', '1sigma', '2sigma'] = 'nominal', maxiter: int = 10000, optimiser_arguments: Dict[str, Any] | None = None) → float | List[float]

Compute the upper limit for the parameter of interest (POI), denoted as \(\mu\).

Parameters:

• expected (ExpectationType, default observed) –

  Specifies the type of expectation for the fitting algorithm and p-value computation.

  • observed: Computes p-values using post-fit prescription, assuming experimental data as the truth (default).

  • aposteriori: Computes expected p-values using post-fit prescription, assuming experimental data as the truth.

  • apriori: Computes expected p-values using pre-fit prescription, assuming the Standard Model (SM) as the truth.

• confidence_level (float, default 0.95) – Confidence level for the upper limit, representing \(1 - CL_s\). Must be between 0 and 1. Default is 0.95.

• low_init (Optional[float], default 1.0) –

  Initial lower limit for the search algorithm. If None, it is determined by \(\hat\mu + 1.5\sigma_{\hat\mu}\). Default is 1.0.

  Note

  \(\sigma_{\hat\mu}\) is determined via sigma_mu() function.

• hig_init (Optional[float], default 1.0) –

  Initial upper limit for the search algorithm. If None, it is determined by \(\hat\mu + 2.5\sigma_{\hat\mu}\). Default is 1.0.

  Note

  \(\sigma_{\hat\mu}\) is determined via sigma_mu() function.

• expected_pvalue (Literal["nominal", "1sigma", "2sigma"], default "nominal") –

  In case of aposteriori and apriori expectation, specifies the type of expected p-value for upper limit calculation.

  • "nominal": Computes the upper limit for the central p-value. Returns a single value.

  • "1sigma": Computes the upper limit for the central p-value and \(1\sigma\) fluctuation from background. Returns 3 values.

  • "2sigma": Computes the upper limit for the central p-value and \(1\sigma\) and \(2\sigma\) fluctuation from background. Returns 5 values.

  Note

  For expected=spey.ExpectationType.observed, expected_pvalue argument will be overwritten to "nominal".

• allow_negative_signal (bool, default True) – Allows for negative signal values, changing the computation of the test statistic. Default is False.

• maxiter (int, default 10000) – Maximum number of iterations for the optimiser. Default is 10000.

• optimiser_arguments (Dict, default None) – Additional arguments for the optimiser used to compute the likelihood and its maximum. Default is None.

Returns:

• A single value representing the upper limit for the nominal case.

• A list of values representing the upper limits for the central value and statistical deviations (for “1sigma” and “2sigma” cases). The order is: \(-2\sigma\), \(-1\sigma\), central value, \(1\sigma\), \(2\sigma\).

Return type:

Union[float, List[float]]

Raises:

AssertionError – If the confidence level is not between 0 and 1.

remove(analysis: str) → None

Remove an analysis from the stack.

Parameters:

analysis (Text) – unique identifier of the analysis to be removed.

Raises:

AnalysisQueryError – If the unique identifier does not match any of the statistical models in the stack.

sigma_mu(poi_test: float, expected: ExpectationType = observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float

If available, \(\sigma_\mu\) will be computed through Hessian of negative log-likelihood see spey.StatisticalModel.sigma_mu_from_hessian() for details. However, if not available it will be estimated via \(q_{\mu,A}\)

\[\sigma^2_A = \frac{(\mu - \mu^\prime)^2}{q_{\mu,A}}\quad , \quad q_{\mu,A} = -2\ln\lambda_A(\mu)\]

see eq. (31) in [arXiv:1007.1727]

Parameters:

• poi_test (float, default 1.0) – parameter of interest, \(\mu\).

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

• test_statistics (Text, default “qtilde”) –

  test statistics.

  • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).

    Warning

    Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.

  • 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).

  • 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).

• kwargs –

  keyword arguments for the optimiser.

  • init_pars (List[float], default None): initial parameters for the optimiser

  • par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

value of the variance on \(\mu\).

Return type:

float

significance(expected: ExpectationType = observed, **kwargs) → Tuple[float, float, List[float], List[float]]

Compute the discovery of a positive signal. See [arXiv:1007.1727] eq. (53). and sec. 5.1.

Parameters:

• expected (ExpectationType) –

  Sets which values the fitting algorithm should focus and p-values to be computed.

  • observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).

  • apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

    Note

    Since aposteriori and observed are both represent post-fit computation the result will be the same. The only difference can be seen via prefit, apriori, computation.

• kwargs –

  keyword arguments for the optimiser.

  • init_pars (List[float], default None): initial parameters for the optimiser

  • par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

(\(\sqrt{q_{0,A}}\), \(\sqrt{q_0}\), p-values and expected p-values)

Return type:

Tuple[float, float, List[float], List[float]]

property statistical_models: Tuple[StatisticalModel]

Accessor to the statistical model stack

Returns:

statistical model stack as an ntuple.

Return type:

Tuple[StatisticalModel]

Hypothesis testing

test_statistics.qmu(mu, muhat, max_logpdf, ...)	Test statistic \(q_{\mu}\), see eq.
test_statistics.qmu_tilde(mu, muhat, ...)	Alternative test statistics, \(\tilde{q}_{\mu}\), see eq.
test_statistics.q0(mu, muhat, max_logpdf, logpdf)	Discovery test statistics, \(q_{0}\) see eq.
test_statistics.get_test_statistic(test_stat)	Retreive the test statistic function
test_statistics.compute_teststatistics(mu, ...)	Compute test statistics
upper_limits.ComputerWrapper(computer)	Wrapper for the computer function to track inputs and outputs
upper_limits.find_poi_upper_limit(...[, ...])	Find upper limit for parameter of interest, \(\mu\)
upper_limits.find_root_limits(computer[, ...])	Find upper and lower bracket limits for the root finding algorithm
utils.pvalues(delta_test_statistic, ...)	Calculate the p-values for the observed test statistic under the signal + background and background-only model hypotheses.
utils.expected_pvalues(...)	Calculate the \(p\) values corresponding to the median significance of variations of the signal strength from the background only hypothesis \(\mu=0\) at \((-2,-1,0,1,2)\sigma\).
distributions.AsymptoticTestStatisticsDistribution(shift)	The distribution the test statistic in the asymptotic case.
distributions.EmpricTestStatisticsDistribution(samples)	Create emprical distribution.
asymptotic_calculator.compute_asymptotic_confidence_level(...)	Compute p values i.e. \(p_{s+b}\), \(p_b\) and \(p_s\).
toy_calculator.compute_toy_confidence_level(...)	Compute confidence limits i.e. \(CL_{s+b}\), \(CL_b\) and \(CL_s\).

Gradient Tools

value_and_grad(statistical_model[, ...])	Retreive function to compute negative log-likelihood and its gradient.
hessian(statistical_model[, expected, data])	Retreive the function to compute Hessian of negative log-likelihood

Default Backends

Distributions

Poisson(loc)	Poisson distribution
Normal(loc, scale[, weight, domain])	Normal distribution
MultivariateNormal(mean, cov[, weight, domain])	Multivariate normal distribution
MainModel(loc[, cov, pdf_type])	Main statistical model, modelled as Poisson distribution which has a variable lambda.
ConstraintModel(pdf_descriptions)	Constraint term modelled as a Gaussian distribution.

Default PDFs

DefaultPDFBase(signal_yields, ...[, ...])	Default PDF backend base
UncorrelatedBackground(signal_yields, ...[, ...])	Interface for uncorrelated background uncertainties.
CorrelatedBackground(signal_yields, ...[, ...])	Correlated multi-region statistical model.
ThirdMomentExpansion(signal_yields, ...[, ...])	Simplified likelihood interface with third moment expansion.
EffectiveSigma(signal_yields, ...[, ...])	Simplified likelihood interface with variable Gaussian.
third_moment.third_moment_expansion(...[, ...])	Construct the terms for third moment expansion.
third_moment.compute_third_moments(...[, ...])	Assuming that the uncertainties are modelled as Gaussian, it computes third moments using Bifurcated Gaussian with asymmetric uncertainties.

Simple PDFs

simple_pdf.Poisson(signal_yields, ...[, ...])	Poisson distribution without uncertainty implementation.
simple_pdf.Gaussian(signal_yields, ...)	Gaussian distribution for uncorrelated likelihoods.
simple_pdf.MultivariateNormal(signal_yields, ...)	Multivariate Gaussian distribution.

Developer Tools

Exceptions

AnalysisQueryError	Analysis query exception
MethodNotAvailable	If the method is not available for a given backend
NegativeExpectedYields	Negative expected yields exception
PluginError	Invalid plugin exception
UnknownCrossSection	Unknown cross-section exception
UnknownTestStatistics	Unknown test statistics exception
CalculatorNotAvailable	Unavailable calculator Exception
CanNotFindRoots	Unable to find roots of the function
UnknownComputer	Unknown computation base
CombinerNotAvailable	Unavailable combination routine exception

Logging

disable_logging	Temporary disable logging implementation
capture_logs	Context manager that captures log records emitted while inside the context and suppresses duplicate messages.