Usage¶
This page shows the functions that call the LLM, their options and their errors. To install EBaMl and set an API key first, see Setup.
from interpret.glassbox import ExplainableBoostingClassifier
from ebaml import DescribeEBMRequest, DescribeGraphRequest, PromptOptions, describe_ebm, describe_graph
ebm = ExplainableBoostingClassifier().fit(X, y)
print(describe_graph(DescribeGraphRequest(ebm, feature_index=0)))
print(describe_ebm(DescribeEBMRequest(ebm, num_sentences=30, prompt=PromptOptions(dataset_description="..."))).text)
describe_ebm returns an EBMDescription. Its field text is the description. Its fields skipped_terms and
skipped_features name the interaction terms and the features without a graph that the description leaves out. The text
says so too, and the same names reach the log as interactions_skipped and features_skipped warnings.
A graph can be turned into text with EBMGraph.to_text and back with EBMGraph.from_text, which also reads a graph
embedded in a longer prompt. It needs the confidence bounds, and it only reads continuous features.
Each function takes one request: DescribeGraphRequest, DescribeEBMRequest or ExplainInstanceRequest. Three keyword-only fields set the options. graph=GraphOptions(...) sets the options of EBMGraph.from_ebm and
EBMGraph.to_text, for example max_tokens. prompt=PromptOptions(...) sets the descriptions that the prompts use
(expert_description, graph_description, dataset_description and task_description). model= selects the LLM: see
Choosing the model. The requests, GraphOptions, PromptOptions and ModelOptions are importable from ebaml. Token usage reported by the provider is logged at INFO on the ebaml.llm.descriptors logger.
A multiclass classifier has one graph series per class. GraphOptions(classes=[...]) selects the classes to describe,
because the text of a model with many classes is long. In every bin, the scores have the same mean over all the classes
of the model. Only the differences between the classes change the predicted probabilities.
The multiclass notebook shows each option on the iris data. The model kinds notebook compares the graphs and the local
explanations of a binary classifier, a multiclass classifier and a regressor.
from ebaml import GraphOptions
multiclass = ExplainableBoostingClassifier(interactions=0).fit(X, y_multiclass)
print(describe_graph(DescribeGraphRequest(multiclass, feature_index=0, graph=GraphOptions(classes=["young", "old"]))))
Explaining one prediction¶
from ebaml import DescribeEBMRequest, ExplainInstanceRequest, describe_ebm, explain_instance
ebm = ExplainableBoostingClassifier(interactions=0).fit(X, y) # binary classifier, no interactions
global_explanation = describe_ebm(DescribeEBMRequest(ebm)).text # computed once, prepended to every local prompt
explanation = explain_instance(
ExplainInstanceRequest(ebm, X.iloc[0].to_dict(), global_explanation, audience="layperson")
)
print(explanation.counterfactual) # what would have to change to flip the prediction
print(explanation.sufficient_reason) # which inputs settle the prediction on their own
print(explanation.completions) # how certain the outcome is as inputs become known
The facts are computed exactly from the scores and confidence intervals of the EBM (ebaml.local); the LLM only puts
them in words. They are the three queries of arXiv:2010.12265: minimum change
required, minimum sufficient reason and count completions. audience="layperson" (default) gives no model numbers;
audience="data_scientist" quotes scores, probabilities and confidence intervals. Subclass ebaml.Audience to add a
reader with its own rules. Pass language="French" (any language name) to get the explanations in that language. It supports every kind of model
without interactions, and explanation.facts holds the numbers behind the text. The Local explanations page explains what each explanation means, the options and the limits,
and the local explanations notebook walks through them on the Titanic data.
Errors¶
Every error that ebaml raises on purpose derives from ebaml.EBaMlError. Use except EBaMlError to catch the errors
of ebaml apart from the errors of other code. Each error for a bad value also derives from ValueError, so an
except ValueError catches it too.
| Error | ValueError |
Raised when |
|---|---|---|
InvalidOptionError |
yes | An option is not one of its valid values, for example an unknown Normalization. |
OptionRangeError |
yes | A number option is outside its range, for example confidence_level=1. |
InvalidInstanceError |
yes | The instance to explain lacks a value, or has a category that the EBM did not see. |
UnsupportedModelError |
yes | A local explanation gets an EBM with interactions, or one that is not a classifier or a regressor. A graph gets classes, and the model has one output. model_kind gets an EBM that is not fitted. |
GraphTextError |
yes | EBMGraph.from_text cannot read the text of a graph. |
EBMGraphError |
no | The base class of the errors of a graph. |
EBMGraphShapeError |
yes | The bins of a graph do not fit its scores, the scores and the standard deviations have different shapes, or a boolean format gets more than two bins. |
EBMGraphTokenLimitError |
no | The text of a graph is over the token budget, and its format cannot be simplified. |
EBMGraphSimplificationError |
no | The text of a graph is still over the token budget at the largest simplification. |
TokenFitError |
yes | The token weights cannot be fitted from an empty list of samples, or a weight is negative or not finite. |
SDKUnavailableError |
no | baml_sdk is missing, and ebaml cannot run baml generate. The error also derives from ImportError. |
LLMRequestError |
no | The base class of the errors of a provider. Its message lists the steps that fix the cause. |
MissingAPIKeyError |
no | A provider needs an API key and the environment has none. |
APIKeyRejectedError |
no | A provider answers a request with "unauthorized" or "forbidden". |
RateLimitedError |
no | A provider limits the rate of the requests. |
ProviderUnreachableError |
no | A request does not reach the provider. |
EBMGraphShapeError, EBMGraphTokenLimitError and EBMGraphSimplificationError derive from EBMGraphError. The four provider errors derive from LLMRequestError. ebaml exports all of these classes.
Behaviour¶
- Model:
openrouter/freepicks a free model for each request, so the model can change between calls and between the turns of one conversation. Quality and speed vary with it. - Retries: transient failures (rate limits, network errors, 5xx) are retried up to 3 times. Requests the provider rejects are not retried.
- Concurrency:
describe_ebmdescribes features concurrently, at mostmax_concurrencyat a time (default 8). Free models are rate limited, so lower it (describe_ebm(DescribeEBMRequest(ebm, max_concurrency=2))) if you see rate-limit errors. - Progress:
describe_ebm(DescribeEBMRequest(ebm, progress_bar=True))shows a progress bar on the error stream. It counts one step for each feature and one for the summary. If the error stream is not a terminal, the bar writes one line for each step, so a log file or an agent that polls the console sees the progress. The benchmark andfit_token_weights.pyshow a bar by default. Use--no-progressto turn it off. - Token budget: there is no tokenizer before a call, so the check estimates the tokens of a graph text from its
words and its digits. The default weights are 4.13 tokens for each word and 0.27 for each digit. They come from 100
graph texts that one current model counted. The estimate has a margin above the fit, so it is more often too high
than too low. Other models count other numbers of tokens, so the estimate is a guide. Going a little over
max_tokensonly warns. Going more thanoverbudget_percent(default 50) over makesto_textsimplify the graph, or raise if it cannot. The provider's own error is the real limit.describe_graph,describe_ebmandexplain_instancepass the check. A direct call ofEBMGraph.to_textchecks nothing, unless you passcheck_budget, for exampleebaml.check_token_budget. That check needs no LLM and no key. To choose the weights, pass aTokenEstimatorinstead. To use the weights of each model and to learn them, passGraphOptions(token_learner=TokenLearner(learning=True)). The learner starts each model at the weights of a table that ships with the package. Then it moves them toward the tokens of the graphs in your real calls. The weights persist in~/.cache/ebaml/token_weights.json. Learning is off by default, because learned weights change the graph text from one call to the next.scripts/llm/token_weights.pyadds the weights of a new model to the table.
The model argument selects the LLM of a call: see Choosing the model.