Skip to content

API reference

Import every public name from ebaml, for example from ebaml import describe_ebm. The sections below name the module that defines each name.

Describing graphs and models

ebaml.llm.descriptors

Describe EBM graphs and models using LLMs.

The prompts and the chat conversations live in baml_src/talk_to_ebm.baml; this module extracts the graphs from the EBM, converts them to text, and calls the generated baml_sdk functions. Each function takes one request from ebaml.llm.requests, and the model field of the request chooses the LLM of the call.

EBMDescription

The description of an EBM, and the parts of the model that it leaves out.

A reader of text learns from the text itself what the description leaves out. A program reads the same information from the two lists, with no need to parse a log or the text.

Attributes:

Name Type Description
text str

The description. Pass it as global_explanation to explain_instance.

skipped_terms list[str]

The names of the interaction terms that the description leaves out. The list is empty for a model without interactions.

skipped_features list[str]

The names of the features that have no main-effect term, because exclude or remove_terms left them out. The description has no graph for them.

LocalExplanation

The explanation of one prediction: three kinds of explanation in words, and the facts they come from.

describe_graph(request)

Ask the LLM to describe a graph. Uses chain-of-thought reasoning.

A reply cut off by max_tokens is retried with a simplified graph, up to 4 times. The logged usage leaves out the cut-off attempts, because the provider error carries no token count. It is a lower bound, and the usage log event has the number of those attempts in truncated_attempts. The description covers the main effect of the feature. The prompt names the interaction terms of the feature, so that the description says that it leaves them out, and an interactions_skipped warning names them too.

Parameters:

Name Type Description Default
request DescribeGraphRequest

The EBM, the index of the feature, the number of sentences, the options of the graph, the prompts and the model.

required

Returns:

Type Description
str

The description of the graph.

sample_graph_tokens(graph, model=None, *, prompt=_DEFAULT_PROMPT_OPTIONS)

Send one request with the text of a graph and return its token samples.

The request is the first turn of describe_graph. Each sample has the body of the request and the input tokens that the provider counted. Set max_output_tokens in the model options to a small number, such as 16, to keep the cost low: a reply that the limit cuts off is not an error here. scripts/llm/token_weights.py uses the function to fit the token weights of a model.

Parameters:

Name Type Description Default
graph str

The text of the graph, as EBMGraph.to_text makes it.

required
model ModelOptions | str | None

The LLM of the call, as ModelOptions or the "provider/model" shorthand. None selects OpenRouter's openrouter/free.

None
prompt PromptOptions

The descriptions that the prompt uses.

_DEFAULT_PROMPT_OPTIONS

Returns:

Type Description
list[TokenSample]

One sample for each request that the provider counted. The list is empty when the provider reports no count.

chat_with_mcp(command, arguments, model=None, *, max_steps=12)

Chat in the terminal with an agent that reads an EBM through an MCP server.

BAML starts the server once and keeps it for the whole chat. Each message that you type gets an answer from the LLM, which calls the tools of the server as it needs them. The chat reads standard input and ends on an empty line.

Parameters:

Name Type Description Default
command str

The program that starts the MCP server, for example the path of ebaml-mcp.

required
arguments Sequence[str]

The arguments of the program, for example ["--model", "model.skops", "--data", "train.parquet"].

required
model ModelOptions | str | None

The LLM of the chat, as ModelOptions or the "provider/model" shorthand. None selects OpenRouter's openrouter/free.

None
max_steps int

The most rounds of tool calls for one answer.

12

Returns:

Type Description
int

The number of questions that the chat answered.

describe_ebm(request)

Ask the LLM to describe an EBM.

Each feature graph is described like in describe_graph (concurrently, at most max_concurrency at a time, each retried with a simplified graph if the reply is cut off by max_tokens), and then the LLM summarizes the descriptions. The descriptions skip the interaction terms, and a feature that exclude or remove_terms left out, because it has no graph. The summary prompt names both, so that the text says what it leaves out. An interactions_skipped and a features_skipped warning name them too. The logged usage leaves out the cut-off attempts, because the provider error carries no token count. It is a lower bound, and the usage log event has the number of those attempts in truncated_attempts.

Parameters:

Name Type Description Default
request DescribeEBMRequest

The EBM, the number of sentences, the number of features described at the same time, whether to show a progress bar on the error stream (one step for each feature and one for the summary; without a terminal, the bar writes one line for each step), the options of the graph, the prompts and the model.

required

Returns:

Type Description
EBMDescription

The description of the EBM, and the names of the interaction terms and the features that it leaves out.

explain_instance(request)

Explain one prediction of an EBM with three explanations, computed exactly and put in words by the LLM.

The facts come from ebaml.local.compute_local_facts: the minimum change that flips the prediction (counterfactual), a minimum set of inputs that fixes it (sufficient reason), and the share of completions of a partially known input that get the same prediction (count completions). The global explanation of the model, e.g. the output of describe_ebm, is prepended to each prompt.

Parameters:

Name Type Description Default
request ExplainInstanceRequest

The EBM, the instance, the global explanation, the audience, the number of sentences, the language, the local options, the prompts and the model.

required

Returns:

Type Description
LocalExplanation

The three explanations and the facts they come from.

Options

ebaml.llm.options

Options of the functions in ebaml.llm.descriptors.

Each class holds the settings of one part of the work: GraphOptions for the graph and its text, PromptOptions for the prompts, ModelOptions for the LLM. ebaml.local.LocalOptions holds the settings of the local facts. A default of an option is the default of the function that reads it, so a call with no options does what the function does alone.

GraphOptions

Options of EBMGraph.from_ebm and EBMGraph.to_text, and the token learner of the budget check.

Attributes:

Name Type Description
normalization Normalization | str

See EBMGraph.from_ebm.

use_feature_bounds bool

See EBMGraph.from_ebm.

classes Sequence[str] | None

See EBMGraph.from_ebm. Only a multiclass classifier accepts it.

include_description bool

See EBMGraph.to_text.

feature_format FeatureType | str | None

See EBMGraph.to_text.

x_axis_precision int | None

See EBMGraph.to_text.

y_axis_precision AxisPrecision

See EBMGraph.to_text.

confidence_bounds bool

See EBMGraph.to_text.

confidence_level float

See EBMGraph.to_text.

max_tokens int

See EBMGraph.to_text.

overbudget_percent int

See EBMGraph.to_text.

token_learner TokenLearner | None

The token estimators of the budget check, and the weights that they learn from the calls. None checks the budget with the default weights and learns nothing.

PromptOptions

The descriptions that the prompts use. A value of None keeps the BAML default.

Attributes:

Name Type Description
expert_description str | None

Who the LLM plays.

graph_description str | None

What a graph is and how to read it.

dataset_description str | None

What the data is about. A good description helps the LLM.

task_description str | None

What the model predicts.

ModelOptions

The LLM that a call uses. A field of None keeps the default of the call.

The key of the model never sits in this class. api_key_env names the environment variable that holds it, and EBaMl reads the variable only when it sends a request. BAML lists the valid providers. describe_ebm and the other functions raise InvalidOptionError before the first paid call if BAML does not know the provider.

Attributes:

Name Type Description
provider str

The BAML provider prefix, such as "openai", "anthropic", "openrouter" or "ollama".

model str

The name of the model at the provider.

api_key_env str | None

The name of the environment variable that holds the key. None selects the default variable of the provider.

base_url str | None

The address of the server, for a gateway, a local server or Azure. None selects the address of the provider.

temperature float | None

The sampling temperature. None selects 0.7, or 0.0 for the benchmark.

max_output_tokens int | None

The most tokens of one reply. A reasoning model counts its hidden tokens in this limit. None selects 12000, or 16000 for the first turn of a conversation and for the benchmark.

headers Mapping[str, str]

Headers to add to each request.

parse(value) classmethod

Read the "provider/model" shorthand, or pass a ModelOptions on.

The string splits at the first / only, so "openrouter/openrouter/free" is the provider openrouter with the model openrouter/free.

Parameters:

Name Type Description Default
value ModelOptions | str

The options, or the shorthand.

required

Returns:

Type Description
ModelOptions

The options.

Raises:

Type Description
InvalidOptionError

If the string has no /, or its provider or model is empty.

Examples:

>>> options = ModelOptions.parse("openrouter/openai/gpt-4o-mini")
>>> options.provider, options.model
('openrouter', 'openai/gpt-4o-mini')

Retry after a cut-off reply

ebaml.llm.retry

Retry an LLM call with a smaller graph when the reply is cut off.

This module holds the policy only. It imports no LLM client, so the caller passes the call, the truncation test, the token estimate and the graph rebuild. ebaml.llm.descriptors passes the real baml_sdk functions.

ShrinkingGraphCall

The call and the services that call_with_shrinking_graph needs.

Attributes:

Name Type Description
call Callable[[str], R]

Sends the graph text to the LLM and returns the reply.

graph str

The text of the graph for the first call.

error_type type[E]

The class of the error that call raises when the LLM call fails.

is_truncated Callable[[E], bool]

Tells whether an error means that the reply hit the max_tokens limit.

estimate_tokens Callable[[str], int]

Estimates the number of tokens of a text.

rebuild Callable[[int], str]

Returns the graph text again for a token budget. It raises EBMGraphSimplificationError or EBMGraphTokenLimitError when the graph cannot fit.

on_truncated Callable[[E], None]

Called with the error of each attempt that was cut off, the last one included, before the retry or the raise. The caller counts the attempts with it, because the error carries no token usage.

label str

The name of the feature, for the log.

call_with_shrinking_graph(request)

Call the LLM with a graph text. If the reply is cut off, ask again with a smaller graph, up to 4 times.

Each retry asks request.rebuild for a graph of 75% of the previous estimated size. The function raises the error of the LLM call if the error is not a truncation, if 4 retries failed, or if the graph cannot shrink.

Parameters:

Name Type Description Default
request ShrinkingGraphCall[R, E]

The call, the first graph text, and the services that the retry uses.

required

Returns:

Type Description
R

The reply of the first call that succeeds.

Local explanations

ebaml.local.facts

Deterministic local explanations of one EBM prediction.

The three explanations are the explainability queries of Barceló, Monet, Pérez and Subercaseaux, "Model Interpretability through the Lens of Computational Complexity" (arXiv:2010.12265):

  • Minimum change required (counterfactual): the fewest inputs to change so that the prediction flips.
  • Minimum sufficient reason: the fewest inputs that fix the prediction whatever the other inputs are.
  • Count completions: the share of completions of a partially known input that get the same prediction.

An EBM without interactions is a sum of one score per feature. The decision of a prediction is a margin: a sum of one term per feature, positive while the decision holds (ebaml.local.local_kinds). The first two explanations are solved exactly by sorting the features by how much they move the margin, and the third by sampling the unknown features from their training distribution with a fixed seed. A decision with several margins, and the counterfactual of a chosen class, are covering problems that sorting does not solve. A CoveringSolver of ebaml.local.covering_solvers solves them. Nothing here calls an LLM, and nothing here renders text: ebaml.local.fact_text puts the facts in words for the prompts in baml_src/local_explanations.baml.

The proof that the sorting algorithms are exact is in proofs/papers/local-explanations/paper.typ. It covers a binary classifier, a multiclass classifier and a regressor, which use the same algorithms on the margin of their decision. For a covering problem, the paper states what a result of a solver proves and what it does not.

The confidence of the model enters through confidence_level, as a number z of standard deviations. Each claim names the scores that move by z deviations, always in the direction that makes the claim harder to prove:

  • A counterfactual moves the score of the new bin of each changed feature against the flip. The features that do not change keep their fitted scores.
  • A sufficient reason moves every score against the decision: a fixed feature at its bin, and a free feature at its worst plausible bin.
  • The interval of the output moves the total score. Its deviation is that of a sum of independent errors.
  • A margin of a multiclass classifier is the difference of two class scores. Its deviation is the sum of the two deviations, because an EBM stores no covariance between two classes.

A claim holds under its own model only. confidence_level=0 gives z = 0, so no score moves.

compute_local_facts(ebm, instance, options=None)

Compute the three local explanations of one prediction of a classifier or a regressor.

The classifier is binary or multiclass. The function raises an UnsupportedModelError for an EBM that is not a classifier or a regressor, and for an EBM with interactions. It raises an InvalidInstanceError for an instance that lacks a value or has an unknown category. A regressor explains the answer to LocalOptions.decision.

A counterfactual and a sufficient reason are the smallest sets only if their minimality says so. Sorting proves it for a decision with one margin. A decision with several margins and the counterfactual of a target class depend on LocalOptions.search.

Parameters:

Name Type Description Default
ebm ExplainableBoostingModel

The fitted EBM.

required
instance Mapping[str, FeatureValue]

The value of every feature, by feature name.

required
options LocalOptions | None

Confidence level, sampling and support options.

None

Returns:

Type Description
LocalFacts

The prediction, the contributions and the three explanations.

Raises:

Type Description
OptionRangeError

If the confidence level is not in [0, 1).

Audiences of the local explanations

ebaml.local.audience

The audience of a local explanation: who reads it decides which facts the LLM gets and how it is told to write.

Audience is the base class. A subclass fills in the rules of one kind of reader:

  • guidance is the instruction that tells the LLM how to write for this reader.
  • contribution, prediction, change, flip, class_output, margin, guarantee, support, lower_bound, share and sample return the keys that the audience adds to the facts of the prompt. Each is abstract, so the author of an audience decides on every model number.
  • flip, guarantee, support and sample state the conditions of the facts: whether the scores moved by their uncertainty, whether rare values were left out, and how large the sample is. The prompts read these keys, so the text claims a condition only when the options of the facts set it.

Layperson and DataScientist are the two audiences that ship with the package. To change one rule, subclass one of them and override one method. Pass an instance of any subclass as audience= to explain_instance or to facts_to_texts.

Audience

Bases: ABC

The base class of the audiences: one subclass for each kind of reader.

facts_to_texts builds the keys that all audiences share. The methods here return the keys that one audience adds. All methods are abstract, so a new audience cannot leak a model number that its author forgot. To change one rule of a built-in audience, subclass Layperson or DataScientist.

Examples:

To change only the language of the explanation, pass language= to explain_instance. An audience is for a reader with its own rules. This one is a chief executive: no model numbers, and the bottom line first:

from ebaml.local.audience import Layperson


class Executive(Layperson):
    def guidance(self) -> str:
        return (
            "The reader is a chief executive who has one minute, so open with the bottom line in one sentence. "
            "Then name the one or two inputs that matter most, because the reader decides on the result."
        )
guidance() abstractmethod

Return the instruction that tells the LLM how to write for this reader, and why.

contribution(contribution, prediction) abstractmethod

Return the keys that describe the size of one contribution.

prediction(prediction, confidence_level) abstractmethod

Return the keys that describe how sure the model is of the prediction.

change(change) abstractmethod

Return the keys that describe the size of one change of the counterfactual.

flip(confidence_level) abstractmethod

Return the keys that tell how sure the flip of the counterfactual is.

class_output(output, output_kind) abstractmethod

Return the key that tells the output after the changes: the probability of a class, or a predicted value.

margin(reason, output_kind) abstractmethod

Return the keys that describe the margin of the sufficient reason.

guarantee(confidence_level) abstractmethod

Return the keys that tell whether the sufficient reason holds under the uncertainty of the scores.

support(min_support) abstractmethod

Return the keys that tell whether the counterfactual and the sufficient reason left out rare values.

lower_bound(lower_bound, items) abstractmethod

Return the keys that give the fewest items ("changes" or "features") that any set needs.

share(share) abstractmethod

Return the share of completions with the same prediction.

sample(n_samples) abstractmethod

Return the keys that tell how large the sample behind the shares of the completions is.

Layperson

Bases: Audience

Replace every model number by a word, so the LLM has no number to repeat.

DataScientist

Bases: Audience

Give the scores, the probabilities and the confidence interval, rounded.

likelihood_word(probability)

Return the words for a probability, from almost certain to almost impossible.

Parameters:

Name Type Description Default
probability float

The probability, in [0, 1].

required

Returns:

Type Description
str

The words. Each word starts at its threshold: 0.8 is very likely.

strength_word(score)

Return the word for the size of a score, from strongly to slightly.

Parameters:

Name Type Description Default
score float

The score. Only its size counts, so -0.6 and 0.6 give the same word.

required

Returns:

Type Description
str

strongly from 0.5, moderately from 0.2, else slightly.

sample_size_word(n_samples)

Return the word for the size of a sample, from large to small.

Parameters:

Name Type Description Default
n_samples int

The number of samples.

required

Returns:

Type Description
str

large from 10000 samples, moderate from 1000, else small.

round_value(value)

Round a number to three decimals, the precision that the prompts show.

Parameters:

Name Type Description Default
value float

The number.

required

Returns:

Type Description
float

The rounded number.

points_towards(contribution, prediction)

Tell whether a contribution pushes towards the prediction.

A score points towards the prediction if it raises the score while the decision says that the score is high, and if it lowers the score otherwise. The decision says that the score is high when the score is above the threshold of the decision: the log-odds zero for a classifier. A score on the threshold belongs to the low side.

Parameters:

Name Type Description Default
contribution Contribution

The contribution of one feature.

required
prediction Prediction

The prediction.

required

Returns:

Type Description
bool

True if the contribution supports the prediction.

resolve_audience(audience)

Return the audience for an Audience or for the name of a built-in one.

Parameters:

Name Type Description Default
audience Audience | str

An audience, or "layperson" or "data_scientist".

required

Returns:

Type Description
Audience

The audience.

Raises:

Type Description
InvalidOptionError

If the name is not the name of a built-in audience.

Text of the local explanations

ebaml.local.fact_text

Put the local facts of one EBM prediction in words, as JSON texts for the LLM prompts.

ebaml.local computes the facts. This module only renders them, for one Audience (ebaml.local.audience): a layperson gets words in place of the model numbers, so the LLM has no number to repeat, and a data scientist gets the numbers. The prompts that read the texts are in baml_src/local_explanations.baml.

FactTexts

The JSON text of each explanation, as the prompts read them.

facts_to_texts(facts, audience=_DEFAULT_AUDIENCE)

Render the facts as one JSON text per explanation, for the prompts.

The audience decides which keys each fact gets. For a layperson the model numbers (scores, probabilities, intervals) are replaced by words, so the LLM has none to repeat. A data scientist gets the numbers.

Parameters:

Name Type Description Default
facts LocalFacts

The facts of the prediction.

required
audience Audience

Who will read the explanation. Pass an instance of any subclass of Audience.

_DEFAULT_AUDIENCE

Returns:

Type Description
FactTexts

The texts of the prediction, the counterfactual, the sufficient reason and the completions.

MCP server

ebaml.mcp.server

Create the MCP server, register the tools and start it from the command line.

Each tool is a thin adapter. It calls a function of a lower layer and turns the result into JSON text. A package error (EBaMlError) becomes a tool error with its message.

create_server(store, llm=None)

Create the server and register the tools.

Parameters:

Name Type Description Default
store ModelStore

The session.

required
llm ModelOptions | None

The LLM. Without it, the server has no LLM tools.

None

Returns:

Type Description
MCPServer

The server.

serve(*, model, data=None, llm=None, transport=Transport.STDIO, host='127.0.0.1', port=8000, output_dir=None, edits=None, labels=None)

Load the EBM and run the MCP server.

Parameters:

Name Type Description Default
model Path

EBM in a skops file, which runs no code, or in a pickled or joblib file, which does.

required
data Path | None

Parquet or CSV file with the rows for the row argument.

None
llm str | None

PROVIDER/MODEL of the LLM tools. Without it the server has none.

None
transport Transport

How the server talks to the client.

STDIO
host str

The address that the streamable-http transport listens on. Use 0.0.0.0 to accept other computers.

'127.0.0.1'
port int

The port that the streamable-http transport listens on.

8000
output_dir Path | None

The directory where the tool save_model writes the edited model and its history. The server creates it if it does not exist. Without it, the server has no save_model tool.

None
edits Path | None

The history file of an earlier session (*.history.json). The server replays it, so the work goes on. The server refuses a history that started from another model.

None
labels str | None

The column of the data that holds the true outcome of each row. The effect tools compute the metrics from it.

None

main(argv=None)

Run the ebaml-mcp command.

Parameters:

Name Type Description Default
argv Sequence[str] | None

The arguments. The default is the command line.

None

ebaml.mcp.model_store

The state of one server session: the EBM, the optional data and the optional LLM options.

The user gives the paths at start. No tool argument names a path, so an agent cannot make the server load a file.

RevisionRead

A copy of the model at one revision, and the state of the session when the tool read it.

Attributes:

Name Type Description
model ExplainableBoostingModel

A copy of the model at info.revision. A reader can use it with no lock, because an edit never changes it.

info RevisionInfo

The fixed name of the revision, the head, and the number of edits that the head holds.

ModelStore

The EBM of the session and what the tools need besides it.

The store never changes the loaded model. The edit session works on a copy, so each read of a revision gives the model with the edits of that revision.

Attributes:

Name Type Description
original ExplainableBoostingModel

The fitted EBM that the user loaded. The constructor takes it as ebm.

data Frame | None

The rows that the row argument of the explain tool indexes. Wrap a pandas or polars frame with narwhals.from_native.

session EditSession | None

The edits of the session. The default is a new session on original, or None for a model that the edits do not support (a model with a link that EBaMl cannot read). Pass a replayed session to continue the work of an earlier session.

output_directory Path | None

The directory where save_model writes. The user gives it at start. Without it, the server saves nothing.

label_column str | None

The column of data that holds the true outcome of each row. The effect tools compute the metrics from it. Without it, they count the rows whose prediction changes.

read(revision=CURRENT)

Read the model at a revision. The read never changes the model of the session.

Parameters:

Name Type Description Default
revision str

original, current or the id of a revision of the history.

CURRENT

Returns:

Type Description
RevisionRead

A copy of the model at the revision, and the state of the session. current becomes the id of the head.

Raises:

Type Description
EditError

If the model has no edit session and the revision is not original or current. The session raises the same error for a revision that it does not hold.

labels()

Read the true outcome of each row of the data.

Returns:

Type Description
NDArray[generic] | None

The values of the label column, or None if the server has no label column.

Raises:

Type Description
DataFileError

If the server has a label column but no data, or the data has no such column.

editing()

Give the edit session.

Returns:

Type Description
EditSession

The session that the edit tools use.

Raises:

Type Description
UnsupportedModelError

If the model is one that the edits do not support. The message of EditSession says which.

described(revision)

Give the kept description of the model if it describes this revision.

Parameters:

Name Type Description Default
revision str

The fixed name of a revision, as RevisionRead.info.revision gives it.

required

Returns:

Type Description
str | None

The output of describe_ebm, or None if the store keeps none for this revision.

keep_description(revision, text)

Keep the description of a revision, but only if the head is still this revision.

An edit during the slow LLM call moves the head. The description then describes a model that is no longer the current one, so the store drops it.

Parameters:

Name Type Description Default
revision str

The fixed name of the revision that the text describes.

required
text str

The output of describe_ebm.

required
feature_index(feature)

Find the index of a feature by its name.

Parameters:

Name Type Description Default
feature str

The name of the feature.

required

Returns:

Type Description
int

The index of the feature in feature_names_in_. Edits never change the features of the model.

Raises:

Type Description
InvalidInstanceError

If the model has no feature with this name.

row_instance(row)

Read one row of the data as an instance of the model.

Parameters:

Name Type Description Default
row int

The position of the row in the data of --data.

required

Returns:

Type Description
dict[str, FeatureValue]

The value of each feature of the model.

Raises:

Type Description
InvalidInstanceError

If the server has no data, the row does not exist, or the data lacks a feature.

load_model(path)

Load an EBM from a skops file, or from a pickled or joblib file, of the path that the user gave.

A skops file is a zip archive. The loader trusts only the EBM classes, so a skops file runs no code. A pickled or joblib file runs code from the file. Call this function only with a path from the command line of the user, never with a path from a tool argument.

Parameters:

Name Type Description Default
path PathLike

The file of the fitted EBM.

required

Returns:

Type Description
ExplainableBoostingModel

The EBM.

Raises:

Type Description
UnsupportedModelError

If the file does not hold an EBM, or a skops file holds an untrusted type.

load_data(path)

Read the rows from a Parquet file, or from a CSV file.

The function reads the first and the last bytes of the file, not its extension. A file that starts and ends with the Parquet mark PAR1 is a Parquet file, and Parquet keeps the type of each column. A file of UTF-8 text is a CSV file, and polars infers the type of each column from the text.

Parameters:

Name Type Description Default
path PathLike

The Parquet file, or the CSV file.

required

Returns:

Type Description
Frame

The rows.

Raises:

Type Description
DataFileError

If the file is empty, is cut off, is neither Parquet nor UTF-8 CSV, or the reader fails. The message names the file and says how to save it. Polars reads a binary file as a CSV file of no rows, so the check of the bytes comes first.

ebaml.mcp.read_tools

The tools that read an EBM: its summary, its features and the text of a graph.

Each function calls code of a lower layer and returns an attrs class. The server turns the class into JSON.

ModelSummary

The facts about the model that an agent needs first.

Attributes:

Name Type Description
kind str

BinaryClassifier, MulticlassClassifier or Regressor.

outputs list[str]

The names of the outputs of the model.

classes list[ClassLabel]

The class labels, in the order of the scores. The list is empty for a regressor.

score_meaning str

What a score of a graph means.

intercept list[float]

The intercept: one value, or one value per class for a multiclass classifier.

feature_count int

The number of features that have a graph.

skipped_terms list[str]

The interaction terms. The tools skip them.

session RevisionInfo

The revision that the summary reads, the head, and the number of edits that the head holds.

FeatureInfo

One feature of the model.

FeatureList

The features with a graph, the most important first, and the revision that the list reads.

model_summary(store, revision=CURRENT)

Summarize the model.

Parameters:

Name Type Description Default
store ModelStore

The session.

required
revision str

original, current or the id of a revision of the history.

CURRENT

Returns:

Type Description
ModelSummary

The kind, the outputs, the intercept, the number of features and the interaction terms that the tools skip.

list_features(store, revision=CURRENT)

List the features with a graph, the most important first.

Parameters:

Name Type Description Default
store ModelStore

The session.

required
revision str

The revision to list. See model_summary.

CURRENT

Returns:

Type Description
FeatureList

The name, the type and the importance of each feature, sorted by importance, and the revision that the list reads.

graph_text(store, feature, *, revision=CURRENT, normalization=Normalization.NONE, x_axis_precision=None, y_axis_precision=None, max_tokens=3000)

Write the graph of a feature as text.

Parameters:

Name Type Description Default
store ModelStore

The session.

required
feature str

The name of the feature.

required
revision str

original, current or the id of a revision of the history.

CURRENT
normalization Normalization | str

See EBMGraph.from_ebm.

NONE
x_axis_precision int | None

See EBMGraph.to_text.

None
y_axis_precision int | None

See EBMGraph.to_text.

None
max_tokens int

See EBMGraph.to_text.

3000

Returns:

Type Description
str

A line that names the revision and the edits that the head holds, then the text of EBMGraph.to_text.

ebaml.mcp.explain_tools

The tool that explains one prediction with the exact local facts. It calls no LLM.

PredictionExplanation

The facts of one prediction and their texts.

Attributes:

Name Type Description
facts LocalFacts

The prediction, the contributions and the three local explanations.

texts FactTexts

The facts as the JSON texts that the prompts read, for the audience of the call.

session RevisionInfo

The revision that the facts explain, the head, and the number of edits that the head holds.

PredictionRequest

What to explain, and how.

Attributes:

Name Type Description
instance (Mapping[str, FeatureValue], optional)

The value of every feature, by feature name.

row (int, optional)

The position of a row of the data of --data. Give exactly one of instance and row.

confidence_level float, default=0.95

See LocalOptions.confidence_level.

audience str, default="layperson"

The name of a built-in audience. It decides the keys of the texts.

revision str, default="current"

original, current or the id of a revision of the history. The facts use the model at this revision.

explain_prediction(store, request)

Compute the local facts of one prediction.

Parameters:

Name Type Description Default
store ModelStore

The session.

required
request PredictionRequest

The instance or the row, the revision, the confidence level and the audience.

required

Returns:

Type Description
PredictionExplanation

The facts and their texts.

Raises:

Type Description
InvalidInstanceError

If the request gives both or none of instance and row.

ebaml.mcp.llm_tools

The tools that ask the LLM to describe the model. The server registers them only when the user gives --llm.

describe_feature(store, llm, feature)

Ask the LLM to describe the graph of one feature.

Parameters:

Name Type Description Default
store ModelStore

The session.

required
llm ModelOptions

The LLM.

required
feature str

The name of the feature.

required

Returns:

Type Description
str

The output of describe_graph.

describe_model(store, llm)

Ask the LLM to describe the whole model at the head. The store keeps the text for the revision that it describes.

The LLM call runs on a revision with no lock. If an edit moves the head during the call, the store drops the text, so the next call describes the new head.

Parameters:

Name Type Description Default
store ModelStore

The session.

required
llm ModelOptions

The LLM.

required

Returns:

Type Description
str

The output of describe_ebm.

Graphs

ebaml.graphs.ebm_graph

Graphs for EBM models.

TokenBudget

Bases: Protocol

The result of a token budget check.

estimated_tokens property

The estimated number of tokens of the text.

status property

"ok" within max_tokens, "warn" above it, and "over" beyond overbudget_percent.

BudgetCheck

Bases: Protocol

A function that estimates the tokens of a text and compares them with a budget.

ebaml.graphs.token_budget.check_token_budget is such a function.

__call__(*, text, max_tokens, overbudget_percent)

Compare the estimated tokens of text with max_tokens, and with the tolerance above it.

EBMGraph

A data structure for the graphs of Explainable Boosting Machines.

A graph of a binary classifier or a regressor has one score per bin, and classes is empty. A graph of a multiclass classifier has one row per bin with one score per class, and classes names the columns.

from_ebm(ebm, feature_index, normalization=Normalization.NONE, *, use_feature_bounds=True, classes=None) classmethod

Extract the graph of a feature from an Explainable Boosting Machine.

This is a low-level function. It does not return the final format in which the graph is presented to the LLM.

The purpose of this function is to extract the graph from the internals of the EBM and return it in a format that is easy to work with.

Parameters:

Name Type Description Default
ebm ExplainableBoostingModel

The EBM model.

required
feature_index int

The index of the feature in the EBM. A feature that exclude or remove_terms left out has no main-effect term, and raises UnsupportedModelError.

required
normalization Normalization | str

How to normalize the graph (shift on the y-axis).

NONE
use_feature_bounds bool

If True, the first and last x-axis bins are the min and max values of the feature stored in the EBM. If false, the first and last value are -inf and inf.

True
classes Sequence[str] | None

The classes to keep in the graph of a multiclass classifier, in the order to show them. If not provided, the graph has every class. The text of a model with many classes is long, so a subset keeps it short. A model with one output raises UnsupportedModelError for this option, and so does a model of an unknown kind for any call. Scores that do not match the outputs of the model raise EBMGraphShapeError.

None

Returns:

Type Description
EBMGraph

The graph.

Raises:

Type Description
InvalidOptionError

If the feature type is not continuous or nominal.

from_enum

If the normalization option is not valid.

EBMGraphShapeError

If the number of bins does not match the number of scores.

Notes

A multiclass classifier has one score per bin and per class. Softmax does not change when the same constant is added to the scores of all the classes of a bin, so the graph centres the scores of each bin over all the classes of the model, before it selects classes. The standard deviations stay those of the EBM. The normalization then shifts all the scores by one constant, so the differences between the classes of a bin do not change. proofs/papers/multiclass-scores/paper.typ proves that these graphs keep the predictions of the EBM.

simplify(min_variation_per_cent=0.0)

Simplify the graph: remove the redundant (flat) bins.

Parameters:

Name Type Description Default
min_variation_per_cent float

The degree of simplification. If min_variation_per_cent>0, the function simplifies the graph by removing bins that correspond to a less than min_variation_per_cent change in the score, considering the overall min/max difference of score for the feature as 100%. A bin of a multiclass graph merges only if the score of every class changes that little.

0.0

Returns:

Name Type Description
EBMGraph Self

The simplified graph.

Raises:

Type Description
InvalidOptionError

If the feature type is not continuous.

OptionRangeError

If min_variation_per_cent is not between 0 and 1.

to_body_text(*, feature_format, x_axis_precision, y_axis_precision, confidence_bounds, confidence_level)

Render the lines of text (name, type, means and optional bounds) that follow the preamble.

Parameters:

Name Type Description Default
feature_format FeatureType

The format of the feature.

required
x_axis_precision int | None

The precision of the values on the x-axis. If None, the precision is not adjusted.

required
y_axis_precision int

The precision of the values on the y-axis.

required
confidence_bounds bool

Whether to include confidence bounds.

required
confidence_level float

The desired confidence level of the bounds.

required

Returns:

Type Description
str

The textual representation of the graph, without the preamble. A multiclass graph has a Class: line before the means and the bounds of each class.

to_text(*, include_description=True, feature_format=None, x_axis_precision=None, y_axis_precision=None, confidence_bounds=True, confidence_level=0.95, max_tokens=3000, overbudget_percent=50, check_budget=None)

Convert the graph to text. This is the text that we then pass to the LLM.

The function takes care of a variety of different formatting issues that can arise in the process of converting a graph to text.

Parameters:

Name Type Description Default
include_description bool

Whether to include a short descriptive preamble that describes the graph to the LLM. The preamble of a multiclass graph also says how to read the series of the classes.

True
feature_format FeatureType | str | None

The format of the feature (continuous, categorical, boolean). If not provided, auto-detection is attempted based on the feature type and the x-axis values. A boolean format needs exactly two x-axis values, and raises an EBMGraphShapeError otherwise.

None
x_axis_precision int | None

The precision of the values on the x-axis. If not provided, the precision is not adjusted.

None
y_axis_precision AxisPrecision

The precision of the values on the y-axis. If not provided or "auto", the precision is auto-detected based on the range of the y-axis

None
confidence_bounds bool

Whether to include confidence bounds.

True
confidence_level float

The desired confidence level of the bounds.

0.95
max_tokens int

The token budget of the text of the graph. The token count is a rough estimate (there is no tokenizer before the LLM call), so exceeding the budget only emits a warning.

3000
overbudget_percent int

How far above max_tokens (in percent) the estimate may go before the graph is considered obviously too large. Beyond that bound the function simplifies the graph, or raises if it cannot. The LLM provider's own error remains the real limit.

50
check_budget BudgetCheck | None

Estimates the tokens of a text and compares them with max_tokens and overbudget_percent, for example ebaml.graphs.token_budget.check_token_budget. If not provided, the text is not checked: the function writes it once, at the first simplification level, and max_tokens and overbudget_percent have no effect.

None

Returns:

Type Description
str

The textual representation of the graph.

Raises:

Type Description
EBMGraphSimplificationError

If the graph cannot be simplified enough to fit into the max_tokens limit.

EBMGraphTokenLimitError

If the graph exceeds the max_tokens limit even after simplification.

from_text(text) classmethod

Parse the textual representation of a graph back into a graph. The inverse of to_text.

See ebaml.graphs.graph_text.parse_graph_text for the details of the format.

Parameters:

Name Type Description Default
text str

The text of the graph, with the confidence bounds included.

required

Returns:

Type Description
EBMGraph

The graph.

Terms of the features

ebaml.graphs.terms

The term of an EBM that holds the main effect of each feature.

interpret keeps the scores, the standard deviations, the bin weights and the importances by term, and the names, the types and the bins by feature. Term i is not always feature i: a feature that exclude leaves out at fit time, or that remove_terms removes later, has no term, and the terms after it move. So other code finds the term of a feature here, and does not index the term attributes with the index of a feature.

main_term(ebm, feature_index)

Find the term that holds the main effect of a feature.

Parameters:

Name Type Description Default
ebm ExplainableBoostingModel

The fitted EBM.

required
feature_index int

The index of the feature in feature_names_in_.

required

Returns:

Type Description
int

The index of the term in term_features_, term_scores_ and the other term attributes.

Raises:

Type Description
UnsupportedModelError

If the feature has no main-effect term, because exclude or remove_terms left it out.

main_effect_features(ebm)

List the features that have a main-effect term.

Parameters:

Name Type Description Default
ebm ExplainableBoostingModel

The fitted EBM.

required

Returns:

Type Description
list[int]

The index of each such feature in feature_names_in_, in increasing order.

Token budget

ebaml.graphs.token_budget

The token budget of a text: an estimate of its tokens, compared with a limit.

No tokenizer runs before the LLM call, so the estimate comes from the counts of the text. TokenEstimator turns the words, bytes and digits of a text into a number of tokens, and adds a margin, so that an estimate that is too low is rarer than one that is too high. A text that the estimate calls too large only gets a simpler graph, but a text that it calls small enough can make the provider refuse the request. The provider error stays the real limit. The module needs no baml_sdk, so code that calls no LLM can check a budget too.

The estimator can learn. TokenEstimator.observe moves the weights toward the token counts of real requests, by recursive least squares with a prior at the starting weights. Learning is off by default, because learned weights change the estimate, and so the graph text, from one call to the next.

check_token_budget = TokenEstimator() module-attribute

The budget check with the default weights.

The estimator never learns, so it gives the same status for the same text in every call. Call it with text, max_tokens and overbudget_percent. It is a BudgetCheck, so EBMGraph.to_text accepts it as check_budget. Pass a TokenEstimator instead to choose the weights or to learn them.

Examples:

>>> str(check_token_budget(text="1.5 2.5 " * 12, max_tokens=100, overbudget_percent=50).status)
'warn'
>>> str(check_token_budget(text="1.5 2.5 " * 12, max_tokens=1000, overbudget_percent=50).status)
'ok'
>>> str(check_token_budget(text="1.5 2.5 " * 12, max_tokens=50, overbudget_percent=50).status)
'over'

BudgetStatus

The estimated tokens of a text, and how they compare with the budget.

Attributes:

Name Type Description
estimated_tokens int

The estimated number of tokens of the text, with the margin.

status BudgetLevel

"ok" within max_tokens, "warn" above it, and "over" beyond overbudget_percent above it.

EstimatorState

What a learning estimator knows after its samples. TokenEstimator.restore takes it back.

Attributes:

Name Type Description
weights TokenWeights

The learned weights.

covariance tuple[float, ...]

The nine entries of the 3 by 3 matrix of the recursive least squares, row by row. It holds how sure the estimator is about each weight.

samples_seen int

The number of samples that the estimator learned from.

TokenEstimator

Estimate the tokens of a text, and compare the estimate with a budget.

An instance is a BudgetCheck, so EBMGraph.to_text(check_budget=estimator) works. The estimate is an upper bound: the fit plus margin times relative_spread of the fit.

Attributes:

Name Type Description
weights TokenWeights

The starting weights, and the center of the prior of the learning.

relative_spread float

The root mean square of the relative error of weights. It sets the size of the margin.

margin float

How many spreads the estimate lies above the fit.

learning bool

Whether observe changes the weights. It is off by default.

prior_samples float

How many samples of a typical graph the prior of weights is worth. A larger value makes a sample move the weights less.

Examples:

>>> estimator = TokenEstimator()
>>> str(estimator(text="1.5 2.5 " * 12, max_tokens=100, overbudget_percent=50).status)
'warn'
current_weights property

The weights that the estimate uses now. They equal weights until a sample changes them.

samples_seen property

The number of samples that changed the weights.

__attrs_post_init__()

Start the learning at the prior.

estimate(text)

Estimate the tokens of a text, with the margin, rounded up.

Parameters:

Name Type Description Default
text str

The text.

required

Returns:

Type Description
int

The estimated number of tokens. It is 0 for a text of no words.

Examples:

>>> TokenEstimator(weights=TokenWeights(words=2.0, bytes=0.0, digits=0.0), relative_spread=0.0).estimate(
...     "a b c"
... )
6
__call__(*, text, max_tokens, overbudget_percent)

Compare the estimated tokens of a text with a budget and with the tolerance above it.

The method makes an estimator a BudgetCheck, so EBMGraph.to_text(check_budget=estimator) works.

Parameters:

Name Type Description Default
text str

The text.

required
max_tokens int

The budget.

required
overbudget_percent int

How far above max_tokens, in percent, the estimate may go before the status is "over".

required

Returns:

Type Description
BudgetStatus

The estimate and the status.

observe(sample)

Move the weights toward the token count of one real request, if learning is on.

The step is one of recursive least squares with the prior at weights. A projection then keeps every weight at zero or above. The method is safe to call from several threads.

Parameters:

Name Type Description Default
sample TokenSample

The counts of the text of a request and the input tokens that the provider reported for it.

required

Returns:

Type Description
bool

Whether the sample changed the weights. It is False when learning is off.

Examples:

>>> TokenEstimator().observe(TokenSample(TextFeatures(words=10, bytes=50, digits=5), 60))
False
>>> learner = TokenEstimator(learning=True)
>>> learner.observe(TokenSample(TextFeatures(words=10, bytes=50, digits=5), 60))
True
state()

Take a copy of what the estimator learned.

Returns:

Type Description
EstimatorState

The learned weights, the matrix of the recursive least squares and the number of samples.

restore(state)

Take back what state made, for example after a restart.

Parameters:

Name Type Description Default
state EstimatorState

A value that state returned.

required

Returns:

Type Description
TokenEstimator

The estimator itself.

estimate_tokens(text)

Estimate the tokens of a text with the default weights, with the margin, rounded up.

Parameters:

Name Type Description Default
text str

The text.

required

Returns:

Type Description
int

The estimated number of tokens.

Examples:

>>> estimate_tokens("")
0
>>> estimate_tokens("0.1 0.2 0.3 0.4") > estimate_tokens("0.1 0.2")
True

ebaml.graphs.token_weights

The word, byte and digit counts of a text, and the weights that turn them into a token estimate.

The estimate of the tokens of a text is words * w_words + bytes * w_bytes + digits * w_digits. Graph text is mostly digits and punctuation, so a digit count separates it from prose better than a character count. The three counts grow together, so a plain least-squares fit can give a weight below zero. fit_weights keeps every weight at zero or above. The module needs no tokenizer, no file and no baml_sdk.

TextFeatures

The counts of a text that the token estimate reads.

Attributes:

Name Type Description
words int

The number of parts of the text that whitespace separates.

bytes int

The number of bytes of the UTF-8 encoding of the text.

digits int

The number of characters of the text that are digits.

of(text) classmethod

Count the words, bytes and digits of a text.

Parameters:

Name Type Description Default
text str

The text.

required

Returns:

Type Description
TextFeatures

The counts.

Examples:

>>> TextFeatures.of("x: 12 é")
TextFeatures(words=3, bytes=8, digits=2)
as_vector()

Return the counts in the order of TokenWeights.as_vector.

Returns:

Type Description
tuple[float, ...]

The words, the bytes and the digits.

TokenWeights

The tokens that one word, one byte and one digit add to the estimate of a text.

Attributes:

Name Type Description
words float

The tokens for each word.

bytes float

The tokens for each byte.

digits float

The tokens for each digit.

as_vector()

Return the weights in the order of TextFeatures.as_vector.

Returns:

Type Description
tuple[float, ...]

The weights of the words, the bytes and the digits.

estimate(features)

Estimate the tokens of a text from its counts.

Parameters:

Name Type Description Default
features TextFeatures

The counts of the text.

required

Returns:

Type Description
float

The estimated number of tokens, before any margin.

Examples:

>>> TokenWeights(words=2.0, bytes=0.5, digits=1.0).estimate(TextFeatures(words=3, bytes=10, digits=2))
13.0

TokenSample

One real request: the counts of its text and the input tokens that the provider reported for it.

Attributes:

Name Type Description
features TextFeatures

The counts of the text of the messages of the request.

tokens int

The input tokens that the provider counted, with the cached tokens included. It is at least 1.

ModelKey

The provider and the name of the model that answered a request.

Attributes:

Name Type Description
provider str

The provider that the BAML runtime resolved, for example "openrouter".

model str

The model at the provider, for example "openai/gpt-6-luna".

WeightsEntry

The weights of one model, and what they are worth.

Attributes:

Name Type Description
weights TokenWeights

The weights that fit the model.

relative_spread float

The root mean square of the relative error of weights on the samples that fitted them.

samples int

The number of samples that fitted weights.

update_allowed bool

Whether a learning estimator may change weights. A model whose entry is final sets it to False.

prose_bytes_per_token float

The bytes of English prose that the model counts as one token, such as the prompt template. A request holds the graph and also prose, and the learner takes the prose away to learn the weights of the graph alone.

WeightsTable

The default weights and the weights of the models that were fitted.

Attributes:

Name Type Description
default WeightsEntry

The entry for a model that the table does not list.

models Mapping[ModelKey, WeightsEntry]

The entry of each model that was fitted.

entry_for(key)

Look up the entry of a model.

Parameters:

Name Type Description Default
key ModelKey | None

The model, or None when it is not known.

required

Returns:

Type Description
WeightsEntry

The entry of the model, or default when the table does not list it.

with_entry(key, entry)

Add the entry of a model, or replace the one that the table has.

Parameters:

Name Type Description Default
key ModelKey

The model.

required
entry WeightsEntry

Its entry.

required

Returns:

Type Description
WeightsTable

A new table. The table itself does not change.

from_bytes(data) classmethod

Read a table from its JSON text.

Parameters:

Name Type Description Default
data bytes

The JSON text that to_bytes writes.

required

Returns:

Type Description
WeightsTable

The table.

Raises:

Type Description
TokenFitError

If the text is not JSON, or lacks a field of the table.

to_bytes()

Write the table as JSON text, with the models in order, so that the text is the same for the same table.

Returns:

Type Description
bytes

The JSON text.

fit_weights(samples)

Fit the weights by least squares, with every weight at zero or above.

The fit tries each set of features that can have a weight above zero. It solves the least-squares problem for the set, and keeps the set with the smallest squared error among those whose weights are all at zero or above. No proof backs that this is the best fit with weights of zero or above.

Parameters:

Name Type Description Default
samples Sequence[TokenSample]

The samples to fit.

required

Returns:

Type Description
TokenWeights

The weights that fit the samples, with every weight at zero or above.

Raises:

Type Description
TokenFitError

If samples is empty.

Examples:

>>> samples = [
...     TokenSample(TextFeatures(words=words, bytes=0, digits=digits), 2 * words + digits)
...     for words, digits in [(1, 5), (4, 1), (3, 3)]
... ]
>>> [round(weight, 6) for weight in fit_weights(samples).as_vector()]
[2.0, 0.0, 1.0]

relative_spread(weights, samples)

Measure how far the estimates are from the counts, as the root mean square of the relative error.

Parameters:

Name Type Description Default
weights TokenWeights

The weights to judge.

required
samples Sequence[TokenSample]

The samples to judge them on.

required

Returns:

Type Description
float

The root mean square of (estimate - tokens) / tokens. It is 0.0 for no samples.

Examples:

>>> relative_spread(TokenWeights(words=1.0, bytes=0.0, digits=0.0), [TokenSample(TextFeatures(1, 0, 0), 2)])
0.5

ebaml.graphs.request_text

The text of the messages in the body of an LLM request.

The tokenizer of a provider counts the prompt, not the JSON around it. The body of a request holds the model name, the sampling parameters and the structure of the messages besides the text. So the token learner reads the text of the messages only. Providers shape the body differently (OpenAI, Anthropic and Gemini differ), so the function collects the text under the keys that carry it. It needs no baml_sdk.

message_text(body)

Collect the text of the messages from the body of a request, one part on each line.

Parameters:

Name Type Description Default
body str

The JSON text of the request body.

required

Returns:

Type Description
str

The text of the messages. It is empty when the body is not JSON or holds no text.

Examples:

>>> message_text('{"model": "m", "messages": [{"role": "user", "content": "hello there"}]}')
'hello there'
>>> body = '{"system": "be brief", "messages": [{"role": "user", "content": [{"type": "text", "text": "hi"}]}]}'
>>> print(message_text(body))
be brief
hi

ebaml.llm.token_learning

Token estimators that learn from the token counts of real LLM calls, one for each model.

The BAML runtime reports the input tokens of each request and the model that answered. TokenLearner keeps one TokenEstimator for each model. It starts each from the shipped table of weights, and moves it toward the tokens of the graph in the requests when learning is on. The tokens of the prose in a request (the template and earlier replies) are taken away first. Learning is off by default, because learned weights change the estimate, and so the text of a graph, from one call to the next.

When learning is on, the weights persist between sessions in a JSON file. By default the file is $XDG_CACHE_HOME/ebaml/token_weights.json, or ~/.cache/ebaml/token_weights.json. The learner reads and writes the file only when learning is on. A model that the table marks as final (update_allowed false) never learns.

RequestSample

Bases: Protocol

One counted request, as the BAML runtime reports it. baml_sdk.TokenSample is such a sample.

provider property

The provider that answered.

model property

The model that answered.

body property

The JSON body of the request. It never holds a header.

input_tokens property

The input tokens that the provider counted, with the cached tokens included.

TokenLearner

Keep a token estimator for each model, and learn its weights from the counts of real requests.

Pass a learner as GraphOptions(token_learner=...). The budget check of describe_graph and describe_ebm then uses the estimator of the model that you chose, and the calls feed their token counts back to it. The BAML runtime names the model that answered, which can differ from the one that you chose (openrouter/free picks a model for each request). So a learner learns the weights of each model that answered. Its estimate before a call uses the model that you chose, and the default weights when the table does not list it.

Attributes:

Name Type Description
learning bool

Whether the estimators change their weights. It is off by default.

persist bool

Whether learned weights persist in path. It has an effect only when learning is on.

path PathLike | None

The file of the learned weights. None selects default_weights_path().

table WeightsTable

The weights that each estimator starts from. The default is the table that ships with the package.

prior_samples float

See TokenEstimator.prior_samples.

margin float

See TokenEstimator.margin.

estimator_for(key)

Get the estimator of a model, and create it from the table on the first request.

Parameters:

Name Type Description Default
key ModelKey | None

The model, or None when it is not known. None gets the default entry of the table.

required

Returns:

Type Description
TokenEstimator

The estimator of the model. The learner keeps it, so later calls get the same object.

observe(samples, graph)

Teach the estimators the token counts of the graph in requests, and save what they learned.

A request holds the graph and also prose: the prompt template and, in later turns, earlier replies. The provider counts both. The method takes the prose away, at the bytes for each token that the table gives for the model, and teaches the estimator what is left for the graph. A request that does not hold graph as it is, or that has no tokens left, gives no sample. A model whose entry is final learns nothing. The method does nothing when learning is off.

Parameters:

Name Type Description Default
samples Iterable[RequestSample]

The counted requests, for example counted.usage.samples.

required
graph str

The text of the graph that the requests carried.

required

Returns:

Type Description
int

The number of samples that changed an estimator.

save()

Write the learned weights to the file. Nothing happens when learning or persistence is off.

A file that cannot be written logs a warning and does not stop the call.

default_table() cached

Read the table of weights that ships with the package.

Returns:

Type Description
WeightsTable

The default weights and the weights of the models that were fitted.

default_weights_path()

Name the file where learned weights persist when the learner gets no path.

Returns:

Type Description
Path

$XDG_CACHE_HOME/ebaml/token_weights.json. Without the variable, ~/.cache/ebaml/token_weights.json.

Text of the graphs

ebaml.graphs.graph_text

Helpers for the textual representation of EBM graphs: the JSON lines, and how to read a graph text back.

ParsedGraph

The fields of a graph that parse_graph_text reads from a text. They are the fields of EBMGraph.

xy_to_json(x_vals, y_vals)

Convert a sequence of x_vals and y_vals to a json string.

Parameters:

Name Type Description Default
x_vals Sequence[str | tuple[float | str, float | str]]

The x values. The graph text passes interval ends as rounded strings.

required
y_vals Iterable[float | str]

The y values. The graph text passes them as rounded strings.

required

Returns:

Type Description
str

A json string representation of the x and y values. The keys are escaped as JSON strings. The y values are written as they are, to keep their exact digits.

Examples:

>>> xy_to_json([("0.0", "1.5"), "red"], ["0.25", "-1.0"])
'{"(0.0, 1.5)": 0.25, "red": -1.0}'
>>> print(xy_to_json(['Master "B"'], ["0.5"]))
{"Master \"B\"": 0.5}

parse_str_tuple_to_float_tuple(str_tuple)

Parse a string tuple to a float tuple.

Parameters:

Name Type Description Default
str_tuple str

The string tuple to parse.

required

Returns:

Type Description
Interval

The parsed float tuple.

Raises:

Type Description
GraphTextError

If the text is not two numbers in brackets.

Examples:

>>> parse_str_tuple_to_float_tuple("(0.5, 2)")
(0.5, 2.0)
>>> parse_str_tuple_to_float_tuple("(a, b)")
Traceback (most recent call last):
...
ebaml.core.exceptions.GraphTextError: '(a, b)' is not an interval of two numbers.

parse_graph_text(text)

Read the fields of a graph from its textual representation. The inverse of EBMGraph.to_text.

The text may contain other lines (such as the preamble or an LLM prompt around the graph). Only the lines that EBMGraph.to_text writes are read: the name, the type, the means and both confidence bounds. The standard deviations are estimated from the bounds with the normal approximation, at the confidence level that the bounds are labeled with, so they are only as precise as the rounding of the text.

A graph of a multiclass classifier has one series per class: a Class: line, then the means and the bounds of that class. The scores and the standard deviations then have one row per bin and one column per class.

Parameters:

Name Type Description Default
text str

The text of the graph, with the confidence bounds included.

required

Returns:

Type Description
ParsedGraph

The feature name, the feature type, the x-axis intervals, the scores, the standard deviations and the classes.

Raises:

Type Description
GraphTextError

If a line of the graph is missing, is not valid JSON, the means and the bounds do not share the same intervals, or a multiclass text has a series before its first Class: line.

InvalidOptionError

If the feature type is not continuous.

Types

ebaml.core.custom_types

Types for EBM models, and the coercion of options to them.

Normalization

Bases: StrEnum

Normalization options for EBM graphs.

FeatureType

Bases: StrEnum

Feature types for EBM graphs.

MonotoneDirection

Bases: StrEnum

The direction of a monotone graph.

Direction

Bases: StrEnum

The direction that an edit gives a graph. AUTO lets the edit pick the better MonotoneDirection.

BudgetLevel

Bases: StrEnum

How an estimated number of tokens compares with the budget.

Transport

Bases: StrEnum

How an MCP server talks to its client.

normalize_feature_type(feature_type)

Normalize a feature type to the FeatureType enum.

Parameters:

Name Type Description Default
feature_type str

The feature type to normalize.

required

Returns:

Type Description
FeatureType

The normalized feature type.

Raises:

Type Description
from_enum

If an invalid feature type is provided.

coerce_enum(enum_class, value)

Convert a value (a member or its string value) to a member of an enum.

Parameters:

Name Type Description Default
enum_class type[E]

The enum class.

required
value E | str

The member or the value to convert.

required

Returns:

Type Description
E

The enum member.

Raises:

Type Description
from_enum

If the value is not a member of the enum.

normalize_feature_types(feature_types, reject_set=None)

Normalize a list of feature types to the FeatureType enum.

Parameters:

Name Type Description Default
feature_types Iterable[str]

The feature types.

required
reject_set Set[FeatureType] | None

Set of feature types to reject. If a feature type is in this set, the function raises an InvalidOptionError.

None

Returns:

Type Description
list[FeatureType]: List of normalized feature types.

Raises:

Type Description
InvalidOptionError

If an invalid feature type is provided or if a feature type is in the reject set.

Exceptions

ebaml.core.exceptions

Exceptions for EBM models.

Every error that ebaml raises on purpose derives from EBaMlError, so except EBaMlError catches all of them. An error for a bad value also derives from ValueError, so code that catches ValueError keeps working.

EBaMlError

Bases: Exception

The base class of the errors that ebaml raises.

InvalidOptionError(option, valid_options)

Bases: EBaMlError, ValueError

Exception raised for invalid options.

Build the error message from the invalid option and the valid ones.

Parameters:

Name Type Description Default
option str

The invalid option provided.

required
valid_options Collection[str]

The options that are valid.

required
from_enum(option, validation_enum) classmethod

Create an InvalidOptionError from an enum.

Parameters:

Name Type Description Default
option str

The invalid option provided.

required
validation_enum type[Enum]

The enum class that defines the valid options.

required

Returns:

Type Description
InvalidOptionError

An instance of InvalidOptionError with a message indicating the invalid option and the valid options.

OptionRangeError

Bases: EBaMlError, ValueError

Exception raised when a number option is outside its range.

EBMGraphError

Bases: EBaMlError

Exception raised for errors in the EBM graph processing.

EBMGraphSimplificationError

Bases: EBMGraphError

Exception raised when an EBM graph cannot be simplified enough to fit into the max_tokens limit.

EBMGraphTokenLimitError

Bases: EBMGraphError

Exception raised when an EBM graph exceeds the max_tokens limit even after simplification.

EBMGraphShapeError

Bases: EBMGraphError, ValueError

Exception raised when the bins of an EBM graph do not fit the scores or the requested format.

GraphTextError

Bases: EBaMlError, ValueError

Exception raised when the textual representation of a graph cannot be read.

UnsupportedModelError

Bases: EBaMlError, ValueError

Exception raised when a function does not support the kind of EBM, such as one with interactions.

DataFileError

Bases: EBaMlError, ValueError

Exception raised when a data file is empty, is neither Parquet nor UTF-8 CSV, or fails to read.

InvalidInstanceError

Bases: EBaMlError, ValueError

Exception raised when the instance to explain lacks a value or has a value that the EBM did not see.

SDKUnavailableError

Bases: EBaMlError, ImportError

Exception raised when the generated baml_sdk package is missing and ebaml cannot generate it.

The message lists the steps that fix the cause. The error also derives from ImportError, so code that catches the import error of a missing package catches it too.

LLMRequestError

Bases: EBaMlError

Exception raised when the provider of the LLM fails a request.

The message names the cause and lists the steps that fix it. The error that the BAML runtime raised is the __cause__.

MissingAPIKeyError

Bases: LLMRequestError

Exception raised when a provider needs an API key and the environment has none.

APIKeyRejectedError

Bases: LLMRequestError

Exception raised when a provider answers a request with "unauthorized" or "forbidden".

RateLimitedError

Bases: LLMRequestError

Exception raised when a provider limits the rate of the requests.

ProviderUnreachableError

Bases: LLMRequestError

Exception raised when a request does not reach the provider.

TokenFitError

Bases: EBaMlError, ValueError

Exception raised when the token weights cannot be fitted or read, for example from an empty list of samples.

SearchLimitError

Bases: EBaMlError

Exception raised when a covering search ends at its limit with no set and no proof that no set exists.

EditError

Bases: EBaMlError, ValueError

Exception raised when an edit of an EBM cannot run, for example an edit that needs an order on a nominal feature.

The message names the cause and lists the steps that fix it.

SelectionError

Bases: EditError

Exception raised when the bins of an edit do not exist, or the feature type does not allow the selection.

StaleRevisionError

Bases: EBaMlError

Exception raised when the head of an edit session is not the revision that the caller expected.

The message names the current head. The caller reads the session again and decides what to do. The session never retries by itself.

UnreviewedEditError

Bases: EBaMlError

Exception raised when a save needs the review of an edit that nobody reviewed.

HistoryError

Bases: EditError

Exception raised when the history of an edit session cannot be read, or does not fit the model.

Benchmark

ebaml.benchmark

Benchmark of how well an LLM reads the graphs of Explainable Boosting Machines.

Port of the benchmark/ of TalkToEBM. The questions are stored in benchmark/data as JSON lists of [prompt, solution] pairs. Each question is answered in two LLM turns (see answer_benchmark_question in baml_src/talk_to_ebm.baml): the model reasons step by step, then states its final answer in a format that this module parses and compares with the solution. TalkToEBM's notebooks printed the answers next to the solutions and compared them by eye; here the comparison is automatic. The module imports no LLM client: the caller passes the function that asks the question, so the parsing and the scoring run without the generated baml_sdk.

Usage

Bases: Protocol

The tokens that the provider reported for a reply.

input_tokens property

The tokens of the prompt.

output_tokens property

The tokens of the reply.

Reply

Bases: Protocol

The reply of the LLM to a question, with the tokens that it used.

text property

The final answer of the LLM.

usage property

The tokens that the provider reported.

AskQuestion

Bases: Protocol

A function that asks the LLM a question of the benchmark.

baml_sdk.answer_benchmark_question_sync is such a function.

__call__(*, question, answer_format)

Ask the question and ask for the final answer in the format of answer_format.

Task

Bases: StrEnum

The tasks of the benchmark. The value is the name of the data file.

answer_format property

How the model is asked to state its final answer, to complete "answering with ...".

Returns:

Type Description
str

The description of the answer format of the task.

Question

A question of the benchmark.

Outcome

The answer of the LLM to a question.

TaskResult

The outcomes of a task, with the tokens that the provider reported for them.

correct property

The number of questions that were answered correctly.

Returns:

Type Description
int

The number of correct outcomes.

accuracy property

The fraction of the questions that were answered correctly.

Returns:

Type Description
float

The accuracy, between 0 and 1. A task with no outcomes has the accuracy 0.

load_questions(task, data_dir)

Load the questions of a task.

Parameters:

Name Type Description Default
task Task

The task.

required
data_dir str | PathLike[str]

The directory with the data files of the benchmark.

required

Returns:

Type Description
list[Question]

The questions, in the order of the file.

parse_answer(task, text)

Parse the answer to a question of a task, or its solution.

Parameters:

Name Type Description Default
task Task

The task, which sets the format of the answer.

required
text str

The text of the answer.

required

Returns:

Type Description
Answer | None

The parsed answer, or None if the text does not hold an answer in the format of the task.

is_correct(task, response, solution)

Whether the response of the LLM states the solution.

The comparison is made on the parsed answers, so the formatting of the response (a or (a), 0.50 or .5, markdown, a trailing period) does not matter, but the values must be exactly the same.

Parameters:

Name Type Description Default
task Task

The task, which sets the format of the answer.

required
response str

The final answer of the LLM.

required
solution str

The solution of the question.

required

Returns:

Type Description
bool

True if the response holds an answer, and it is the solution.

Examples:

>>> is_correct(Task.MONOTONICITY, "The graph is **increasing**.", "increasing")
True
>>> is_correct(Task.READ_VALUE, "The answer is 0.50.", "0.5")
True
>>> is_correct(Task.READ_VALUE, "The answer is 0.25.", "0.5")
False

run_task(task, questions, ask, *, progress=False)

Ask the LLM the questions of a task, one at a time, and score its answers.

The questions are asked sequentially, because the free models are rate limited.

Parameters:

Name Type Description Default
task Task

The task, which sets the answer format and the scoring.

required
questions Sequence[Question]

The questions to ask, for example from load_questions.

required
ask AskQuestion

Asks the LLM one question, for example baml_sdk.answer_benchmark_question_sync. It chooses the model.

required
progress bool

Whether to show a progress bar of the questions on the error stream. If that stream is not a terminal, the bar writes one line for each question.

False

Returns:

Type Description
TaskResult

The outcomes and the token usage. If there are no questions, there are no outcomes and the accuracy is 0.

format_results(results)

Format the results of the tasks as a table.

Parameters:

Name Type Description Default
results Sequence[TaskResult]

The results, one per task.

required

Returns:

Type Description
str

A table with a row per task: the correct answers, the accuracy and the tokens used.