Skip to main content

Introduction

Zeno is an interactive platform for data management and model evaluation. It combines a Python API with an interactive user interface to empower you to discover, explore, and analyze the performance of your models across diverse use cases. Zeno can be used for any data type or task with modular views for everything from object detection to audio transcription.

See the Quickstart to set up Zeno and learn about the core concepts.

Key Features​

Zeno uses a Python decorator API to generate data and model information that is used to scaffold the interactive evaluation UI.

Exploration UI​

Zeno's Exploration UI is the main interface for evaluation. It lets you slice and dice your data to quickly see how your model works for different types of instances. You can pick different models and metrics to see model performance.

The modular instance view supports any data type or task. Existing views can be found here, and new views can be written for unsupported data types.

Report UI​

The Report page is used to compare models across slices. You can create reports for groups of instance and see how new models compare with sparkline visualizations and trends. To catch potential regressions, you can create tests of expected metrics for slices. Reports can be shared and exported as PDFs.

Python API​

The Python API consists of four core decorator functions that you can use to plug your models in and generate information for evaluation. The @model functions return model outputs for any Python-based model or API, which can then be evaluated with @metric functions. To test more diverse use cases @distill functions can be used to create new metadata columns based on raw data instances.

Example Zeno API functions
@metric
def accuracy(df, ops):
    return df[ops.label_column] 
                == df[ops.output_column]

@distill
def brightness(df, ops):
    return lum(df[ops.data_column]))

Why Zeno?​

Zeno helps you move beyond relying on aggregate metrics and spot-checking model outputs. Instead, it allows you to develop a deep and quantitative understanding of how your model behaves.

Zeno supports your workflow - it is model and data agnostic, and requires just a single Python function to start working. It also lets you test diverse mode use cases, from potential fairness concerns to robustness checks. Lastly, Zeno lets you compare your models and detect potential regressions as you release updates.