Skip to main content

Getting Started

Zeno is an interactive AI evaluation platform for exploring, debugging, and sharing how your AI systems perform. Evaluate any task and data type with Zeno's modular views which support everything from chatbot conversations to object detection and audio transcription.

tip

Explore public Zeno reports and projects on Zeno Hub to see what's possible with Zeno!

Creating a Project​

The core of Zeno are projects, which consist of a base evaluation dataset and any number of AI system outputs. We'll walk through creating your first project step by step, which will result in this final project example:

Complete Example
from zeno_client import ZenoClient, ZenoMetric
import pandas as pd

client = ZenoClient("YOUR API KEY HERE")

df = pd.DataFrame(
  {
      "id": [1, 2, 3],
      "text": [
          "I love this movie!",
          "I hate this movie!",
          "This movie is ok.",
      ],
      "label": ["positive", "negative", "neutral"],
  }
)

# Add any additional columns you want to do analysis across.
df["input length"] = df["text"].str.len()

# Create a project.
project = client.create_project(
name="Sentiment Classification",
view="text-classification",
metrics=[
    ZenoMetric(name="accuracy", type="mean", columns=["correct"]),
  ]
)

# Upload the data.
project.upload_dataset(df, id_column="id", data_column='text', label_column="label")

# Create a system DataFrame.
df_system = pd.DataFrame(
  {
    "output": ["positive", "negative", "negative"],
  }
)

# Create an id column to match the base dataset.
df_system["id"] = df_system.index

# Measure accuracy for each instance, which is averaged by the ZenoMetric above.
df_system["correct"] = (df_system["output"] == df["label"]).astype(int)

proj.upload_system(df_system, name="System A", id_column="id", output_column="output")

Creating a Zeno Account​

If you don't have a Zeno account already, create one on Zeno Hub. After logging in to Zeno Hub, generate your API key by clicking on your profile at the top right to navigate to your account page.

Using the Python API​

Zeno projects are created and managed using the zeno-client Python library:

pip install zeno-client

We can now initialize a client with the API key which we will use to create projects and upload data.

from zeno_client import ZenoClient, ZenoMetric
import pandas as pd

# Initialize a client with our API key.
client = ZenoClient("YOUR API KEY HERE")

Setting up Your Data​

Zeno projects take data and model outputs as Pandas DataFrames. In this example, we will upload a toy example of sentiment classification with 3 instances and ground truth labels:

...

# Put all data in a Pandas DataFrame
df = pd.DataFrame(
    {
        "id": [1, 2, 3],
        "text": [
            "I love this movie!",
            "I hate this movie!",
            "This movie is ok.",
        ],
        "label": ["positive", "negative", "neutral"],
    }
)

# Add any additional columns you want to do analysis across.
df["input length"] = df["text"].str.len()
tip

Every DataFrame in Zeno requires a unique id_column that identifies each instance. This is used to match the base dataset with system outputs.

Initializing a Project​

We can now initialize a project using the client and upload our base dataset:

...

project = client.create_project(
    name="Sentiment Classification",
    view="text-classification",
    metrics=[
        ZenoMetric(name="accuracy", type="mean", columns=["correct"]),
    ]
)

project.upload_dataset(df, id_column="id", data_column='text', label_column="label")

We named our project Sentiment Classification and specified to use the text_classification view. We also added an accuracy metric which takes the mean of the correct column that will be present in the system outputs we upload later.

tip

Learn more about Zeno's view specification in the instance view docs. You can find the default supported views here.

Uploading AI System Outputs​

Lastly, we can upload our AI system outputs to the project. We add an id column to match the base dataset and a correct column that measures accuracy for each instance.

...

df_system = pd.DataFrame(
    {
        "output": ["positive", "negative", "negative"],
    }
)

# Create an id column to match the base dataset.
df_system["id"] = df_system.index

# Measure accuracy for each instance, which is averaged by the ZenoMetric above.
df_system["correct"] = (df_system["output"] == df["label"]).astype(int)

proj.upload_system(df_system, name="System A", id_column="id", output_column="output")

You can now navigate to the project URL in Zeno Hub to see the uploaded data and metrics and start exploring your AI system's performance!

Quickstart with Zeno Build​

Zeno Build is a collection of example projects for common AI and ML tasks. Check out some common Zeno Build notebooks: