Use ai.embed with pandas

The ai.embed function converts text into vector embeddings that represent meaning. Use embeddings to search, group, and compare content by meaning instead of exact wording.

Note

• This article covers ai.embed with pandas. For PySpark, see Use ai.embed with PySpark.
• For all AI Functions and prerequisites, see AI Functions overview.
• Change default configuration for AI Functions with pandas.

Overview

The ai.embed function extends the pandas Series class.

To generate vector embeddings of each input row, call the function on either a pandas Series or a text column of pandas DataFrame.

The function returns a pandas Series that contains embeddings, which can be stored in a new DataFrame column.

Syntax

df["embed"] = df["col1"].ai.embed()

Parameters

None.

Returns

The function returns a pandas Series that contains embeddings as numpy array of float-32 values for each input row. The array size depends on the embedding model dimensions, which are configurable in AI Functions.

Example

# This code uses AI. Always review output for mistakes.

df = pd.DataFrame([
        "This duvet, lovingly hand-crafted from all-natural fabric, is perfect for a good night's sleep.",
        "Tired of friends judging your baking? With these handy-dandy measuring cups, you'll create culinary delights.",
        "Enjoy this *BRAND NEW CAR!* A compact SUV perfect for the professional commuter!"
    ], columns=["descriptions"])
    
df["embed"] = df["descriptions"].ai.embed()
display(df)

Output:

Related content

• Use ai.embed with PySpark.
• Learn more about AI Functions.
• Change default configuration for AI Functions with pandas.
• Understand billing for AI Functions.