Create a Prompt flow with Semantic Kernel

Article
12/14/2023

In the planner article, we demonstrated how you can use the sequential planner to automatically use the math plugin to answer word problems provided by the user. If you began testing your planner with additional inputs, however, you may have noticed that it doesn't always produce the desired results. In this article, we'll show you how you can create a Prompt flow that runs your plugins and planners so that you can easily test, evaluate, and deploy them in following articles.

At the end of this article, you'll have a Prompt flow that can answer questions to math problems using Semantic Kernel.

Semantic Kernel running inside of Prompt flow

If you want to see the final solution to this article, you can check out the following samples in the public documentation repository. Use the link to the previous solution if you want to follow along.

Note

Today Prompt flow is only available in Python, so this article will only show how to use Prompt flow to evaluate plugins using Python.

Language	Link to previous solution	Link to final solution
C#	Not applicable	Not available
Java	Not applicable	Not available
Python	Open solution in GitHub	Open solution in GitHub

Create a new Prompt flow

In this tutorial, we'll be creating a flow that uses the math plugin we created in the pervious tutorial. We'll use the code-first approach to creating a Prompt flow. If you want full documentation on how to use the code-first approach, please refer to Prompt flow's open source documentation.

If you would like to use the Azure portal to create your flow, you can follow the How to develop an evaluation flow tutorial in the Azure documentation.

Install Prompt flow

Install the promptflow and promptflow-tools by running the following command in your terminal.
```
pip install promptflow promptflow-tools
```
Validate that the installation was successful by running the following command in your terminal.
```
# should print promptflow version, e.g. "0.1.0b3"
pf -v
```

Install the Prompt flow VS Code extension

We also recommend installing the Prompt flow VS Code extension to help you create and test Prompt flows directly from within VS Code.

Install the latest stable version of VS Code
Install the VS Code Python extension
Install the Prompt flow for VS Code extension

With the VS Code extension, you'll be able to view your Prompt flow in a visual editor, as well as test your Prompt flow directly from within VS Code.

A graph of a Prompt flow

Use the CLI to create a new Prompt flow

Prompt flow has three different types of flows: standard, chat, and evaluation. We'll first create a flow that will run the math plugin we created in the previous tutorial using the standard flow type. This will allow us to test the flow and make sure it's working as expected.

To create a standard flow, navigate to the root of the previously completed solution and run the following command in your terminal. If you haven't completed the previous tutorial, you can use the link at the top of this article to download the completed previous solution.

pf flow init --flow perform_math

This will create a new folder with the name of the flow you specified along with some boilerplate code. You can now open this folder in VS Code to view the files that were created. The most important files to note are the following:

flow.dag.yaml – The definition of the flow; includes inputs/outputs, nodes, tools and variants.
.promptflow/flow.tools.json – Provides metadata for the nodes found in the flow.dag.yaml file.
Source code files (.py, .jinja2) – The custom scripts that are referenced by Prompt flow.
requirements.txt – The Python package dependencies for this flow; used for installing packages into the Azure environment.

Visualize the flow in VS Code

If you want to see what your flow looks like in a visual editor, you can open the flow.dag.yaml file in VS Code and click the Visual editor link in the top left corner of the editor.

A graph of a Prompt flow in the VS Code extension

This will open the flow in a visual editor where you can see the nodes and their connections.

Edit the new Prompt flow

Now that we have a starter flow, we can start customizing it to use the math plugin we created in the previous tutorial. We'll start by deleting the existing nodes and creating a new node that will call the math plugin.

Delete the existing nodes

Navigate to the visual editor
Select the delete icon (the trash can) for the hello_prompt and echo_my_prompt nodes.
Delete the hello.jinja2 and hello.py files from the flow folder.

Create a new node

To run our math plugin, we'll need to create a new node that will actually call the plugin using a Semantic Kernel planner. We'll use the Python node type to do this.

Navigate to the visual editor
Select the + icon to create a new node
Select the Python node type
Name your new node math_planner
Select New file.

After completing these steps, you should see a new node in the visual editor with the name math_planner along with a new Python file called math_planner.py in the flow folder.

Wiring up the new node

In the visual editor, you'll likely see an error icon. This is because our node doesn't have any inputs or outputs defined. We'll need to define these so that Prompt flow knows how to wire up the nodes.

To do this, follow these steps:

Navigate to the visual editor
Rename the output_prompt output variable to result
Change the value of the result output variable to ${math_planner.output}
Set the value of the input1 string of the math_planner node to ${inputs.text}

After completing these steps, you should see that the error icon has disappeared and the nodes are now connected.

A valid graph of a Prompt flow

If you want, you can also edit the flow directly in the flow.dag.yaml file. The following code shows the updated flow.dag.yaml file. If you are running into issues with the visual editor, you can copy and paste this code into your flow.dag.yaml file.

id: template_standard_flow
name: Template Standard Flow
environment:
  python_requirements_txt: requirements.txt
inputs:
  text:
    type: string
outputs:
  result:
    type: string
    reference: ${math_planner.output}
nodes:
- name: math_planner
  type: python
  source:
    type: code
    path: math_planner.py
  inputs:
    input1: ${inputs.text}

Calling the math plugin with the planner

Now that we have our nodes wired up, we can add the code to call the math plugin. To do this, we'll need to update the math_planner.py file with the code from the planner tutorial. Below is the updated code for the math_planner.py file.

When creating a node for Prompt flow, it's important to use the @tool decorator to indicate that the function is a tool that can be called by Prompt flow.

import asyncio
from promptflow import tool

import semantic_kernel as sk
from semantic_kernel.planning.sequential_planner import SequentialPlanner
from plugins.MathPlugin.Math import Math as Math
from promptflow.connections import (
    AzureOpenAIConnection,
)

from semantic_kernel.connectors.ai.open_ai import (
    AzureChatCompletion,
    AzureTextCompletion,
)

@tool
def my_python_tool(
    input: str,
    deployment_type: str,
    deployment_name: str,
    AzureOpenAIConnection: AzureOpenAIConnection,
) -> str:
    # Initialize the kernel
    kernel = sk.Kernel(log=sk.NullLogger())

    # Add the chat service
    if deployment_type == "chat-completion":
        kernel.add_chat_service(
            "chat_completion",
            AzureChatCompletion(
                deployment_name,
                endpoint=AzureOpenAIConnection.api_base,
                api_key=AzureOpenAIConnection.api_key,
            ),
        )
    elif deployment_type == "text-completion":
        kernel.add_text_completion_service(
            "text_completion",
            AzureTextCompletion(
                deployment_name,
                endpoint=AzureOpenAIConnection.api_base,
                api_key=AzureOpenAIConnection.api_key,
            ),
        )

    planner = SequentialPlanner(kernel=kernel)

    # Import the native functions
    math_plugin = kernel.import_plugin(Math(), "MathPlugin")

    ask = "Use the available math functions to solve this word problem: " + input

    plan = asyncio.run(planner.create_plan_async(ask))

    # Execute the plan
    result = asyncio.run(kernel.run_async(plan)).result

    for index, step in enumerate(plan._steps):
        print("Function: " + step.plugin_name + "." + step._function.name)
        print("Input vars: " + str(step.parameters.variables))
        print("Output vars: " + str(step._outputs))
    print("Result: " + str(result))

    return str(result)

You'll also need to copy and paste your plugins folder from the previous tutorial into the flow folder so that the math plugin is available to the flow. Below is the updated directory structure for the flow.

12-Evaluate-with-Prompt-Flow
└───perform_math
|   └───.promptflow
|   |   └───flow.tools.json
|   └───plugins
│   |   └───MathPlugin
|   |       └───Math.py
|   └───.gitignore
|   └───data.jsonl
|   └───flow.dag.yaml
|   └───hello.jinja2
|   └───hello.py
|   └───requirements.txt
└───main.py

Performing an ad-hoc test of your Prompt flow

Our flow is now ready to be tested. To do this, complete the following:

Navigate to the visual editor.
Set the value of the text input variable to What is 2 plus 3?
Select the Run all button (the double play icon) in the top right corner of the visual editor.

Within the terminal, you should see the following output.

{'result': '5.0'}

If you want to test your flow using the SDK, you can also write a simple Python script to do this.

from promptflow import PFClient
from promptflow.entities import AzureOpenAIConnection, OpenAIConnection
from dotenv import dotenv_values

#################################################

# Load the configuration from the .env file
config = dotenv_values(".env")
deployment_type = config.get("AZURE_OPEN_AI__DEPLOYMENT_TYPE", None)
if deployment_type == "chat-completion":
    deployment_name = config.get("AZURE_OPEN_AI__CHAT_COMPLETION_DEPLOYMENT_NAME", None)
elif deployment_type == "text-completion":
    deployment_name = config.get("AZURE_OPEN_AI__TEXT_COMPLETION_DEPLOYMENT_NAME", None)

Next steps

Now that you have a flow that can run your plugin, you can start to test where it's performing well and where it's not. Testing each case individually can be time consuming, so in the next article, we'll show you how you can run your flow on a large amount of data to see how well it performs with a wide range of user requests.

Run batches with Prompt flow