Share via

Send media attachments with Bot Framework SDK

  • Article

APPLIES TO: SDK v4

Messages exchanged between user and bot can contain media attachments, such as images, video, audio, and files. The Bot Framework SDK supports the task of sending rich messages to the user. To determine the type of rich messages a channel (Facebook, Slack, and so on) supports, consult the channel's documentation for information about limitations.

Note

The Bot Framework JavaScript, C#, and Python SDKs will continue to be supported, however, the Java SDK is being retired with final long-term support ending in November 2023.

Existing bots built with the Java SDK will continue to function.

For new bot building, consider using Microsoft Copilot Studio and read about choosing the right copilot solution.

For more information, see The future of bot building.

Prerequisites

Send attachments

To send the user content like an image or a video, you can add an attachment or list of attachments to a message.

See Design the user experience for examples of available cards.

See also What is the size limit of a file transferred using channels? in the FAQ.

All of the source code shown in this section is based on the Handling attachments sample.

The Attachments property of the Activity object contains an array of Attachment objects that represent the media attachments and rich cards attached to the message. To add a media attachment to a message, create an Attachment object for the reply activity and set the ContentType, ContentUrl, and Name properties.

To create the reply message, define the text and then set up the attachments. Assigning the attachments to the reply is the same for each attachment type, however the various attachments are set up and defined differently, as seen in the following snippets. The code below is setting up the reply for an inline attachment:

Bots/AttachmentsBot.cs

C# 
{
    reply = MessageFactory.Text("This is an inline attachment.");

Next, we look at the types of attachments. First is an inline attachment:

Bots/AttachmentsBot.cs

C# 
{
    var imagePath = Path.Combine(Environment.CurrentDirectory, @"Resources", "architecture-resize.png");
    var imageData = Convert.ToBase64String(File.ReadAllBytes(imagePath));

    return new Attachment
    {
        Name = @"Resources\architecture-resize.png",
        ContentType = "image/png",
        ContentUrl = $"data:image/png;base64,{imageData}",
    };
}

Then, an uploaded attachment:

Bots/AttachmentsBot.cs

C# 
{
    if (string.IsNullOrWhiteSpace(serviceUrl))
    {
        throw new ArgumentNullException(nameof(serviceUrl));
    }

    if (string.IsNullOrWhiteSpace(conversationId))
    {
        throw new ArgumentNullException(nameof(conversationId));
    }

    var imagePath = Path.Combine(Environment.CurrentDirectory, @"Resources", "architecture-resize.png");

    var connector = turnContext.TurnState.Get<IConnectorClient>() as ConnectorClient;
    var attachments = new Attachments(connector);
    var response = await attachments.Client.Conversations.UploadAttachmentAsync(
        conversationId,
        new AttachmentData
        {
            Name = @"Resources\architecture-resize.png",
            OriginalBase64 = File.ReadAllBytes(imagePath),
            Type = "image/png",
        },
        cancellationToken);

    var attachmentUri = attachments.GetAttachmentUri(response.Id);

    return new Attachment
    {
        Name = @"Resources\architecture-resize.png",
        ContentType = "image/png",
        ContentUrl = attachmentUri,
    };
}

Lastly, an internet attachment:

Bots/AttachmentsBot.cs

C# 
    {
        // ContentUrl must be HTTPS.
        return new Attachment
        {
            Name = @"Resources\architecture-resize.png",
            ContentType = "image/png",
            ContentUrl = "https://docs.microsoft.com/en-us/bot-framework/media/how-it-works/architecture-resize.png",
        };
    }
}

If an attachment is an image, audio, or video, the Connector service will communicate attachment data to the channel in a way that enables the channel to render that attachment within the conversation. If the attachment is a file, the file URL will be rendered as a hyperlink within the conversation.

Send a hero card

Besides simple image or video attachments, you can attach a hero card, which allows you to combine images and buttons in one object, and send them to the user. Markdown is supported for most text fields, but support may vary by channel.

To compose a message with a hero card and button, you can attach a HeroCard object to a message.

The following source code is from the Handling attachments sample.

Bots/AttachmentsBot.cs

C# 

      private static async Task DisplayOptionsAsync(ITurnContext turnContext, CancellationToken cancellationToken)
      {
          // Create a HeroCard with options for the user to interact with the bot.
          var card = new HeroCard
          {
              Text = "You can upload an image or select one of the following choices",
              Buttons = new List<CardAction>
              {
                  // Note that some channels require different values to be used in order to get buttons to display text.
                  // In this code the emulator is accounted for with the 'title' parameter, but in other channels you may
                  // need to provide a value for other parameters like 'text' or 'displayText'.
                  new CardAction(ActionTypes.ImBack, title: "1. Inline Attachment", value: "1"),
                  new CardAction(ActionTypes.ImBack, title: "2. Internet Attachment", value: "2"),
                  new CardAction(ActionTypes.ImBack, title: "3. Uploaded Attachment", value: "3"),
              },
          };

          var reply = MessageFactory.Attachment(card.ToAttachment());
          await turnContext.SendActivityAsync(reply, cancellationToken);

Process events within rich cards

To process events within rich cards, use card action objects to specify what should happen when the user selects a button or taps a section of the card. Each card action has a type and value property.

To function correctly, assign an action type to each clickable item on a hero card. This table lists and describes the available action types and what should be in the associated value property.

The messageBack card action has a more generalized meaning than the other card actions. See the Card action section of the Activity schema for more information about the messageBack and other card action types.

Type Description Value
call Initiates a phone call. Destination for the phone call in this format: tel:123123123123.
downloadFile Downloads a file. The URL of the file to download.
imBack Sends a message to the bot, and posts a visible response in the chat. Text of the message to send.
messageBack Represents a text response to be sent via the chat system. An optional programmatic value to include in generated messages.
openUrl Opens a URL in the built-in browser. The URL to open.
playAudio Plays audio. The URL of the audio to play.
playVideo Plays a video. The URL of video to play.
postBack Sends a message to the bot, and may not post a visible response in the chat. Text of the message to send.
showImage Displays an image. The URL of the image to display.
signin Initiates an OAuth sign-in process. The URL of the OAuth flow to initiate.

Hero card using various event types

The following code shows examples using various rich card events.

For examples of all the available cards, see the Using cards sample.

Cards.cs

C# 
public static HeroCard GetHeroCard()
{
    var heroCard = new HeroCard
    {
        Title = "BotFramework Hero Card",
        Subtitle = "Microsoft Bot Framework",
        Text = "Build and connect intelligent bots to interact with your users naturally wherever they are," +
               " from text/sms to Skype, Slack, Office 365 mail and other popular services.",
        Images = new List<CardImage> { new CardImage("https://sec.ch9.ms/ch9/7ff5/e07cfef0-aa3b-40bb-9baa-7c9ef8ff7ff5/buildreactionbotframework_960.jpg") },
        Buttons = new List<CardAction> { new CardAction(ActionTypes.OpenUrl, "Get Started", value: "https://docs.microsoft.com/bot-framework") },
    };

    return heroCard;
}

Cards.cs

C# 
public static SigninCard GetSigninCard()
{
    var signinCard = new SigninCard
    {
        Text = "BotFramework Sign-in Card",
        Buttons = new List<CardAction> { new CardAction(ActionTypes.Signin, "Sign-in", value: "https://login.microsoftonline.com/") },
    };

    return signinCard;
}

Send an Adaptive Card

While you can use the message factory to create a message that contains an attachment (of any sort), an Adaptive Card is one specific type of attachment. Not all channels support Adaptive Cards, and some channels may only partially support Adaptive Cards. For example, if you send an Adaptive Card in Facebook, the buttons won't work while texts and images work well. The message factory is a Bot Framework SDK helper class used to automate creation steps for you.

Adaptive Cards are an open card exchange format enabling developers to exchange UI content in a common and consistent way. However, not all channels support Adaptive Cards.

The Adaptive Cards Designer provides a rich, interactive design-time experience for authoring adaptive cards.

Note

You should test this feature with the channels your bot will use to determine whether those channels support adaptive cards.

To use Adaptive Cards, be sure to add the AdaptiveCards NuGet package.

The following source code is from the Using cards sample.

Cards.cs

This example reads the Adaptive Card JSON from a file and adds it as an attachment.

C# 
public static Attachment CreateAdaptiveCardAttachment()
{
    // combine path for cross platform support
    var paths = new[] { ".", "Resources", "adaptiveCard.json" };
    var adaptiveCardJson = File.ReadAllText(Path.Combine(paths));

    var adaptiveCardAttachment = new Attachment()
    {
        ContentType = "application/vnd.microsoft.card.adaptive",
        Content = JsonConvert.DeserializeObject(adaptiveCardJson),
    };

    return adaptiveCardAttachment;
}

Messages can also include multiple attachments in a carousel layout, which places the attachments side by side and allows the user to scroll across.

The following source code is from the Using cards sample.

Dialogs/MainDialog.cs

First, create the reply and define the attachments as a list.

C# 
// Cards are sent as Attachments in the Bot Framework.
// So we need to create a list of attachments for the reply activity.
var attachments = new List<Attachment>();

// Reply to the activity we received with an activity.
var reply = MessageFactory.Attachment(attachments);

Then add the attachments and set the layout type to carousel. Here we're adding them one at a time, but feel free to manipulate the list to add the cards however you prefer.

C# 
// Display a carousel of all the rich card types.
reply.AttachmentLayout = AttachmentLayoutTypes.Carousel;
reply.Attachments.Add(Cards.CreateAdaptiveCardAttachment());
reply.Attachments.Add(Cards.GetAnimationCard().ToAttachment());
reply.Attachments.Add(Cards.GetAudioCard().ToAttachment());
reply.Attachments.Add(Cards.GetHeroCard().ToAttachment());
reply.Attachments.Add(Cards.GetOAuthCard().ToAttachment());
reply.Attachments.Add(Cards.GetReceiptCard().ToAttachment());
reply.Attachments.Add(Cards.GetSigninCard().ToAttachment());
reply.Attachments.Add(Cards.GetThumbnailCard().ToAttachment());
reply.Attachments.Add(Cards.GetVideoCard().ToAttachment());

Once the attachments are added, you can send the reply just like any other.

C# 
// Send the card(s) to the user as an attachment to the activity
await stepContext.Context.SendActivityAsync(reply, cancellationToken);

Code sample for processing Adaptive Card input

The following sample shows one way to use Adaptive Card inputs within a bot dialog class. It extends the hero cards sample by validating the input received in the text field from the responding client. You first need to add the text input and button functionality to the existing adaptive card by adding the following code just before the final bracket of adaptiveCard.json, located in the resources folder:

JSON 
"actions": [
  {
    "type": "Action.ShowCard",
    "title": "Text",
    "card": {
      "type": "AdaptiveCard",
      "body": [
        {
          "type": "Input.Text",
          "id": "text",
          "isMultiline": true,
          "placeholder": "Enter your comment"
        }
      ],
      "actions": [
        {
          "type": "Action.Submit",
          "title": "OK"
        }
      ]
    }
  }
]

The ID of the text input field is set to "text". When the user selects OK, the message the Adaptive Card generates will have a value property that has a property named text that contains the information the user entered in the text input field of the card.

Our validator uses Newtonsoft.json to first convert this to a JObject, and then create a trimmed text string for comparison. So add:

C# 
using System;
using System.Linq;
using Newtonsoft.Json.Linq;

to MainDialog.cs and install the latest stable NuGet package of Newtonsoft.Json. In the validator code, we added the logic flow into the code comments. This ChoiceValidator method is placed into the Using cards sample just after the closed brace public for declaration of MainDialog:

C# 
private async Task ChoiceValidator(
    PromptValidatorContext promptContext,
    CancellationToken cancellationToken)
{
    // Retrieves Adaptive Card comment text as JObject.
    // looks for JObject field "text" and converts that input into a trimmed text string.
    var jobject = promptContext.Context.Activity.Value as JObject;
    var jtoken = jobject?["text"];
    var text = jtoken?.Value().Trim();

    // Logic: 1. if succeeded = true, just return promptContext
    //        2. if false, see if JObject contained Adaptive Card input.
    //               No = (bad input) return promptContext
    //               Yes = update Value field with JObject text string, return "true".
    if (!promptContext.Recognized.Succeeded && text != null)
    {
        var choice = promptContext.Options.Choices.FirstOrDefault(
        c => c.Value.Equals(text, StringComparison.InvariantCultureIgnoreCase));
        if (choice != null)
        {
            promptContext.Recognized.Value = new FoundChoice
            {
                Value = choice.Value,
            };
            return true;
        }
    }
    return promptContext.Recognized.Succeeded;
}

Now above in the MainDialog declaration change:

C# 
// Define the main dialog and its related components.
AddDialog(new ChoicePrompt(nameof(ChoicePrompt)));

to:

C# 
// Define the main dialog and its related components.
AddDialog(new ChoicePrompt(nameof(ChoicePrompt), ChoiceValidator));

This will invoke your validator to look for Adaptive Card input each time a new choice prompt is created.

Test the Using Cards bot

  1. Run the Using cards sample locally and open the bot in the Bot Framework Emulator.
  2. Follow the prompts in the bot to display a card type, such as an Adaptive Card.

Next steps

Add buttons to guide user action