Tutorial: Enable inline image support in your Chat app

The Chat SDK is designed to work with Microsoft Teams seamlessly. Specifically, Chat SDK provides a solution to receive inline images sent by users from Microsoft Teams. Currently this feature is only available in the Chat SDK for JavaScript and C#.

Add inline image support

Inline images are images that are copied and pasted directly into the send box of the Teams client. For images that were uploaded via the "Upload from this device" menu or via drag-and-drop, such as images dragged directly to the send box in Teams, you need to refer to this tutorial to enable it as the part of the file sharing feature. (See the section "Handling Image Attachment.") To copy an image, the Teams user can either use their operating system's context menu to copy the image file and then paste it into the send box of their Teams client or use keyboard shortcuts.

The Chat SDK for JavaScript provides previewUrl and url for each inline image. Note that some GIF images fetched from previewUrl might not be animated, and a static preview image may be returned instead. Developers are expected to use the url if the intention is to fetch animated images only.

In this tutorial, you learn how to enable inline image support using the Azure Communication Services Chat SDK for JavaScript.

Sample Code

Find the finalized code of this tutorial on GitHub.

Prerequisites

  • You've gone through the quickstart - Join your chat app to a Teams meeting.
  • Create an Azure Communication Services resource. For details, see Create an Azure Communication Services resource. You need to record your connection string for this tutorial.
  • You've set up a Teams meeting using your business account and have the meeting URL ready.
  • You're using the Chat SDK for JavaScript (@azure/communication-chat) 1.4.0 or latest. See here.

Goal

  1. Be able to render preview images in the message thread
  2. Be able to render full scale image upon click on preview images

Handle inline images for new messages

In the quickstart, we've created an event handler for chatMessageReceived event, which would be trigger when we receive a new message from the Teams user. We have also appended incoming message content to messageContainer directly upon receiving the chatMessageReceived event from the chatClient like this:

chatClient.on("chatMessageReceived", (e) => {
   console.log("Notification chatMessageReceived!");

   // check whether the notification is intended for the current thread
   if (threadIdInput.value != e.threadId) {
      return;
   }

   if (e.sender.communicationUserId != userId) {
      renderReceivedMessage(e.message);
   }
   else {
      renderSentMessage(e.message);
   }
});
   
async function renderReceivedMessage(message) {
   messages += '<div class="container lighter">' + message + '</div>';
   messagesContainer.innerHTML = messages;
}

From incoming event of type ChatMessageReceivedEvent, there's a property named attachments, which contains information about inline image, and it's all we need to render inline images in our UI:

export interface ChatMessageReceivedEvent extends BaseChatMessageEvent {
  /**
   * Content of the message.
   */
  message: string;

  /**
   * Metadata of the message.
   */
  metadata: Record<string, string>;

  /**
   * Chat message attachment.
   */
  attachments?: ChatAttachment[];
}

export interface ChatAttachment {
  /** Id of the attachment */
  id: string;
  /** The type of attachment. */
  attachmentType: ChatAttachmentType;
  /** The name of the attachment content. */
  name?: string;
  /** The URL where the attachment can be downloaded */
  url?: string;
  /** The URL where the preview of attachment can be downloaded */
  previewUrl?: string;
}

export type ChatAttachmentType = "image" | "unknown";

Now let's go back to the previous code to add some extra logic like the following code snippets:

chatClient.on("chatMessageReceived", (e) => {
  console.log("Notification chatMessageReceived!");
  // check whether the notification is intended for the current thread
  if (threadIdInput.value != e.threadId) {
     return;
  }
   
  if (e.sender.communicationUserId != userId) {
    renderReceivedMessage(e);
  } else {
    renderSentMessage(e.message);
  }
});

async function renderReceivedMessage(e) {
  const messageContent = e.message;

  const card = document.createElement('div');
  card.className = 'container lighter';
  card.innerHTML = messageContent;
  
  messagesContainer.appendChild(card);
  
  // filter out inline images from attchments
  const imageAttachments = e.attachments.filter((e) =>
    e.attachmentType.toLowerCase() === 'image');
  
  // fetch and render preview images
  fetchPreviewImages(imageAttachments);
  
  // set up onclick event handler to fetch full scale image
  setImgHandler(card, imageAttachments);
}

function setImgHandler(element, imageAttachments) {
  // do nothing if there's no image attachments
  if (!imageAttachments.length > 0) {
    return;
  }
  const imgs = element.getElementsByTagName('img');
  for (const img of imgs) {
    img.addEventListener('click', (e) => {
      // fetch full scale image upon click
      fetchFullScaleImage(e, imageAttachments);
    });
  }
}

async function fetchPreviewImages(attachments) {
  if (!attachments.length > 0) {
    return;
  }
  // since each message could contain more than one inline image
  // we need to fetch them individually 
  const result = await Promise.all(
      attachments.map(async (attachment) => {
        // fetch preview image from its 'previewURL'
        const response = await fetch(attachment.previewUrl, {
          method: 'GET',
          headers: {
            // the token here should the same one from chat initialization
            'Authorization': 'Bearer ' + tokenString,
          },
        });
        // the response would be in image blob we can render it directly
        return {
          id: attachment.id,
          content: await response.blob(),
        };
      }),
  );
  result.forEach((imageResult) => {
    const urlCreator = window.URL || window.webkitURL;
    const url = urlCreator.createObjectURL(imageResult.content);
    // look up the image ID and replace its 'src' with object URL
    document.getElementById(imageResult.id).src = url;
  });
}

Noticing in this example, we've created two helper functions - fetchPreviewImages and setImgHandler - where the first one fetches preview image directly from the previewURL provided in each ChatAttachment object with an auth header. Similarly, we set up a onclick event for each image in the function setImgHandler, and in the event handler, we fetch a full scale image from property url from the ChatAttachment object with an auth header.

Another thing we need to do is to expose token on to the global level since we need to construct an auth header with it. So we need to modify the following code:

// new variable for token string
var tokenString = '';

async function init() {

   ....
   
   let tokenResponse = await identityClient.getToken(identityResponse, [
      "voip",
      "chat"
	]);
	const { token, expiresOn } = tokenResponse;
   
   // save to token string
   tokenString = token;
   
   ...
}

To show full scale image in an overlay, we need to add a new component as well:


<div class="overlay" id="overlay-container">
   <div class="content">
      <img id="full-scale-image" src="" alt="" />
   </div>
</div>

with some CSS:


/* let's make chat popup scrollable */
.chat-popup {

   ...
   
   max-height: 650px;
   overflow-y: scroll;
}

 .overlay {
    position: fixed; 
    width: 100%; 
    height: 100%;
    background: rgba(0, 0, 0, .7);
    top: 0;
    left: 0;
    z-index: 100;
 }

.overlay .content {
   position: fixed; 
   width: 100%;
   height: 100%;
   text-align: center;
   overflow: hidden;
   z-index: 100;
   margin: auto;
   background-color: rgba(0, 0, 0, .7);
}

.overlay img {
   position: absolute;
   display: block;
   max-height: 90%;
   max-width: 90%;
   top: 50%;
   left: 50%;
   transform: translate(-50%, -50%);
}

#overlay-container {
   display: none
}

Now we have an overlay set up, it's time to work on the logic to render full scale images. Recall that we've created an onClick event handler to call a function fetchFullScaleImage:


const overlayContainer = document.getElementById('overlay-container');
const loadingImageOverlay = document.getElementById('full-scale-image');

function fetchFullScaleImage(e, imageAttachments) {
  // get the image ID from the clicked image element
  const link = imageAttachments.filter((attachment) =>
    attachment.id === e.target.id)[0].url;
  loadingImageOverlay.src = '';
  
  // fetch the image
  fetch(link, {
    method: 'GET',
    headers: {'Authorization': 'Bearer ' + tokenString},
  }).then(async (result) => {
   
    // now we set image blob to our overlay element
    const content = await result.blob();
    const urlCreator = window.URL || window.webkitURL;
    const url = urlCreator.createObjectURL(content);
    loadingImageOverlay.src = url;
  });
  // show overlay
  overlayContainer.style.display = 'block';
}

One last thing we want to add is the ability to dismiss the overlay when clicking on the image:

loadingImageOverlay.addEventListener('click', () => {
  overlayContainer.style.display = 'none';
});

Now we've concluded all the changes we need to render inline images for messages coming from real time notifications.

Run the code

Webpack users can use the webpack-dev-server to build and run your app. Run the following command to bundle your application host on a local webserver:

npx webpack-dev-server --entry ./client.js --output bundle.js --debug --devtool inline-source-map

Demo

Open your browser and navigate to http://localhost:8080/. Enter the meeting URL and the thread ID. Send some inline images from Teams client like this:

A screenshot of Teams client shown a sent message reads: Here are some ideas, let me know what you think! The message also contains two inline images of room interior mockups.

Then you should see the new message being rendered along with preview images:

A screenshot of sample app shown an incoming message with inline images being presented.

Upon clicking the preview image by the Azure Communication Services user, an overlay would be shown with the full scale image sent by the Teams user:

A screenshot of sample app shown an overlay of a full scale image being presented.

In this tutorial, you learn how to enable inline image support using the Azure Communication Services Chat SDK for C#.

Prerequisites

Goal

  • Grab the previewUri for inline image attachments

Handle inline images for new messages

In the quickstart, we poll for messages and append new messages to the messageList. We build on this functionality later to include parsing and fetching of the inline images.

  CommunicationUserIdentifier currentUser = new(user_Id_);
  AsyncPageable<ChatMessage> allMessages = chatThreadClient.GetMessagesAsync();
  SortedDictionary<long, string> messageList = [];
  int textMessages = 0;
  await foreach (ChatMessage message in allMessages)
  {
      if (message.Type == ChatMessageType.Html || message.Type == ChatMessageType.Text)
      {
          textMessages++;
          var userPrefix = message.Sender.Equals(currentUser) ? "[you]:" : "";
          var strippedMessage = StripHtml(message.Content.Message);
          messageList.Add(long.Parse(message.SequenceId), $"{userPrefix}{strippedMessage}");
      }
  }

From incoming event of type ChatMessageReceivedEvent, there's a property named attachments, which contains information about inline image, and it's all we need to render inline images in our UI:

public class ChatAttachment
{
    public ChatAttachment(string id, ChatAttachmentType attachmentType)
    public ChatAttachmentType AttachmentType { get }
    public string Id { get }
    public string Name { get }
    public System.Uri PreviewUrl { get }
    public System.Uri Url { get }
}

public struct ChatAttachmentType : System.IEquatable<AttachmentType>
{
    public ChatAttachmentType(string value)
    public static File { get }
    public static Image { get }
}

As an example, the following JSON is an example of what ChatAttachment might look like for an image attachment:

"attachments": [
    {
        "id": "9d89acb2-c4e4-4cab-b94a-7c12a61afe30",
        "attachmentType": "image",
        "name": "Screenshot.png",
        "url": "https://contoso.communication.azure.com/chat/threads/19:9d89acb29d89acb2@thread.v2/images/9d89acb2-c4e4-4cab-b94a-7c12a61afe30/views/original?api-version=2023-11-03",
        "previewUrl": "https://contoso.communication.azure.com/chat/threads/19:9d89acb29d89acb2@thread.v2/images/9d89acb2-c4e4-4cab-b94a-7c12a61afe30/views/small?api-version=2023-11-03"
      }
]

Now let's go back and replace the code to add extra logic to parse and fetch the image attachments.

  CommunicationUserIdentifier currentUser = new(user_Id_);
  AsyncPageable<ChatMessage> allMessages = chatThreadClient.GetMessagesAsync();
  SortedDictionary<long, string> messageList = [];
  int textMessages = 0;
  await foreach (ChatMessage message in allMessages)
  {
      // Get message attachments that are of type 'image'
      IEnumerable<ChatAttachment> imageAttachments = message.Content.Attachments.Where(x => x.AttachmentType == ChatAttachmentType.Image);

      // Fetch image and render
      var chatAttachmentImageUris = new List<Uri>();
      foreach (ChatAttachment imageAttachment in imageAttachments)
      {
          client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", communicationTokenCredential.GetToken().Token);
          var response = await client.GetAsync(imageAttachment.PreviewUri);
          var randomAccessStream = await response.Content.ReadAsStreamAsync();
          await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, async () =>
          {
              var bitmapImage = new BitmapImage();
              await bitmapImage.SetSourceAsync(randomAccessStream.AsRandomAccessStream());
              InlineImage.Source = bitmapImage;
          });

          chatAttachmentImageUris.Add(imageAttachment.PreviewUri);
      }

      // Build message list
      if (message.Type == ChatMessageType.Html || message.Type == ChatMessageType.Text)
      {
          textMessages++;
          var userPrefix = message.Sender.Equals(currentUser) ? "[you]:" : "";
          var strippedMessage = StripHtml(message.Content.Message);
          var chatAttachments = chatAttachmentImageUris.Count > 0 ? "[Attachments]:\n" + string.Join(",\n", chatAttachmentImageUris) : "";
          messageList.Add(long.Parse(message.SequenceId), $"{userPrefix}{strippedMessage}\n{chatAttachments}");
      }

Noticing in this example, we grab all attachments from the message of type 'Image' and then we fetch each one of the images. We must use our 'Token' in the 'Bearer' portion of the request header for authorization purposes. Once the image is downloaded, we can assign it to the 'InlineImage' element of the view.

We also include a list of the attachment URIs to be shown along with the message in text message list.

Demo

  • Run the application from the IDE.
  • Enter a Teams meeting link.
  • Join meeting.
  • Admit user on the Teams side.
  • Send a message from the Teams side with an image.
  • The url included with the message is displayed in the message list and the last received image rendered at the bottom of the window.

Next steps

For more information, see the following articles: