Pragmatic Development https://docs.microsoft.com/archive/blogs/pragdev/feed.xml Customer focused development Wed, 17 Apr 2019 03:43:04 GMT en-US hourly 1 Enterprise Bot Architecture https://docs.microsoft.com/archive/blogs/pragdev/enterprise-bot-architecture Sat, 10 Feb 2018 00:23:34 GMT https://blogs.msdn.microsoft.com/pragdev/?p=76 Summary Many organizations are seeing the productivity gains of utilizing bots across the enterprise.  In many cases a department will create a bot as a proof of concept that turns out to be a success and other departments quickly see value in applying bot technology to their domain. However, this can lead to issues if there is not a cohesive enterprise bot strategy and architecture in place.  In this post we will look at some of the scaling issues enterprises face as they deploy more bots and a sample architecture that can help alleviate those issues and maximize the productivity gains for the enterprise.

Typical Scenario

Acme Co. is a large, global company with about 50,000 employees world wide. They have a help desk that services all their employees and even provides first level support for their customers.  Their help desk is staffed with a mix of full time employees and contractors from a help desk service provider.  Call volume has been relatively consistent for the past several years with the occasional spikes when new technology or products are introduced. In the coming year, Acme is migrating most of their user base to the latest version of the Windows operating system and deploying two large overhauls to their customer facing applications..  Based on historical patterns this will cause a significant spike in calls to the help desk and incur additional expenses and lower customer satisfaction.  To proactively address these issues, Acme Co. develops a bot to allow employees and customers to find key information and perform several self-help tasks.  In addition, Acme Co. takes the time to incorporate password reset functionality into their bot as their metrics show password resets are the number one reason for help desk calls (this is very typical across all industries). Acme Co., quickly creates a MVP and performs a controlled rollout.  Their rollout shows the bot will meet or exceed their expectations and they perform a complete rollout to their internal and external user base.  As they perform their migration to Windows 10 and deploy their updates to their customer applications, they not only see the expected increase mitigated, but an actual decline in overall call volume as 90% of the password reset cases can now be handled by the bot.  The project is a success and several other teams are keen on utilizing bots in their area and make plans to deploy MVPs. Fast forward several months, and there are more than half-dozen bots deployed at Acme Co.  In addition to the original help desk bot, which was expanded to cover more topics, there are bots for human resource, legal department, IT infrastructure, travel department, and a few more.  Several more teams at Acme Co. are looing to have their bots go to production by the end of the year.

Problems Arise

Quickly Acme’s CTO realizes that bot technology has helped, but the ad-hoc deployment of bots across the enterprise has created several problems that are creating architectural and technical debt.

User Experience

The user experience for the employees is starting to suffer.  Users have to remember different bots and their locations on the intranet to access them.  In addition, some bots allow access through additional channels such as SMS.  When users connect to various bots, there is no consistent dialog flows and expected behaviors.  Users have to learn the quirks of each bot individually.  Bots from various departments also have very inconsistent user interfaces that the users sometimes find confusing.

Discovery

Users are quickly finding that they don’t know what bots are out there until some time has passed.  While each bot is announced on email and the intranet home site for a time, users are inundated with various messaging and often time miss these announcements.  They typically find out about bots through word of mouth.  This can be very difficult for employees in the remote offices.  This means the enterprise is not getting the maximum productivity from their bots.

Bot Maintainability

While various teams did share some code, there is no single bot code repository with templates for the teams to reuse.  As one team utilizes a new feature in the core bot framework or fixes a bug in their bot, the other bots do not get the benefit of this work.  When a department starts up a new bot they end up using another department's bot as a starting point, but they quickly diverge.  lastly, many teams can get funding to develop a bot through the Acme’s global business services or through a partner, but they do not have the funding or skillsets on their team to maintain the bot through its lifecycle.

Inconsistent Security

The bots all have various levels of security and have to go through more comprehensive and expensive audits since much of their code base is unique.  Some bots have enabled channels other than the Intranet to allow users to connect to the bot through SMS or messaging systems such as Skype.

Inconsistent API Access

Many of the bots need to interface to internal or 3rd party systems.  They have not been consistent in how they integrate with these systems.  Some of the bots utilize the systems’ available APIs directly, others utilize a micro-service architecture, and some directly access databases.  This has caused a lot of additional development and in some cases tight system coupling and dependencies that are undesirable long term. A visual representation of what this disjointed architecture would be similar to this: image

Finding A Solution

A solid enterprise bot architecture and strategy should be able to solve all the problems listed above and provide some additional benefits such as quick deployment of new bots into the eco-system.  The approach we will present here is to create a special bot that will act as the single point of entry into the enterprise bot eco-system.  We will call this bot the master bot, and all the other bots will now be child bots. The master bot will be the single point where ACL is handled and will provide a consistent user interface back to the user.  API access will be done through micro-services, Azure Logic Apps, or a combination.  All the child bots will use a shared bot template from a central repository.  The revised architecture would look something like this: image

Managing The Conversation

The master bot will be responsible for managing the conversation with the end user and routing the conversation to the appropriate child bot.  While talking to a child bot, the conversation data still flows through the master bot so that it can look for commands or cues that it needs to step into the conversation and perhaps route the conversation to a different bot. A simple approach that scales well with this approach is to formulate discussions around various topics.  Each child bot will register itself with the master bot and tell the master bot which topic(s) it can handle.  The master bot can then present a list of common or all topics to the user to help guide them to the right child bot.  The master bot can then look for a command such as ‘switch topics” or “show topics” as a cue to step back into the conversation and take over.

Resolving Core Issues

User Experience

The user experience is greatly improved by having the master bot provide a consistent user interface regardless of which bot is accessed.  Company branding is easily enforced and updated across all the bots.  The master bot can provide a common set of commands that will always  be available allowing the user to quickly perform command actions.  The master bot can provide a consistent set of user interface controls across all channels that the  bots can rely upon.  The child bots still have quite a bit of flexibility in defining custom cards with adaptive cards.

Discovery

Discovery of new bots and their capabilities is greatly enhanced in this model.  Since users will be going to the master bot for many tasks, when a new bot is deployed the master bot can display a new welcome message informing the user of a new capability.  In addition, when the user asks to see more topics, new bots will automatically show up in the list.

Bot Maintainability & Creation

By sharing a common template, the individual child bots can receive updates by syncing with the core source repository.  They will get core bug fixes, new features, and security enhancements as a result.  By having this shared template, departments can create bots at their own speed and deploy them into the eco-system easily.

Managed Security

The master bot and the child bots can have security easily managed by standard security tools.  For example, each child bot can be registered in Azure Active Directory and the master bot can simply query AAD with the current user authorization to see what bots the user should have access to.  The master bot would also not display topics for bots that the current user does not have access to.  This makes for management of access to the bots very easy to control and fits within existing security practices and tools.

Inconsistent API Access

By using a common template and utilizing Azure Logic Apps or custom micro services, the various bots will have a consistent way to get access to other systems and reduce coupling between the bots and those systems.   A common class library in the template can handle various tasks such as managing access tokens and services accounts by using services such as Azure Key Vault.  The bot writer is simply exposed a simple API to the necessary system and does not have to worry about the details.

In Action

Here is an example video of how a master / child bot pattern would work: [video width="365" height="794" mp4="https://msdnshared.blob.core.windows.net/media/2018/02/masterBotSample.mp4"][/video]

Final Thoughts

Bots are becoming a valuable way for businesses to enhance productivity and generate new revenue.  As bots continue to be deployed across enterprises we will see the creation and refinement of new enterprise wide patterns to manage bots in the enterprise.  By leveraging these patterns early on, enterprises will avoid significant technical debt and user confusion. Sample open source code for the master bot pattern above will be available a few weeks after this posting has been made available.  Please subscribe to get notified when the code is made available!]]> Bridging The Gap Between Technology & Business (part 2) https://docs.microsoft.com/archive/blogs/pragdev/bridging-the-gap-between-technology-business-part-2 Tue, 06 Feb 2018 22:37:58 GMT https://blogs.msdn.microsoft.com/pragdev/?p=35 In the first part of this series we discussed achieving productivity savings for an organization.  In this part, we will focus on an utilizing technology to grow revenue for an organization.  There is no better way to serve most clients than by helping achieve growth for their business.  If you can align a technology solution with business growth, you will quickly become a valued partner.

Revenue

Lets take a look at a real world scenario to why business growth is should always be a focus item when servicing your clients or employer.  This is example is from a real world fortune 500 company that is known throughout the world.  Currently I am working on several different projects for this company that revolve around productivity, but I am pushing hard to get into areas of revenue growth for reason’s which will become obvious.  Here is a break down of this company’s past and predicted (numbers are slightly different to protect client identity, but overall trend is accurate):

Revenue 2012: $49 billion
Revenue 2016: $43 billion
Revenue 2020 (predicted): $33 – $35 billion

This company has faced a challenge in the market place.  Its primary line of products have fallen out of favor due to a shift in consumers’ attitudes towards healthier lifestyles.  Even though the company has a fairly diverse set of products that are attractive across all demographics, they have not been able to maintain overall market share.  This company is going through sizable layoffs every quarter to stay ahead of the revenue erosion to try to maintain margin.

Productivity gains are are a critical part of this company’s plan, but it is readily apparent that this company cannot cut costs only to survive.  They will have to find additional revenue growth or at least stabilize existing revenue quickly otherwise shareholders will eventually force the company to take drastic actions.

Fundamentals

As a technologist, what can we propose to this company to help stabilize it’s business? At first it may seem like we will have to get extremely deep into their product mix, research, marketing, and overall strategy to make a difference.  Eventually, that would be a great place to be as a partner, but you can still provide value by solving the basic problems any company in this situation has:

  • Stop the bleeding of existing customers
  • Identify potential new customers
  • Identify potential new investments / acquisitions that will grow revenue

Each one of these items can be solved by various technology solutions, but all of them will require the company to undergo fundamental change in the way they conduct business.  The technology can show them the path, but in many cases it will be an uncomfortable path for the customer to take and there will be resistance to change.  To be successful you will need to overcome that resistance while implementing the technology solution.

Some solutions to each of these key challenges are:

  • Execute a churn analysis and implement a customer retention strategy
  • Market and customer segmentation analysis to identify underserved customer segments
  • Predictive market and competitor analysis to identify new strategic opportunities

Technology can help with each of these analysis to give more accurate and timely results.  In many cases the summary analysis can be done with publicly available data.  Coming to a client meeting with such an analysis can go a long way to showing the customer that you invested in seeing them succeed and not just your own interest.  Be prepared to get some feedback that your are missing key pieces of information to make the analysis accurate (which is probable).  Take the opportunity to offer to include that data in the analysis and give a quick turnaround.

Implementation

The initial set of analysis that you would want to perform for the client are well known and easily transferrable across different clients and industries.  Our Azure marketplace has many Microsoft and 3rd party models already created to perform the various analysis and you will just need to plug in the relevant data and desired metrics for your particular clients.

Some key models to utilize:

  • Customer predictive churn analysis models.  For these you will need to some sample data from the client most likely.
  • Customer segmentation and trend models.  Various consumer research firms make subsets of data available that you can utilize (i.e. Nielsen).  They also provide standard customer segmentation models that you will want to use so that you are talking in the same terms as the client’s marketing and sales teams. In addition government and health agencies also provide such data.  You may also find valuable datasets on sites such as Kaggle.com
  • Predictive models to show changes in customer behavior at a market level.  You can utilize various aggregate sales data sources and sources typically used by financial firms to gauge new investments.  In particular you want to highlight smaller companies that appear to moving fast in small, but growing segments.  You want your clients to invest in companies when they are worth $100 million dollars and not when they are worth $5 billion and have already saturated a market niche.

Bringing It Together

Ultimately you bring all of this together into a concise story for your customer.  For many situations the following statement would be a good starting point:

Our goal is to improve your key business metrics by identifying productivity and revenue generation opportunities through application of key technologies to analyze your internal performance metrics and the behavior of current and potential customers in the marketplace

From the initial engagement through maintenance of deployed machine learning models, you want to be consistent in your message that our success is always tied to the success of our clients.  Don’t sell technology solutions, sell customer success stories!

]]> Bridging The Gap Between Technology & Business https://docs.microsoft.com/archive/blogs/pragdev/bridging-the-gap-between-technology-business Wed, 18 Oct 2017 21:35:20 GMT https://blogs.msdn.microsoft.com/pragdev/?p=26 If there is one lesson many technology professionals need to learn is that you must always ensure the technology activities are aligned with the business goals and objectives.  At first this seems easy, but in the vast majority of the cases, technologists are not looking to the fundamental business needs.  Technologists tend to focus at the project or program level and not at the larger impact to the business.  The primary drivers for business can be distilled down to productivity and revenue, and technologists need to understand how their projects and activities impact those two areas to have consistent success in delighting clients and managing their careers.   In this first part of the examining the relationship between technology and business, the impact to productivity will be explored.

Productivity

Do more with less is a simple statement, but it has significant ramifications for businesses.  Successful businesses have to keep tight eye on costs and want to find the equilibrium of just the right amount of resources to produce the output demanded without causing issues in the output that would reduce revenue.  Resource expenses include labor, recurring technology costs, office space, capital expenditures, etc…  Every technology activity at a business will have a direct impact to productivity in some fashion.

Businesses expect to improve operational efficiency year over year, which will result in productivity gains.  Typically a company will decide how they want to apply those gains by either reducing operational costs or investing in new work activities to help drive new revenue or further productivity gains.  Unfortunately, this does often lead to a reduction in force if the business has not identified new activities for surplus labor to take on or if the skillsets of the labor do not match what is needed for the new work.  Surplus labor may b allocated to new work activities to further increase productivity, but eventually the gains will be realized by reducing expenses to improve margins, pay out dividends, etc…

Example Scenario

So how does technology play into this? Let’s look at an example of a current technology being applied to a common business problem:

ACME Co, has enlisted its technology partner, Blue Co, to automate its help desk through the use of the Microsoft Bot Framework.  Based on a use case analysis, the bot should be able to reduce tickets that need human intervention by 30-40%.

On the surface this seems like a straightforward project that will yield good results for Acme and the users of the help desk.  However, much more detail is needed to fully understand the impact of this project on the productivity of the business.  For the sake of analysis, we will assume the majority of the calls are for password resets and related issues (which is actually very typical in the real world). Here is some additional information that will needed to do a full analysis of the impacts to productivity of this project:

  • Average cost to the company per call to the help desk.
  • Average cost of labor for the callers to the help desk
  • How much is the partner charging to build the system
  • The frequency and clustering of the calls over time (any externalities causing patterns such as quarterly updates?)
  • Any future technology rollouts that would impact the frequency of the calls (For example, biometric security?)
  • How much will it cost to communicate to the help desk users that they should use the bot?
  • How effective will the communication be and how resistant to change will the user base be?
  • Any new recurring costs for the bot deployment (cloud costs, networking, etc..)

Financial Analysis

Lets look at an extreme example where Acme’s primary callers to the help desk are lawyers that bill $450 a hour and the help desk employees cost the company $15 a hour.  In a typical year, there are 10,000 calls to the help desk to reset passwords. The communication costs are fixed at $0.10 cents a minute.  Current call analysis shows that when a lawyer calls the help desk for a password reset it takes approximately 90 seconds for the agent to verbally verify the identity of the lawyer an reset the password.  It is shown that lawyers that were shown the bot take approximately 60 seconds to utilize the bot to do the password reset (they have to open a browser, navigate to the bot, and proceed through several security questions).  The cost to host and support the bot is fixed at $10,000 a year

So the initial cost of a password reset for a call versus the bot would be:

($15 + $450) / (3600 * 90) + (.10 * 1.5) = $11.78 * 10,000 = $117,800 year

$450 / 3600 * 60 +  (10,000 / $10,000) = $8.50 * 10,000 = $85,000 year

The cost per year has been reduced 28% which adds up to a yearly cost savings of $32,800 if we can fully realize the productivity by reducing help desk agent hours worked (either by reducing staff levels or hours given to existing staff)).  Blue Co., is charging a development fee of $20,000 to build and fully deploy the bot in 90 days.  Internal communications is expected to cost $5000.  So it appears the help desk bot will have a quick turnaround on ROI of less than 12 months after it is deployed.

Everything looks good! Or does it? Lets take a look at this analysis again when taking into account these factors:

  • Similar rollouts have shown only 75% of users will utilize the new system after a year and the remaining 25% will insist on calling the helpdesk for issues that could be resolved by the bot.
  • Acme Co, is rolling out Windows 10 within the next 18 months and looks to have its deployment complete within 24 months to be lined up with its next hardware refresh.  The simplified bio-metric security is expected to reduce password reset related issues by over 90%.

Users behavior patterns can be very consist across companies and industries, so lets revisit our financial outlook with projected user behavior taken into account.

Our costs of operating the bot now become:

($11.78 * 2500) + ($8.50 * 7500) =  $93,200 year

Our yearly cost savings have now dropped to $24,600 a year.  Our total development and internal communication costs are $25,000 so now it will take just over a year for the project to deliver a return on the initial investment if everything goes right and there are no delays or variance in user behavior.  However, we have the Windows 10 rollout starting within 18 months and is expected to be complete in 24 months.  This means that the bot project only has a full 6 months of actual returns, with almost all returns of the project eliminated 6 months after that.

Conclusion

If you were a business analyst or program manager at Acme Co, what would you now do with the bot automation project after running the above analysis.  The decision is pretty clear that the bot project is a very low return on investment once all the other information is taken into account.  Even if Acme Co is looking at using automation of the help desk as a first step into utilization of bots, it would still make sense to find another area to use the technology that will have longer usefulness and productivity returns.

Many projects are proposed and even completed with only completing the initial part of the analysis.  In many cases the people doing the analysis may not have access or knowledge of the other activities going on in the business that will impact the ultimate value of the project.  This is why it is important to dig deeper when analyzing the business impact of a project and recognize when you don’t have all the necessary information to gauge the true impact of the project.

Coming Soon: Part 2 – Discussion on revenue.

]]>