This example scenario shows how using a dedicated search service can dramatically increase the relevance of search results for your e-commerce customers.
Architecture
Download a Visio file of this architecture.
Workflow
This scenario covers an e-commerce solution where customers can search through a product catalog.
- Customers navigate to the e-commerce web application from any device.
- The product catalog is maintained in an Azure SQL Database for transactional processing.
- Azure Cognitive Search uses a search indexer to automatically keep its search index up-to-date through integrated change tracking.
- Customer's search queries are offloaded to the Azure Cognitive Search service, which processes the query and returns the most relevant results.
- As an alternative to a web-based search experience, customers can also use a conversational bot in social media or straight from digital assistants to search for products and incrementally refine their search query and results.
- Optionally, the skillset feature can be used to apply artificial intelligence for even smarter processing.
Components
- App Services - Web Apps hosts web applications allowing autoscale and high availability without having to manage infrastructure.
- SQL Database is a general-purpose relational database-managed service in Microsoft Azure that supports structures such as relational data, JSON, spatial, and XML.
- Azure Cognitive Search is a search-as-a-service cloud solution that provides a rich search experience over private, heterogeneous content in web, mobile, and enterprise applications.
- Bot Service provides tools to build, test, deploy, and manage intelligent bots.
- Cognitive Services lets you use intelligent algorithms to see, hear, speak, understand, and interpret your user needs through natural methods of communication.
Alternatives
- You could use in-database search capabilities, for example, through SQL Server full-text search, but then your transactional store also processes queries (increasing the need for processing power) and the search capabilities inside the database are more limited.
- You could host the open-source Apache Lucene (on which Cognitive Search is built) on Azure Virtual Machines, but then you're back to managing Infrastructure-as-a-Service (IaaS) and don't benefit from the many features that Cognitive Search provides on top of Lucene.
- You could also consider deploying Elasticsearch from the Azure Marketplace, which is an alternative and capable search product from a third-party vendor, but also in this case you're running an IaaS workload.
Other options for the data tier include:
- Azure Cosmos DB - Microsoft's globally distributed, multi-model database. Azure Cosmos DB provides a platform to run other data models such as MongoDB, Cassandra, Graph data, or simple table storage. Cognitive Search also supports indexing the data from Azure Cosmos DB directly.
Scenario details
Search is the primary mechanism through which customers find and ultimately purchase products, making it essential that search results are relevant to the intent of the search query, and that the end-to-end search experience matches that of search giants by providing near-instant results, linguistic analysis, geo-location matching, filtering, faceting, autocomplete, hit highlighting, and so on.
Imagine a typical e-commerce web application with product data stored in a relational database like SQL Server or Azure SQL Database. Search queries are often handled inside the database using LIKE queries or Full-Text Search features. By using Azure Cognitive Search instead, you free up your operational database from the query processing and you can easily start taking advantage of those hard-to-implement features that provide your customers with the best possible search experience. Also, because Cognitive Search is a platform as a service (PaaS) component, you don't have to worry about managing infrastructure or becoming a search expert.
Potential use cases
This solution is optimized for the retail industry.
Other relevant use cases include:
- Finding real estate listings or stores near the user's physical location (for the facilities and real-estate industry).
- Searching for articles in a news site or looking for sports results, with a higher preference for more recent information (for the sports, media, and entertainment industries).
- Searching through large repositories for document-centric organizations, like policy makers and notaries.
Ultimately, any application that has some form of search functionality can benefit from a dedicated search service.
Considerations
These considerations implement the pillars of the Azure Well-Architected Framework, which is a set of guiding tenets that can be used to improve the quality of a workload. For more information, see Microsoft Azure Well-Architected Framework.
Scalability
The pricing tier of the Azure Cognitive Search service is used mainly for capacity planning as it defines the maximum storage you get and how many partitions and replicas you can provision. Partitions allow you to index more documents and get higher write throughputs, whereas replicas provide more Queries-Per-Second (QPS) and High Availability.
You can dynamically change the number of partitions and replicas but it's not possible to change the pricing tier, so you should carefully consider the right tier for your target workload. If you need to change the tier anyway, you'll need to provision a new service side by side and reload your indexes there, at which point you can point your applications at the new service.
Availability
Cognitive Search provides a 99.9% availability SLA for reads (that is, querying) if you have at least two replicas, and for updates (that is, updating the search indexes) if you have at least three replicas. Therefore you should provision at least two replicas if you want your customers to be able to search reliably, and 3 if actual changes to the index should also be considered high availability operations.
If there's a need to make breaking changes to the index without downtime (for example, changing data types, deleting or renaming fields), the index will need to be rebuilt. Similar to changing service tier, this means creating a new index, repopulating it with the data, and then updating your applications to point at the new index.
Security
Cognitive Search is compliant with many security and data privacy standards, which makes it possible to be used in most industries.
For securing access to the service, Cognitive Search uses two types of keys: admin keys, which allow you to perform any task against the service, and query keys, which can only be used for read-only operations like querying. Typically, the application that performs the search doesn't update the index, so it should only be configured with a query key and not an admin key (especially if the search is performed from an end-user device like script running in a web browser).
Search Relevance
How successful your e-commerce application is depends largely on the relevance of the search results to your customers. Carefully tuning your search service to provide optimal results based on user research, or relying on built-in features such as search traffic analysis to understand your customer's search patterns allows you to make decisions based on data.
Typical ways to tune your search service include:
- Using scoring profiles to influence the relevance of search results, for example, based on which field matched the query, how recent the data is, the geographical distance to the user, ...
- Using Microsoft provided language analyzers that use an advanced Natural Language Processing (NLP) stack to better interpret queries
- Using custom analyzers to ensure your products are found correctly, especially if you want to search on non-language based information like a product's make and model.
Cost optimization
Cost optimization is about looking at ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Overview of the cost optimization pillar.
To explore the cost of running this scenario, all the services mentioned above are pre-configured in the cost calculator. To see how the pricing would change for your particular use case change the appropriate variables to match your expected usage.
We've provided three sample cost profiles based on amount of traffic you expect to handle:
- Small: In this profile, we're using a single
Standard S1Web App to host the website, the free tier of the Azure Bot service, a singleBasicsearch service, and aStandard S2SQL Database. - Medium: Here we're scaling up the Web App to two instances of the
Standard S3tier, upgrading the search service to aStandard S1tier, and using aStandard S6SQL Database. - Large: In the largest profile, we use four instances of a
Premium P2V2Web App, upgrade the Azure Bot service to theStandard S1tier (with 1.000.000 messages in Premium channels), use two units of theStandard S3search service, and aPremium P6SQL Database.
Deploy this scenario
To deploy a version of this scenario, you can follow this step-by-step tutorial that provides a .NET sample application that runs a job search web site. It demonstrates most of the Azure Cognitive Search features discussed thus far.
Contributors
This article is maintained by Microsoft. It was originally written by the following contributors.
Principal author:
- Jelle Druyts | Principal Customer Engineer
Next steps
To learn more about Azure Cognitive Search, visit the documentation center, check out the samples, or see a full-fledged demo site in action.
To find out more about other Azure components, see these resources:
- What is Azure SQL Database?
- App Service overview
- Azure Bot Service documentation
- What are Azure Cognitive Services?