1. Chart 1 – Total amount and total margin move linearly as expected, though as mentioned later there are several customers of interest who fall in negative margin territory. DIG DEEPER.
2. Chart 2 – Number of orders doesn’t really have a relationship with total amount. We have customers with more than 400 orders processed with total amounts spread over a wide range from 20k to 50k. UNCELAR PICTURE.
3. Chart 3 – Percentage of total orders that are discounted falls as one moves towards the right territory of large total amount. Conversely, one can see that for orders with amount less than 10k, the percent of orders discounted could be anywhere between 0 -100%. UNCLEAR.
4. Chart 4 – This clearly shows that customers whose last orders came in quite a while ago do a lot less business with Contoso. Conversely, customers who do a lot of business with Contoso have placed orders quite recently. NO SURPSIES HERE
5. Chart 5 – This one also clearly shows that discount percent on discounted orders has a lot of variation only when we look at marginal customers. When we look at customers who do significant business with Contoso, mean discount is quite close to zero. HEALTHY
6. Chart 6 – This one also clearly shows that customers who do a lot less business with Contoso can have a lot of gap between their orders. Conversely customers who have little gap between orders are more likely to make high spend with Contoso. NO SURPRISES HERE

1. About 100,000 customers buy in any given year, some just once, others many times
2. Customers are divided into 20 different markets for easy management
3. Customers are also divided into customer types depending on products they buy
4. Customer accounts themselves are ordered in a hierarchy since many customers use different accounts to order based on sites they order from, or their own ownership structure etc.

1. Do some basic business analysis - get to know the customer, get to know the data, do some simple charts to see how revenue changes, margins change, what products sell the most etc.,
2. Then do customer segmentation to group similar customers together and away from other groups,
3. Then identify some segments that are more price conscious than others,
4. Develop a machine learning model that calculates price elasticity
5. Identify high selling products (X), that most customers buy
6. Use this ML model to estimate price elasticity per segment for high selling products
7. Then find a couple frequently bought together products (Y) with each of the high selling products using FBT model
8. Find a couple recommended products (Z) for each of the high selling products using recommendation machine learning model
9. Take X,Y,Z together and estimate cross-price elasticity using price elasticity model; so you can say what happens to demand for Y and Z when price for X is lowered
10. Then predict demand for X,Y,Z using demand forecast model that takes into account price elasticity and cross price-elasticity
11. Now you know how much of each is likely to sell for different price points of X, so now calculate revenue, margin (assuming you have costs)
12. Offer this to a price professional who has the knowledge & authority to decide which price to chose for which segment of customers for next promotion
13. Go next step, and use some optimization capability to automatically optimize, but in our experience, pricing experts want to have control so they are not looking to automate this step
14. Run the actual promotion, collect actual sales data and feed it to all the models so they stay current
15. In the end, use customer churn machine learning model to find customers at risk, see what segment they belong to, estimate if price is putting them at risk, evaluate if they should be offered something exclusively in next promotion run

A little pricing theory

The pricing data schema and use case

Input data preparation

Using the pricing services

Building the model and examining elasticities

Interpreting Cross-Elasticities

Simulating the effect of promotion

• Data and Model  (Build the model)
• Elasticity              (Retrieve dataframe and visual of self-elasticities)
• Cross-Elasticity    (Retrieve dataframe and visual of cross-elasticities)
• Forecasts             (Forecasted demands at current, fixed prices)
• Demand Curve    (Forecasted demand at varied prices)
• Promotion Effect (Effect on hypothetical promo on all related products)
• Retrospective      (How good were past forecasts? Confidence bounds.)
• Outliers                (Examine worst forecast misses.)

Pricing engine Azure architecture

Deployment

The data science: elasticity model

Core methodology

• Seasonal indicators
• The price itself, in absolute, and relative to other products in the same category
• Lagged changes in price and demand
• The recent trend in the product’s category prices and demands
• Indicator variables for the product category and site, channel
• The recent demand and pricing trends at the site and in the channel
• Indicator variables for the products category
• Various interactions between these variables, especially between price change and site, channel and category

Data acquisition and cleaning

Conclusions

HTS parameter search space

ts_method        <- c("arima", "ets", "rw")
hts_method       <- c("bu", "comb", "tdgsa", "tdgsf", "tdfp")
comb_hts_weights <- c("sd", "none", "nseries”)
search_space     <- expand.grid(ts_method, hts_method, comb_weights, stringsAsFactors = FALSE)

Iterative optimization

Architecture

# Sets computing context to Spark
rxSetComputeContext(RxSpark())

rxResult <- rxExec(run_hierarchical_forecast, 
                   rxElemArg(search_space), 
                   … )

Results

References

1. Package 'hts'
2. Forecasting: principles and practice - Chapter 9.4 Forecasting hierarchical or grouped time series
3. Sample R Script
4. HDInsight
5. Microsoft R Server

• Product Recommendations can be seen only on the cloud POS
• Product Recommendations can be seen only on the MPOS in online mode
• These can be seen when you view details of a product, they will appear automatically next to 'related products'
• These will also appear on the customer detail page
• These can be seen when you are making transactions on cloud POS or on MPOS when it's in online mode

1. Azure data factory - for sending data (catalog & usage) to the blob storage
2. Azure blob storage - for storing raw and processed data
3. Hive scripts - for cleansing raw data to reduce the bias from 'unidentified' customers buying through POS in the scenario when you want the API to make personalized recommendations
4. All the components for spinning up the recommendation API itself
5. Some other components for making sure all services involved are properly authenticated and have the right credentials

1. When your environment is being created through LCS, if you chose you want to use machine learning, recommendations related components are installed in your environment

1. Go to entity store, find the entity 'Retail sales' and hit refresh. This will pick the demo data (or your data) from your operational DB and move it to Entity store
2. Go to Retail parameters --> Machine learning and turn 'enable recommendations' ON
3. If you will like to see recommendations while making POS transactions, go to screen layout designer and drop recommendation control to transactions screen
4. Run the global configuration job - 1110 (to see recommendations on POS) and channel configuration job 1070 (to reflect changes made to POS screen layout designer)

1. When you refresh the entity store entity, following happens
   • Data in the format required by the API is extracted from AX DB and send to Entity store
   • This data is then used by ADF copy activity and sent to blob storage
   • If there is cleansing required, Hive scripts are run as part of ADF activities and cleansed data is stored in blob storage
   • Data from blob storage is used by the API to train a recommendation model
2. When you turn 'enable recommendations' and run the configuration jobs following happens
   • Model credentials and ID are picked up from the API and stored in AX DB, in the web.config for AOS as well as in the retail server
   • Model credentials and ID are made available to CRT so calls for product recommendations from CPOS and modern POS can be honored

1. MEO
2. DutchCrafters
3. JJ Food

1. MEO
2. DutchCrafters
3. JJ Food

Provisioning a Data Science VM

Microsoft R Server

Example use case: exploration of product demand data

in.file <- file.path('C:/shared/demand_dataset.csv')
xdf.file <- "demand_dataset.xdf"

# Read data from csv 
dataset <- rxImport(inData = in.file,
                    outFile = xdf.file,
                    overwrite = TRUE,
                    type = "text",
                    missingValueString = "N/A",
                    stringsAsFactors = TRUE)

> dim(dataset)
[1] 41034690       20

# Get variable info
rxGetVarInfo(dataset)

> # Get summary stats
> rxSummary(~ volume, data = dataset)

Rows Processed: 41034690 
Call:
rxSummary(formula = ~volume, data = dataset, reportProgress = 1)

Summary Statistics Results for: ~volume
Data: dataset (RxXdfData Data Source)
File name: demand_dataset.xdf
Number of valid observations: 41034690 

Name        Mean     StdDev   Min Max       ValidObs MissingObs
volume      48214.28 219421.2 0   135384480 41034690 0

# Load dplyrXdf package
library(dplyrXdf)

# Aggregate volume over time (per yyyy-mm-dd)
volume.by.ymd <- dataset %>%
    select(date, volume) %>%
    group_by(date) %>%
    summarise(volume = sum(volume)) %>%
    as.data.frame

# Plot the aggregated data
ggplot(volume.by.ymd, aes(date, volume)) + geom_line() + ggtitle("Daily volume over time")

# Aggregate volume over time (per yyyy-mm)
volume.by.ym <- volume.by.ymd %>%
    select(date, volume) %>%
    mutate(date = as.Date(as.yearmon(date))) %>%
    group_by(date) %>%
    summarise(volume = sum(volume)) %>%
    as.data.frame

# plot the aggregated data
ggplot(volume.by.ym, aes(date, volume)) + geom_line() + ggtitle("Monthly volume over time")

Further Reading

References

1. Introduction to the cloud-based Data Science Virtual Machine for Linux and Windows
2. Data Science Virtual Machine - Product page
3. Provision the Microsoft Data Science Virtual Machine
4. Ten things you can do on the Data science Virtual Machine
5. Microsoft R Server
6. dplyrXdf - R package for xdf data manipulation