Product Displays (larger grocer)

A national grocery chain wants to test 4 new product display layouts to determine which layout drives the most sales. However, sales are also influenced by the Store Manager, some managers are better at merchandising than others, and the week because sales naturally vary from week to week (promotions, pay cycles, weather, etc.).

Product Displays (different grocer)

A national grocery chain wants to test 4 new product display layouts to determine which layout drives the most sales. However, sales are also influenced by the Store Manager, some managers are better at merchandising than others. The stores will test each display only on weekends so that the baseline sales are equivalent for each layout.

Product Displays (new grocer)

A new grocery store wants to test 4 new product display layouts to determine which layout drives the most sales. They only have one store manager and will test each display twice using only Fridays since they know their customers shopping habits are similar on each Friday.

Verizon Plan Pricing

Verizon has recently implemented mix and match plans for each line. In order to optimize the plan pricing they tested 3 different versions of the “Get More” Plan. They tested the three different versions over the course of one month and measured the number of plan sign-ups during this time.

plan sign_up
A 0
A 0
A 0
A 0
A 0
A 0 
library(tidyverse)
df %>% group_by(plan, sign_up) %>% summarise(n=n()) %>% mutate(n/sum(n))
## # A tibble: 6 × 4
## # Groups:   plan [3]
##   plan  sign_up     n `n/sum(n)`
##   <chr>   <int> <int>      <dbl>
## 1 A           0  4763     0.973 
## 2 A           1   134     0.0274
## 3 B           0  4823     0.965 
## 4 B           1   177     0.0354
## 5 C           0  4857     0.971 
## 6 C           1   143     0.0286
test<-chisq.test(table(df$plan, df$sign_up))
test
## 
##  Pearson's Chi-squared test
## 
## data:  table(df$plan, df$sign_up)
## X-squared = 6.3038, df = 2, p-value = 0.04277
test$residuals
##    
##              0          1
##   A  0.2211868 -1.2475558
##   B -0.3536140  1.9944827
##   C  0.1347173 -0.7598436

Direct Mail Experiment

An article in the International Journal of Research in Marketing describes an experiment to test new ideas to increase direct mail sales by the credit card division of a financial services company. They know from experience that the interest rates are an important factor in attracting potential customers so they have decided to focus on factors involving both interest rates and fees. The factors tested are as follows:

The company measured the total responses to each offer.

A B responses
-1 -1 24
1 -1 33
-1 1 21
1 1 22
-1 -1 24
1 -1 33
-1 1 23
1 1 24

Model

reg<-lm(responses~(A+B)^2, data=df) #this is the same as lm(response.rate~A*B, data=df)
summary(reg)
## 
## Call:
## lm(formula = responses ~ (A + B)^2, data = df)
## 
## Residuals:
##          1          2          3          4          5          6          7 
## -1.316e-14  3.938e-16 -1.000e+00 -1.000e+00  1.305e-14 -5.344e-16  1.000e+00 
##          8 
##  1.000e+00 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  25.5000     0.3536  72.125 2.21e-07 ***
## A             2.5000     0.3536   7.071  0.00211 ** 
## B            -3.0000     0.3536  -8.485  0.00106 ** 
## A:B          -2.0000     0.3536  -5.657  0.00481 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1 on 4 degrees of freedom
## Multiple R-squared:  0.9747, Adjusted R-squared:  0.9557 
## F-statistic: 51.33 on 3 and 4 DF,  p-value: 0.001192
library(sjPlot)
plot_model(reg, type="int")

Social Media Ad Testing

Vuori tested ads on Instagram. They ran an a/b test with two different versions of an ad. The response was a measure of user engagement on a scale of 0 to 100. The company also paid for user data from Instagram so they could have insight on who was engaging with the ad.

ad engagement age device
A 85.51612 24 Apple
A 62.30687 45 Apple
A 69.48172 46 Apple
A 54.56918 31 Apple
A 98.10323 36 Apple
A 50.54667 17 Apple

Test of the Ad Effectiveness

boxplot(engagement~ad)

t.test(engagement~ad, data=df)
## 
##  Welch Two Sample t-test
## 
## data:  engagement by ad
## t = -2.6043, df = 1977.9, p-value = 0.009274
## alternative hypothesis: true difference in means between group A and group B is not equal to 0
## 95 percent confidence interval:
##  -2.8685058 -0.4041082
## sample estimates:
## mean in group A mean in group B 
##        75.01723        76.65354

Test of the Ad Effectiveness by Age

hist(age)

A<-(df$age<35)
B<-(df$age>=35)
t.test(df$engagement[A]~df$ad[A])
## 
##  Welch Two Sample t-test
## 
## data:  df$engagement[A] by df$ad[A]
## t = -1.887, df = 1210.2, p-value = 0.0594
## alternative hypothesis: true difference in means between group A and group B is not equal to 0
## 95 percent confidence interval:
##  -3.11425835  0.06063418
## sample estimates:
## mean in group A mean in group B 
##        75.22132        76.74813
t.test(df$engagement[B]~df$ad[B])
## 
##  Welch Two Sample t-test
## 
## data:  df$engagement[B] by df$ad[B]
## t = -1.8253, df = 763.05, p-value = 0.06835
## alternative hypothesis: true difference in means between group A and group B is not equal to 0
## 95 percent confidence interval:
##  -3.7827944  0.1375878
## sample estimates:
## mean in group A mean in group B 
##        74.68846        76.51106

Test of device usage by Age

t.test(df$age~df$device)
## 
##  Welch Two Sample t-test
## 
## data:  df$age by df$device
## t = 0.99765, df = 1588.9, p-value = 0.3186
## alternative hypothesis: true difference in means between group Android and group Apple is not equal to 0
## 95 percent confidence interval:
##  -0.4426229  1.3589472
## sample estimates:
## mean in group Android   mean in group Apple 
##              33.52823              33.07006