Soft Drink Displays-Larger Company

A soft drink distributor knows that end-aisle displays are an effective way to increase sales of the product. There are several ways to design these displays, varying the text, the colors, and the visual images. The marketing group has designed three new end-aisle displays and wants to test their effectiveness. They have identified 15 stores of similar size to participate in the study. Each store will test one of the displays for a period of one month.

Store Design.Display Percent.Increase.in.Sales
1 1 5.43
2 1 5.71
3 1 6.22
4 1 6.01
5 1 5.29
6 2 6.24
7 2 6.71
8 2 5.98
9 2 5.66
10 2 6.60
11 3 8.79
12 3 9.20
13 3 7.90
14 3 8.15
15 3 7.55

Soft Drink Displays–Small Company

A smaller company is also interested in testing these three soft drink displays. This company chooses to test these displays in each of 5 stores. This smaller company knows there are store to store differences but they are not interested in studying those differences.

Store Design.Display Percent.Increase.in.Sales
1 1 5.43
1 2 6.24
1 3 8.79
2 1 5.71
2 2 6.71
2 3 9.20
3 1 6.22
3 2 5.98
3 3 7.90
4 1 6.01
4 2 5.66
4 3 8.15
5 1 5.29
5 2 6.60
5 3 7.55

Store Display–New Company

A new, small company wishes to execute the same type of test for the Soft Drink display. This company has 3 stores to run the tests. They will use 3 months, 1 month per display, to run the tests. The company knows there are store to store and month to month differences but they wish to make conclusions regardless of store and month.

Store Month Design.Display Percent.Increase.in.Sales
1 1 1 5.43
1 2 2 6.24
1 3 3 8.79
2 1 2 6.71
2 2 3 9.20
2 3 1 5.71
3 1 3 7.90
3 2 1 6.22
3 3 2 5.66

Spotify Pricing and Options

Spotify has recently implemented differential pricing based on the type of plan (family, individual, etc). In order to optimize the plan pricing they tested 3 different pricing/options combinations of the Family Plan. They tested the three different versions over the course of one month and measured the number of plan sign-ups during this time.

plan sign_up
A 0
A 0
A 0
A 0
A 0
A 0 
library(tidyverse)
df %>% group_by(plan, sign_up) %>% summarise(n=n()) %>% mutate(n/sum(n))
## # A tibble: 6 × 4
## # Groups:   plan [3]
##   plan  sign_up     n `n/sum(n)`
##   <chr>   <int> <int>      <dbl>
## 1 A           0  4763     0.973 
## 2 A           1   134     0.0274
## 3 B           0  4823     0.965 
## 4 B           1   177     0.0354
## 5 C           0  5051     0.971 
## 6 C           1   149     0.0287
test<-chisq.test(table(df$plan, df$sign_up))
test
## 
##  Pearson's Chi-squared test
## 
## data:  table(df$plan, df$sign_up)
## X-squared = 6.2939, df = 2, p-value = 0.04298
test$residuals
##    
##              0          1
##   A  0.2207380 -1.2451572
##   B -0.3540657  1.9972432
##   C  0.1329796 -0.7501225

Direct Mail Experiment

An article in the International Journal of Research in Marketing describes an experiment to test new ideas to increase direct mail sales by the credit card division of a financial services company. They know from experience that the interest rates are an important factor in attracting potential customers so they have decided to focus on factors involving both interest rates and fees. The factors tested are as follows:

The company measured the number of responses to each offer.

A B response.rate
-1 -1 0.0245333
1 -1 0.0336000
-1 1 0.0216000
1 1 0.0229333
-1 -1 0.0249333
1 -1 0.0338667
-1 1 0.0232000
1 1 0.0244000 
reg<-lm(responses~(A+B)^2, data=df)
summary(reg)
## 
## Call:
## lm(formula = responses ~ (A + B)^2, data = df)
## 
## Residuals:
##    1    2    3    4    5    6    7    8 
## -2.0 -1.5 -8.0 -7.5  2.0  1.5  8.0  7.5 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  261.250      2.812  92.912 8.05e-08 ***
## A             25.750      2.812   9.158 0.000789 ***
## B            -31.000      2.812 -11.025 0.000385 ***
## A:B          -19.500      2.812  -6.935 0.002270 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 7.953 on 4 degrees of freedom
## Multiple R-squared:  0.9845, Adjusted R-squared:  0.9728 
## F-statistic:  84.5 on 3 and 4 DF,  p-value: 0.0004501
library(sjPlot)
plot_model(reg, type="int")

AirBnB Hidden vs. Shown Price

Months ago Airbnb ran this A/B test on their listing pages where they showed price totals for a given date range. This was one step beyond the already visible day rates (in the control). They also used default dates for the total calculation when someone didn’t enter a date, or user specified dates when users expressed them. They measured user spend for those who booked. They also have user data for its customers obtained by tracking cookie including user age and the device used to book the listing.

A-control
A-control
B
B
spend price age device
582.74 A 24 Apple
234.60 A 45 Apple
342.23 A 46 Apple
118.54 A 31 Apple
771.55 A 36 Apple
58.20 A 17 Apple

Test of the Ad Effectiveness

boxplot(spend~price, data=df)

t.test(spend~price, data=df)
## 
##  Welch Two Sample t-test
## 
## data:  spend by price
## t = -9.4873, df = 1927.3, p-value < 2.2e-16
## alternative hypothesis: true difference in means between group A and group B is not equal to 0
## 95 percent confidence interval:
##  -125.90310  -82.76738
## sample estimates:
## mean in group A mean in group B 
##        425.2584        529.5936

Test of the Ad Effectiveness by Age

hist(age)

A<-(df$age<35)
B<-(df$age>=35)
t.test(df$spend[A]~df$price[A])
## 
##  Welch Two Sample t-test
## 
## data:  df$spend[A] by df$price[A]
## t = -7.2659, df = 1158.7, p-value = 6.79e-13
## alternative hypothesis: true difference in means between group A and group B is not equal to 0
## 95 percent confidence interval:
##  -131.04133  -75.31825
## sample estimates:
## mean in group A mean in group B 
##        428.3197        531.4995
t.test(df$spend[B]~df$price[B])
## 
##  Welch Two Sample t-test
## 
## data:  df$spend[B] by df$price[B]
## t = -6.1126, df = 764.18, p-value = 1.562e-09
## alternative hypothesis: true difference in means between group A and group B is not equal to 0
## 95 percent confidence interval:
##  -140.56539  -72.22695
## sample estimates:
## mean in group A mean in group B 
##        420.3268        526.7229

Test of device usage by Age

t.test(df$age~df$device)
## 
##  Welch Two Sample t-test
## 
## data:  df$age by df$device
## t = 0.99765, df = 1588.9, p-value = 0.3186
## alternative hypothesis: true difference in means between group Android and group Apple is not equal to 0
## 95 percent confidence interval:
##  -0.4426229  1.3589472
## sample estimates:
## mean in group Android   mean in group Apple 
##              33.52823              33.07006

Madewell Bundles

Madewell entices shoppers by introducing special sets of discounting clothing. They are currently testing three different bundle names to entice customers to purchase a certain amount. They are currently running this test in four different demographic markets, West, East, Midwest and South. They would like to figure out which bundle name produces the highest user sales regardless of the test region. The data below contains purchase amounts for those who shopped each bundle in each region.

Head of the data

##   stylist luxurious onthego Region
## 1  190.67    193.42  201.69   East
## 2  185.43    190.61  196.42   East
## 3  192.53    192.75  198.84   East
## 4  189.25    205.17  198.81   East
## 5  190.71    197.74  199.10   East
## 6  188.21    199.73  199.07   East

Tests

##               Df  Sum Sq Mean Sq F value Pr(>F)    
## bundle         2   79592   39796   108.7 <2e-16 ***
## Residuals   7197 2635504     366                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##               Df  Sum Sq Mean Sq F value Pr(>F)    
## bundle         2   79592   39796    2476 <2e-16 ***
## Region         3 2519876  839959   52259 <2e-16 ***
## Residuals   7194  115629      16                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##                        diff       lwr       upr        p adj
## onthego-luxurious  2.885246  2.613946  3.156545 4.682366e-12
## stylist-luxurious -5.152946 -5.424245 -4.881646 4.682366e-12
## stylist-onthego   -8.038192 -8.309491 -7.766892 4.682366e-12