Electronic Arts Inc. (EA) is a California-based video game company which, in terms of revenue, is the second-largest gaming company in the Americas and Europe. Notable games created by EA include Apex Legends, Sims, Star Wars Battlefront and the EA Sports suite of games (FIFA, NBA Live, etc.). With every new game released, a large amount of revenue is generated by “pre-orders” in which a customer purchases the game in advance of the release date to ensure they receive their copy as soon as it becomes available. On November 17, 2017 Star Wars Battlefront II was released. Prior to this release EA experimented with the pre-order promotional banner shown on their website. In particular they altered the location of the promotional banner, whether the banner was a photo or video, whether the banner showed the game’s price, whether the banner advertised the game’s downloadable content (i.e., purchasable expansions that provide additional/ enriched gameplay), and whether the banner offered a “Pre-Order Now” 10% discount. The experimental factors and their levels are summarized in the following table:
A total of 32 experimental conditions (banners) were created from the
32 unique combinations of the factor levels, and each banner was shown
separately to 500 different randomly selected visitors to www.ea.com.
For each of these customers, their purchase total (the total dollars
spent during their visit to the website) was recorded. Interest lies in
determining which banner maximizes average-purchase-total and
understanding which of the investigated factor(s) significantly drive
pre-order purchases. Below is the head() of the data.
library(tidyverse)
ea<-read.csv("I:\\Classes\\ISA 633\\Exams\\Spring 2026\\Exam 2\\ea.csv")
head(ea)## banner.position banner.type price.shown dlc.advertised discount spend
## 1 1 -1 1 1 -1 47.93
## 2 -1 -1 1 1 -1 36.82
## 3 -1 1 -1 1 -1 45.94
## 4 1 -1 1 1 -1 46.00
## 5 -1 1 -1 -1 1 46.42
## 6 1 1 1 1 1 56.99
## Rows: 16,000
## Columns: 6
## $ banner.position <int> 1, -1, -1, 1, -1, 1, 1, -1, 1, -1, 1, 1, 1, 1, 1, -1, …
## $ banner.type <int> -1, -1, 1, -1, 1, 1, -1, 1, -1, -1, 1, 1, 1, 1, -1, 1,…
## $ price.shown <int> 1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, 1, 1, 1, -1, 1…
## $ dlc.advertised <int> 1, 1, 1, 1, -1, 1, -1, 1, 1, -1, 1, -1, 1, 1, -1, 1, 1…
## $ discount <int> -1, -1, -1, -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1…
## $ spend <dbl> 47.93, 36.82, 45.94, 46.00, 46.42, 56.99, 46.97, 44.94…
Re-label factors for easy-to-read output.
ea$A<-ea$banner.position
ea$B<-ea$banner.type
ea$C<-ea$price.shown
ea$D<-ea$dlc.advertised
ea$E<-ea$discountModel 1:
##
## Call:
## lm(formula = spend ~ A * B * C * D * E, data = ea)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.9786 -0.6700 -0.0010 0.6778 3.7917
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.500e+01 7.907e-03 5691.170 <2e-16 ***
## A 4.998e+00 7.907e-03 632.154 <2e-16 ***
## B 5.000e+00 7.907e-03 632.311 <2e-16 ***
## C 1.113e-02 7.907e-03 1.408 0.1592
## D 1.599e-02 7.907e-03 2.023 0.0431 *
## E -7.355e-03 7.907e-03 -0.930 0.3523
## A:B 2.693e-03 7.907e-03 0.341 0.7335
## A:C -4.589e-03 7.907e-03 -0.580 0.5617
## B:C 4.414e-03 7.907e-03 0.558 0.5767
## A:D 8.093e-03 7.907e-03 1.023 0.3061
## B:D -1.362e-02 7.907e-03 -1.722 0.0851 .
## C:D -8.737e-04 7.907e-03 -0.111 0.9120
## A:E -6.111e-03 7.907e-03 -0.773 0.4396
## B:E 9.987e-01 7.907e-03 126.310 <2e-16 ***
## C:E 4.960e-03 7.907e-03 0.627 0.5305
## D:E -1.176e-03 7.907e-03 -0.149 0.8817
## A:B:C 4.150e-03 7.907e-03 0.525 0.5997
## A:B:D -2.791e-03 7.907e-03 -0.353 0.7241
## A:C:D -5.622e-03 7.907e-03 -0.711 0.4770
## B:C:D 1.025e-03 7.907e-03 0.130 0.8969
## A:B:E -9.057e-03 7.907e-03 -1.146 0.2520
## A:C:E -5.304e-03 7.907e-03 -0.671 0.5024
## B:C:E -1.280e-02 7.907e-03 -1.619 0.1055
## A:D:E -1.014e-02 7.907e-03 -1.283 0.1995
## B:D:E 7.378e-03 7.907e-03 0.933 0.3508
## C:D:E -7.109e-03 7.907e-03 -0.899 0.3686
## A:B:C:D 9.189e-03 7.907e-03 1.162 0.2452
## A:B:C:E -2.550e-04 7.907e-03 -0.032 0.9743
## A:B:D:E -6.375e-05 7.907e-03 -0.008 0.9936
## A:C:D:E -5.110e-03 7.907e-03 -0.646 0.5181
## B:C:D:E -6.660e-03 7.907e-03 -0.842 0.3996
## A:B:C:D:E -4.269e-03 7.907e-03 -0.540 0.5893
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1 on 15968 degrees of freedom
## Multiple R-squared: 0.9808, Adjusted R-squared: 0.9808
## F-statistic: 2.63e+04 on 31 and 15968 DF, p-value: < 2.2e-16
Model 2:
reg<-lm(spend~A*B*C*D*E-A:B:D-D:E-B:C:D-A:B:C:E-A:B:D:E-C:D-A:B-A:B:C:D:E-B:C-A:B:C-A:C-A:C:D:E-A:C:E-A:C:D-C:E-B:C:D:E-C:D:E-B:D:E-A:E-A:B:E-A:D-A:B:C:D-A:D:E-B:C:E-C-B:D, data=ea)
summary(reg)##
## Call:
## lm(formula = spend ~ A * B * C * D * E - A:B:D - D:E - B:C:D -
## A:B:C:E - A:B:D:E - C:D - A:B - A:B:C:D:E - B:C - A:B:C -
## A:C - A:C:D:E - A:C:E - A:C:D - C:E - B:C:D:E - C:D:E - B:D:E -
## A:E - A:B:E - A:D - A:B:C:D - A:D:E - B:C:E - C - B:D, data = ea)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.9438 -0.6726 -0.0026 0.6762 3.7846
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 44.999923 0.007905 5692.382 <2e-16 ***
## A 4.998424 0.007905 632.289 <2e-16 ***
## B 4.999669 0.007905 632.446 <2e-16 ***
## D 0.015994 0.007905 2.023 0.0431 *
## E -0.007355 0.007905 -0.930 0.3522
## B:E 0.998726 0.007905 126.336 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.9999 on 15994 degrees of freedom
## Multiple R-squared: 0.9808, Adjusted R-squared: 0.9808
## F-statistic: 1.631e+05 on 5 and 15994 DF, p-value: < 2.2e-16
Check assumptions:
Interaction Plot:
library(ggeffects)
library(ggplot2)
eff <- ggpredict(reg, terms = c("B", "E"))
eff_df<-as.data.frame(eff)
eff_df$B <- eff_df$x
eff_df$E <- eff_df$group
ggplot(eff_df,
aes(x = x, y = predicted,
group = E, linetype = E, shape = E, color=E)) +
geom_point(size = 2) +
geom_line(aes(color = E)) +
#facet_wrap(C~D, labeller = label_both) +
scale_linetype_manual(values = c("solid", "dashed")) +
labs(
x = "B",
y = "Spend",
#linetype = NULL,
#shape = NULL
) +
theme_bw()Course Hero is an education technology company which hosts an online repository of course-specific study resources for tens of thousands of courses at post-secondary institutions. Complete access is blocked by a “paywall” and is only available to paying subscribers. Given that Course Hero is a for-profit company, interest lies in maximizing the conversion rate from “free” to “premium” accounts. As such an experiment is conducted which alters various aspects of the “premium” service to determine which offer maximizes conversion rates. In particular, when a user visits the pricing webpage, they are offered one of 12 versions of the “premium” services. The different versions are based on differing values of the price ($9.95/month or $10/month), the number of Tutor Questions they are permitted to ask (10, 20 or 40) and whether a 6-month free trial is offered (Yes or No). Each of these 12 versions is separately offered to 1000 different randomly selected users currently subscribed to the “free” tier. For each user in the experiment the binary indicator convert/non-convert is recorded.
Here is the data.
ch<-read.csv("I:\\Classes\\ISA 633\\Exams\\Spring 2026\\Exam 2\\coursehero.csv")
ch$reps<-rep(1000, 12)
ch## price tutor free_trial conversion reps
## 1 10.00 20 No 199 1000
## 2 10.00 40 No 217 1000
## 3 9.95 10 Yes 206 1000
## 4 10.00 10 No 219 1000
## 5 9.95 10 No 221 1000
## 6 10.00 40 Yes 220 1000
## 7 9.95 20 Yes 225 1000
## 8 9.95 40 No 232 1000
## 9 10.00 10 Yes 205 1000
## 10 9.95 20 No 254 1000
## 11 9.95 40 Yes 237 1000
## 12 10.00 20 Yes 211 1000
Analysis 1:
## Df Sum Sq Mean Sq F value Pr(>F)
## price 1 901.3 901.3 4.280 0.107
## tutor 1 334.3 334.3 1.588 0.276
## free_trial 1 120.3 120.3 0.571 0.492
## price:tutor 1 37.1 37.1 0.176 0.696
## price:free_trial 1 133.3 133.3 0.633 0.471
## tutor:free_trial 1 178.1 178.1 0.846 0.410
## price:tutor:free_trial 1 18.0 18.0 0.086 0.785
## Residuals 4 842.4 210.6
Analysis 2:
mod<-glm(cbind(conversion, reps-conversion)~price*tutor*free_trial, data=ch, family="binomial")
anova(mod, test="Chisq")## Analysis of Deviance Table
##
## Model: binomial, link: logit
##
## Response: cbind(conversion, reps - conversion)
##
## Terms added sequentially (first to last)
##
##
## Df Deviance Resid. Df Resid. Dev Pr(>Chi)
## NULL 11 14.7839
## price 1 5.2451 10 9.5389 0.02201 *
## tutor 1 1.9420 9 7.5969 0.16346
## free_trial 1 0.7005 8 6.8964 0.40260
## price:tutor 1 0.1804 7 6.7160 0.67105
## price:free_trial 1 0.7360 6 5.9800 0.39093
## tutor:free_trial 1 1.0683 5 4.9117 0.30134
## price:tutor:free_trial 1 0.0968 4 4.8148 0.75564
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Analysis 3:
## Df Sum Sq Mean Sq
## price 1 901.3 901.3
## tutor 2 396.5 198.3
## free_trial 1 120.3 120.3
## price:tutor 2 547.2 273.6
## price:free_trial 1 133.3 133.3
## tutor:free_trial 2 178.2 89.1
## price:tutor:free_trial 2 288.2 144.1
Analysis 4:
mod<-glm(cbind(conversion, reps-conversion)~price*tutor*free_trial, data=ch, family="binomial")
anova(mod, test="Chisq")## Analysis of Deviance Table
##
## Model: binomial, link: logit
##
## Response: cbind(conversion, reps - conversion)
##
## Terms added sequentially (first to last)
##
##
## Df Deviance Resid. Df Resid. Dev Pr(>Chi)
## NULL 11 14.7839
## price 1 5.2451 10 9.5389 0.02201 *
## tutor 2 2.3143 8 7.2246 0.31438
## free_trial 1 0.7006 7 6.5240 0.40259
## price:tutor 2 3.1204 5 3.4036 0.21010
## price:free_trial 1 0.7365 4 2.6672 0.39079
## tutor:free_trial 2 1.0746 2 1.5926 0.58432
## price:tutor:free_trial 2 1.5926 0 0.0000 0.45100
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Design 1:
## A B C D E F
## 1 -1 1 -1 -1 1 -1
## 2 -1 -1 -1 -1 1 1
## 3 1 1 -1 1 -1 1
## 4 1 -1 1 1 1 -1
## 5 1 -1 1 -1 -1 -1
## 6 1 1 -1 1 1 -1
## 7 -1 1 -1 -1 -1 1
## 8 -1 1 -1 1 1 1
## 9 1 -1 -1 1 -1 -1
## 10 1 -1 1 1 -1 1
## 11 -1 -1 -1 1 1 -1
## 12 1 1 1 1 -1 -1
## 13 1 1 1 -1 -1 1
## 14 -1 1 1 -1 1 1
## 15 -1 -1 -1 1 -1 1
## 16 -1 -1 1 -1 1 -1
## 17 -1 1 1 -1 -1 -1
## 18 -1 1 -1 1 -1 -1
## 19 -1 -1 1 -1 -1 1
## 20 -1 1 1 1 1 -1
## 21 1 -1 -1 -1 -1 1
## 22 -1 1 1 1 -1 1
## 23 1 1 1 -1 1 -1
## 24 -1 -1 -1 -1 -1 -1
## 25 -1 -1 1 1 1 1
## 26 1 -1 -1 1 1 1
## 27 1 -1 1 -1 1 1
## 28 1 1 -1 -1 1 1
## 29 1 1 1 1 1 1
## 30 1 1 -1 -1 -1 -1
## 31 -1 -1 1 1 -1 -1
## 32 1 -1 -1 -1 1 -1
## class=design, type= FrF2
##
## A:B = C:D:E:F
## A:C = B:D:E:F
## A:D = B:C:E:F
## A:E = B:C:D:F
## A:F = B:C:D:E
## B:C = A:D:E:F
## B:D = A:C:E:F
## B:E = A:C:D:F
## B:F = A:C:D:E
## C:D = A:B:E:F
## C:E = A:B:D:F
## C:F = A:B:D:E
## D:E = A:B:C:F
## D:F = A:B:C:E
## E:F = A:B:C:D
## A:B:C = D:E:F
## A:B:D = C:E:F
## A:B:E = C:D:F
## A:B:F = C:D:E
## A:C:D = B:E:F
## A:C:E = B:D:F
## A:C:F = B:D:E
## A:D:E = B:C:F
## A:D:F = B:C:E
## A:E:F = B:C:D
Design 2:
## A B C D E F
## 1 1 -1 1 -1 -1 1
## 2 1 -1 -1 1 1 -1
## 3 -1 1 -1 1 1 -1
## 4 -1 -1 1 1 1 1
## 5 -1 1 1 1 -1 -1
## 6 1 -1 -1 -1 1 1
## 7 1 1 -1 -1 -1 -1
## 8 -1 1 -1 -1 1 1
## 9 -1 -1 -1 1 -1 1
## 10 1 1 1 -1 1 -1
## 11 -1 -1 -1 -1 -1 -1
## 12 -1 1 1 -1 -1 1
## 13 1 1 -1 1 -1 1
## 14 1 1 1 1 1 1
## 15 -1 -1 1 -1 1 -1
## 16 1 -1 1 1 -1 -1
## class=design, type= FrF2
##
## A = B:C:E = B:D:F
## B = A:D:F = A:C:E
## C = D:E:F = A:B:E
## D = C:E:F = A:B:F
## E = C:D:F = A:B:C
## F = C:D:E = A:B:D
## A:B = C:E = D:F
## A:C = A:D:E:F = B:C:D:F = B:E
## A:D = A:C:E:F = B:C:D:E = B:F
## A:E = A:C:D:F = B:D:E:F = B:C
## A:F = A:C:D:E = B:C:E:F = B:D
## C:D = A:B:C:F = A:B:D:E = E:F
## C:F = A:B:C:D = A:B:E:F = D:E
## A:C:D = A:E:F = B:C:F = B:D:E
## A:C:F = A:D:E = B:C:D = B:E:F
A large e-commerce company is seeking to improve the efficiency of its warehouse operations by reducing the average time required to fulfill customer orders. To do this, the company conducts a designed experiment in one of its fulfillment centers, evaluating seven operational factors that may influence performance, including staffing levels (A), picking methods (B), conveyor speed (C), packing resources (D), routing algorithms (E), shift timing (F), and inventory layout (G). Because testing all possible combinations would be too costly and disruptive, the company implements a fractional factorial design that allows it to efficiently study the effects of these factors over a limited number of experimental runs. Each experimental condition is replicated across three time periods (e.g., different shifts) to account for natural variability in order volume and worker performance. During each run, the warehouse operates under a specific combination of factor settings for a fixed time period, and the response is measured as the average order fulfillment time (in minutes per order). The goal of the experiment is to identify which factors have the greatest impact on processing time (lower is better) while obtaining reliable estimates of variability to support statistical inference and decision-making.
Below is the treatment combinations they used an the alias structure.
## A B C D E F G
## 1 -1 1 -1 1 1 -1 1
## 2 1 1 -1 1 -1 1 -1
## 3 1 1 1 1 1 1 1
## 4 -1 -1 1 1 1 1 -1
## 5 -1 1 1 1 -1 -1 -1
## 6 -1 -1 1 -1 1 -1 1
## 7 -1 1 1 -1 -1 1 1
## 8 -1 -1 -1 -1 -1 -1 -1
## 9 1 -1 -1 -1 1 1 1
## 10 1 1 -1 -1 -1 -1 1
## 11 1 -1 -1 1 1 -1 -1
## 12 1 1 1 -1 1 -1 -1
## 13 1 -1 1 -1 -1 1 -1
## 14 1 -1 1 1 -1 -1 1
## 15 -1 -1 -1 1 -1 1 1
## 16 -1 1 -1 -1 1 1 -1
## class=design, type= FrF2
##
## A = B:C:E = B:D:F = C:D:G = E:F:G
## B = A:C:E = A:D:F = C:F:G = D:E:G
## C = A:D:G = B:F:G = D:E:F = A:B:E
## D = A:C:G = B:E:G = C:E:F = A:B:F
## E = A:F:G = B:D:G = C:D:F = A:B:C
## F = A:E:G = B:C:G = C:D:E = A:B:D
## G = A:C:D = A:E:F = B:C:F = B:D:E
## A:B = C:E = D:F
## A:C = B:E = D:G
## A:D = B:F = C:G
## A:E = B:C = F:G
## A:F = B:D = E:G
## A:G = C:D = E:F
## B:G = C:F = D:E
## A:B:G = A:C:F = A:D:E = B:C:D = B:E:F = C:E:G = D:F:G
Initial model.
##
## Call:
## lm.default(formula = y ~ A + B + C + D + E + F + G + A:B + A:C +
## A:D + A:E + A:F + A:G + B:G + A:B:G, data = d)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.04348 -0.54389 0.00693 0.49561 1.59804
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.9373659 0.1358138 21.628 < 2e-16 ***
## A -0.1145710 0.1358138 -0.844 0.4052
## B 1.2982530 0.1358138 9.559 6.75e-11 ***
## C 1.0347270 0.1358138 7.619 1.11e-08 ***
## D -0.0691760 0.1358138 -0.509 0.6140
## E 0.1619169 0.1358138 1.192 0.2419
## F 0.9127380 0.1358138 6.721 1.37e-07 ***
## G 1.2337611 0.1358138 9.084 2.25e-10 ***
## A:B 0.1768868 0.1358138 1.302 0.2021
## A:C -0.2541936 0.1358138 -1.872 0.0704 .
## A:D -0.1382280 0.1358138 -1.018 0.3164
## A:E -0.0009176 0.1358138 -0.007 0.9947
## A:F 0.1102944 0.1358138 0.812 0.4227
## A:G 0.0498524 0.1358138 0.367 0.7160
## B:G 1.0742393 0.1358138 7.910 5.01e-09 ***
## A:B:G 0.0495510 0.1358138 0.365 0.7176
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.9409 on 32 degrees of freedom
## Multiple R-squared: 0.9161, Adjusted R-squared: 0.8767
## F-statistic: 23.28 on 15 and 32 DF, p-value: 4.874e-13
Final model.
##
## Call:
## lm.default(formula = y ~ A + B + C + F + G + B:G, data = d)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.9621 -0.5164 -0.1308 0.6236 1.8010
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.9374 0.1356 21.669 < 2e-16 ***
## A 0.8854 0.1356 6.532 7.58e-08 ***
## B 1.2983 0.1356 9.577 5.13e-12 ***
## C 1.0347 0.1356 7.633 2.13e-09 ***
## F 0.9127 0.1356 6.733 3.93e-08 ***
## G 1.2338 0.1356 9.102 2.16e-11 ***
## B:G 1.0742 0.1356 7.925 8.42e-10 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.9391 on 41 degrees of freedom
## Multiple R-squared: 0.9034, Adjusted R-squared: 0.8893
## F-statistic: 63.94 on 6 and 41 DF, p-value: < 2.2e-16
In an A/B test with a binary response of convert or not, we assume the true proportion of arm \(a\) will behave according to the following distribution, which is updated as the data for the test arrives.
\[ p_a|\text{data} \sim \text{Beta}(a_a+s_a, b_a+f_a) \] We will use \(\text{Beta}(1,1)\) as the prior distribution.
In an A/B test with a continuous response, average spend per day, we assume the mean behaves according to the following distribution, which is updated as the data for the test arrives.
\[\mu_a|data \sim N(m_a, s^2_a) \] and where
\[s^2_a=(\frac{1}{s^2_0}+\frac{n_a}{\sigma^2})^{-1} \]
\[m_a=s^2_a(\frac{m_0}{s^2_0}+\frac{n_a\bar{y}_a}{\sigma^2}) \] We will use this prior distribution \[\mu_a \sim N(m_0, s^2_0) \] for arm \(a\) and assume \(s_0\)=10 and \(m_0\)=0. From historical data we know \(\sigma^2=5\).
A Ridesharing company is comparing a new dynamic pricing algorithm (B) to its current pricing system (A). Twenty regions are randomly assigned to sequences AB or BA, where each region uses one algorithm in week 1 and the other in week 2. The response is average revenue per ride-hour. A reset between weeks minimizes carryover effects.
## Region Sequence Algorithm_A Algorithm_B
## 1 R01 AB 47.62466 52.71540
## 2 R02 AB 44.72172 48.47720
## 3 R03 AB 47.12813 50.53385
## 4 R04 AB 49.96928 51.40051
## 5 R05 AB 42.40624 48.89336
## 6 R06 AB 43.31671 50.46330
## 7 R07 AB 49.97448 53.32944
## 8 R08 AB 48.95457 53.91624
## 9 R09 AB 43.76918 47.44991
## 10 R10 AB 44.78331 50.21645
## 11 R11 BA 43.76694 47.46932
## 12 R12 BA 42.90785 52.30878
## 13 R13 BA 45.03456 50.28498
## 14 R14 BA 39.03753 43.20030
## 15 R15 BA 40.19999 41.47066
## 16 R16 BA 44.14953 46.80910
## 17 R17 BA 40.44803 46.35196
## 18 R18 BA 45.72881 47.49752
## 19 R19 BA 42.21943 44.57257
## 20 R20 BA 41.22759 42.44746
rs$diff <- ifelse(
rs$Sequence == "AB",
0.5 * (rs$Algorithm_A- rs$Algorithm_B),
0.5 * (rs$Algorithm_B- rs$Algorithm_A) #mu_A-mu_B
)##
## Two Sample t-test
##
## data: diff by Sequence
## t = -8.5263, df = 18, p-value = 9.776e-08
## alternative hypothesis: true difference in means between group AB and group BA is not equal to 0
## 95 percent confidence interval:
## -5.137663 -3.106312
## sample estimates:
## mean in group AB mean in group BA
## -2.237369 1.884619