4  Simulating Games with R

Author

Affiliation

Ryan McShane, Ph.D.

The University of Chicago

4.0.1 Quick Aside: Rainbow Parentheses

1. In RStudio, go to “Tools” \(\rightarrow\) “Global Options” \(\rightarrow\) “Code” \(\rightarrow\) “Display”
2. Check the “Use rainbow parentheses” option.

Now, it should be easier to check when parentheses match/not!

4.1 Simulating Blackjack

4.1.1 Code Source: Nolan and Lang, Case Studies in Data Science with R

The Card Players, Paul Cézanne, 1890-1892, Metropolitan Museum of Art

4.1.2 What is Blackjack?

• Card game, goal: sum cards as close to 21 without going over
• A few nuances to card value (e.g., Ace can be 1 or 11)
• Start with 2 cards, build up one card at a time
• Lots of different strategies (also based on dealer’s cards)

4.1.3 Establish Deck and Shuffling

deck = rep(c(1:10, 10, 10, 10), 4)

shuffle_decks = function(n) {
  all_deck = rep(deck, n)
return(sample(all_deck))  
}

\({}\)

Testing handValue

shuffle_decks(1)
##  [1]  3  3  1  6  6  2 10  7  4  4  4  2  3  1 10  7 10 10  8  9  5  1 10  9 10
## [26] 10 10 10  8  2 10  5  6  4 10  5  8  9  1 10  9  2  8  5 10  3  6 10  7 10
## [51]  7 10
shuffle_decks(3)
##   [1]  6  3 10  8 10  8 10  1  6 10  4  8  3 10 10  9 10  6 10 10  3  9 10  5  6
##  [26]  9  4  5  4 10  1 10  8 10  6  5 10  5  7  1 10  7  2  2  6  8 10  5 10 10
##  [51]  2  8 10 10 10  3  6  1 10 10  4  8  9  7 10  1  6  3  8  1  3  2  7 10 10
##  [76]  7  1  7  9  4 10  8  2  2  6  4  3  8  1  5 10  4 10  9  3  8  2 10  4  2
## [101]  2  1 10  2  5 10  9  5  7 10  7 10  3  3  5  2  5  7 10 10  3 10  7  9  5
## [126]  7  9  1 10 10 10 10  9 10  4  8 10  5  6  1 10  9  1 10  2  9 10  4  6  3
## [151]  4  4 10  6  7 10

4.1.4 Given Hand, Calculate Value

handValue = function(cards) {
  value = sum(cards)
       # Check for an Ace and change value if it doesn't bust
  if (any(cards == 1) && value <= 11) 
    value = value + 10
       # Check bust (set to 0); check black jack (set to 21.5)
  if (value > 21) 
    return(0) 
  else if (value == 21 && length(cards) == 2)  
    return(21.5) # Blackjack
  else 
    return(value)
}

Testing handValue

handValue(c(10,4))
## [1] 14
handValue(c(10, 1))
## [1] 21.5
handValue(c(10,10))
## [1] 20

4.1.5 Better Test of handValue

test_cards = list(
    c(10, 1), 
    c(10, 5, 6), 
    c(10, 1, 1), 
    c(7, 6, 1, 5), 
    c(3, 6, 1, 1), 
    c(2, 3, 4, 10), 
    c(5, 1, 9, 1, 1),
    c(5, 10, 7), 
    c(10, 9, 1, 1, 1)
  ) 
expected_values = c(21.5, 21, 12, 19, 21, 19, 17, 0, 0)
# apply the function handValue to test_cards
actual_values = sapply(X = test_cards, FUN = handValue)
identical(expected_values, actual_values)
## [1] TRUE

4.1.6 Explicit Outcome Evaluation (Outputs Multiplier of Wager)

winnings = function(dealer, players) {
  if (dealer > 21) {  # Dealer=Blackjack, ties players with Blackjack
    output =  -1 * (players <= 21)
  } else if (dealer == 0) { # Dealer busts - all non-busted players win
    output = 1.5 * (players > 21) +
               1 * (players <= 21 & players > 0) +
              -1 * (players == 0) 
  } else {            # Dealer 21 or below, all players > dealer win
    output = 1.5 * (players > 21) +  
               1 * (players <= 21 & players > dealer) +
              -1 * (players <= 21 & players < dealer) 
  }
return(output)
}

Testing winnings v1

winnings(dealer = 17, players = c(20, 21.5, 14, 0, 21))
## [1]  1.0  1.5 -1.0 -1.0  1.0

4.1.7 More Compact Outcome Evaluation (Outputs Multiplier of Wager)

winnings = function(dealer, players) {
  score = (players > dealer & players > 21)  * 1.5 +  # Blackjack
          (players > dealer & players <= 21) *  1  +  # win
          (players < dealer | players == 0)  * -1     # lose
return(score)
}

winnings(dealer = 17, players = c(20, 21.5, 14, 0, 21))
## [1]  1.0  1.5 -1.0 -1.0  1.0

4.1.8 Testing winnings

test_vals = c(0, 16, 19, 20, 21, 21.5)

testWinnings =
  matrix(data = c(-1,  1,  1,  1,  1, 1.5,
                  -1,  0,  1,  1,  1, 1.5,
                  -1, -1,  0,  1,  1, 1.5,
                  -1, -1, -1,  0,  1, 1.5,
                  -1, -1, -1, -1,  0, 1.5,
                  -1, -1, -1, -1, -1, 0 ), 
         nrow = length(test_vals), byrow = TRUE)
dimnames(testWinnings) = list(
    dealer = test_vals, 
    player = test_vals
  )

testWinnings
##       player
## dealer  0 16 19 20 21 21.5
##   0    -1  1  1  1  1  1.5
##   16   -1  0  1  1  1  1.5
##   19   -1 -1  0  1  1  1.5
##   20   -1 -1 -1  0  1  1.5
##   21   -1 -1 -1 -1  0  1.5
##   21.5 -1 -1 -1 -1 -1  0.0

Nested for loop approach

# make the matrix the right size, but init. to NA
check_for = testWinnings * NA
for (i in seq_along(test_vals)) {
  for (j in seq_along(test_vals)) {
    check_for[i, j] = 
      winnings(test_vals[i], test_vals[j])
  }
}
identical(check_for, testWinnings)
## [1] TRUE

apply family of functions approach: outer

check_out = outer(FUN = winnings, 
                    X = test_vals, Y = test_vals)
identical(check_out, testWinnings)
## [1] FALSE
attributes(check_out) = attributes(testWinnings)
# identical once we ignore attributes
# which were for readability of testWinnings
identical(check_out, testWinnings)
## [1] TRUE

4.1.9 Function for Getting More Cards

# Shoe is where the deck or decks are held
shoe = function(m = 1) {
  output = sample(x = deck, size = m, replace = TRUE)
return(output)
}

new_hand = function(shoe, cards = shoe(2), bet = 1) {
  list(bet = bet, shoe = shoe, cards = cards)
}

Example Usage of Function

myCards = new_hand(shoe, bet = 7)
myCards
## $bet
## [1] 7
## 
## $shoe
## function (m = 1) 
## {
##     output = sample(x = deck, size = m, replace = TRUE)
##     return(output)
## }
## 
## $cards
## [1] 4 2

4.1.10 Actions with effects (hit, stand, dd)

hit

Receive another card

hit = function(hand) {
  hand$cards = c(hand$cards, hand$shoe(1))
  hand
}

hit(myCards)$cards
## [1] 4 2 8

stand

Stay with current cards

stand = function(hand) hand

stand(myCards)$cards
## [1] 4 2

double down (dd)

Double the bet after receiving exactly one more card

dd =  function(hand) {
  hand$bet = hand$bet * 2
  hand = hit(hand)
  stand(hand)
}

dd(myCards)$cards
## [1] 4 2 4
dd(myCards)$bet
## [1] 14

4.1.11 Action with Effect: Split a Pair (splitPair)

Create two different hands from initial hand with two cards of the same value

splitPair = function(hand) {
  list( new_hand(hand$shoe, 
             cards = c(hand$cards[1], hand$shoe(1)),
             bet = hand$bet),
        new_hand(hand$shoe, 
             cards = c(hand$cards[2], hand$shoe(1)),
             bet = hand$bet))   
}

Split Result

splitHand = splitPair(myCards)
splitHand[[1]]$cards
## [1] 4 5
splitHand[[2]]$cards
## [1] 2 7

4.1.12 Example of (Manual) Game

Initialize Game

set.seed(470)
dealer = new_hand(shoe)
player = new_hand(shoe)

Player Plays

player$cards
## [1]  5 10
player = hit(player)
player$cards
## [1]  5 10  9

Dealer Plays

dealer$cards
## [1] 2 3
dealer = hit(dealer)
dealer$cards
## [1] 2 3 3

Who wins?

dealer$cards
## [1] 2 3 3
player$cards
## [1]  5 10  9
handValue(dealer$cards)
## [1] 8
handValue(player$cards)
## [1] 0
winnings(
    dealer = handValue(dealer$cards), 
    players = handValue(player$cards)
  )
## [1] -1

4.1.13 Simple Strategy

strategy_simple = function(mine, dealerFaceUp) {
  if (handValue(dealerFaceUp) > 6 && handValue(mine) < 17) 
     do_this = "H" 
  else 
     do_this = "S"
return(do_this)
}

Improved Simple Strategy

strategy_simple = function(mine, dealerFaceUp) {
  if (handValue(mine) == 0) {
    do_this = "S"
  } else if (handValue(dealerFaceUp) > 6 && handValue(mine) < 17) 
     do_this = "H" 
  else 
     do_this = "S"
return(do_this)
}

4.1.14 Dealer Strategy

Dealer needs to achieve at least a 17 no matter what

dealer_cards = function(shoe) {
  cards = shoe(2)
  while (handValue(cards) < 17 && handValue(cards) > 0) {
    cards = c(cards, shoe(1))
  }
return(cards)
}

dealer_cards(shoe)
## [1] 5 9 1 8

4.1.15 Playing a Full Hand

play_hand = function(shoe, strategy, 
      hand = new_hand(shoe), dealer = dealer_cards(shoe)) {
  face_up_card = dealer[1]
  action = strategy(hand$cards, face_up_card)
  while (action != "S" && handValue(hand$cards) != 0) {
    if (action == "H") {
      hand = hit(hand)
      action = strategy(hand$cards, face_up_card)
    } else {
      stop("Unknown action: should be one of S, H")
    }
  }
  money_back = winnings(handValue(dealer), handValue(hand$cards)) * hand$bet
return(money_back)
}

Play a Few Hands

set.seed(4000)
play_hand(shoe, strategy = strategy_simple)
## [1] 1
play_hand(shoe, strategy = strategy_simple, hand = new_hand(shoe, bet = 7))
## [1] -7

4.1.16 Repeated Games

reps = 5
# our "bankroll"
money = 20
for (i in seq_len(length.out = reps)) {
  money = money + play_hand(shoe, strategy_simple)
  print(money)
}
## [1] 21
## [1] 20
## [1] 19
## [1] 18
## [1] 18

4.2 Key Takeaways from Hands-on Programming in R

4.2.1 Objects

• Objects provide a structured approach to programming.
  • By defining a dataset as a custom object, a developer can easily create multiple similar objects and modify existing objects within a program.
  • In R, objects can store data using vectors, lists, matrices, arrays, and data frames.
  • We can modify our stored data (in place) through using the <- or = operators, which is useful in cleaning smaller data sets. However, with larger datasets we can use logical tests and boolean operators to modify the values.
• A perfect example of using objects is when creating a die. die = c(1, 2, 3, 4, 5, 6). In this case, die is our object, and it is storing an atomic vector which is grouping some sort of data values together with c.

4.2.2 Functions

• Functions are self-contained modules of code that accomplish a specific task.
  • Functions usually take in data, process it, and return a result.
  • Once a function is written, it can be used over and over and over again.
  • Functions are an easy way to abstract away repeatable intermediate processes in your code.
  • Using functions makes code more clear for a 3rd party as well as for the person programming.
    
• An example of a function is sample(x, size) which takes two arguments x, the elements to choose from, size, the size of the sample, and returns a random sample of size size made up of elements from x.

4.2.3 Environments & Scope

• R keeps multiple environments; the largest being the global environment that keeps objects that are called in the command line. R will create smaller environments which contain subsets of functions. Objects in smaller environments are not accessible from the global environment. Once a function has completed its commands and returned its results, the intermediate objects used within that function goes away.

• Don’t walk on the flowers!

4.2.4 Loops and Vectorization

• Loops can be computationally expensive when a number of complex processes are repeated over and over. Use this ranking of speed when deciding on a looping structure in R:
  1. Vectorized functions (including the apply family, like sapply, lapply, outer, and replicate, as well as functions re-written by base::Vectorize())
  2. for loop (repeat a certain number of times)
  3. while loop (repeat until a certain condition is met)
  4. Recursion (x |> f() |> ... |> f() until problem is solved)

• R is designed for vectorization of these processes, meaning that the complex processes can be performed simultaneously for all of the items in a dataset as opposed to looping over each of them individually.

Calculating Expected Value, \(\mathrm{E}[X] = \sum_i^n x_ip(x_i)\)

• Calculating \(\mathrm{E}[X]\) with a Loop: Multiply each \(p(x_i)\) with each \(x_i\) and then add each term. This requires \(n\) separate iterations.

• Calculating \(\mathrm{E}[X]\) with Vectorization: Take the dot product of \(x\) and \(p(x)\), \(\mathbf x \cdot p(\mathbf x)\). This requires just one calculation.

4.2.5 Larger Projects

• Larger projects can be tricky to manage. Remembering what everything does and making sure every edge case is accounted for can be hard. Using the following tactics can help:
  • Use # to add comments to your code. Writing what each chunk of code is intended to perform is incredibly helpful for larger projects. This will help you quickly remember where you left off the last time you were working on the project and will allow you and other members of your team to be on the same page.
  • Break projects into many intermediate steps. Large projects can be daunting but if you map out what are the intermediate steps that make up this larger problem and write functions for each one of them, the project seems much more manageable.
    
  • Test and debug often. Writing large amounts of code in between testing will make it especially tricky to identify where your bugs are coming from. Testing more often mitigates this problem.

4.3 Qwixx Mini-Project Part I: Write an R Program to Run a Game of Qwixx without Human Intervention

4.3.1 Project Stages

1. First, you’ll reconstruct the functions a previous student had programmed with some guidance. Refer to the rules frequently!
2. Second, you’ll brainstorm a strategy – essentially, you’ll be writing an outline of an algorithm to play the game of Qwixx.
3. Finally, you’ll implement your brainstormed strategy in an R program – I’ll supply a script with the completed functions from part 1 (in case you didn’t complete this portion).
   • Your strategy will compete against prior strategies, including the “default” strategy you’ll be programming yourself soon.
   • It should be relatively easy to get a passing grade, but your strategy will need to be a genuine attempt to win to get a perfect score on the project. The autograder will tell you how your strategy stacks up against prior strategies.
   • When the strategies have all been submitted, your strategies will compete against each other in a large simulation. This part is just for fun!

4.3.2 Default strategy (very beatable!)

• Should default to “penalty” only if there is no possible move.
• Should pick a single move that places an “x” in the leftmost box. If there’s a tie, draw the color at random.
• Should never put two “x”s down on the same roll.
• Should only place an “x” while they’re the active roller.