Prisoners' Dilemma
Cooperate
Cooperate
1
2
Defect
Defect
2
2
3
0
0
3
1
1
There's only one Nash equilibrium of this game.
Does it feel right? Do you live in a world in which nobody ever cooperates?
pollev.com/chrismakler
The Economics of Ongoing Relationships
Christopher Makler
Stanford University Department of Economics
Econ 51: Lecture 12
Today's Agenda
Part 1: Discrete Strategies
Part 2: Continuous Strategies
Finitely-repeated games
Evaluating infinite payoffs
Infinitely-repeated games
Collusion in Cournot Duopoly
Big Ideas
The ability to do so relies on
the ability to credibly promise/threaten something in the future,
and a high enough value placed on future payoffs
In ongoing relationships,
you can achieve things you cannot achieve in one-shot games.
What are we modeling?
-
Players who have ongoing relationships:
- Delta Airlines and American Airlines
- Managers and employees
- Spouses, roommates, friends
- Members of political parties
- Decisions they make have an immediate impact but also an impact on the relationship.
- How much greater is GDP because of the level of social capital (trust and other investments in relationships) is higher? How much worse off are we if that trust is eroded?
Finitely Repeated Games
-
Suppose there is some stage game G, which is repeated for T periods, with the payoffs being the sum of the payoffs in each period.
-
Trivial result: it is always a SPNE if a NE is played in every period.
-
Nontrivial result: there may be SPNE in which non-NE strategy profiles are played in early periods.
-
Key insight: this requires multiple Nash equilibria, which gives the ability to credibly promise a reward in the last period
1
2
5
2
,
0
0
,
2
4
,
0
0
,
T
M
L
C
B
R
0
6
,
0
0
,
0
0
,
9
0
,
7
9
,
If this game were played once,
what are the Nash equilibria?
pollev.com/chrismakler
Recall: Strategy Definition
A strategy is a complete, contingent plan of action for a player in a game.
This means that every player
must specify what action to take
at every decision node in the game tree!
A strategy space is the set of all strategies available to a player.
What is player 2's strategy space?
pollev.com/chrismakler
What is player 1's strategy space?
pollev.com/chrismakler
1
2
5
2
,
0
0
,
2
4
,
0
0
,
T
M
L
C
B
R
0
6
,
0
0
,
0
0
,
9
0
,
7
9
,
What are the strategy spaces of the players if the game is played twice?
Remember: a strategy is a complete, contingent plan of how to play the game. It has to specify what to do after every possible history.
Each strategy must specify:
- What to play in the first period.
- What to play in the second period
for every possible outcome in the first period
Each strategy must specify:
- What to play in the first period.
- What to play in the second period
for every possible outcome in the first period
Remember: a strategy is a complete, contingent plan of how to play the game. It has to specify what to do after every possible history.
Each strategy must specify:
- What to play in the first period.
- What to play in the second period
for every possible outcome in the first period
1
2
T
M
L
C
B
R
L
C
R
L
C
R
Each strategy must specify:
- What to play in the first period.
- What to play in the second period
for every possible outcome in the first period
What are the strategy spaces of the players if the game is played twice?
1
2
T
M
L
C
B
R
L
C
R
L
C
R
Example strategy for player 1:
- Play T in the first period.
- In the second period, play T again after any history in which player 2 played L; after every other history, play M.
Note that this has to specify 10 things -- what to do initially, and what to do after each of the 9 histories of what could happen in the first period!
What are the subgames of this game?
1
2
T
M
L
C
B
R
L
C
R
L
C
R
Example strategy for player 1:
- Play T in the first period.
- In the second period, play T again after any history in which player 2 played L; after every other history, play M.
Note that this has to specify 10 things -- what to do initially, and what to do after each of the 9 histories of what could happen in the first period!
What are the subgames of this game?
1
2
T
M
L
C
B
R
L
C
R
L
C
R
What are the subgames of this game?
Remember: in SPNE, the strategies must specify a NE in every subgame.
There were two NE of this game:
(T, L) and (M, C).
So, either (T, L) or (M, C) must be played in each of the 9 possible subgames in the second round in a subgame-perfect Nash equilibrium.
1
2
T
M
L
C
B
R
L
C
R
L
C
R
What are the subgames of this game?
Remember: in SPNE, the strategies must specify a NE in every subgame.
There were two NE of this game:
(T, L) and (M, C).
So, either (T, L) or (M, C) must be played in each of the 9 possible subgames in the second round in a subgame-perfect Nash equilibrium.
(T, L) or (M, C)
(T, L) or (M, C)
(T, L) or (M, C)
(T, L) or (M, C)
(T, L) or (M, C)
(T, L) or (M, C)
(T, L) or (M, C)
(T, L) or (M, C)
(T, L) or (M, C)
1
2
5
2
,
0
0
,
2
4
,
0
0
,
T
M
L
C
B
R
0
6
,
0
0
,
0
0
,
9
0
,
7
9
,
Let's look at the game again.
Both players would like to be able to play (B, R), but it wouldn't be possible in a one-shot game: player 1 would have an incentive to play T if they knew player 2 was going to play R.
But if it's played twice, can player 2 make any kind of credible promise to player 1 about how they'll act in the second round?
Consider player 2's strategy:
- In the first round, play R.
- If (B,R) was played in the first round, play L; otherwise play C.
1
2
5
2
,
0
0
,
2
4
,
0
0
,
T
M
L
C
B
R
0
6
,
0
0
,
0
0
,
9
0
,
7
9
,
Consider player 2's strategy:
- In the first round, play R.
- If (B,R) was played in the first round, play L; otherwise play C.
Assuming player 2 does this, what is player 1's best response in the second round if (B,R) was played in the first round?
What is player 1's best response in the second round if anything other than (B,R) was played in the first round?
- Play T, since player 2 will play L.
- Play M, since player 2 will play C.
1
2
5
2
,
0
0
,
2
4
,
0
0
,
T
M
L
C
B
R
0
6
,
0
0
,
0
0
,
9
0
,
7
9
,
Consider player 2's strategy:
- In the first round, play R.
- If (B,R) was played in the first round, play L; otherwise play C.
Player 1's best response:
- In the first round, ___________________.
- If (B,R) was played in the first round, play T; otherwise play M.
play B!
If player 1 plays B in the first period, they'll get 7, and then 5 in the second period when (T, L) is played, for a total of 12.
If player 1 plays T in the first period, they'll get 9, and then 2 in the second period when (T, L) is played, for a total of 11.
Infinitely Repeated Games
Infinitely Repeated Games
- A stage game G is repeated an infinite number of times.
- Can't just sum up the payoffs (would be infinite!)
- Approach: use discounting, like we did in week 2
V(\pi_0,\pi_1,\pi_2,\pi_3, \cdots) = \pi_0 + \delta \pi_1 + \delta^2 \pi_2 + \delta^3 \pi_3 \cdots
V(x) = x + \delta x + \delta^2 x + \delta^3 x + \cdots
The present value of a stream of payoffs
(\(\pi_0\) now, \(\pi_1\) in the next period, \(\pi_2\) two periods from now, etc)
may be given by the infinite sum
The present value of a stream of payoffs of \(x\) in every period is:
V(x) = x + \delta [x + \delta x + \delta^2 x + \cdots]
Factor out \(\delta\) from the second and following terms:
What is this expression?
V(x) = x + \delta V(x)
(1-\delta)V(x) = x
V(x) = {x \over 1 - \delta}
The present value of a stream of payoffs of \(x\) in every period is:
\displaystyle{V(x,x,x,x, \cdots) = {x \over 1 - \delta}}
The present value of a stream of payoffs of \(z\) in every period,
starting next period, is:
\displaystyle{V(0,z,z,z, \cdots) = \delta \left[{z \over 1 - \delta}\right]}
The present value of getting
payoff \(y\) now, and then payoff \(z\) in every period thereafter, is:
\displaystyle{V(y,z,z,z, \cdots) = y + \delta \left[{z \over 1 - \delta}\right]}
Write these formulas down!
Strategies in Infinitely Repeated Games
- Strategies must specify what to do in each period,
as a function of the history of all previously taken actions - We often group action histories to make strategy description simpler:
SPNE in Infinitely Repeated Games
- Can't do backward induction!
- Instead: "guess a strategy profile, and "check" that it is a Subgame Perfect Nash Equilibrium
- A simple SPNE: play the Nash Equilibrium of the stage game in every period, for any history
- Can we have a SPNE
in which there is cooperation?
‟Grim Trigger" Strategy (a.k.a. ‟Grudger")
Suppose your opponent is playing this strategy. Is playing this strategy a best response?
Cooperate
Cooperate
1
2
Defect
Defect
5
5
8
0
0
8
2
2
What is your payoff if you too play Grim Trigger?
What is your most profitable deviation?
Get a payoff of 2 in every period.
\displaystyle{V(5,5,5,5, \cdots) = {5 \over 1 - \delta}}
Get a payoff of 3 once, then 1 in every period.
\displaystyle{V(8,2,2,2, \cdots) = 8 + \delta \left[{2 \over 1 - \delta}\right]}
‟Grim Trigger" Strategy (a.k.a. ‟Grudger")
Suppose your opponent is playing this strategy. Is playing this strategy a best response?
What is your payoff if you too play Grim Trigger?
What is your most profitable deviation?
\displaystyle{V(5,5,5,5, \cdots) = {5 \over 1 - \delta}}
\displaystyle{V(8,2,2,2, \cdots) = 8 + \delta \left[{2 \over 1 - \delta}\right]}
When is GT a best response to GT?
\displaystyle{{5 \over 1 - \delta} > 8 + \delta \left[{2 \over 1 - \delta}\right]}
Multiply both sides by \(1 - \delta\):
5 > 8(1-\delta) + 2\delta
6\delta > 3
\delta > {1 \over 2}
If people are sufficiently patient (value future payoffs highly enough), then cooperation can be sustained using a "grim trigger" strategy.
Collusion
What is firm 2's best response function?
P(q_1,q_2) = 14 - (q_1+q_2)
c_2(q_2) = 2q_2
\pi_2(q_2) = P(q_1 + q_2)q_2 - c_2(q_2)
= (14 - q_1 - q_2)q_2 - 2q_2
= 14q_2 - q_1q_2 - q_2^2 - 2q_2
\pi'_2(q_2) = 14-q_1 - 2q_2 - 2 = 0
q_2^* = BR_2(q_1) = 6-\frac{1}{2}q_1
\text{total revenue}
\text{total cost}
\text{marginal revenue}
\text{marginal cost}
2
P
P(q_2|q_1) = 14-q_1 - q_2
MR(Q) = 14-q_1 - 2q_2
MC(q_2) = 2
6-\frac{1}{2}q_1
14-q_1
q_2
"Firm 2's Residual Demand Curve"
Firm 2's "best response function"
Cournot Best Response Functions
BR_1(q_2) = 6 - {1 \over 2}q_2
Firm 1's
best response function
BR_2(q_1) = 6 - {1 \over 2}q_1
Firm 2's
best response function
q_1
q_2
BR_1(q_2) = 6 - {1 \over 2}q_2
BR_2(q_1) = 6 - {1 \over 2}q_1
Nash equilibrium (a.k.a. Cournot equilibrium) is a pair of quantities \((q_1,q_2)\) such that
q_1 = BR_1(q_2)
q_2 = BR_2(q_1)
Profits in Cournot Equilibrium
In Cournot equilibrium, \(q_1 = q_2 = 4\).
\text{Combined profits = }\underbrace{P(q_1,q_2)q_1 - c(q_1)}_\text{Firm 1's profit} + \underbrace{P(q_1,q_2)q_2 - c(q_2)}_\text{Firm 2's profit}
= (14 - q_1 - q_2)q_1 - 2q_1 + (14 - q_1 - q_2)q_2 - 2q_2
Is this the best they could do? Suppose they each produced some amount \(q\).
What value of \(q\) would maximize their combined profits?
\text{Combined profits if they each produce 4 }= \underbrace{(14 - 4 - 4)4 - 2 \times 4}_{\pi_1(4,4) = 16}
+ \underbrace{(14 - 4 - 4)4 - 2\times 4}_{\pi_2(4,4) = 16}
\text{Combined profits if they each produce }q= \underbrace{(14 - q - q)q - 2q}_{\pi_1(q,q)} + \underbrace{(14 - q - q)q - 2q}_{\pi_2(q,q)}
\text{Combined profits if they each produce 4 }= \underbrace{(14 - 4 - 4)4 - 2 \times 4}_{\pi_1(4,4) = 16}
+ \underbrace{(14 - 4 - 4)4 - 2\times 4}_{\pi_2(4,4) = 16}
\text{Combined profits if they each produce }q= \underbrace{(14 - q - q)q - 2q}_{\pi_1(q,q)} + \underbrace{(14 - q - q)q - 2q}_{\pi_2(q,q)}
= 2[(14 - q - q)q - 2q]
= 2[14q - 2q^2 - 2q]
= 2[12q - 2q^2]
= 24q - 4q^2
Take derivative, set it equal to zero:
24 - 8q = 0
\Rightarrow q^* = 3
\text{Combined profits if they each produce 3 }= \underbrace{(14 - 3 - 3)3 - 2 \times 3}_{\pi_1(3,3) = 18} + \underbrace{(14 - 3 - 3)3 - 2\times 3}_{\pi_2(3,3) = 18}
\text{Combined profits if they each produce 4 }= \underbrace{(14 - 4 - 4)4 - 2 \times 4}_{\pi_1(4,4) = 16}
+ \underbrace{(14 - 4 - 4)4 - 2\times 4}_{\pi_2(4,4) = 16}
\text{Combined profits if they each produce 3 }= \underbrace{(14 - 3 - 3)3 - 2 \times 3}_{\pi_1(3,3) = 18} + \underbrace{(14 - 3 - 3)3 - 2\times 3}_{\pi_2(3,3) = 18}
But...if the other firm was going to produce 3, what should you produce?
BR_i(q_{-i}) = 6 - {1 \over 2}q_{-i}
BR_i(3) = 6 - {1 \over 2}3 = 4.5
\pi_i(4.5,3) = 20.25
COLLUSION
If both produce 3...
BETRAYAL
If you produce 4.5 while they produce 3...
NASH EQUILIBRIUM
If both produce 4
x = 18
y = 20.25
z = 16
\displaystyle{V(x,x,x,x, \cdots) = {x \over 1 - \delta}}
\displaystyle{V(y,z,z,z, \cdots) = y + \delta \left[{z \over 1 - \delta}\right]}
If both firms play a "grim trigger" strategy, when can they sustain collusion?
pollev.com/chrismakler
COLLUSION
If both produce 3...
BETRAYAL
If you produce 4.5 while they produce 3...
NASH EQUILIBRIUM
If both produce 4
x = 18
y = 20.25
z = 16
\displaystyle{V(18,18,18, \cdots) = {18 \over 1 - \delta}}
\displaystyle{V(20.25,16,16, \cdots) = 20.25 + \delta \left[{16 \over 1 - \delta}\right]}
\displaystyle{{18 \over 1 - \delta} > 20.25 + \delta \left[{16 \over 1 - \delta}\right]}
18 > 20.25(1-\delta) + 16\delta
4.25\delta > 2.25
\delta > {2.25 \over 4.25} = {9 \over 17} \approx 0.53
What did we show this week?
- Tuesday: some Nash equilibria are not subgame perfect, because they depend on non-credible threats or promises.
- Today: some non-Nash outcomes of a stage game (e.g. cooperation in a Prisoners' Dilemma) can be sustained if the players have an ongoing relationship.
- Sunday: homework for this week due
- Next Monday: review session (time/place TBD, def not on zoom)
- Next Wednesday: Game Theory Checkpoint I
(60 minutes long, in Hewlett 201 at either 5:30 or 7:00)
Econ 51 | 11 | The Economics of Ongoing Relationships
By Chris Makler
Econ 51 | 11 | The Economics of Ongoing Relationships
Sequential Games of Complete and Perfect Information
