Nash vs ChatGPT: who wins?

I have always been intrigued by AI's problem-solving capabilities. As per this report by the Analytica India Magazine, Gemini 2.5 pro outperformed the JEE (entrance exam to the Indian Institute of Technology) topper, scoring 336.2 out of 360 marks. As an educator, this excites me because students now have an additional tool to practice problem solving and verify the accuracy of their solutions.

In this context, I decided to put AI to the test to solve some problems based on game-theoretic (multi-agent) models. I asked GPT-4o to solve two problems sourced from Game of Strategy (3rd edition) by Dixit et al, which is a standard textbook prescribed across multiple business schools to discuss applications of game theory. The results were surprising!

Problem 1: The survivor game/ the stick game/ the cookie jar game

Prompt: Two players, A and B, play the following game with a jar of 100 cookies. The players take turns, and A goes first. Each time it is a player's turn, she takes between 1 and 10 cookies out of the jar. The player whose move empties the jar wins. (a) If both players play optimally, who will win the game? Explain.

For this problem, chatgpt gave an accurate answer with a detailed explanation (given below)

Chatgpt's response:

In this game, Player A and Player B take turns removing cookies from a jar containing 100 cookies. The goal is to be the player who takes the last cookie from the jar. Each player can take between 1 and 10 cookies on their turn.

To determine the winner when both players play optimally, we can analyze the game using a backward induction approach:

1. Winning and Losing Positions:

▪ A position is a winning position if the player whose turn it is can force a win with optimal play.

▪ A position is a losing position if every possible move leaves the opponent in a winning position.

2. Analyzing the Game:

▪ If there are 0 cookies left, the player who just played loses (no cookies to take).

▪ If there are 1 to 10 cookies left, the current player can take all the cookies and win.

▪ If there are 11 cookies left, no matter how many cookies (1 to 10) the current player takes, they will leave the opponent with a winning position (1 to 10 cookies). Thus, 11 is a losing position.

3. Continuing the Analysis:

▪ From 12 to 21 cookies, the current player can take cookies to leave the opponent with 11 cookies (a losing position). Thus, positions 12 to 21 are winning positions.

▪ For 22 cookies, any move (taking 1 to 10 cookies) will leave the opponent with a winning position (12 to 21 cookies). Therefore, 22 is a losing position.

▪ This pattern continues, establishing losing positions at intervals of 11 cookies (11, 22, 33, ..., 99).

4. Conclusion:

▪ The losing positions are 11, 22, 33, 44, 55, 66, 77, 88, 99.

▪ Since 100 is not a losing position (it is a winning position), Player A can take cookies in such a way to leave Player B in a losing position.

Final Outcome:

Player A will win the game if both players play optimally. Player A can always make a move that leaves Player B with a losing position (11, 22, 33, ..., 99).

I was impressed with the accuracy of the answer and the lucidity of the explanation (to the extent where I wondered if I would have been able to explain this in class with this level of clarity).

Impressed by this experience, I attempted problem number 2, which involved finding the Nash Equilibrium by creating a payoff matrix.

Problem 2: An old lady is looking for help crossing the street. Only one person is needed to help her; more are ok, but not better than one. You and I are the two people in the vicinity who can help; we have to choose simultaneously whether to do so. Each of us will derive pleasure worth a 3 from her success (regardless of who helps her). But each one who goes to help will bear a cost of 1, this being a value of our time in helping. If neither player helps, then the payoff for each player is zero. Set this up as a game with a payoff table and find all pure strategy Nash equilibria.

ChatGPT did a great job converting the real-life context into a payoff table to analyse and find the equilibrium outcome.

However, ChatGPT couldn't figure out all the equilibrium outcomes. As it turns out, this game has two equilibrium outcomes {N, H} and {H, N}. But the AI engine suggested only (N, H) as the equilibrium solution (response shown below)

To this, I responded that the above answer seems incorrect. At this point, the GPT was able to correct itself and offer both (N, H) and (H, N) as the equilibrium solutions. I was quite impressed with its ability to correct itself.

At this point, I wanted to check if it was giving the correct solution based on the actual logic, or if it was just a coincidence. Hence, I further prompted, "Are you sure about this answer?" If the AI engine had identified the logic, it should have reproduced the same result (as the result was accurate this time).

However, to my surprise, it changed the solution and gave a very inaccurate solution that there are no pure-strategy equilibrium solutions to this problem (response given below)

My observation from this experiment:

The AI engine did a far superior job of solving problem 1 compared to problem 2. Problem 1 was relatively straightforward since it is a sequential game where players make decisions sequentially, and solving it backwards simplifies the analysis. Problem 2, however, involved analysing a situation where agents took decisions simultaneously.

From this observation, I conjecture that AI's current ability to solve problems involving simultaneous decisions is less developed compared to its ability to solve problems involving sequential decisions.

Evidently, it is not a good idea to rely blindly on the solutions provided by AI engines (especially when it comes to solving complex problems), as they could provide a confidently worded, yet inaccurate solution (as seen in the above case).

Limitations: I used only one engine, and I am unsure if the results would have been any better had I used a different AI engine. Hence, one might question the generalizability of this conclusion.

Problems were sourced from the book "Game of Strategy" by Dixit et al., a highly recommended resource for learning game-theoretic applications in business and other contexts. (https://wwnorton.com/books/Games-of-Strategy/)