Yomi Layer 3: Knowing the Mind of the Opponent
Sirlin
This is not really how Yomi works.Yomi is the Japanese word reading, as in reading the mind of the opponent. If you can condition your enemy to act in a certain way, you can then use his own instincts against him (a concept from the martial art of Judo). Paramount in the design of competitive games is the guarantee to the player that if he knows what his enemy will do, there is some way to counter it.
What happens, though, when your enemy knows that you know what he will do? He needs a way to counter you. He's said to be on another level than you, or another "Yomi Layer." You knew what he would do (yomi), but he knew that you knew (Yomi Layer 2). What happens when you know that he knows that you know what he will do (Yomi Layer 3)? You'll need a way to counter his counter. And what happens when he knows that you know....
Sound like a joke that could never happen in real gameplay of an actual game? Surprise: it's quite common in strategy games. The reason has to do with conditioning the opponent and the inequality of risk/reward in these guessing games (see my article on Rock, Paper, and Scissors in Strategy Games).
Before we get into how ordinary human minds can become entangled in complicated guessing games, let's look at what needs to be there to create these guessing games at all. The designer's tendency might be to create moves and counters. Then create counters to counters, then counters to counters to counters, then counters to those, and so on. Actually, a game need only support counters up to Yomi Layer 3, since Yomi Layer 4 can loop around back to Yomi Layer 0.
Let's say I have a move (we'll call it "m") that's really, really good. I want to do it all the time. (Here's where the inequality of risk/reward comes in. If all my moves are equally good, this whole thing falls apart.) The "level 0" case here is discovering how good that move is and doing it all the time. Then, you will catch on and know that I'm likely to do that move a lot (Yomi Layer 1), so you'll need a counter move (we'll call it "c1"). You've stopped me from doing m. You've shut me down. I need a way to stop you from doing c1. I need a counter to your counter, or "c2."
Now you don't know what to expect from me anymore. I might do m, or I might do c2. Interestingly, I probably want to do m, but I just do c2 to scare you into not doing c1 anymore. Then I can sneak in more m.
You don't have adequate choices yet. I can alternate between m and c2, but all you have is c1. You need a counter to c2, which we'll call c3. Now we each have two moves.
Me: m, c2
You: c1, c3.
Now I need a counter to c3. The tendency might be to create a c4 move, but it's not necessary. The move m can serve as my c4. Basically, if you expect me to do my counter to your counter (rather than my original good move m), then I don't need a counter that; I can just do go ahead and do the original move...if the game is designed that way. Basically, supporting moves up Yomi Layer 3 is the minimum set of counters needed have a complete set of options, assuming Yomi Layer 4 wraps around back to Layer 0.
This is surely sounding much more confusing than it is, so let's look at an actual example from Virtua Fighter 3 (which will almost certainly confuse you even more).
Example of Yomi Layer 3 from Virtua Fighter 3
Let's say Akira knocks down Pai. As Pai gets up, she can either do a rising attack (these attacks have the absolute highest priority in the game) or she can do nothing. A high rising attack will stop any attack that Akira does as she gets up, but if Akira expects this, he can block and retaliate with a guaranteed throw. Pai does the rising kick and Akira predicts this and blocks. Now the guessing game begins.
Akira would like to do his most damaging throw (that's his m), and be done with it. Even though the throw is guaranteed here, all throws can be escaped for zero damage if the defender expects the throw and enters the throw reverse command. The throw is guaranteed to start but Pai might reverse it. In fact, Pai is well aware that a throw is guaranteed here (it's common knowledge), and it's only obvious that Akria will do his most damaging throw. After all, this situation has happened a hundred times before against a hundred Akiras and they all do the same thing. It's really conditioning, not strategy, that tells Pai she needs to do a throw escape here (that's her c1). In fact, it will become her natural, unthinking reaction after a while.
Akira is tired of having his throw escaped again and again. He decides to be tricky by doing one of his very powerful moves such as a double palm, a reverse body check, a two fisted strike, or a shoulder ram (we'll just lump all those into c2). Why does a big, slow move work in this situation? First of all, if Pai does her throw escape and there is no throw to escape, the escape becomes a throw attempt. If her opponent is out of range or otherwise unthrowable for some reason, her throw attempt becomes a throw whiff. She grabs the air and is vulnerable for a moment. One important rule in VF is that you cannot throw an opponent during the startup phase or the hitting phase of a move. So if Akira does a big, powerful move, he is totally unthrowable until after the hitting phase of the move is over and he enters recovery (retracting his arm or leg).
Years after this article was written, a newer version of Akira still wants to do a throw ("m"), but instead opts for Double Palm ("c2").
Back to our story. Akira is tired of getting his throw escaped all day, so he does standard counter to any throw: a big, powerful move. This c2 move does a decent amount of damage, by the way. The next time this whole situation arises, Pai doesn't know what to do. Her instincts tell her to reverse the throw, but if she does, she is vulverable to Akira's powerful move. Rather than go for the standard reverse, Pai does her c3 move: she simply blocks. By blocking, she'll take no damage from the Akira's powerful move, and depending on exactly which move it was, she'll probably be able to retaliate.
So what does Akira do if he expects this? In fact, he needs no c4 move since his original throw (m) is the natural counter to a blocking opponent. A throw is a special kind of move that grabs an enemy and does damage regardless of whether they are blocking. It's specifically designed to be used against an opponent in block who is afraid of an attack.
In summary,
Akira has: throw; powerful move.
Pai has: throw escape; block.
As I tried to show, it's actually pretty reasonable to expect players to be thinking on Yomi Layer 3, 4 or even higher. It's because conditioning makes doing the throw escape an unthinking, natural reaction. But against a clever opponent, you'll have to think twice about doing a standard throw escape, or blocking. The Akira player will do the occasional powerful, slow move just to put his enemy off balance and abandon his instinct to escape the throw. Then Akira can go back to his original goal: land the throw.
Another very interesting property is "beginner's luck." Notice that a beginner Akira in this situation will go for the throw, since that works on other beginners who haven't learned to throw escape. The beginner Akira will never land the throw on an intermediate player, though, since the intermediate player knows to always throw escape. But strangely, the beginner will sometimes land the throw on the expert, since the expert is aware of the whole guessing game and might block rather than throw escape. Of course, the expert will soon learn that beginner is, in fact, a beginner and then he'll be able to yomi almost every move.
Just as a final note on Virtua Fighter to further demonstrate the complexity of its guessing games, I actually greatly simplified the example above. I left out, for example, that Pai could attack with a fast move rather than block. And Akira has another c2 move besides a slow, powerful move. He can also do what's called a "kick-guard cancel" or "kg." This means he can press kick, which will make him unthrowable until his kick reaches recovery phase. If Pai tries to throw, she'll whiff. But then Akira can cancel the kick before it even gets to the hitting phase. Now he's free to act and take advantage of Pai's whiffed throw vulnerability. Now, Akira has a guaranteed throw, putting him back in the exact same situation he began in.
This paper clip guy always seems to know exactly what people are thinking.
The catch is that if Akira does kg-cancel and then goes for the throw he originally wanted to do, Pai will probably not have time to react with a throw escape. It's just too fast. She'd have to be on the next yomi layer. She'd have to expect Akira to throw, enter a throw escape, see the kg-cancel, then immediately enter her next guess (probably an attack or throw escape). Any hesitation and she'd be thrown.
Suddenly forcing the opponent to think on the next yomi layer than were expecting is usually a way to blind them. Even if the opponent was ready for the guessing game I described above, if you do the "kg" trick and suddenly force a new guessing game they weren't ready for, they are likely to do an automatic response that's easy to predict becase they won't have the time or presence of mind to do anything else.
The point I'm making here is that despite Virtua Fighter's absurd complexity, players really are able to think on the levels I'm hinting at. Playing such a game and successfully landing a move because you knew he knew you knew he would do a particular move is the greatest feeling in the world. So design counters and counters-to-counters, and so on, but know that making Yomi Layer 4 the same as Layer 0 allows you to only design counters up to Yomi Layer 3.
Reader Comments (20)
Hi Sirlin. That was a nice read. I was wondering if any fighting game AI has been built upon this method of thinking? Not directly reacting to the game inputs of the player, but coming up with a set of solutions based on its situation and doing it's own guessing.
One solution computers can implement in a Yomi situation is the Nash equilibrium. This uses random numbers to make the computer's mind 100% impossible to read. For example, in rock-paper-scissors, the computer generates a random number - 1, 2, or 3.
1 ---> Rock
2 ---> Paper
3 ---> Scissors
What should the player do? It doesn't matter.
-----------------------------------------------------------
Or, you can try to Yomi them. The problem with computer-Yomi is it's always at one level. For example, the computer could make the rps-society style guess that the player will prefer:
Beginning ----> Rock
Winning -----> Scissors
Losing -----> Paper
Then it will play
Beginning -----> Paper
Losing ------> Rock
Winning ----> Scissors
And will be destroyed by
Begging -----> Scissors
Winning ----> Paper
Losing ------> Rock
--------------------------------
Computer Yomi is too easy to outsmart.
This is assuming that the computer makes the guess of the player playing a certain move through statistical data gathered from its play experiences thus far, correct? What is not to say that the computer can gather further data from its failures of a Yomi Layer 1 move to create a Yomi Layer 3 decision?
Because, there's still a code in their. Players can figure out what that is and predict it with 100% accuracy.
I'm not entirely convinced. I'm going to diverge away from RPS since my original question was about AI in fighting games and I don't think RPS is a good enough context. I will be using the notations from the article: m, c1, c2, c3.
Given the lifetime experiences of that particular computer AI, it will eventually come up with one set of Yomi Layer 1 decisions and one set of Yomi Layer 3 decisions for every situation. Using that information, if a player has a tendency to throw a lot of c2 moves in a given situation; let's say m (30%), c2 (70%), then the computer will use c1 (30%), c3 (70%) in those situations. Not entirely predictable, but the player "could" favor that statistic to his odds until it evens out to 50%, 50% in those situations.
And let's assume the RNG is good enough to meet our needs since we're in a real-time situation.
Play this:
m c2 m c2 m c2 m c2 m c2
Performance each round, scale of 0 to 1:
1/2
1
1/2
2/3
1/2
3/5
1/2
4/7
1/2
5/9
You just did better then 1/2. The computer could have done better by playing randomly.
If utilizing yomi is essentially guessing what your opponent is guessing you're guessing and so on then as far as AI goes you'd need to keep track of how often the player uses M, C2, C2 in response to C1, and M in response to C3 in each "yomi series". You can use these numbers to help create a probability of which response they player is likely to give. Of course the player can use those numbers to trick the AI into setting itself up to be exposed to an attack (which can happen even with a human opponent) so you can throw in a wild card, maybe a 25% possibility, that will cause the AI to use a response contradicting the probability numbers (i.e. there is a 25% chance it will choose C3 instead of C2 as its response, for example). Now the player has the threat of the AI "guessing" (randomly) that the player knows that it knows the player's probable actions and the AI might counter the player's counter-counter.
yomi could lead to some unnusual paradoxes.
quote: JWK5
If utilizing yomi is essentially guessing what your opponent is guessing you're guessing
it is not, it's deciding on what your opponent is going to do, based on logical reasoning, not guessing ^^,
Your statement about how players must guess which Yomi layer their opponent is acting on reminds me of a term used in the game Epic Mafia. Players created the term WIFOM, or "Wine in Front of Me" to describe infinite loops of reverse psychology, where a player may be telling the truth, or may be lying to appear to be telling the truth, or trying to appear as if they're lying so that people will then believe they're telling telling the truth, etc. Not sure if this is really relevant, but I found the similarity interesting.
Good to put a concrete name to the concept; thanks for sharing. Fencing has precisely this layered concept, codified in the "Tactical Wheel".
Attacks and their counters increase in complexity around the wheel until the final counter is a simple direct attack. In high level matches the tactics can whirl clockwise around the tactical wheel as each fencer tries to stay a step ahead of the their opponent's plan. Heady stuff.
After reading this article many times, I still find it rather difficult to understand how it works, especially if the game is turn-based. It seems to me that a game which focuses on yomi will need to be in simultaneous action or in real time.
In a game of rock-paper-scissors, the players' moves are done in simultaneous action. So, if you predict that I will use rock, you will play paper to counter me. If I know that you know I will play rock and will play paper, I can then play scissors, which makes me a layer above you in terms of yomi.
What about in a turn based game? Lets assume that this turn based game is a card game where cards are hidden from the opponent. Lets say on my turn, I play a rock. On your turn, you will obviously play paper. You already saw my move during my turn so of course you will play the counter to my move. There is no guessing games at all! There are people who point out that as long as there is hidden info, there will be yomi since you will try to predict what card is in your opponent's hand. But I am not sure that is this prediction considered yomi. In most turn based game that I play, it is difficult to even have yomi layer 3. Even if the prediction thing is considered yomi, there is probably only up to yomi layer 1 or 2.
I hope some of you can give me some explaination about this thing. Thanks.
@CCGer
I believe yomi is most notably for real-time situations, but it can be used in turn-based situations. You're right about your rps turn-based example, because if you saw that I play rock, you would play paper every time. There's no guessing or prediction there. But that kind of turn-based example has no yomi involved at all. The first thing that comes to mind with turn-based yomi is texas hold-em' poker.
If I made a big bet pre-flop a bunch of times, which inevitably at the end of the hand it is revealed that I had a really big hand (pocket aces, pocket kings, something really good), you might assume I have something equally as good if I make another big bet pre-flop. This is my "m", as I bet big so you hopefully call my bet and lose more money to my much better hand than yours. So if I make another big bet, you should probably fold if you believe that I really do have a good hand. This is your c1, because more often than not, my monster hand will beat your not so monster hand. You're saving yourself money this way in the long run. Again, that's your c1.
But maybe it becomes revealed that recently when I make big bets pre-flop, I actually have bad or decent cards. This is my c2, because now I countered you folding to my big bets, because I could practically just bet every single hand I get, making you fold every single time if you were stuck on yomi level 1. So now the next time I make a big pre-flop bet, you aren't very sure if I have an awesome hand or a bad hand. If you have a good read that I probably have a bad hand, you will call my bet, which is your c3. If I happened to have a great hand, then we are back to yomi level 0 with my "m".
Let me reword the last two sentences of my previous post.
If you have a good read that I probably have a bad hand, you will call my big bet, which is your c3. If I happened to have a great hand in this very same instance and I know I programmed you to start to believe that I am possibly make a big bet with bad hands (c2), then I could throw out a big bet, predicting that you will probably call my bet because you probably believe that my hand isn't good when I bet anymore. That's back to "m" or yomi level 0.
In something as simple as RPS, that's probably true, but when you start throwing in unequal payoffs and in particular, valuation, then you can start predicting your opponent even in turn based situation. For instance, in a board game with an auction mechanic, you can try to predict how much your opponent wants the item being auctioned and drive him right up to that point but not further (assuming you don't want it). He might guess that you don't want it and stick you with it a low price that was still too high for you or he might think you actually want it and so he'll keep increasing his bid even though you're just bidding him up for no real reason. In such a situation, the ability to look at his position, his style, and his current bids and correctly predict how much he actually values the item is essentially Yomi.
Edit: I guess Poker is probably an easier example of turn based Yomi.
Another thing I can think of is "a rock" being not a single card but a combination of cards. With your first card you give an indication that you're going to use rock, but with your second/third/fourth card you can turn that right around to scissors. At some point It might involve a sort of a stack mechanic like in Magic: The Gathering.
Or in the cases it works it wasn't because of yomi but just good old luck. I'd like to see if there is data about yoming vs simple chance. I'm guessing they won't be that different.
klaasvaak: of course it's much different than simple chance, that is the entire point. Some of the best fighting game players are known for their yomi, meaning they get reads way way more often than simple chance. Playing against a Virtua Fighter opponent whose on a higher yomi layer than you is totally demoralizing. It's not like they are just rolling dice and equally able to beat you as some bad player who chooses stuff randomly.
Trip said:
Another thing I can think of is "a rock" being not a single card but a combination of cards. With your first card you give an indication that you're going to use rock, but with your second/third/fourth card you can turn that right around to scissors. At some point It might involve a sort of a stack mechanic like in Magic: The Gathering.
Can you give me some examples of how a combination of cards in MTG becomes a "rock" or "scissors"? And how do you counter them with yomi? Are there yomi layer 3 in those situations? I still find yomi difficult to understand in turn-based games.
Thanks.
For MTG the classic yomi situation is Counterspell. You want to play your big game winning card (m), and are tired of losing when I counter it (c1). If you predict I am going to counter you can play a cheaper card (c2), hoping to beat me next turn and ideally make me waste my Counterspell. But if I KNOW all that, I can make your Game Winning Card rot in your hand just by leaving two islands untapped and smirking at you (c3), even when I have nothing. In poker terms I am bluffing by representing the hand "Counterspell". The response is to play your card and win the game (m)...