Poker Bots Evolution – From History To The Future
20 minutes
Posted by: Ivan
Online poker has come a long way since its somewhat humble beginnings. In fact, when virtual tables first appeared in 1998, no one could even imagine just how far it would go.
Over the next couple of decades, online poker has turned into a multi-billion industry that has seen countless technical innovations.
It is not a surprise that everyone has their own interest and different ways to reach it:
- Players have mostly been focusing on poker training to get better and make more money
- Companies spent their time developing better poker software and bigger tournaments
- Scientists spent their time developing poker bots and algorithms to counter this game
The last group is what we are going to cover today. These scientists, or at least most of them, weren’t motivated by potential financial gains from the table.
A coincidence or not, online poker turned out to be an excellent platform for testing new and improved Artificial Intelligence concepts that couldn’t be properly applied in games such as chess or even Go.
This is how the history of poker bots began and grew to advanced software solutions that you can find today.
To make things easier, we created an infographic that can quickly show you the development of this sector, but if you want to learn even more details, feel free to read the whole article.
Due to its very complex decision tree, poker, especially Texas Hold’em, represents a very demanding environment. Developing a functional poker bot that could stand up to competent human opponents has long been dismissed as a pipe dream.
Today, however, these AI bots playing poker that can hold their own and even come out ahead against top human players. So what should you know?
Early Days of Poker Bots
Although scientists have been toying with the idea of poker robots since the mid-1980s, the first important breakthroughs in this field happened in the 2000s.
Up until then, a majority of the AI research was focused on chess, the game that was an excellent platform for such research.
The problem with chess, however, is that there is a finite number of options in the game (as big as it might be).
With computers becoming faster and more robust, it was only a matter of time when AI would become superior to human opponents while playing chess.
It did happen eventually, and these days even the best of the best, such as the young champion Magnus Carlsen, admit that it is pointless to play against computers.
So, scientists had to find a new game to turn to.
As the popularity of poker was growing in the 2000s, a bulk of researchers started to turn that way. No-Limit Hold’em was a new and exciting platform that provided a challenge, and scientists were looking to take it.
The World Series of Poker AI Robots
The first significant event to mark the rise of AI poker bots happened in 2005.
Golden Nugget organized the World Series of Poker Robots in 2005 as the first public poker bots event.
This was an interesting event gathering many scientists who traveled from all over the globe to test their bots. These AI players were pitched against each other in the event that was free to enter and had a prize pool of $100,000.
Eventually, there was just one robot still “standing” after the match. It was a piece of software engineering featuring a rather unimaginative name – PokerProBot.
Then the bot went on to square off against Phil ‘Unabomber’ Laak in an exhibition match. Although the game lasted for a while, Laak was able to come out ahead and claim the victory.
Phil Laak explained that he was able to recognize many leaks in the software that allowed him to get away with kind of plays he couldn’t pull against human opponents.
Still, even during these early days, it was clear bots could become a big part of the online poker future.
The creator of PokerProBot Print Given even admitted that his AI was raking in money playing at real money online poker sites at the time. This was long before poker rooms were actively trying to get rid of poker bots, so he was able to get away with it.
Heads Up Poker with Polaris Bot
In the next few years, the research of poker bots would become more severe as universities and research centers started to get involved. The University of Alberta (UoA) was one of the first ones, and they’ve made some of the most significant contributions to the development of poker AI.
Their first major effort came in the form of Polaris, a limit poker bot that took on a couple of human opponents in a heads up Limit Hold’em match.
The first player to go up against the machine was once again Phil Laak, and the second opponent selected for this challenge was Ali Eslami. These test matches happened in 2007, and the bot lost 2:1, with one draw.
In 2008, a new and improved version of Polaris took on six human opponents, and this time around, the machine was victorious, with three wins, one tie, and two losses.
However, Polaris was only able to perform well in Limit Hold’em. Its capabilities in the no-limit variation of the game were quite limited and no match for even a solid human player.
Limit Hold’em is a much easier game to solve mathematically with fewer options, so no one was too surprised that a computer would perform well in it. Even preflop hand selection is much easier, were in No-Limit games you want to have a solid starting hand poker cheat sheet.
But No-Limit Hold’em? That was a whole different story. Given so many possibilities on each and every street, it just seemed impossible that a piece of software, no matter how good, could compete with capable human players.
After all, there is more than just pure math to it: adjusting to your opponent, understanding their play style, and adopting new strategies to counter that style effectively.
Next Generation of Poker Bots
After the events of 2008, the situation with artificial intelligence poker bots seemed to have died down a bit. There hadn’t been many new developments worth mentioning, and it seemed almost as if scientists have moved on to different projects that didn’t involve poker.
As it turned out, several researches were going on behind the scenes, but developers were waiting for the right time to come out with the information. It happened in 2015 when the poker public was introduced to Cepheus.
Cepheus: HU Limit Poker Solved?
Cepheus was a project developed by a research team from the University of Alberta. After putting in a lot of hard work behind the scenes, they finally came out with a poker bot that they claimed essentially solved heads up limit poker games. So, once again, this was an LHE bot.
The group behind Cepheus managed to develop a strategy that was very close to Nash equilibrium.
It created a setup where even playing optimally against the bot would only produce a very small, virtually negligible win-rate.
The way developers went about it was by giving the program a chance to learn all the possible situations in a heads up scenario. The total number was 316 quadrillion possibilities, and it took Cepheus over two months to learn all this powered by 200 processors. In the end, the database grew to 11 terabytes.
Although this poker bot was pretty much unbeatable, some experts, including David Sklansky, emphasized that this didn’t make it the best HU Limit Hold’em player.
While Cepheus won’t lose and it will win in the long run against most players, it lacks the ability to adapt that human players have. So, a good human will beat a bad player much faster than Cepheus.
This was an essential step for scientists as they successfully created pretty much unbeatable poker bot. It was discussed in the twoplustwo forum at large. However, this wasn’t that big of a deal for the poker community.
Heads up Limit Hold’em was hardly ever a popular variation, so even if the bot were much better, it wouldn’t represent much of a threat to the poker economy.
Crossing the Limit with Claudico
Although some touted Cepheus as something that would change the game of online poker forever, most players and experts didn’t share this opinion.
Differences between Limit and No-Limit Hold’em are vast, and the fact that a bot could analyze poker hands well in the limit variation wasn’t much of an indicator that scientists were close to solving NLHE.
But then came 2015 and Claudico.
For the first time, researchers presented a heads up no limit bot.
This was a huge leap, and the poker public watched with great interest what would come of it.
During the late April and the first week of next month, a special event was set up at Rivers Casino in Pittsburgh. The idea of the event was to see if Claudico was ready to take on some of the best human players. It was the ultimate test for this poker bot.
Four top heads up players were selected to participate in this challenge:
- Doug Polk
- Dong Kim
- Bjon Li
- Jason Les
The rules were that the bot would play 750 hands against each of the opponents every single day, for a total of 80,000 hands (20,000 hands per player).
The setup was designed to minimize the influence of short-term luck. Each day, there were two sets of identical hands prepared. In one match, Claudico would play with set A against the opponent playing with set B. In the second match, roles were reversed, so the human would play set A while the computer played with set B.
There was a lot of debate about who would come out on top. The team behind the poker-playing AI gave its poker bot about a 50% chance to beat the players.
Polk, on the other hand, was convinced that Claudico couldn’t hold its own against some of the best HU specialists in this group.
In the end, the “humanity” won by a decent margin of over 700,000 chips.
However, professor Sandholm, who was leading the team Claudico, said that the final outcome wasn’t enough to declare a decisive victory and instead declared the match a “statistical tie.”
In the aftermath of the match, Polk commented on the bot’s performance and said the AI played well in certain situations. However, he also explained he noticed some illogical moves, especially with regards to bet sizing.
Sometimes, Claudico would make extremely large or unusually small bets in spots where no human would do that. Regardless of that, Claudico was able to perform reasonably well against human opponents
Enter Libratus: Turning the Tide for Poker Bots
After player beating Claudico, poker fans around the world breathe the sigh of relief. Computers were still not able to beat the humans in No-Limit Texas Hold’em.
But, in 2017, the team of scientists from Pittsburgh’s Carnegie Mellon University (CMU) came up with a brand new poker bot. This time around, it was called Libratus, freely translated as “balanced.”
Although it used some of the programming elements from Claudico, Libratus was a brand new and greatly improved poker-playing AI.
Once again, a challenge was issued to humanity.
Players selected this time around included two names from the Claudico match (Les and Kim) and two new players: Jimmy Chou and Daniel McAulay.
The setup was similar to the first match, but the total number of hands was raised to 120,000 to get more statistically relevant results. All four players got to split $200,000 prize pool as well: $20,000 each, and the rest would be divided based on their performance against the AI.
Players believed they’d have no problems defeating the poker bot once again, but they were in for a big surprise.
It turned out Libratus was far superior to Claudico. It would play matches during the day but spent nights analyzing the games and improving its strategy.
In the end, Libratus took a decisive victory with over 1.7 million in chips ahead of human players.
More importantly, it managed to beat every individual player. Dong Kim was the best of the team humans, with a loss of just 85,000.
For the developers, this was a huge success as it showed that the fundamental programming of the AI bot was very solid. It meant that poker software could be used in many areas other than poker. The game has just been a testing platform for bigger goals.
But for the poker community at large, a bot such as this wasn’t welcome news. If the AI were able to beat some of the best players, it would probably be able to dominate online tables.
Luckily, Libratus was still very demanding on resources and not something an average person could get their hands on and run on their home PC. Additionally, the poker bot was limited just to heads up games, which still make up a relatively small fragment of the overall online poker market.
But all of that was about to change.
Going Multiway: Pluribus Poker Bot
The latest milestone in the development of artificial intelligence poker bots came in 2019, and it was pretty much the embodiment of many online players’ deepest fears. Named Pluribus, this was a 6-max bot, and it was actually pretty good!
Pluribus was another product of researchers from Carnegie Mellon University, but this time around, they put their forces together with the Facebook AI lab to produce the ultimate poker playing computer bot.
The developers took a fascinating approach with Pluribus, which is quite frightening to the players if you think about it.
Instead of teaching it poker by force, they simply taught it the rules of Texas Hold’em and then let it learn everything about the game playing against itself.
The end result was better than expected.
The new poker bot took on human players in two different scenarios:
- In one, Pluribus played against five human opponents.
- In the other, a human player was pitched against five different iterations of the AI.
Some of the players involved in this epic match were Greg Merson and Nick Petrangelo, alongside several other names, all with more than $1,000,000 in winnings. So, Pluribus was going against strong opponents.
In the end, though, it didn’t matter.
In the first scenario, five humans vs. one AI, Pluribus played 10,000 hands across 12 days. When it was all said and done, Pluribus was showing the win-rate of 48 mill-big blinds per game.
In the second round, Chris Ferguson and Darren Elias played 5,000 hands each against five bots, and once again, the AI triumphed.
Perhaps Pluribus didn’t crush the human competition, but it did come out victorious and showed that poker bots are reaching the next level. Many of the pros that participated in the challenge stated later that Pluribus was really tough to play against – more robust than most humans they had to face.
But there is another thing that makes this poker bot different from all that came before it.
Pluribus doesn’t require a huge amount of resources. It can run on two CPUs and requires less than 128 GB of RAM. So, it can easily be set up and do its thing on a cloud server at the cost of around $150 a month. So that might be the first real sign of danger for online players.
Poker Bots Today: Real Danger for the Games?
This lengthy overview of how poker bots have developed over the years provides some valuable insights about how these programs actually operate and what you can expect from them going into the future, which is going to be very different from what we have seen.
Back in the early days of online poker, discovering a primitive bot at your table was a license to print money.
These bots were terrible and easy to beat, where everyone can do that by learning a couple of essential poker tips. Times have changed, however, and you have to adapt to the new situation.
Poker has been turned almost into a science, largely thanks to the players who spent countless hours trying to figure out the best way to play the game.
Various solvers and other software out there can be very useful for those wanting to learn, but they also make poker more AI-friendly.
For some time now, poker bots have been present at online tables, especially in cash games. Although these cases are often tucked away as rooms don’t like bad publicity, there is more than enough evidence out there that bots do exist.
Some of them aren’t as good as the others, but the latest developments in this field indicate that bots will only get better moving forward.
The question is: should bots be allowed, or should rooms do everything they can to stop them?
A majority of players are against bots and don’t want them in their games – and I must agree with the majority on this one. While you could say that bots are fair game as they aren’t cheating (i.e., they’re playing by the rules of the game), that’s an oversimplification, and in my book is almost the same as making online poker rigged.
Poker, in any of its numerous forms, is a game of people.
It’s not just about math and numbers, at least not if you’re a human. Playing poker also means dealing with your emotions, handling bad beats, hiding your tells, and physically enduring lengthy sessions fighting fatigue and discomfort.
Poker bots don’t have to deal with any of this, so they have a huge starting advantage.
For them, the game is a simple sum of 1s and 0s. They don’t care if their bluff doesn’t go through three times in a row. They’ll have no fear of trying it again if it is mathematically correct.
Poker bots never hesitate to pull the trigger based on how the session has been going on thus far, and that is a rare quality for human players.
A human player can never play perfect poker. Even if they were capable of doing all the math just as well as Pluribus, the bot would still have an advantage because of all these other factors.
A human being can develop high levels of control over their emotions and can do a lot to become physically fitter, but they’ll never reach the point where these things don’t exist. If they did, I guess they’d almost become a new life form.
Fighting a (Losing) Battle?
Poker rooms realize that real players don’t want to play against machines. At this point, it almost becomes an equivalent of playing with a stacked deck. But their efforts to battle bots are often quite limited.
The reason for this is twofold:
- First of all, with AI poker bots getting better, detecting them gets harder and harder.
- Many rooms, especially smaller ones, have limited resources they can put into fighting bots.
That’s why it’s not uncommon to hear stories about bot rings on many lesser-known sites, and especially on unregulated poker apps.
Secondly, poker rooms want and need traffic. Players, artificial or real, take up seats at the tables and help generate the rake. If a room is struggling with player numbers, they might be more lenient on bots.
It may not be the right thing to do, but if these businesses are faced with a decision to allow poker bots or go bankrupt, I am sure you can guess what they pick.
At this point, it does seem like poker bots might become a part of online poker. There are many rumors that AI players are already quite present at the virtual felt, and this isn’t likely to change.
Looking at trends in other industries, it would be somewhat unrealistic to expect online poker would somehow go back to what it used to be a decade ago. Operators may try to do their part in fighting bots, but at the end of the day, there is only so much they can do.
What does this mean for the future of online poker?
It is tough to say. It’s unlikely people will stop playing online just because there are some bots in the mix unless it becomes a huge issue, and the AI floods the internet.
It is worth mentioning that even if you have a bot at your table, it surely does not mean that you will lose.
If there are recreational players, the bot will take only part of their money, and serious players can surely play profitably alongside these bots.
However, professionals may be less incentivized to play as this would make a significant dent in their expected winnings and poker bankrolls. But no one can say anything for sure yet.
Detecting & Countering Poker Bots
Artificial intelligence poker bots are getting better by the day, which means that filtering them out from actual players is becoming harder and harder. You may be able to recognize a lower quality bot on your own, but top-of-the-shelf stuff isn’t as easy to spot.
Often, bots operating in the same ring will have very similar tendencies in similar situations with regards to bet sizing, frequencies, etc. SO there is a couple of ways you can actually spot it:
- If you find players who have identical stats over meaningful hand sample size and plays the same strategy in all situations, this might be worth looking into
- If you find players who always play GTO poker strategy and never deviate from it, that is very suspicious. No human player can play it perfectly all the time, so that could be a sign as well.
- If someone always takes the same time to commit the action on all decisions (however, that is easy to program)
- If the player never responds to a chat even when you announce your suspicions (it does not mean it is a bot, but I would react if someone raises such doubts)
- If a player is constantly playing unbelievably long sessions or an unrealistic number of tables
Something like this does require a lot of time and effort, which is probably why many bots go undetected for some time.
When discovered, poker rooms will usually act against these bots and ban them. Sometimes they’ll even confiscate whatever funds they can from these accounts and reimburse some of the affected players.
So, the best way to counter poker bots in today’s environment is for the players to work together on uncovering them and bringing them down.
Other than this, you could try to figure out if a player you suspect to be a bot has certain leaks in their game and exploit them. I wouldn’t recommend doing this, though, as it could easily backfire unless you’re very experienced with this type of analysis.
It is much easier to mark any accounts you think could be bots and stay away from them as much as you can.
The good news for tournament players is that bots haven’t entered the MTT scene that much just yet. Tournament poker is more complicated because of continually changing circumstances, and the high variance that’s inherent to MTTs doesn’t make it that appealing.
So, if you’re worried about bots, you could also consider switching to tournaments for a while to see how everything develops.
Summary: Stay Alert but Don’t Panic
Yes, poker bots are real, and yes, they are getting better and more sophisticated every single day.
However, there is no reason to panic just yet.
Online poker isn’t being conquered by the robots. A majority of your opponents are made of flesh and blood, so in order to beat them, you just have to get better in your poker game.
Even if you face what you suspect might be a bot, don’t panic and try to think about bots as solid players that you want to avoid.
If you can, don’t sit in the same games with them or try not to get involved in questionable hands. And, if you feel like there are too many bots where you usually play, you can always switch to a different site, since bots don’t riddle most of them.
You can connect with Ivan via his LinkedIn profiile and follow him on Twitter.
- Poker Squeeze Play – Which Hands Make the Most Sense for Squeezing? - August 10, 2023
- How to Play Low Pocket Pairs in Texas Hold’em - July 29, 2023
- How to Make Deep Runs in MTTs More Often - July 22, 2023
