Both programs took big steps toward the endgame in the last several years. In January 2015, Bowling’s team published a paper showing how it had solved heads-up limit hold’em, a two-person poker game that is simpler than no-limit hold’em because of restrictions on how players can bet. Sandholm and Brown, a Ph.D. student who has been working with him on poker AI for the last five years, held their first “Brains v. AI” competition against top humans at Rivers Casino several months later. Their bot, named Claudico, lost $732,000 over 80,000 hands played against four professional players. Sandholm said the match was close enough to call a draw, a claim that at least one player disputed.
Sandholm and Brown say there are several general areas their AI has improved since then. Claudico played well in the early stages, but tended to make mistakes at the end of hands. It bluffed at the wrong moments, and had trouble accounting for how the odds of the game changed based on the cards that it knew had been removed from the deck. In its simplest form, this is the reasoning that says that if there are two kings on the table and you have two kings, your opponent can't have any. Libratus has improved in all those areas. Its creators remain coy on some other areas, like specifically how it chooses to make adjustments based on what it learns over the course of a day of play.
All the details of Libratus will eventually be revealed when its creators publish their findings. This kind of academic work tends to filter into real-world poker in various ways. The Annual Computer Poker Competitions have included entrants that also play in cash games, according to Brown. Bowling said his research papers are popular on message boards for people building bots. “There’s this whole separate group of people reading these papers and trying to understand them,” he said.
Billings joined the poker industry in 2008. He’s one of a handful of people who did so after leaving the University of Alberta's program. Most of them have been hired by companies that run gaming platforms. Richard Gibson struck out on his own, starting a company called Robot Shark Gaming that built AI programs for studying and playing strategic games, and then a fantasy sports company called SportsBid.
Gibson was finishing up a Ph.D. in 2013 when a group of professional players approached him, offering to pay for software they could use in training. Gibson was only given one person’s name, never met any of his clients in person, and isn’t sure how many people were in the group. “Even though they weren’t using it to gamble online, there was a stigma,” he said.
Gibson built multiple programs, and said he designed the software to demonstrate the effectiveness of various strategies; it couldn’t play on its own. In his most lucrative year, Gibson made about $100,000 on that project, and his clients paid another $20,000 to $30,000 in fees related to the computing power it took to run the software.
The anonymous pros weren’t Gibson’s only clients. In one case, he said someone paid him tens of thousands of dollars to spend about six months building a lightweight poker bot. He didn’t ask much about how it would be used — he didn’t want to know — but the design pointed to a specific application. “My clients wanted a standalone thing they could load onto their laptop,” he said. “I imagine they’re trying to play online with them.”
At the end of each night at Rivers, Les and his fellow poker pros would order takeout and pore over data about the day’s action in search of Libratus’s weaknesses. Early in the month, they woke up each morning optimistic that they had some new tricks. “There were specific exploits we identified in the first few days,” said Les. “We attacked them and attacked them, and now they’re gone.”
Libratus was also making adjustments. During the day, the program split its computing power between playing the hands in front of it and what Sandholm described as “continuous strategy improvement.” At night, the program focused entirely on strategy, using 600 nodes of the supercomputer, the equivalent of about 3,330 high-end Macbooks working in tandem.
In poker, as in other games that AI has played at the top levels, computers have developed strategies that filter back to human players. Les said he’s trying to figure out how to adapt some of Libratus's irregular betting behavior to his own game. It’s hard. "We just simply do not have the mental capacity to do it,” he said.