AI performance

Members can make suggestions for improving the site and improving the forum as well as submit bug reports to be reviewed by our support team here.
Post Reply
Message
Author
User avatar
kestasjk
Developer
Developer
Posts: 518
Joined: Tue Mar 14, 2017 8:13 pm
Location: Perth, Australia
Karma: 775
Contact:

AI performance

#1 Post by kestasjk » Sun Dec 11, 2022 5:24 am

Hi all,

With FAIR winding down their AI research they’ve decommissioned their FairBot which used to run our FvA GvI games. As a stopgap I subbed in the MILA bot just to see if it worked at all, and it looks like it does and at least plays sensibly.

However I’ve never played an FvA game to any level of competence, so I don’t know if “FairBot” is effectively broken now. Can someone decent at FvA/GvI give an AI game a go and let me know how it goes?


Note we are trying to host the FAIR no-press bots ourselves, just like we’re hosting the MILA bots right now, as soon as possible to provide a higher difficulty level.
We also want to provide CICERO games asap, but that will be trickier due to the heavy hardware requirements. In order to be the first site to offer this, which I think webDiplomacy should be, we may need to have a system where we integrate into AWS/Azure so that we can quickly spin up a 56 GPU system to play a game for just the brief periods we need.

In this way we might be able to have CICERO full-text games, but we’d need to figure out how much the IaaS costs would be since these cloud services are often a lot pricier than you would guess. You think okay I’ll spin up 56 CPUs, use it for a few minutes, then shut it down, that should cost a few minutes, but they always have tricks where the minimum time period is an hour so that you get these huge unexpected bills.

Anyway any feedback on FvA/GvI performance would be appreciated.
4

Trigfea63
Gold Donator
Gold Donator
Posts: 330
Joined: Fri Apr 02, 2021 11:17 pm
Karma: 76
Contact:

Re: AI performance

#2 Post by Trigfea63 » Tue Dec 27, 2022 4:21 pm

I'm not sure how I missed this post, but I'm happy to provide some feedback on the new Fairbot. It's not broken, first of all. In fact, it's response time is quite a bit faster. The old Fairbot would often take 10-15 minutes to enter orders in the mid-to-late game stages. The new Fairbot enters orders more or less immediately, similar to the Kestas Bot AI in GvI.

The quality of the AI's play has dropped considerably. The old Fairbot was close to unbeatable. You had to play a near-perfect game, and even then, you normally wouldn't win. The new one is a decent enough player, but it doesn't challenge you in the same way. A good FvA player will beat it almost every time.

If I could recommend an ideal to shoot for, it would be something closer to the old Fairbot, but not quite so strong. After a while, it gets demoralizing to lose game after game. :( Even better, if it's feasible, would be a field in the Create-a-Game form where you can choose the AI's level.
1

dipperjay
Gold Donator
Gold Donator
Posts: 81
Joined: Sun Dec 31, 2017 4:32 am
Karma: 52
Contact:

Re: AI performance

#3 Post by dipperjay » Tue Dec 27, 2022 11:56 pm

Trigfea63 wrote:
Tue Dec 27, 2022 4:21 pm
I'm not sure how I missed this post, but I'm happy to provide some feedback on the new Fairbot. It's not broken, first of all. In fact, it's response time is quite a bit faster. The old Fairbot would often take 10-15 minutes to enter orders in the mid-to-late game stages. The new Fairbot enters orders more or less immediately, similar to the Kestas Bot AI in GvI.

The quality of the AI's play has dropped considerably. The old Fairbot was close to unbeatable. You had to play a near-perfect game, and even then, you normally wouldn't win. The new one is a decent enough player, but it doesn't challenge you in the same way. A good FvA player will beat it almost every time.

If I could recommend an ideal to shoot for, it would be something closer to the old Fairbot, but not quite so strong. After a while, it gets demoralizing to lose game after game. :( Even better, if it's feasible, would be a field in the Create-a-Game form where you can choose the AI's level.
I completely concur with Trigfea63’s post, would love to see a stronger AI compared to the current level, but not as unbeatable as the old Fairbot.
1

User avatar
kestasjk
Developer
Developer
Posts: 518
Joined: Tue Mar 14, 2017 8:13 pm
Location: Perth, Australia
Karma: 775
Contact:

Re: AI performance

#4 Post by kestasjk » Tue Jan 03, 2023 11:20 am

Yep well spotted. With FB wrapping things up unfortunately they stopped hosting the bots.

I'm not sure exactly what happened, whether they cut them off or our new proxy interfered somehow, or the site traffic increased, but the bots shut down in sets.
At the same time the old MILA bots that the original AI bot developer created were getting slower and slower due to the larger number of games going on.

What I did is got the MILA bots running on our server instead, which I had been working on in the background for a while, and replaced the other bot AIs with the MILA AI until I can get the FairBot etc models up and running ourselves.
I also spoke to Philip Paquette about possible ways to make his bot easier for newcomers, which was suggested on the DiplomacyGames podcast, and he said there is a "temperature" setting of some kind that should do just the job.


The dev priority right now is getting the multi-accounter / meta-gamer / relationship etc business sorted, but I've written back to Meta about getting access to these bots (which they suggested themselves), and if it's similar to how the MILA bots are implemented I might be able to get them back up fairly quickly.

I'll rename it to FairBot once it is FairBot again.
1

Post Reply

Who is online

Users browsing this forum: No registered users and 76 guests