Bots are broken tonight
-
- Posts: 10
- Joined: Thu Dec 22, 2022 9:03 am
- Karma: 4
- Contact:
Bots are broken tonight
Bots don't seem to be entering moves tonight. I played an all-bots AI game, and nothing happened (with 3-day turns.)
I then started a game with five-minute turns, invited humans, and used "fill with bots". The bots used up all their missed-turns and then crashed into civil disorder.
(Kinda fun playing against one other guy with a bunch of sleeping units, but not what I wanted to play, of course!)
I then started a game with five-minute turns, invited humans, and used "fill with bots". The bots used up all their missed-turns and then crashed into civil disorder.
(Kinda fun playing against one other guy with a bunch of sleeping units, but not what I wanted to play, of course!)
- kestasjk
- Developer
- Posts: 522
- Joined: Tue Mar 14, 2017 8:13 pm
- Location: Perth, Australia
- Karma: 788
- Contact:
Re: Bots are broken tonight
Thanks; we had to update the server and the docker bot container can't connect out, trying to resolve it now
- kestasjk
- Developer
- Posts: 522
- Joined: Tue Mar 14, 2017 8:13 pm
- Location: Perth, Australia
- Karma: 788
- Contact:
Re: Bots are broken tonight
Yeah some weird firewall issue, but once it starts it's okay so the bots are now running again. Looks like they're catching up now
-
- Posts: 10
- Joined: Thu Dec 22, 2022 9:03 am
- Karma: 4
- Contact:
Re: Bots are broken tonight
Yup, so far so good! Also, thank you for your work on making bots that play pretty well. Super fun and helpful to be able to play many, many rounds of Dip. (Looking forward to playing against Cicero when that finally happens. Don't discount a paid edition of WebDip -- it'll be worth it!)
- kestasjk
- Developer
- Posts: 522
- Joined: Tue Mar 14, 2017 8:13 pm
- Location: Perth, Australia
- Karma: 788
- Contact:
Re: Bots are broken tonight
Yeah I really like having bots on the site to play against, and as a software dev it's good to learn more about Docker and AI in a hands-on way.ShipOfTheseus wrote: ↑Sat Feb 04, 2023 6:11 pmYup, so far so good! Also, thank you for your work on making bots that play pretty well. Super fun and helpful to be able to play many, many rounds of Dip. (Looking forward to playing against Cicero when that finally happens. Don't discount a paid edition of WebDip -- it'll be worth it!)
Based on the recent diplomacygames interview with Noam I'm less hopeful Cicero will be able to be run at scale on the site. He said that even running the model takes 56 GPUs, and I was previously hoping based on the code that that was just the requirement for training. (Usually training takes crazy processing power, but running the model is okay)
I think the best reasonable outcome is that we have a Cicero instance that only plays a small number of games then caps out, and only a limited number of messages per game. That way it won't be able to play live, it'll be 56x slower with 1/56th the processing power, but at least people will be able to play against it.
Edit: Hmm, I get this when running a single Cicero bot:
Code: Select all
RuntimeError: CUDA out of memory. Tried to allocate 32.00 MiB (GPU 0; 7.79 GiB total capacity; 5.54 GiB already allocated; 30.25 MiB free; 6.06 GiB reserved in total by PyTorch)
![Sad :(](./images/smilies/icon_e_sad.gif)
Edit: Yep looks like searchbot runs quickly enough that at 1-2 bots should be able to run on a GPU, not sure if I can multiplex it because it uses ~3GB GPU memory and my card only has 8GB.
I tried diplodocus and it does work, but takes minutes to generate orders.
Making progress though.. I've got a new bot account FairBot2 that is running from my machine doing FvA games. It's using searchbot, which I'm not sure if that's the right one.
Also I noticed there's no GvI support in the current repo; did bots ever play GvI?
Note this won't be super-stable since it's running from my home machine in Australia, this is just for testing an evaluation. But please give it a go and let me know your thoughts. I'm also interested to replace one or two of the MILA bots to try and tweak the difficulties etc (both higher and lower).
1
- kestasjk
- Developer
- Posts: 522
- Joined: Tue Mar 14, 2017 8:13 pm
- Location: Perth, Australia
- Karma: 788
- Contact:
Re: Bots are broken tonight
Tried my luck running a searchbot in a 7 player game while running the FvA bot but no it needs more memory than I've got available, need to look at the options
Re: Bots are broken tonight
Kestas, this is excellent, thank you for spending the time to get this up and running again!
I've started a game against Fairbot 2, and it seems like the old "FB" so far. It's only 1902, so a bit hard to tell. One difference I've noticed already, it seems to want to play a strange opening as Austria, to ALB, RUM, and GAL. Not just in my game. I've seen MerryBBuck use this opening with some success against the old FB, but it only works if you catch your opponent off guard.
I'll update you on this thread once I've had a chance to play a few games against Fairbot 2. It might be a few days. I will say, it's much faster in processing moves than the old FB.
The advanced Meta AI never played GvI. GvI has always been the old Univ. of Montreal bots. I know, because when I got tired of repeated whippings at the hands of FB, I would play a game of GvI where I could usually win.![Very Happy :-D](./images/smilies/icon_e_biggrin.gif)
I've started a game against Fairbot 2, and it seems like the old "FB" so far. It's only 1902, so a bit hard to tell. One difference I've noticed already, it seems to want to play a strange opening as Austria, to ALB, RUM, and GAL. Not just in my game. I've seen MerryBBuck use this opening with some success against the old FB, but it only works if you catch your opponent off guard.
I'll update you on this thread once I've had a chance to play a few games against Fairbot 2. It might be a few days. I will say, it's much faster in processing moves than the old FB.
The advanced Meta AI never played GvI. GvI has always been the old Univ. of Montreal bots. I know, because when I got tired of repeated whippings at the hands of FB, I would play a game of GvI where I could usually win.
![Very Happy :-D](./images/smilies/icon_e_biggrin.gif)
- kestasjk
- Developer
- Posts: 522
- Joined: Tue Mar 14, 2017 8:13 pm
- Location: Perth, Australia
- Karma: 788
- Contact:
Re: Bots are broken tonight
Ah great news, glad to hear it’s at least an improvement. Interested to hear if you think it’s definitely behaving differently or not over time, if so I can try to look into what I might be doing different.
Interesting/concerning to hear it’s faster too.. I was not expecting that, I can’t think why as they had a supercomputer and I’ve got a 2080..
There’s a lot to look into now that I can get them running locally; getting tuneable difficulty, different personalities, what the performance bottlenecks are, what would the ideal Diplomacy AI machine be and what can you run at what cost.
I’m looking into getting a dedicated data processing Tesla GPU that’s designed for AI workloads/datacenters. I’d like to have MILA with easy/medium/hard for live 1v6 games, searchbot for FvA, searchbot/diplodocus for non-live 1v6 games, diplodocus for civil disorders, and one Cicero that perhaps joins one long-phase anonymous game at any time that people can know it was Cicero after the game ends (but I don’t want to neglect human games)
Interesting/concerning to hear it’s faster too.. I was not expecting that, I can’t think why as they had a supercomputer and I’ve got a 2080..
There’s a lot to look into now that I can get them running locally; getting tuneable difficulty, different personalities, what the performance bottlenecks are, what would the ideal Diplomacy AI machine be and what can you run at what cost.
I’m looking into getting a dedicated data processing Tesla GPU that’s designed for AI workloads/datacenters. I’d like to have MILA with easy/medium/hard for live 1v6 games, searchbot for FvA, searchbot/diplodocus for non-live 1v6 games, diplodocus for civil disorders, and one Cicero that perhaps joins one long-phase anonymous game at any time that people can know it was Cicero after the game ends (but I don’t want to neglect human games)
Re: Bots are broken tonight
At least that opening has a name! Maybe Fairbot 2 knows something about the "Warthog" that we don't, ha ha.Trigfea63 wrote: ↑Tue Feb 07, 2023 8:49 amOne difference I've noticed already, it seems to want to play a strange opening as Austria, to ALB, RUM, and GAL. Not just in my game. I've seen MerryBBuck use this opening with some success against the old FB, but it only works if you catch your opponent off guard.
http://uk.diplom.org/pouch/Online/Openings/The Library of Diplomacy Openings wrote: RUMANIAN GAMBIT, BALKAN ROADHOG VARIANT
The variant of the Rumanian Gambit which sends Vienna to Galicia.
Tony Swinnerton's Diplomatic Pouch article, "Playing the Key Lepanto," recommends it in conjunction with an unusual Italian/Austrian alliance opening.
RUMANIAN GAMBIT, WARTHOG VARIANT
The Warthog is Richard Sharp's name for the rarely-used Austrian opening (0.87%): F Tri-Alb, A Vie-Gal and, A Bud-Rum. Richard writes that it is "a new name for something resembling a Hedgehog, but not as pretty."
[RUSSIAN FROLIC]
Allen Wells' name for the Austro-Turkish opening based on A Vie-Gal, A Bud-Rum, A Con H, A Smy-Arm, F Ank-Bla. The key here is A Con H which, by eliminating Turkey's 1901 risk to Serbia, permits Austria to move safely to Rum. Discussed in issue 29 of Diplomacy World.
See my full Profile:
http://www.webdiplomacy.net/profile.php?userID=17421
http://www.webdiplomacy.net/profile.php?userID=17421
-
- Posts: 617
- Joined: Fri Sep 29, 2017 3:06 pm
- Location: Manchester, UK
- Karma: 768
- Contact:
Re: Bots are broken tonight
This is called the Russian Opening (Albanian Variation) in FvA. Both my 2017 openings tier list and RoganJosh's (better) 2020 tier list put it at the very lowest tier that's still considered viable in high level play.
Very surprised to hear that the bot uses it, or even that Merry managed to use it with success against the old bot - it seemed just straight up suboptimal to me.
Very surprised to hear that the bot uses it, or even that Merry managed to use it with success against the old bot - it seemed just straight up suboptimal to me.
Re: Bots are broken tonight
Very interesting. In your 2017 article, you say, "This opening is strictly worse than the standard Russian opening because it immediately loses the game against any Piedmont opening." And that is what happened here. I played the Alpine. Now the game is at 1902 builds, and I feel like I've already won (although, if this is the real Fairbot, it's dangerous to speak too soon):captainmeme wrote: ↑Tue Feb 07, 2023 6:24 pmThis is called the Russian Opening (Albanian Variation) in FvA. Both my 2017 openings tier list and RoganJosh's (better) 2020 tier list put it at the very lowest tier that's still considered viable in high level play.
https://webdiplomacy.net/board.php?gameID=533605
Here is an example, with Merry as Austria securing the victory in 1903:captainmeme wrote: ↑Tue Feb 07, 2023 6:24 pmVery surprised to hear that the bot uses it, or even that Merry managed to use it with success against the old bot - it seemed just straight up suboptimal to me.
https://webdiplomacy.net/board.php?game ... #gamePanel
- kestasjk
- Developer
- Posts: 522
- Joined: Tue Mar 14, 2017 8:13 pm
- Location: Perth, Australia
- Karma: 788
- Contact:
Re: Bots are broken tonight
FYI the FvA bot will be down for a few hours as I need to do some work I can only do on my Windows OS and currently I can only run the bots on Ubuntu. (Going to try to run in WSL on Windows now that I know how on Ubuntu)
I've got a graphics card upgrade that was able to run searchbot, but even running one searchbot is a lot slower than the MILA bots. Really starting to appreciate how fast the MILA bots are, and understanding why FAIR turned the bots off so quickly after the research was complete. Going to have to explore some options and try different things out to see what we can get running sustainably
Edit: I've got it running under my main Windows OS now, also I've found the old DORA bot and the original FvA bot as it was when FAIR were running it, I can see that they made it search a lot deeper since it's just 1v1 which might explain some of the different behavior
Definitely the FAIR bots are going to be too slow for live play, but if we rotate the same bot instance across multiple bot accounts the FAIR bots will be able to play longer term games. Exciting though that with the new card I'm just barely able to fit an unmodified Cicero 1v6 into memory, and it's generating text:
Which means we'll probably be able to run full press AI here before long ![Surprised :o](./images/smilies/icon_e_surprised.gif)
I've got a graphics card upgrade that was able to run searchbot, but even running one searchbot is a lot slower than the MILA bots. Really starting to appreciate how fast the MILA bots are, and understanding why FAIR turned the bots off so quickly after the research was complete. Going to have to explore some options and try different things out to see what we can get running sustainably
Edit: I've got it running under my main Windows OS now, also I've found the old DORA bot and the original FvA bot as it was when FAIR were running it, I can see that they made it search a lot deeper since it's just 1v1 which might explain some of the different behavior
Definitely the FAIR bots are going to be too slow for live play, but if we rotate the same bot instance across multiple bot accounts the FAIR bots will be able to play longer term games. Exciting though that with the new card I'm just barely able to fit an unmodified Cicero 1v6 into memory, and it's generating text:
Code: Select all
I0210 20:09:01 [parlai_message_handler:621] Pseudo orders for TURKEY: PseudoOrders({'S1901M': {'TURKEY': ('F ANK - BLA', 'A SMY - CON', 'A CON - BUL'), 'RUSSIA': ('F STP/SC - BOT', 'A MOS - UKR', 'A WAR - GAL', 'F SEV - BLA')}, 'F1901M': {'TURKEY': ('F ANK - BLA', 'A BUL - GRE', 'A CON - BUL'), 'RUSSIA': ('F BOT - SWE', 'F SEV S A UKR - RUM', 'A UKR - RUM', 'A GAL - VIE')}})
I0210 20:09:01 [parlai_message_handler:1055] Applying initial message prompt for [82301] TURKEY -> RUSSIA: [0] TURKEY -> ITALY: Hey Italy!
I think the IT can be a really effective alliance when it’s done well, specifically when Turkey focuses on armies and after dealing with Austria, Italy builds fleets and heads west. Would you be down for something like that?
I0210 20:09:01 [parlai_message_handler:1055] Applying initial message prompt for [82301] TURKEY -> RUSSIA: [1500] TURKEY -> ENGLAND: Hey England, just reaching out to establish comms! I’d love to share intel as the game goes on if you’d be down to do the same, I reckon we could help one another out quite a bit!
I0210 20:09:02 [parlai_message_handler:556] Running message filtering on message "Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?".
I0210 20:09:02 [classifiers:667] Skipping nonsense classifier hvm_with_no_message_history for message proposal Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?, due to it not having order history to rely on.
I0210 20:09:02 [classifiers:652] Skipping nonsense classifier hvm_singleseed_2Mphasesorderhistory_150000steps for message proposal Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?, due to it not having message history to rely on.
I0210 20:09:02 [classifiers:652] Skipping nonsense classifier hvm_singleseed_seed2_2Mphasesorderhistory_2Mphasesstatehistory_150000steps for message proposal Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?, due to it not having message history to rely on.
I0210 20:09:02 [classifiers:652] Skipping nonsense classifier hvm_no_speaker_dialogue for message proposal Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?, due to it not having message history to rely on.
I0210 20:09:02 [classifiers:652] Skipping nonsense classifier hvm_4MPhasesOrderHistory_1epoch_seed3 for message proposal Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?, due to it not having message history to rely on.
I0210 20:09:02 [classifiers:652] Skipping nonsense classifier hvm_no_speaker_dialogue_bilateral_fixed for message proposal Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?, due to it not having message history to rely on.
I0210 20:09:02 [classifiers:652] Skipping nonsense classifier hvwm for message proposal Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?, due to it not having message history to rely on.
I0210 20:09:02 [classifiers:652] Skipping nonsense classifier hvm_NEW_justifications_2seeds for message proposal Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?, due to it not having message history to rely on.
I0210 20:09:02 [classifiers:652] Skipping nonsense classifier hvm_3seeds for message proposal Hey Russia, hoping we can come to an arrangement about the Black Sea. Would you be open to an arranged bounce?, due to it not having message history to rely on.
![Surprised :o](./images/smilies/icon_e_surprised.gif)
1
Re: Bots are broken tonight
Kestas, I don't think Fairbot2 is the same AI as the old (2021-22) Fairbot (now "FB"). There's the opening. Fairbot2 has played that Russian/Albanian opening in all 21 of its games as Austria. Even beyond that, it's not super-difficult to predict Fairbot2's moves and outguess it. FB was brutal on the guesses. And there's the win percentage. FB was kind of unbeatable. It played 5,592 games and won 80% of them. So far, Fairbot2 is winning 37%. It's only finished 30 games. Still, I think it's statistically impossible that an 80%-win-percentage AI could fare so poorly against mere humans. ![Very Happy :-D](./images/smilies/icon_e_biggrin.gif)
![Very Happy :-D](./images/smilies/icon_e_biggrin.gif)
- kestasjk
- Developer
- Posts: 522
- Joined: Tue Mar 14, 2017 8:13 pm
- Location: Perth, Australia
- Karma: 788
- Contact:
Re: Bots are broken tonight
Got it, really like the direct feedback, we’ll need lots of that to refine things. I’ll try switching it to running the original FvA bot that does a lot deeper search and we’ll see what happensTrigfea63 wrote: ↑Sat Feb 11, 2023 8:38 pmKestas, I don't think Fairbot2 is the same AI as the old (2021-22) Fairbot (now "FB"). There's the opening. Fairbot2 has played that Russian/Albanian opening in all 21 of its games as Austria. Even beyond that, it's not super-difficult to predict Fairbot2's moves and outguess it. FB was brutal on the guesses. And there's the win percentage. FB was kind of unbeatable. It played 5,592 games and won 80% of them. So far, Fairbot2 is winning 37%. It's only finished 30 games. Still, I think it's statistically impossible that an 80%-win-percentage AI could fare so poorly against mere humans.![]()
Who is online
Users browsing this forum: No registered users and 270 guests