AI bots on webDiplomacy: Skynet edition!

Site announcements from the moderator team go here.
Message
Author
ColinR
Posts: 5
Joined: Wed May 06, 2020 5:21 pm
Karma: 5
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#161 Post by ColinR » Wed May 06, 2020 5:29 pm

Does anyone know how KestasBot performs against de Jonge and Sierra's D-Brane?

ColinR
Posts: 5
Joined: Wed May 06, 2020 5:21 pm
Karma: 5
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#162 Post by ColinR » Wed May 06, 2020 6:09 pm

ColinR wrote:
Wed May 06, 2020 5:29 pm
Does anyone know how KestasBot performs against de Jonge and Sierra's D-Brane?
... and whether KestasBot is still learning, now against human players on webDiplomacy?

Claesar
Posts: 1965
Joined: Tue Oct 03, 2017 10:34 am
Karma: 1490
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#163 Post by Claesar » Wed May 06, 2020 7:21 pm

ColinR wrote:
Wed May 06, 2020 5:29 pm
Does anyone know how KestasBot performs against de Jonge and Sierra's D-Brane?
No, as far as I know that hasn't been tested.
1

Claesar
Posts: 1965
Joined: Tue Oct 03, 2017 10:34 am
Karma: 1490
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#164 Post by Claesar » Wed May 06, 2020 7:24 pm

ColinR wrote:
Wed May 06, 2020 6:09 pm
ColinR wrote:
Wed May 06, 2020 5:29 pm
Does anyone know how KestasBot performs against de Jonge and Sierra's D-Brane?
... and whether KestasBot is still learning, now against human players on webDiplomacy?
They are not currently learning, but we are still gathering data. There's a non-zero chance they might be updated at some point, but there are no plans for that at the moment.
2

Squigs44
Developer
Developer
Posts: 4003
Joined: Thu Jul 12, 2018 11:50 pm
Location: OKC
Karma: 2010
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#165 Post by Squigs44 » Wed May 06, 2020 8:09 pm

This paper was released at the same time as the bots. The benchmark used was Albert 6.0 and it beat out Albert.
4

ColinR
Posts: 5
Joined: Wed May 06, 2020 5:21 pm
Karma: 5
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#166 Post by ColinR » Thu May 07, 2020 7:29 am

Claesar wrote:
Wed May 06, 2020 7:24 pm
They are not currently learning, but we are still gathering data
The Paquette et al. NIPS paper mentioned that "the supervised agent was able to learn to coordinate support orders while this behaviour appears to deteriorate during self-play training".

Without yet understanding the details, I wondered if this reflected the self-play bot getting stuck in a local maximum due to (non-robust) play against itself - rather than against a more diverse range of opponents. If so, I'd conjecture that learning from its webDiplomacy games could help it get out of this local max.
2

Squigs44
Developer
Developer
Posts: 4003
Joined: Thu Jul 12, 2018 11:50 pm
Location: OKC
Karma: 2010
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#167 Post by Squigs44 » Thu May 07, 2020 8:24 am

ColinR wrote:
Thu May 07, 2020 7:29 am
Claesar wrote:
Wed May 06, 2020 7:24 pm
They are not currently learning, but we are still gathering data
The Paquette et al. NIPS paper mentioned that "the supervised agent was able to learn to coordinate support orders while this behaviour appears to deteriorate during self-play training".

Without yet understanding the details, I wondered if this reflected the self-play bot getting stuck in a local maximum due to (non-robust) play against itself - rather than against a more diverse range of opponents. If so, I'd conjecture that learning from its webDiplomacy games could help it get out of this local max.
Well yes, the point was that when it learned from webdip, it was able to more effectively support others. When it learned from itself, it was less effective at supporting others, and was more effective at winning. The goal of the bot isn't to support, it is to win. Since the bot is in a gunboat setting, it makes sense that supporting other countries wasn't as rewarding.
2

ColinR
Posts: 5
Joined: Wed May 06, 2020 5:21 pm
Karma: 5
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#168 Post by ColinR » Fri May 08, 2020 4:55 pm

We're hosting a virtual Meetup on Wednesday (13 May, 18:30 UK time = 17:30 GMT) to kick-off a Diplomacy AI project.

As the webDiplomacy/Paquette et al. bots are the most impressive I've seen, I'd love it if some of you could join us.

We're using software that allows participants to break into groups (called 'tables'): if any of you can make it, we could have a webDip table, where you explain what you've been doing to interested participants.

To sign up: https://www.meetup.com/Diplomacy-AI-Mee ... 270438476/

A background resources page: https://github.com/rowatc/Diplomacy-AI

Thanks!
2

gnuvag
Posts: 32
Joined: Thu Mar 26, 2020 3:41 am
Karma: 16
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#169 Post by gnuvag » Fri May 29, 2020 2:14 am

Having read this thread, there's still something I don't understand about the bots - do they all share the same AI? Or were they all trained separately and therefore have their own styles?

From what I read above, is it the case that there are only two styles of the AI (I think one more conservative one more risky?), and any bot could adopt either of those styles in any given game? But then additionally there is a bit of randomness from game to game about what moves a bot might make if presented with an identical situation?
2

bo_sox48
Site Admin
Site Admin
Posts: 3901
Joined: Thu Jun 08, 2017 1:01 am
Karma: 2785
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#170 Post by bo_sox48 » Fri May 29, 2020 3:08 am

gnuvag wrote:
Fri May 29, 2020 2:14 am
Having read this thread, there's still something I don't understand about the bots - do they all share the same AI? Or were they all trained separately and therefore have their own styles?

From what I read above, is it the case that there are only two styles of the AI (I think one more conservative one more risky?), and any bot could adopt either of those styles in any given game? But then additionally there is a bit of randomness from game to game about what moves a bot might make if presented with an identical situation?
The bots were trained on the same data set, and then they were all "retrained" together as we made some improvements to fill some gaps in their play. Because we can stagger their usage of these two rounds of training, and because they are trained based on human orders and not always the "optimal" or analytically perfect moveset in a given scenario, they do not all always play with identical styles, which makes them pretty unpredictable. They do not respond the same way all the time in certain situations, and maybe in a sense play with a human personality.
3

MajorMitchell
Posts: 1396
Joined: Sun Dec 31, 2017 4:05 am
Location: Now Performing Comedic Artist Dusty Balzac Bush Philosopher from Flyblown Gully by the Sea
Karma: 719
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#171 Post by MajorMitchell » Mon Jun 01, 2020 8:34 am

I've just played my first game against a Dalek, the one identifying as Kestas Bot. One v one France v Austria and I had Austria. I won in autumn 08 with a five sc margin over the befuddled bundle of algorithms and s/w coding.
I sent the blighter~bot a friendly global message stating that I came in peace to fight to win and the haughty Dalek maintained a surly silence. Unless these blighter~bots learns how to conduct negotiations in polite company I cannot see them getting very far in the alliance making caper.

MajorMitchell
Posts: 1396
Joined: Sun Dec 31, 2017 4:05 am
Location: Now Performing Comedic Artist Dusty Balzac Bush Philosopher from Flyblown Gully by the Sea
Karma: 719
Contact:

Re: AI bots.. Dalek identifying as Kestas Bot defeated by M~M

#172 Post by MajorMitchell » Mon Jun 01, 2020 8:41 am

I'm inordinately pleased with myself for thrashing the surly Dalek. I know it was unranked but I am wondering whether I can I improve my reliability rating playing the Dalek Bots?

MajorMitchell
Posts: 1396
Joined: Sun Dec 31, 2017 4:05 am
Location: Now Performing Comedic Artist Dusty Balzac Bush Philosopher from Flyblown Gully by the Sea
Karma: 719
Contact:

Re: AI bots.. Dalek identifying as Kestas Bot defeated again by M~M

#173 Post by MajorMitchell » Mon Jun 01, 2020 3:18 pm

I thought I'd give the bumbling bundle of algorithms and s/w coding a chance to make amends with a second game, same set up. Victory to M~M in autumn 07, so a year sooner and a stunning 21sc to 7sc result, so a fourteen sc margin. I even captured two French home scs, Marseilles and Brest.

Bark
Posts: 34
Joined: Wed Jan 17, 2018 7:01 pm
Karma: 17
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#174 Post by Bark » Tue Jun 02, 2020 8:57 pm

Love the effort of AI gameplay, but my nitpicking continues...

Why does Jane (England) help Skynet (France) instead of me (Italy) in the endgame?

http://webdiplomacy.net/board.php?gameI ... ocache=635

I did nothing provocative against England. France destroyed England, but England helps France solo? In fact, Germany kind of did the same thing too.

As I posted earlier when playing as Austria, only to find Italy, Russia, and Turkey team up against me in '01, it really seems like an unfair collaboration between the AIs. They know enough to support each other, but I've played about three dozen games with them, and this game was the first time any AI ever supported me anywhere.

What's really hilarious is I was trying to support both England and Germany there at the endgame in the East, after NEVER having any issues with them the entire game, but both attacked me! No way that happens in real life with human players...(with all things being equal).

JDawg021
Posts: 1
Joined: Fri Apr 24, 2020 12:19 am
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#175 Post by JDawg021 » Tue Jun 02, 2020 9:17 pm

When do you think Bots will be able to play non-Classic maps?

Squigs44
Developer
Developer
Posts: 4003
Joined: Thu Jul 12, 2018 11:50 pm
Location: OKC
Karma: 2010
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#176 Post by Squigs44 » Tue Jun 02, 2020 10:16 pm

JDawg021 wrote:
Tue Jun 02, 2020 9:17 pm
When do you think Bots will be able to play non-Classic maps?
To be clear, these bots were developed as part of a one time project, with no guarantee for further development. We are very lucky to have them, but we have no control over the training or further development of these bots. The only way we are going to get bots that play on non-Classic maps is if someone with the right skill set volunteers to create them for webdip.
4

Squigs44
Developer
Developer
Posts: 4003
Joined: Thu Jul 12, 2018 11:50 pm
Location: OKC
Karma: 2010
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#177 Post by Squigs44 » Tue Jun 02, 2020 10:26 pm

Bark wrote:
Tue Jun 02, 2020 8:57 pm
Love the effort of AI gameplay, but my nitpicking continues...

Why does Jane (England) help Skynet (France) instead of me (Italy) in the endgame?

http://webdiplomacy.net/board.php?gameI ... ocache=635

I did nothing provocative against England. France destroyed England, but England helps France solo? In fact, Germany kind of did the same thing too.

As I posted earlier when playing as Austria, only to find Italy, Russia, and Turkey team up against me in '01, it really seems like an unfair collaboration between the AIs. They know enough to support each other, but I've played about three dozen games with them, and this game was the first time any AI ever supported me anywhere.

What's really hilarious is I was trying to support both England and Germany there at the endgame in the East, after NEVER having any issues with them the entire game, but both attacked me! No way that happens in real life with human players...(with all things being equal).
So a couple things. First, England's moves are a bit strange, but those moves at the end actually did nothing to change the outcome of the game. Perhaps England could have prevented a French solo if he had really tried, but his moves were no worse than entering all holds in terms of outcome.

Second, I guarantee that the bots cannot communicate with each other. They do not know who the human is, or what moves their neighbors will make. The only "advantage" that they might have is that they all were trained on the same data in the same way, so they "think similarly". This means if there were a lot of games in their training set where France did a particular move and England supported that move, then France is more likely to make that move, and England is more likely to offer that support.

Third, endgames get wonky for bots. Bots have been trained on thousands of games. The beginnings of those games are all pretty similar, with not as many options for how the board looks. So it does really well there. However, as more pieces enter the board, and countries expand past their usual positioning, you get scenarios that may have only shown up once in the training data, or not at all. When you get to a unique endgame, you can get weird stuff happening, because the humans that found themselves in those scenarios did weird things.
2

Bark
Posts: 34
Joined: Wed Jan 17, 2018 7:01 pm
Karma: 17
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#178 Post by Bark » Wed Jun 03, 2020 1:39 pm

Squigs44 wrote:
Tue Jun 02, 2020 10:26 pm
So a couple things. First, England's moves are a bit strange, but those moves at the end actually did nothing to change the outcome of the game. Perhaps England could have prevented a French solo if he had really tried, but his moves were no worse than entering all holds in terms of outcome.

Second, I guarantee that the bots cannot communicate with each other. They do not know who the human is, or what moves their neighbors will make. The only "advantage" that they might have is that they all were trained on the same data in the same way, so they "think similarly". This means if there were a lot of games in their training set where France did a particular move and England supported that move, then France is more likely to make that move, and England is more likely to offer that support.

Third, endgames get wonky for bots. Bots have been trained on thousands of games. The beginnings of those games are all pretty similar, with not as many options for how the board looks. So it does really well there. However, as more pieces enter the board, and countries expand past their usual positioning, you get scenarios that may have only shown up once in the training data, or not at all. When you get to a unique endgame, you can get weird stuff happening, because the humans that found themselves in those scenarios did weird things.
"Endgame gets wonky for robots."

This I agree with. There was actually one game where I came back as Russia with only two units in Turkey to solo win, mainly because England got 17 centers or so, and stopped programming orders in the East. I was able to retake Sevastopol, and the party started.

For this game though, the Spring '12 moves by England and Germany make no sense. At this point, the three of us should be trying to prevent the French solo. My strategic intention was to get next to their Moscow and Warsaw to support them, and they bounced me away, allowing France to land in Livonia. Granted, once I made some headway in Marseilles, Spain, and Portugal, I would eventually stab them for my own solo victory, but for this move, they shouldn't be bouncing me.

I also understand that you currently have no developers working on this anymore, but if you do, my two suggestions would be:

1. Let AIs have the ability to hold a grudge. Ex., France takes England's home centers, England doesn't help France until England gets those home centers back, if at all.

2. Let the AIs have the ability to hear chat from a human player and decide to help or not help. Ex., I chat England to say, "A Warsaw S A Galicia to Silesia," England's AI then takes that into consideration and inputs those orders.

Again, not knocking this effort! I'm in IT too, but not programming. I know it's a thankless job, so I'll say thanks! With that said, there's always room for an upgrade in the future.

CCR
Posts: 57
Joined: Sun Dec 31, 2017 4:51 pm
Karma: 8
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#179 Post by CCR » Wed Jun 03, 2020 11:18 pm

You'd definitely have more fun playing against Alberts on the Daide platform.

They have true skill, i.e.: they play with real AI and not reproducing stupid human behaviour; they can hold lines and thus they play good endgame situations, including stalemating; they weigh alliances and can see instant signs of change toward friendship/enmity from a single previous round; and more.

Squigs44
Developer
Developer
Posts: 4003
Joined: Thu Jul 12, 2018 11:50 pm
Location: OKC
Karma: 2010
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#180 Post by Squigs44 » Thu Jun 04, 2020 12:05 am

CCR wrote:
Wed Jun 03, 2020 11:18 pm
You'd definitely have more fun playing against Alberts on the Daide platform.

They have true skill, i.e.: they play with real AI and not reproducing stupid human behaviour; they can hold lines and thus they play good endgame situations, including stalemating; they weigh alliances and can see instant signs of change toward friendship/enmity from a single previous round; and more.
In gunboat games, the AI we have outperform Albert. So if you are talking about skill, Albert is not more skilled. The question of fun is subjective though, I imagine having a bot that doesn't get wonky in the endgame could be a large factor of fun for some people.
1

Post Reply

Who is online

Users browsing this forum: No registered users and 48 guests