Reliability ratings and locking changes

Post a reply

Confirmation code
Enter the code exactly as it appears. All letters are case insensitive.
Smilies
:points: :-D :eyeroll: :neutral: :nmr: :razz: :raging: :-) ;) :( :sick: :o :? 8-) :x :shock: :lol: :cry: :evil: :?: :smirk: :!:
View more smilies

BBCode is ON
[img] is ON
[flash] is OFF
[url] is OFF
Smilies are ON

Topic review
   

If you wish to attach one or more files enter the details below.

Expand view Topic review: Reliability ratings and locking changes

Re: Reliability ratings and locking changes

by PinguPingu » Fri Jan 06, 2023 2:13 pm

Sorry - how long does it take for non-live penalties to stop contributing to your RR? (I really like the RR system for what it's worth)

Re: Reliability ratings and locking changes

by kestasjk » Fri Jan 06, 2023 1:22 pm

You could do a lot of things.. I wanted to preserve the system jmo put in place as it seemed to be working well. The current system adds some forgiveness for consecutive turns, lets mods forgive turns, differentiates between live games and non-live, etc.
I’m not saying you couldn’t make something like that have similar features.. but you definitely come to learn that any system that tries to sum up any metric like reliability, Diplomacy skill, account-suspiciousness, user-relationship likelihood, etc, it’s always going to be a balance between loads of different trade offs and compromises, and there’ll always be suggestions on how it could be improved in some way or other.

Re: Reliability ratings and locking changes

by learnedSloth » Thu Jan 05, 2023 12:50 pm

kestasjk wrote:
Tue Jan 03, 2023 11:26 am
The new RR calcs are a lot more complex because they put turns into different buckets, and there may still be a bug or two lurking there, but it's O(1) instead of O(N).
Couldn't you just

Code: Select all

newRR = 99% * oldRR + 1% * current_turn
?

Re: Reliability ratings and locking changes

by kestasjk » Tue Jan 03, 2023 11:57 am

kestasjk wrote:
Tue Jan 03, 2023 11:26 am
Navel_Reservist wrote:
Fri Dec 30, 2022 5:41 am
How long does it take for base rate improvements to be reflected in the reliability rating? My base rate has been increasing, and the calculation implies that my reliability rating should be increasing as a result, but it's still stuck at its low point. Is this intentional? Right now the math on my reliability rating page shows this:

"Total:
Reliability Rating: 100% - Base/Total reduction (16.7%) - Non-live penalty (11%) - Live penalty (0%) = Reliability rating (64%) "
I'll take a look asap
So I did a global RR recalc and you went from 72% to 74%:
RR1.png
to
RR2.png
Looks like when you last checked the RR should've been 72% (2 missed turns/12 yearly turns = 16.7%), and when I checked before the global recalc the RR should've been 74% (2 missed turns / 14 yearly turns = 14.3%) which it now is.

Looks like when the yearly turn counter gets incremented it wouldn't trigger a refresh, so it would only update on missed turns. I've fixed that now, thanks for reporting:
https://github.com/kestasjk/webDiplomac ... 6ee7c7f79e

Re: Reliability ratings and locking changes

by kestasjk » Tue Jan 03, 2023 11:26 am

A_Tin_Can wrote:
Sat Dec 24, 2022 8:48 am
While we're on the subject of ratings, my ghost rating seems to have gone down in the last two years, despite me not having played a single game. Is there perhaps something wrong with the calculation?
Hmm, there may well be, I've never looked into how GRs are calculated. Definitely it could be something similar to RRs where they were implemented in a way that was simple and easy to understand but required reading every players entire game history turn by turn for the last year every time a game they were involved in got processed, which contributed to those deadlocking issues that were a big problem for site performance/reliability this time last year.

The new RR calcs are a lot more complex because they put turns into different buckets, and there may still be a bug or two lurking there, but it's O(1) instead of O(N).
Navel_Reservist wrote:
Fri Dec 30, 2022 5:41 am
How long does it take for base rate improvements to be reflected in the reliability rating? My base rate has been increasing, and the calculation implies that my reliability rating should be increasing as a result, but it's still stuck at its low point. Is this intentional? Right now the math on my reliability rating page shows this:

"Total:
Reliability Rating: 100% - Base/Total reduction (16.7%) - Non-live penalty (11%) - Live penalty (0%) = Reliability rating (64%) "
I'll take a look asap

Re: Reliability ratings and locking changes

by A_Tin_Can » Sun Jan 01, 2023 1:45 pm

Thanks Doug, but I'm not sure how your reply is related. I'm talking about the stats reported on the site only.

Re: Reliability ratings and locking changes

by Doug7878 » Sat Dec 31, 2022 12:46 am

@ATC -
I had noticed that the GB and Peak GR ratings changed also for some players when the GR calculations were incorporated into the site at the beginning of 2021. I understand that there were some calculation errors in the old ratings that were corrected at that time. I watch the GB GR changes pretty closely and have seen no unusual changes in gunboat ratings since FWIW.

Re: Reliability ratings and locking changes

by Navel_Reservist » Fri Dec 30, 2022 5:41 am

How long does it take for base rate improvements to be reflected in the reliability rating? My base rate has been increasing, and the calculation implies that my reliability rating should be increasing as a result, but it's still stuck at its low point. Is this intentional? Right now the math on my reliability rating page shows this:

"Total:
Reliability Rating: 100% - Base/Total reduction (16.7%) - Non-live penalty (11%) - Live penalty (0%) = Reliability rating (64%) "

Re: Reliability ratings and locking changes

by A_Tin_Can » Sat Dec 24, 2022 8:48 am

While we're on the subject of ratings, my ghost rating seems to have gone down in the last two years, despite me not having played a single game. Is there perhaps something wrong with the calculation?

Re: Reliability ratings and locking changes

by Crazy Anglican » Thu Dec 22, 2022 6:02 pm

In looking at the RR chart, excused misses count against you?

Re: Reliability ratings and locking changes

by PinguPingu » Thu Dec 22, 2022 12:56 pm

How long dos it take to shed the non-live penalties?

Re: Reliability ratings and locking changes

by kestasjk » Tue Oct 11, 2022 10:52 pm

Oops a divide by 0, always forget something.. Thanks that's fixed now:
Screenshot 2022-10-12 065144.png

Re: Reliability ratings and locking changes

by JECE » Tue Oct 11, 2022 10:31 pm

I just checked and my reliability rating is 0%. I haven't played a game on webDip in a few years, but that seems a little harsh! Could this be an error with the recent changes?

Reliability ratings and locking changes

by kestasjk » Tue Oct 11, 2022 4:46 pm

I've made some changes to reliability ratings; the previous optimizations meant you couldn't really see what was going on with the calcs if they went wrong, but now what's shown on your profile is tied to the actual numbers used in the new calcs.
It seemed like the way it was calculated was built on over time so I tried to simplify the explanation down so it's easier to see where your RR comes from:
Screenshot 2022-10-12 004250.png
Let me know if it's not straightforward or if you notice your numbers seem off. It definitely works when doing a global refresh, but whether it'll work as it goes we'll have to wait and see over time.

I also noticed that deadlocks have been coming back, and it was 90+% due to order submission via the new API that the bots and new UI used. It was locking the entire game instead of just the user record so I've reduced that down. Hopefully there won't be anything to notice here, but I'm interested if anyone got any errors during order submission recently.

Top