I actually hadn't thought of the impact the live game has on RR until Chaqa brought it up. I have a misplaced high affection for my current RR, but I've played a few live games this week. My router occasionally drops and in the time I would need to reset it, grab my phone to go on the G network, etc...I could see those 5 minutes eaten up rather quickly. It's an interesting discussion, especially in terms of weighting the RR for various game modes.

I think the calculation should be simple. The last 1 year of games, irrespective of count. You could also not calculate an RR if there are fewer than say 5 games available. Noobs or players returning after an absence have to re-establish themselves.

I also think there should be a split for Live RR (5,7,10,15 mins?) and "Normal" RR, each gated appropriately.

I agree the system should be simple. Mainly because players should be able to understand the process so they know why their RR went down and what they need to do to improve it. I like Jeff Kuta's proposal.

No. Because then if somebody unreliable leaves the website for a while, they can then come back and hop into high gated games, and go straight back to their old ways.

I think what they mean is that they'd have to play x games before even being able to join RR games.

Yes PF

I see that. Then lets make it simple with Time and #Games. 1 year or 20 games whichever is larger. But I really don't think using the sliding scale is a good idea. The system should be simple and easy to comprehend for anyone. Also the maths behind the RR now is a bit odd to me. Why can't it be simple?

A simple reliability ranking system (simple to understand, at least; I don't know about the coding) would be to tie the ranking impact to the game length. A NMR in a 5 minute game would hurt your ranking less than a NMR in a 24 hours game, which would hurt it less than a 3 day game. This would also make recovery more difficult - you couldn't join a ton of live games and fix your ranking all of a sudden.

Right now, I can't think of this creating bad incentives, but someone else might see something.

So let me pose the current problem that I believe exists with RR.

CD's are held in a table. This table contains a record of all CD's and a new record is added every time a CD occurs in a game. This records includes the user, the gameID, and the turnID. It does NOT contain a date.

The problem is with CD's for cancelled games. Normally you can join on other game tables to find when game with a CD in it ended. But when a game is cancelled all records are removed from those other tables and only the single record in the CD table remains.

So with that in mind, you cannot apply a X time period filter, and applying a past X games filter can only be done based on when the games STARTED by doing an orderby gameID.

So how do you make a formula that is limited in the duration without letting people off free for CD's in cancelled games?

Going forward I'd propose adding a datetime column to the table, but for now I can't think of a good method except running a 1 time query that will add a boolean column to the CD table. Then running a script that will say all CD's that happened in games that started more then 1 year ago are no longer counted if the User has since played X number of games.

Now that 1 time query will be very processor intensive, but that's the best I got. Anyone have a better solution?

I wouldn't have a problem with "starting fresh". Take the existing table, save it, then cull all records prior to a certain game ID number, probably one from a time near when RR became a thing. Then take the table and assign "today's date" to all records so they stick around for some period of time. Then move forward from there with whatever calculation method.

Hmm. That would be a problem.

I think I like ckroberts' idea most so far. There is certainly a large difference between NMRing in a 12h/ phase game and NMRing in a 10 day game.

@JMO: we could simply add a table to the column that is a timestamp, and could grab the start date of all games that weren't cancelled for it for existing CD. It would correct the slight incorrectness of game start time vs. CD time gradually. For games that no longer exist, we could simply have a rough assignment - if the game id is in range x to y, say it's 2 years ago, if it's in range y to z, say it's 1 year ago, and if it's z to now, we could say it's 6 months ago.

This wouldn't correct the issue immediately, but it would allow a roughly approximate representation.

Yeah that's not a bad idea Jeff. I can and probably will end up coding whatever system the team ends up designing on, so as long as something can be agreed on for the CD's in cancelled games everything else isn't too hard to do.

Basically, we'd fill in the dates with slightly incorrect, but generally correct, data that would be replaced over time. I'm guessing that the actual CD timestamp could be found by joining to a game table and seeing when a phase ended - I don't know enough about the DB structure to say for sure, but it seems likely. So we'd have a slightly inaccurate idea of when a cancelled CD happened, but it would only last so long as those games were being counted toward RR - which, with decay, would be at most something like 2 years if we go the long cycle route.

@Chaqa, yeah the CD's for games that aren't cancelled I can join on another table and get the end date of the game so there's no issue there, and it's a small overhead join. The only problem is for the the cancelled games. Which is where it's simply a matter of agreeing on a "fair" way to apply it. So for example my concern is making sure that if someone who hasn't been to the site in 2 years but who CD'ed in 20 out of 20 games that all ended up cancelled comes back, they won't be given a clean slate by whatever method we choose.

That's why I know A_tin_Can is saying he want's the time and year factors, so no one gets off clean by a extended break and no one gets off clean by playing X live games to get eliminated quickly while entering orders to get their RR up.

So how about a hybrid solution? Games older than X time will decay, but only if you play Y amount of games in time period X?

Also, I was looking at some of the oldest commits, and realized I wasn't crazy, and that the +1 button did use to say "Like".

An exponentially smoothed average (https://en.wikipedia.org/wiki/Exponential_smoothing) would take care of the problem of decay. Information is never dropped, it just becomes less and less relevant.

It seems irrelevant to me how long ago the games were played, btw. If you took a break from the site for a while, then you should just play a bunch now to prove that you're reliable. Also, it doesn't seem relevant to me whether you CDed because you were understandably called away or because you didn't respect the game. Either way, you've been unreliable.

I'd vote for keeping a single RR. Making a useful measure requires finding a balance between simplicity and accuracy. Having RR and a non-inflationary skill rating used together should provide the desired set of incentives.

Jmo: Implementation details aren't really important right now. There's no problem there that I don't think we can fix, and I think introducing limitations of the current implementation steers discussion away from "how should we measure players reliability?" and towards "how could we implement reliability measurement?".

I think the former discussion is more important, especially as a lot of players opinions start with "I don't know how easy this would be to code, but".

Also, for CDs where we don't know the date - we mostly know the gameID. We can give an upper bound on the date of that game by walking up the gameIDs until we see a game that wasn't cancelled. Similarly, for the CDs where we don't even know the gameID, an optimistic solution like Jeff's would work. But, I don't particularly want implementation details to be discussed in this thread. I don't think there is (or will be) anything proposed that we can't implement.

Watching the RR scores (and the way people use them) over the last year has given me the following personal criteria for a RR system:

1) Simple. This means that it's easy to understand, without complex maths or exclusions.
2) Interpretable. As in, if someone has 80% RR, then you should have an impression of how likely they are to CD in the current game.
3) Clean. The only statistics that go into it are reliability ones -you don't get a free pass for having a better or worse record, nor do you get any RR improvement for doing other good things for the site.
4) Improvable. Ideally, the path to RR improvement is "play more games. Be reliable in them".

Note that I don't think the current system satisfies this very well - especially #2.

It needs to be representative too. 5 year old games aren't.

Absolutely. Good catch.

Just a tip here ATC, if you're asking someone to re-code RR for you in private chat and they are willing then don't be a jerk when they bring the discussion to the thread devoted to it. Especially when other members get interested in the technical discussion that stemmed from it.

And since you have such a handle the implementation, be my guest.

I think it's important to have two RR ratings, one for "live" games and one for everything else. I just ballparked the number of games finished by phase length:

< 1 hr = ~18500
1-6 hrs = ~180
6-12 hrs = ~700
12-24 hrs = ~6000
1-2 days = ~27000
2-5 days = ~4500
5-10 days = ~500
Total = ~58671

I think it's pretty clear from the distribution that players favor live games or 12 hr to 2 day phases.

What is less clear is what constitutes a live game. There are 5, 7, 10, 15, 20, 30 and 60 (?) minute phase lengths counted in that first filter (the only one available via the interface). A bit more granularity might help. Someone who has the power get those live numbers?

I think ATC makes a valid point. We really want to be discussing the RR we want. The ability to implement should be discussed later so as not to limit a potentially good system based on current technical limitations.

As a side note, it might be worth eliminating those unrealistic phase lengths at 20 minutes to probably 4 or even 6 hours. I don't think those games are fun to play and we shouldn't subject new player to that torture which will just end up in a bunch of CDs and NMRs anyway. No need to set them up to fail.

I can think of several reasons people may want to play hour long phases. Maybe a warning but they shouldn't be eliminated

I'm curious. What are the reasons you can think of? I can't think of any.

They might all be insomniacs.

They might be suffering from serious mental illness.

That's about all I can suggest.

Jmo and I were having a discussion on possible issues, not trying to set things in stone. Really people