Speedrun Leaderboards Discussion Topic (SRL)

Farquaad

2013-08-14 04:52:31 pm

Quote from Svenne:

Last post I will make here. It happened in March, from that point to now there has been three races on Zelda 2 @ SRL.

come on big boy man up and start naming some names if you want to make something out of this statement

Chuckolator

Edit history:

Chuckolator: 2013-08-14 04:54:14 pm

Chuckolator: 2013-08-14 04:54:03 pm

2013-08-14 04:53:35 pm

Quote from Svenne:

Last post I will make here. It happened in March, from that point to now there has been three races on Zelda 2 @ SRL.

If you have an issue with people over an issue that happened almost half a year, why don't you attempt to work with the other relevant people to put the past behind you? All I've read about this issue on here and Twitter is that something happened between you and an undefined SRL party, that the SRL people are unaware of the issue and have tried opening a discussion, and you've kept shooting them down. Keeping up that kind of attitude is fracturing the community for no good reason.

But enough of that, I heard from some people here or there that there's supposed to be a discussion regarding leaderboards?

RockAroundTheClock

2013-08-14 04:57:47 pm

If you hate SRL so much since March, why have you raced over 100 times in various other games since April?
http://speedrunslive.com/profiles/#!/Svenne/1

Twig

Edit history:

Twig: 2013-08-14 05:14:48 pm

Twig: 2013-08-14 05:12:36 pm

Twig: 2013-08-14 05:02:28 pm

2013-08-14 05:00:49 pm

Love leaderboards and massively respect anyone who does all the work to make it a reality.

TimpZ

Edit history:

TimpZ: 2013-08-14 05:28:20 pm

TimpZ: 2013-08-14 05:25:51 pm

TimpZ: 2013-08-14 05:20:39 pm

TimpZ: 2013-08-14 05:16:27 pm

TimpZ: 2013-08-14 05:14:07 pm

TimpZ: 2013-08-14 05:14:05 pm

2013-08-14 05:13:19 pm

I skimmed through the 11 pages and google doc and here's my thoughts:

I won't go into detail, but in general I see a leaderboard as a way of referencing. May it be scientific work, a quote from a book, a segment of a video or a time of a speedrun. Anything that is public information is referenceable and everything not password protected on the internet or similar is public, period. If people have a problem for whatever reason in the future of others referencing their public information, they should work on not having it public. This is how all media works, be it news on the television or wikipedia.

Either way I think this is going to be such a small problem that if problems actually arose they can be worked out on a case-by-case basis, especially with the new forum being planned out.

Now to something I noticed in the google doc that doesn't seem to have been adressed properly; Sonic Adventure 2 is a game that's been released on many platforms; DC, GC, 360, PS3 and PC. PC is fastest real time given you have an adequate computer but GC is to my knowledge a couple seconds faster in-game due to PC being slightly altered. Given that real-time would be standard (as it currently is for some odd reason), would exclusion of load-times be consistent on all platforms? Who would regulate that all data is correct considering everything has to be hand-timed? This is especially important for runs that aren't submitted by the runner themselves.

Sure the chances are that in-game time would be used, but there's other games that I know less about where this might be a problem.

EDIT: Another question, if a game uses ingame-time but also pause buffering, where do you draw the line for it being equaled to a TA of some form? I'm imagining for example if OoT used ingame-time where some might pause buffer extensively throughout the run but real-time would be vastly inaccurate for other reasons.

I skimmed through the 11 pages and google doc and here's my thoughts:

I won't go into detail, but in general I see a leaderboard as a way of referencing. May it be scientific work, a quote from a book, a segment of a video or a time of a speedrun. Anything that is public information is referenceable and everything not password protected on the internet or similar is public, period. If people have a problem for whatever reason in the future of others referencing their public information, they should work on not having it public. This is how all media works, be it news on the television or wikipedia.

Either way I think this is going to be such a small problem that if problems actually arose they can be worked out on a case-by-case basis, especially with the new forum being planned out.

Now to something I noticed in the google doc that doesn't seem to have been adressed properly; Sonic Adventure 2 is a game that's been released on many platforms; DC, GC, 360, PS3 and PC. PC is fastest real time given you have an adequate computer but GC is to my knowledge a couple seconds faster in-game due to PC being slightly altered. Given that real-time would be standard (as it currently is for some odd reason), would exclusion of load-times be consistent on all platforms? Who would regulate that all data is correct considering everything has to be hand-timed? This is especially important for runs that aren't submitted by the runner themselves.

Sure the chances are that in-game time would be used, but there's other games that I know less about where this might be a problem.

EDIT: Another question, if a game uses ingame-time but also pause buffering, where do you draw the line for it being equaled to a TA of some form? I'm imagining for example if OoT used ingame-time where some might pause buffer extensively throughout the run but real-time would be vastly inaccurate for other reasons.

Sonikkustar

2013-08-14 05:23:36 pm

The Loony Bin

Alright, I guess it's my turn to state an opinion. This is a long thread and I haven't really read most of the points, so I apologize if most if this is redundant.

I guess I'll be straight and to the point. If my runs are to be posted on this leaderboard, then I would like to at least be given a heads up before it is put up on there. I think the main reason that most people are against the idea of disallowing a run to be removed is because you aren't given any sort of choice in how your run is managed on the leaderboard. So while there may be no chance of denying your run from making it onto the leaderboards if it is made public, you should at least give the option for the runner to do whatever they want with that run if it shows up on the leaderboards, even if it means them wanting to have it taken out.

I'm sorry if this post makes no sense, but that's honestly all I have to say on the matter. Take care everyone!

CosmoWright

2013-08-14 05:29:51 pm

Quote from TimpZ:

Now to something I noticed in the google doc that doesn't seem to have been adressed properly; Sonic Adventure 2 is a game that's been released on many platforms; DC, GC, 360, PS3 and PC. PC is fastest real time given you have an adequate computer but GC is to my knowledge a couple seconds faster due to PC being slightly altered. Given that real-time would be standard (as it currently is for some odd reason), would exclusion of load-times be consistent on all platforms? Who would regulate that all data is correct considering everything has to be hand-timed? This is especially important for runs that aren't submitted by the runner themselves.

Sure the chances are that in-game time would be used, but there's other games that I know less about where this might be a problem.

EDIT: Another question, if a game uses ingame-time but also pause buffering, where do you draw the line for it being equaled to a TA of some form? I'm imagining for example if OoT used ingame-time where some might pause buffer extensively throughout the run but real-time would be vastly inaccurate for other reasons.

These are issues that tend to be resolved amongst a game's community over time. There are different interpretations of what to do in different cases.

Real Time: simplest solution. no issues with pause abuse. generally the least effort to time it (unless the game gives you a final time, I forgot if SA2 does this).
Game Time: slower to figure out if you have to add up all the levels. game time may fix issues of version loading/speed etc, which could be desirable. pause abuse can get really silly. if gametime has heavy abuse, making up new rules on how to deal with it, or just accept it could both be options.
Time w/o loads: hardest to time, most people will be lazy about it. might be the most fair, but it might be a bit arbitrary to do this on a game with multiple platforms and isn't just pure PC. debatable.

Basically you look at your options, and the people who are running the game make a decision on what the ideal method is. This might change later on if stronger cases are made (such as Super Metroid realtime).

[quote="TimpZ"]Now to something I noticed in the google doc that doesn't seem to have been adressed properly; Sonic Adventure 2 is a game that's been released on many platforms; DC, GC, 360, PS3 and PC. PC is fastest real time given you have an adequate computer but GC is to my knowledge a couple seconds faster due to PC being slightly altered. Given that real-time would be standard (as it currently is for some odd reason), would exclusion of load-times be consistent on all platforms? Who would regulate that all data is correct considering everything has to be hand-timed? This is especially important for runs that aren't submitted by the runner themselves.

Sure the chances are that in-game time would be used, but there's other games that I know less about where this might be a problem.

EDIT: Another question, if a game uses ingame-time but also pause buffering, where do you draw the line for it being equaled to a TA of some form? I'm imagining for example if OoT used ingame-time where some might pause buffer extensively throughout the run but real-time would be vastly inaccurate for other reasons.[/quote]

These are issues that tend to be resolved amongst a game's community over time. There are different interpretations of what to do in different cases.

Real Time: simplest solution. no issues with pause abuse. generally the least effort to time it (unless the game gives you a final time, I forgot if SA2 does this).
Game Time: slower to figure out if you have to add up all the levels. game time may fix issues of version loading/speed etc, which could be desirable. pause abuse can get really silly. if gametime has heavy abuse, making up new rules on how to deal with it, or just accept it could both be options.
Time w/o loads: hardest to time, most people will be lazy about it. might be the most fair, but it might be a bit arbitrary to do this on a game with multiple platforms and isn't just pure PC. debatable.

Basically you look at your options, and the people who are running the game make a decision on what the ideal method is. This might change later on if stronger cases are made (such as Super Metroid realtime).

Onin

2013-08-14 05:50:35 pm

I thought the solution to things like pause buffering and similar issues was simply listing both real time and game time on the run, and leaving it up to the community to decide on which time the ranking should be based.

wersterlobe

2013-08-14 06:01:09 pm

Oh snap, Sonic Adventure 2 got brought up. As far as my knowledge, Dark Story on Sonic Adventure 2 "real time" is held by Talon on PC, but the best "game time" is held by sonicguitar played on GC, and his time was set before the PC version was even released so the obvious "just play on the fastest version" doesn't really apply there.

It has a total time at the end (at least I remember seeing one on PC when Talon was doing runs) and you already know how big a fan I am of game time Wink

Honestly though in those cases, as long as both real time and some form of game time, whether it be removing loads or the actual game timer are both displayed in some manner (40:10 real time (26:10 game time)) it really doesn't matter too much what the "official" community standard is, because all the information is there, so if you prefer one you can find out what the best is regardless.

What I was actually going to post about was how sad I am that I know nothing of coding etc etc, because I'd love to help out this project as much as I can. I've been waiting for leaderboards for a long time and would love to contribute in any way I can and would certainly go through whatever grunt work when it's all up and running Smiley

Oh snap, Sonic Adventure 2 got brought up. As far as my knowledge, Dark Story on Sonic Adventure 2 "real time" is held by Talon on PC, but the best "game time" is held by sonicguitar played on GC, and his time was set before the PC version was even released so the obvious "just play on the fastest version" doesn't really apply there.

It has a total time at the end (at least I remember seeing one on PC when Talon was doing runs) and you already know how big a fan I am of game time ;) Honestly though in those cases, as long as both real time and some form of game time, whether it be removing loads or the actual game timer are both displayed in some manner (40:10 real time (26:10 game time)) it really doesn't matter too much what the "official" community standard is, because all the information is there, so if you prefer one you can find out what the best is regardless.

What I was actually going to post about was how sad I am that I know nothing of coding etc etc, because I'd love to help out this project as much as I can. I've been waiting for leaderboards for a long time and would love to contribute in any way I can and would certainly go through whatever grunt work when it's all up and running :)

Carl Sagan

2013-08-14 06:17:08 pm

I have no idea why I just read this entire thread @_@_@_@_@

I just want to give massive props to everyone who has spent time making the leaderboards a thing. I feel like a lot of time has gone into making the best decisions for things like being able to sort the runs any way you want, letting timing methods be tailored for each game/community, and many other issues that have arisen, and am currently in MAD HYPE MODE for this to roll out.

A couple people seemed to think it will be impossible to moderate/flag/update the leaderboards effectively. I think the success of various wikis (and wikipedia itself) has shown that this is not a concern worth worrying too much about. People in each game community will take it upon themselves to ensure the accuracy of their game on the leaderboards.

As far as the opt-out shitfest goes, it really should not be an argument that publicly posting a video or stream on the internet means that anyone can link to that video or stream -- that is what the internet is.

I have no idea why I just read this entire thread @_@_@_@_@

I just want to give massive props to everyone who has spent time making the leaderboards a thing. I feel like a lot of time has gone into making the best decisions for things like being able to sort the runs any way you want, letting timing methods be tailored for each game/community, and many other issues that have arisen, and am currently in MAD HYPE MODE for this to roll out.

A couple people seemed to think it will be impossible to moderate/flag/update the leaderboards effectively. I think the success of various wikis (and wikipedia itself) has shown that this is not a concern worth worrying too much about. People in each game community will take it upon themselves to ensure the accuracy of their game on the leaderboards.

As far as the opt-out shitfest goes, it really should not be an argument that publicly posting a video or stream on the internet means that anyone can link to that video or stream -- that is what the internet is.

TimpZ

Edit history:

TimpZ: 2013-08-14 06:28:04 pm

TimpZ: 2013-08-14 06:28:00 pm

TimpZ: 2013-08-14 06:27:31 pm

TimpZ: 2013-08-14 06:24:48 pm

2013-08-14 06:22:55 pm

SA2 does give a final time, but I think it excludes deaths which means it would be dumb for SS runs. In its place you have to manually add up all the level times for an accurate in-game counter.

In games like Sonic 1 and 2 (not sure about 3/K) there's spots where pausing the game for a second can load certain bosses quicker and thus save in-game time. I think this could either be considered something like Metroid Prime where you want to watch certain cutscenes because they warp you to a favourable spot, OR an unnecessary manual input that should be disallowed. I've never come across anyone that does it in runs (of S2 at least). But I get what you're saying about community deciding.

But what counts as the community? The elite top of the leaderboards that might have biased opinions or the community as a whole where there might be many uninformed opinions? I can see rifts building where admins or people in general don't see the value of separating a certain category while a majority of serious runners do (such as Zelda 2 I guess?); or the opposite where there's a big community of small time runners doing "Pokemon Blue - Charmander any%" runs or something while the top elite of runners don't agree it's a valid category. The same might be said for Sonic games. Knuckles is the fastest character to use in S3K, all other characters are irrelevant. Most people would proably disagree with the Pokemon example but not Sonic even though it uses the same basic logic. Just a thought experiment.

Also something I don't find quite clear, if a stream goes down in the middle of a run and no local recording is made, or a technical issue arises such that there's no game video feed for a little bit, would that automatically void the record (for leaderboards at least)?

SA2 does give a final time, but I think it excludes deaths which means it would be dumb for SS runs. In its place you have to manually add up all the level times for an accurate in-game counter.

In games like Sonic 1 and 2 (not sure about 3/K) there's spots where pausing the game for a second can load certain bosses quicker and thus save in-game time. I think this could either be considered something like Metroid Prime where you want to watch certain cutscenes because they warp you to a favourable spot, OR an unnecessary manual input that should be disallowed. I've never come across anyone that does it in runs (of S2 at least). But I get what you're saying about community deciding.

But what counts as the community? The elite top of the leaderboards that might have biased opinions or the community as a whole where there might be many uninformed opinions? I can see rifts building where admins or people in general don't see the value of separating a certain category while a majority of serious runners do (such as Zelda 2 I guess?); or the opposite where there's a big community of small time runners doing "Pokemon Blue - Charmander any%" runs or something while the top elite of runners don't agree it's a valid category. The same might be said for Sonic games. Knuckles is the fastest character to use in S3K, all other characters are irrelevant. Most people would proably disagree with the Pokemon example but not Sonic even though it uses the same basic logic. Just a thought experiment.

Also something I don't find quite clear, if a stream goes down in the middle of a run and no local recording is made, or a technical issue arises such that there's no game video feed for a little bit, would that automatically void the record (for leaderboards at least)?

kirkq

Edit history:

kirkq: 2013-08-14 06:41:33 pm

kirkq: 2013-08-14 06:37:31 pm

kirkq: 2013-08-14 06:35:39 pm

kirkq: 2013-08-14 06:34:17 pm

2013-08-14 06:33:56 pm

Die Hard 2013.

I think opt out has run its course for now, so let's try to focus on other issues for a while. I'll revise the wording in the next document revision (Time to be determined) and if you really want to bring up old arguments again at that time feel hesitantly free to do so.

Timing methods will largely be decided by the community by instating game moderators for these games based on community feedback. (Someone somewhere has to do it.) Timing creates a lot of issues in PC games especially. Whether or not SA2B gets split into two separate games might largely be up to the community. What constitutes a separate game has a lot of ambiguity and is probably worth more discussion.

If no recording exists I have no good idea of if the run was cheated or if it was timed well. If there is a period of time with no video it will be addressed on a case by case basis. If you're missing 1 second of video it's quite different than losing 5 minutes. The better solution is to actually record your runs locally and be responsible with your videos instead of relying on twitch saving the video. If you're assuming twitch videos are reliable, you've been a bit misled Tongue

The community is the community. Basically moderators and game moderators are going to be instated with the expectation that they will reflect the community. If they don't, then go to the forums and address your case. People are generally willing to work to mitigate issues when brought up in a reasonable manner; that's one of the assumptions we have to work with. If someone is being really unreasonable they'll probably be removed from power. There is no system that can avoid this potential for bias, but ideally game moderators would not necessarily be the people with the fastest times. It often happens that the people with the fastest times are the most involved with the community though.

I think opt out has run its course for now, so [b][u]let's try to focus on other issues for a while[/u][/b].  I'll revise the wording in the next document revision (Time to be determined) and if you really want to bring up old arguments again at that time feel hesitantly free to do so.

Timing methods will largely be decided by the community by instating game moderators for these games based on community feedback.  (Someone somewhere has to do it.)  Timing creates a lot of issues in PC games especially.  Whether or not SA2B gets split into two separate games might largely be up to the community.  What constitutes a separate game has a lot of ambiguity and is probably worth more discussion.

If no recording exists I have no good idea of if the run was cheated or if it was timed well.  If there is a period of time with no video it will be addressed on a case by case basis.  If you're missing 1 second of video it's quite different than losing 5 minutes.  The better solution is to actually record your runs locally and be responsible with your videos instead of relying on twitch saving the video.  If you're assuming twitch videos are reliable, you've been a bit misled :P

The community is the community.  Basically moderators and game moderators are going to be instated with the expectation that they will reflect the community.  If they don't, then go to the forums and address your case.  People are generally willing to work to mitigate issues when brought up in a reasonable manner; that's one of the assumptions we have to work with.  If someone is being really unreasonable they'll probably be removed from power.  There is no system that can avoid this potential for bias, but ideally game moderators would not necessarily be the people with the fastest times.  It often happens that the people with the fastest times are the most involved with the community though.

CosmoWright

2013-08-14 06:35:22 pm

Really glad to see this discussion turn to more productive things.

Quote from TimpZ:

But what counts as the community? The elite top of the leaderboards that might have biased opinions or the community as a whole where there might be many uninformed opinions? I can see rifts building where admins or people in general don't see the value of separating a certain category while a majority of serious runners do (such as Zelda 2 I guess?); or the opposite where there's a big community of small time runners doing "Pokemon Blue - Charmander any%" runs or something while the top elite of runners don't agree it's a valid category. The same might be said for Sonic games. Knuckles is the fastest character to use in S3K, all other characters are irrelevant. Most people would proably disagree with the Pokemon example but not Sonic even though it uses the same basic logic. Just a thought experiment.

Regarding starters in Pokemon, or even class choices in something like Dark Souls.. these types of situations could be handled by the tagging system. For example if a lot of people find value in doing Charmander runs, maybe a "starter" tag could be applied to the board. This would allow you to see who has the any% glitchless record (full of mostly Squirtles), but you could then filter it by starter and see who has the fastest charmander/bulbasaur run, without it requiring it's own page or separate category. Just an example, not saying this is something that the Pokemon community necessarily wants or anything. As for character choice in S3K, this seems more clear cut as completely separate categories, but it depends again on what people decide on and what the standards are. These statements might sound somewhat vague, but the idea is to have the people who run the game figure out the best ways of sorting these things out, and if there are bigger issues perhaps they could be discussed and examined more closely, and arguments could be made to determine the best course of action. This is one reason we want a forum w/ leaderboards--to facilitate these types of discussions.

Quote:

Also something I don't find quite clear, if a stream goes down in the middle of a run and no local recording is made, or a technical issue arises such that there's no game video feed for a little bit, would that automatically void the record?

One proposed solution for these issues is to apply a "questionable run" note on the leaderboard for these situations. This is expressed more in the document.

Really glad to see this discussion turn to more productive things.

[quote="TimpZ"]But what counts as the community? The elite top of the leaderboards that might have biased opinions or the community as a whole where there might be many uninformed opinions? I can see rifts building where admins or people in general don't see the value of separating a certain category while a majority of serious runners do (such as Zelda 2 I guess?); or the opposite where there's a big community of small time runners doing "Pokemon Blue - Charmander any%" runs or something while the top elite of runners don't agree it's a valid category. The same might be said for Sonic games. Knuckles is the fastest character to use in S3K, all other characters are irrelevant. Most people would proably disagree with the Pokemon example but not Sonic even though it uses the same basic logic. Just a thought experiment.[/quote]
Regarding starters in Pokemon, or even class choices in something like Dark Souls.. these types of situations could be handled by the tagging system. For example if a lot of people find value in doing Charmander runs, maybe a "starter" tag could be applied to the board. This would allow you to see who has the any% glitchless record (full of mostly Squirtles), but you could then filter it by starter and see who has the fastest charmander/bulbasaur run, without it requiring it's own page or separate category. Just an example, not saying this is something that the Pokemon community necessarily wants or anything. As for character choice in S3K, this seems more clear cut as completely separate categories, but it depends again on what people decide on and what the standards are. These statements might sound somewhat vague, but the idea is to have the people who run the game figure out the best ways of sorting these things out, and if there are bigger issues perhaps they could be discussed and examined more closely, and arguments could be made to determine the best course of action. This is one reason we want a forum w/ leaderboards--to facilitate these types of discussions.

[quote]Also something I don't find quite clear, if a stream goes down in the middle of a run and no local recording is made, or a technical issue arises such that there's no game video feed for a little bit, would that automatically void the record?[/quote]
One proposed solution for these issues is to apply a "questionable run" note on the leaderboard for these situations. This is expressed more in the document.

zewing

2013-08-14 07:22:55 pm

just( •_•)>⌐■-■ ..... (⌐■_■)wing it

Charmander runs would be the end of pkmn speedruns

Rakuen

2013-08-14 07:39:49 pm

Weegee Time

Will there be a way for the runner to edit his/her run's information on the leaderboard after submission, whether by themselves or another user? And if this is already planned, could this be explicitly stated in the document so everyone's on the same page?

kirkq

Edit history:

kirkq: 2013-08-14 07:47:43 pm

2013-08-14 07:44:04 pm

Die Hard 2013.

Ideally the run would be attached to the user account for editing if it was their run and separately any moderator could edit it. A run could be attached to an account at a later time manually. How this works for others' runs is more ambiguous. The intent is to not make comments for them or anything, just state methods of play.

Rakuen

Edit history:

Rakuen: 2013-08-14 07:57:23 pm

Rakuen: 2013-08-14 07:51:47 pm

Rakuen: 2013-08-14 07:49:35 pm

Rakuen: 2013-08-14 07:47:00 pm

Rakuen: 2013-08-14 07:46:41 pm

2013-08-14 07:45:42 pm

Weegee Time

Edit: Screw it, kirkq edited his post and I agree with that approach. I also agree with the last statement that only the original runner should be able to edit the more personally meaningful information (ie: run notes). That's really what I was getting at with my question. Smiley

andrewg

Edit history:

andrewg: 2013-08-14 08:26:04 pm

andrewg: 2013-08-14 08:16:24 pm

andrewg: 2013-08-14 08:13:27 pm

2013-08-14 08:11:06 pm

Hi! I'm andrewg!

I have read a bit of the first few pages, and it seems a lot of people don't get that it's just like any other wiki. Say I'm one of the top 100 richest people in the world. Even if I don't want people to know, my name will be on the list, and I wouldn't be able to remove my name (of course, unless I paid many people off). But I think you guys get my point.

Having your name on the leaderboard is just like anyone finding a link to your run and posting it on their website, etc. And I mean, If you make a video public, you likely want people to be able to view it, right? The leaderboard is just a compilation of runs, a wiki. The rank given to the player is obviously not necessarily accurate, but it's a best guess.

Ranks on the leaderboards don't mean anything, again, it's just a list of what's available online.

EDIT: Anyhow, I think this is a great idea, and I think categories should definitely tried to be minimized as much as possible when making the final version. It's way too crazy sometimes.

I have read a bit of the first few pages, and it seems a lot of people don't get that it's just like any other wiki. Say I'm one of the top 100 richest people in the world. Even if I don't want people to know, my name will be on the list, and I wouldn't be able to remove my name (of course, unless I paid many people off). But I think you guys get my point.

Having your name on the leaderboard is just like anyone finding a link to your run and posting it on their website, etc. And I mean, If you make a video public, you likely want people to be able to view it, right? The leaderboard is just a compilation of runs, a wiki. The rank given to the player is obviously not necessarily accurate, but it's a best guess.

Ranks on the leaderboards don't mean anything, again, it's just a list of what's available online.

EDIT: Anyhow, I think this is a great idea, and I think categories should definitely tried to be minimized as much as possible when making the final version. It's way too crazy sometimes.

Patashu

2013-08-14 08:24:18 pm

I think the best description of the timing method used for Sonic 1-3 is 'real time spent in a level'. It's in game time + in game time elasped during every death + every frame spent paused. Will the leaderboards be able to handle runs that are best timed like this?

Maxx

Edit history:

Maxx: 2013-08-14 08:45:28 pm

2013-08-14 08:28:53 pm

Quote from kirkq:

2: We need coders!

This project does not have a lead developer currently. We especially need people familiar with databases, search queries, and forum integration. Give us some idea of your abilities/specialties. For now, you can post here and/or PM me online method of contact if you'd like. I will not be the one ultimately organizing the project from a coding standpoint, but I can pass that information on. We will probably get a github going at some point.

hey there.

i built the ZeldaSpeedRuns leaderboards by myself. i pretty much run ZSR these days (even though it was cosmo's site originally), and i'm pretty familiar with a lot of the SRL communities. i'm also a full-stack web developer at my day job, and i guess i could provide a resume (haha). i feel like i could be pretty helpful when it comes to actually building this stuff.

i would love to lead the development of this project on, but i can't start on anything until sometime next month. i know seasons 2 is coming up soon, so maybe we have time. if you haven't found somebody else by then, count me in for this. in either case, i had a few ideas for how to handle some of the upcoming work.

Quote from kirkq:

- Create SRL forums
- Create SRL logins
- Integrate user accounts for SRL forums, SRL races, SRL leaderboards, and IRC. (This is difficult.)

the biggest issue to start with seems to be some sort of login system. i did see this earlier in the thread:

Quote from nate:

just drop me a line if you're interested in using taiga forum for the account system (either via the sda forum or another, new forum). tf has a nice api that i wrote so that the sda queue and knowledge base could use a single login. there are lots of other goodies in the api too like posting as an arbitrary user, editing posts, sending pms, etc. i can provide the documentation and help you get set up.

this is probably something worth looking into. you'll definitely want to have the forums and the rest of the site (leaderboards, profile, etc.) use the same credentials, so picking out some forum software that can interface well with the rest of your app is important. if it's designed that way (and it sounds like taiga is), then the problem solves itself. otherwise it could be a big hacky mess.

once that's done, the only big issue remaining is irc. for this, i had the idea to authorize one or more irc nicks to submit race results on your behalf. basically, your account on SRL (the website) is your master account. from the site, you can add a nickname that you want to use on irc for races. once it verifies that nobody else has already claimed that nick (and probably verifies that you actually have access to it and you're not just stealing someone else's nick), then all races done using that nick will be associated with your account.

this approach has a lot of advantages imo. it would be a lot easier to build, since you wouldn't have to worry about authenticating through irc using the same credentials as the website (i.e. you could still use nickserv like normal without getting it to talk to the site database). it would also allow the same user to use multiple nicks for racing, which has been a (very minor) problem before. if you have 2 nicks for whatever reason, you could race under either one and both would go to the same account. additionally, if you ever lost one of those nicks, you could revoke its ability to race for you and remove it from your account. this also doesn't tie SRL any more to irc than it already is. so far it works, but if SRL ever did move away from irc, there is no reason the new system couldn't submit race results through the same api.

aside from the login stuff, one of the biggest technical challenges for the ZSR leaderboards was how to store and sort the metadata related to each run. in the google doc in the first post, this sounds like what the "tagging" system refers to. i had a few naive ideas about how much data for a run needs to be available, sortable, and filterable when i started on ZSR's leaderboards, so there were a few design decisions that were definitely mistakes.

making everything about a run (region, console, version, cheats enabled, filename if it matters, game category, character used, etc.) a "tag" is probably the right choice. it allows the most control over the retrieval and display of the data on the boards, and gives you a lot of flexibility for future things that you didn't anticipate. on the other hand, enforcing the integrity of the data becomes hard. to use OoT as an example, you have 3 rom versions, 5 official releases on 4 different consoles, 4 different regions, and 5 different languages. the user should be allowed (and strongly encouraged, maybe required) to submit as much of this information as possible. but v1.0 and v1.1 were only available as ntsc n64 releases, chinese was only available on the ique, german and french were only available on pal releases, etc. so you would need to prevent submissions like 1.1 VC, japanese ique, 1.0 german gamecube, and other things that don't exist. this information needs to be stored somewhere, and it's obviously way different for each game. not every game will even have different "versions" so this needs to be able to support arbitrary restrictions (imagine if the tingle tuner was only available on the japanese wind waker release, for example).

i don't know exactly how to solve that yet, but it's definitely something to think about on the technical side. it's something that needs to work correctly from the start, or it will only make growing the leaderboards to support more games increasingly complex as time goes on.

now for the non-technical discussion: i agree that opting out is not something that needs to be supported. this issue has been discussed to death and just about everything that can be said already has, but i'm just weighing in.

as far as submitting times for other users, i think it is overwhelmingly obvious that runners should submit their own times if possible. random users shouldn't be submitting runs for anyone but themselves, and i would be happy if only a certain higher rank of user (moderator or whatever for a game/community) can submit times for others. the de facto method of submission should be for a runner to submit their own runs. submitting for other people should only be done by some sort of staff-level user and only to fill in notable gaps.

i think that is an issue that some people are having with their runs being posted on the site by others, and it probably needs to be addressed more clearly in the document. assuming i understand it right, runners submit their own runs 99% of the time, including comments and whatever else they want. if a runner has a notable run, has no interest in posting it themselves, and a reasonable amount of time has passed without the runner submitting it, then a trusted moderator of some sort will add the time to the leaderboard in order to keep it accurate and up to date.

that's all i can think of right now. get hype every1

[quote="kirkq"][b]2: We need coders![/b]

This project does not have a lead developer currently.  We especially need people familiar with [u]databases, search queries, and forum integration.[/u]  Give us some idea of your abilities/specialties.  For now, you can post here and/or PM me online method of contact if you'd like.  I will not be the one ultimately organizing the project from a coding standpoint, but I can pass that information on.  We will probably get a github going at some point.[/quote]

hey there.

i built the [url="http://zeldaspeedruns.com/leaderboards"]ZeldaSpeedRuns leaderboards[/url] by myself. i pretty much run ZSR these days (even though it was cosmo's site originally), and i'm pretty familiar with a lot of the SRL communities. i'm also a full-stack web developer at my day job, and i guess i could provide a resume (haha). i feel like i could be pretty helpful when it comes to actually building this stuff.

i would love to lead the development of this project on, but i can't start on anything until sometime next month. i know seasons 2 is coming up soon, so maybe we have time. if you haven't found somebody else by then, count me in for this. in either case, i had a few ideas for how to handle some of the upcoming work.

[quote="kirkq"]- Create SRL forums
- Create SRL logins
- Integrate user accounts for SRL forums, SRL races, SRL leaderboards, and IRC.  (This is difficult.)[/quote]

the biggest issue to start with seems to be some sort of login system. i did see this earlier in the thread:

[quote="nate"]just drop me a line if you're interested in using taiga forum for the account system (either via the sda forum or another, new forum). tf has a nice api that i wrote so that the sda queue and knowledge base could use a single login. there are lots of other goodies in the api too like posting as an arbitrary user, editing posts, sending pms, etc. i can provide the documentation and help you get set up.[/quote]

this is probably something worth looking into. you'll definitely want to have the forums and the rest of the site (leaderboards, profile, etc.) use the same credentials, so picking out some forum software that can interface well with the rest of your app is important. if it's designed that way (and it sounds like taiga is), then the problem solves itself. otherwise it could be a big hacky mess.

once that's done, the only big issue remaining is irc. for this, i had the idea to authorize one or more irc nicks to submit race results on your behalf. basically, your account on SRL (the website) is your master account. from the site, you can add a nickname that you want to use on irc for races. once it verifies that nobody else has already claimed that nick (and probably verifies that you actually have access to it and you're not just stealing someone else's nick), then all races done using that nick will be associated with your account.

this approach has a lot of advantages imo. it would be a lot easier to build, since you wouldn't have to worry about authenticating through irc using the same credentials as the website (i.e. you could still use nickserv like normal without getting it to talk to the site database). it would also allow the same user to use multiple nicks for racing, which has been a (very minor) problem before. if you have 2 nicks for whatever reason, you could race under either one and both would go to the same account. additionally, if you ever lost one of those nicks, you could revoke its ability to race for you and remove it from your account. this also doesn't tie SRL any more to irc than it already is. so far it works, but if SRL ever did move away from irc, there is no reason the new system couldn't submit race results through the same api.

aside from the login stuff, one of the biggest technical challenges for the ZSR leaderboards was how to store and sort the metadata related to each run. in the google doc in the first post, this sounds like what the "tagging" system refers to. i had a few naive ideas about how much data for a run needs to be available, sortable, and filterable when i started on ZSR's leaderboards, so there were a few design decisions that were definitely mistakes.

making [i]everything[/i] about a run (region, console, version, cheats enabled, filename if it matters, game category, character used, etc.) a "tag" is probably the right choice. it allows the most control over the retrieval and display of the data on the boards, and gives you a lot of flexibility for future things that you didn't anticipate. on the other hand, enforcing the integrity of the data becomes [i]hard[/i]. to use OoT as an example, you have 3 rom versions, 5 official releases on 4 different consoles, 4 different regions, and 5 different languages. the user should be allowed (and strongly encouraged, maybe required) to submit as much of this information as possible. but v1.0 and v1.1 were only available as ntsc n64 releases, chinese was only available on the ique, german and french were only available on pal releases, etc. so you would need to prevent submissions like 1.1 VC, japanese ique, 1.0 german gamecube, and other things that don't exist. this information needs to be stored somewhere, and it's obviously way different for each game. not every game will even have different "versions" so this needs to be able to support arbitrary restrictions (imagine if the tingle tuner was only available on the japanese wind waker release, for example).

i don't know exactly how to solve that yet, but it's definitely something to think about on the technical side. it's something that needs to work correctly from the start, or it will only make growing the leaderboards to support more games increasingly complex as time goes on.

now for the non-technical discussion: i agree that opting out is not something that needs to be supported. this issue has been discussed to death and just about everything that can be said already has, but i'm just weighing in.

as far as submitting times for other users, i think it is overwhelmingly obvious that runners [i]should submit their own times[/i] if possible. random users shouldn't be submitting runs for anyone but themselves, and i would be happy if [b]only[/b] a certain higher rank of user (moderator or whatever for a game/community) can submit times for others. the de facto method of submission should be for a runner to submit their own runs. submitting for other people should only be done by some sort of staff-level user and only to fill in notable gaps.

i think that is an issue that some people are having with their runs being posted on the site by others, and it probably needs to be addressed more clearly in the document. assuming i understand it right, runners submit their own runs 99% of the time, including comments and whatever else they want. [b]if[/b] a runner has a notable run, has no interest in posting it themselves, and a reasonable amount of time has passed without the runner submitting it, [b]then[/b] a [i]trusted[/i] moderator of some sort will add the time to the leaderboard in order to keep it accurate and up to date.

that's all i can think of right now. get hype every1

kirkq

Edit history:

kirkq: 2013-08-14 08:36:20 pm

2013-08-14 08:35:08 pm

Die Hard 2013.

^This guy is pretty cool.

I'm probably going to bed soon. I have no idea when I'll update the document next.

Maxx

2013-08-14 08:37:05 pm

Quote from Svenne:

Someone said my name, we have had atleast 3 pepole from the Zelda 2 community in here in writing, but you do not care about our community. Sure, I own that site but not like I can force people not to submit to ZSR, they do not submit to it on their own will. And it's not even ZSR they are angry at, but too bad the same people who made that site made SRL.

I would love to see Pro_JN up as #1 on Zelda 2 on the leaderboard so he finally got the recognition he deserve, but we all think you have threated us with too much bad stuff that we, as a whole (community) does not want anything to do with you.

Quote from Svenne:

Last post I will make here. It happened in March, from that point to now there has been three races on Zelda 2 @ SRL.

also fyi i have no idea what svenne's issue with ZSR/SRL is. i've only talked to him once and it was right after i added zelda 2 support to the zsr leaderboards. i definitely didn't threaten him with any "bad stuff"

CosmoWright

2013-08-14 08:42:04 pm

Quote from Maxx:

i would love to lead the development of this project on, but i can't start on anything until sometime next month. i know seasons 2 is coming up soon, so maybe we have time. if you haven't found somebody else by then, count me in for this.

I'm hype and a month doesn't sound like a big delay to get started. There are several things we can accomplish in this time (I will personally build mockups and share once ready) and I know there are definite smaller points of discussion to hammer out in the mean time so we can have the document more fully fledged.

For forums software, I noticed a lot of big sites nowadays are using "XenForo" as their forums software. It seems to be pretty fully featured, not sure if that is necessary. If the taiga forum api that nate was talking about is good then we can probably just use taiga forum. Any other opinions on forums software?

andrewg

2013-08-14 08:44:32 pm

Hi! I'm andrewg!

One major thought!

So, basically timing and version differences:

Runs should be compared directly and converted to directly compare say a J version and US version of the same game. That is, assuming most things are identical for a given game. PAL and NTSC should also be converted in the same manner, but NTSC has sort of become the standard, so this can be tricky. I just think time conversions should be handled properly, and time differences between versions should be compared thoroughly for an accurate leaderboard list.

kirkq

Edit history:

kirkq: 2013-08-14 08:52:58 pm

2013-08-14 08:51:51 pm

Die Hard 2013.

That logic really doesn't work for modern games at all. Just take all the Ocarina of Time version differences for example. Speaking in absolutes for timing doesn't work.

Our idea was to list all versions in "pure time as how it is elected to be measured by the community", then have all the times in the same table (for the most part), and allow you to sort by version. Separately, you list at the top of the category the time differences between versions (maybe this could be in detail on SDA KB if desired functionality.) It is feasible to order the leaderboards by subtracting a preset amount of time for certain versions for simple games. This would take some decent bit of extra coding for generalized functionality. However it cannot be accurately expanded at all to represent games with different amounts of lag or slight physics differences. It's actually the minority case when you can accurately pinpoint that time. It's not the majority case.