Continuation of Roundtable Discussion

Darkwing Duck

2012-11-09 08:16:15 pm

Highly Evolved

Quote from Flip:

Quote from Darkwing Duck:

Quote from Flip:

Hopefully in the near future we can get to the point where submission to posting time is consistently lower.

Would it be possible to give an estimate on how much lower? Six months to five months wouldn't seem to appease the issues with the long gap between completion of a run and posting of a run, while six to, say, three would be a significant dent.

Any estimation i could give you would be pretty arbitrary, tbh. That's why I'm not jumping out here with bold promises. My goal would be closer to 3 months, but only time will tell. As always, how long a run takes to get verified plays a huge part, and nearly all of that is out of my hands.

That's fair enough. There were two major sections of submission that were identified at the roundtable. Verification and post verification. Someone, I think Miles, posted a pastebin showing the amount of time between verification and upload on SDA. The minimum was 44 days, and the average was 80. There was no mode, but I imagine it would be under 80 given the outlier runs that took over 150 days. Could there be an estimation based off of those numbers? If not, that's fine, but while there's been a very high amount of discussion on speeding up the verification portion of submission, but there's been little to no discussion about how the post verification submission process can be sped up. We, the forum members, don't seem to ever get a full idea of what goes into that average of 80 days other than an overview and anecdotes on specific runs and why they're delayed.

Since you're part of the process, it would be great if you could tell us what the realistically ideal amount of time between verification and upload would be, and what you be hoping to get with the automated improvements in that part of the submission process.

[quote="Flip"][quote="Darkwing Duck"][quote="Flip"]Hopefully in the near future we can get to the point where submission to posting time is consistently lower.[/quote]

Would it be possible to give an estimate on how much lower?  Six months to five months wouldn't seem to appease the issues with the long gap between completion of a run and posting of a run, while six to, say, three would be a significant dent.
[/quote]
Any estimation i could give you would be pretty arbitrary, tbh. That's why I'm not jumping out here with bold promises. My goal would be closer to 3 months, but only time will tell. As always, how long a run takes to get verified plays a huge part, and nearly all of that is out of my hands.[/quote]

That's fair enough.  There were two major sections of submission that were identified at the roundtable.  Verification and post verification.  Someone, I think Miles, posted a pastebin showing the amount of time between verification and upload on SDA.  The minimum was 44 days, and the average was 80.  There was no mode, but I imagine it would be under 80 given the outlier runs that took over 150 days.  Could there be an estimation based off of those numbers?  If not, that's fine, but while there's been a very high amount of discussion on speeding up the verification portion of submission, but there's been little to no discussion about how the post verification submission process can be sped up.  We, the forum members, don't seem to ever get a full idea of what goes into that average of 80 days other than an overview and anecdotes on specific runs and why they're delayed.

Since you're part of the process, it would be great if you could tell us what the realistically ideal amount of time between verification and upload would be, and what you be hoping to get with the automated improvements in that part of the submission process.

Paraxade

Edit history:

Paraxade: 2012-11-09 08:31:21 pm

2012-11-09 08:30:33 pm

Just a thought, if we're about knowledge now then obsoleted runs and their comments definitely need to be preserved somewhere.

Not actually that bad of an idea for SDA's direction in the future though.

Tiberius

2012-11-09 08:46:17 pm

Make it so.

Now I've never claimed to be particularly eloquent so I hope you'll bear with me while I try and get my thoughts across here.

With regards to the idea of SDA potentially becoming obsolete due to the potential leaderboards/natural evolution of speedrunning, I do not believe this is the case. While I do agree that there are some people on Twitch etc so make jokes out of SDA (I've had a few in my stream itself) there is also an equal amount for the site and its ideals. I do think there are some small changes that could be made to be slightly more acommodating to the impatient speedrun watcher but it is my personal belief that it is the last think we should is completely revise the basis behind the site, which is what would happen if we tried to track every single record on Twitch.

It's a phrase that is commonly thrown around on here but it rings true to me, SDA is not about records, SDA is about high quality speedruns. One thing I do believe is that the 'quality' in that sentence does not have to apply so strongly to the video quality above all. I was part of a recent verification in which an *extremely* good run (S3&K Tails run to be specific) was very nearly rejected based on Audio quality. While having very nice quality runs is a great thing, I'm sure I'm not the only one who downloads the HQ vids and loves every pixel, having lesser quality runs is not the end of the world, as long as the quality of the run is up to standard.

Because I believe the quality of the run is paramount, I also do not think that the verification process needs to be redefined. Although the idea of a public thread has some merit to it, I am more inclined to believe that it would detract from the prestige of an SDA run. And there is undoubtedly prestige and integrity in having a run on SDA. I personally do not have a run on the site, for a number of gameplay reasons and the secondary reason that I have no capture device. Nonetheless I would equally be against Emulator submission. Running on Emulator is undeniably a great way to practice and run prelim attempts but like people have mentioned, there are minute differences ranging from input method to coping with lag which again, would reduce the prestige of a 'hosted on SDA' video.

Now that we have got that out of the way here are a few of my suggestions.

First and foremost, an incredibly easy way (as far as I'm aware) of letting folk who care less about the video quality getting runs faster: why not leave the link to the verification copy in the posted verification thread? I know for sure that I for one would love to check out some runs that have been posted as an accept but are not yet actually on the site, just as an example Hydlide springs to mind. If there is a Twitch recording of the run a link to that in the verification thread would not be amiss either in my opinion. Add to this a link to the verification topic on the front page so that people can skip there quickly and easily and with almost no effort you have a bunch more SDA ACCEPTED runs that can be viewed, albeit in lesser quality.

A second idea is a way to speed up verification itself (slightly) and is a little more time consuming but equally feasable: Take the idea of the Feedback thread for sales and trades, and apply it to verifiers themselves. I imagine that Flip already has one of these in his head if nothing else and there is no need for it to be public either but the idea is simple, if you offer to verify a run and do your verification comment within say a week of the PM being posted, you get a +1. If you take a month, you get a -1. Arbitrary times there obviously. Then it is a simple matter for Flip or whoever to look at the offers of verification and pick the people who are likely to reliably post in the thread quickly.

Maybe not the best ideas ever and certainly not groundbreaking but I hope they could possibly plant seeds in minds.

Finally: with regards to the supposed rift between SDA and SRL? I've been a small part of both communities for a while now and to be honest, I've never come across anything but respect that the sites have for each other. The problem is that occasionally a person gets shunned (for whatever reason) by one community then goes to the other to complain. These are rare and individual cases however and it's my belief that there is no issue between the two sites, it's essentially just one big community with two seperate divisions of focus.

Thanks for reading all if you've got through this hideous wall of text, hope I've not said anything too ridiculous and to echo several of the sentiments above: Be you an SDA regular, an SRL leaderboard smasher, or a Twitch monster. Love you all.

Now I've never claimed to be particularly eloquent so I hope you'll bear with me while I try and get my thoughts across here.

With regards to the idea of SDA potentially becoming obsolete due to the potential leaderboards/natural evolution of speedrunning, I do not believe this is the case. While I do agree that there are some people on Twitch etc so make jokes out of SDA (I've had a few in my stream itself) there is also an equal amount for the site and its ideals. I do think there are some small changes that could be made to be slightly more acommodating to the impatient speedrun watcher but it is my personal belief that it is the last think we should is completely revise the basis behind the site, which is what would happen if we tried to track every single record on Twitch.

It's a phrase that is commonly thrown around on here but it rings true to me, SDA is not about records, SDA is about high quality speedruns. One thing I do believe is that the 'quality' in that sentence does not have to apply so strongly to the video quality above all. I was part of a recent verification in which an *extremely* good run (S3&K Tails run to be specific) was very nearly rejected based on Audio quality. While having very nice quality runs is a great thing, I'm sure I'm not the only one who downloads the HQ vids and loves every pixel, having lesser quality runs is not the end of the world, as long as the quality of the run is up to standard.

Because I believe the quality of the run is paramount, I also do not think that the verification process needs to be redefined. Although the idea of a public thread has some merit to it, I am more inclined to believe that it would detract from the prestige of an SDA run. And there is undoubtedly prestige and integrity in having a run on SDA. I personally do not have a run on the site, for a number of gameplay reasons and the secondary reason that I have no capture device. Nonetheless I would equally be against Emulator submission. Running on Emulator is undeniably a great way to practice and run prelim attempts but like people have mentioned, there are minute differences ranging from input method to coping with lag which again, would reduce the prestige of a 'hosted on SDA' video.

Now that we have got that out of the way here are a few of my suggestions.

First and foremost, an incredibly easy way (as far as I'm aware) of letting folk who care less about the video quality getting runs faster: why not leave the link to the verification copy in the posted verification thread? I know for sure that I for one would love to check out some runs that have been posted as an accept but are not yet actually on the site, just as an example Hydlide springs to mind. If there is a Twitch recording of the run a link to that in the verification thread would not be amiss either in my opinion. Add to this a link to the verification topic on the front page so that people can skip there quickly and easily and with almost no effort you have a bunch more [b]SDA ACCEPTED[/b] runs that can be viewed, albeit in lesser quality.

A second idea is a way to speed up verification itself (slightly) and is a little more time consuming but equally feasable: Take the idea of the Feedback thread for sales and trades, and apply it to verifiers themselves. I imagine that Flip already has one of these in his head if nothing else and there is no need for it to be public either but the idea is simple, if you offer to verify a run and do your verification comment within say a week of the PM being posted, you get a +1. If you take a month, you get a -1. Arbitrary times there obviously. Then it is a simple matter for Flip or whoever to look at the offers of verification and pick the people who are likely to reliably post in the thread quickly.

Maybe not the best ideas ever and certainly not groundbreaking but I hope they could possibly plant seeds in minds.

Finally: with regards to the supposed rift between SDA and SRL? I've been a small part of both communities for a while now and to be honest, I've never come across anything but respect that the sites have for each other. The problem is that occasionally a person gets shunned (for whatever reason) by one community then goes to the other to complain. These are rare and individual cases however and it's my belief that there is no issue between the two sites, it's essentially just one big community with two seperate divisions of focus.

Thanks for reading all if you've got through this hideous wall of text, hope I've not said anything too ridiculous and to echo several of the sentiments above: Be you an SDA regular, an SRL leaderboard smasher, or a Twitch monster. Love you all.

BlueGlass

2012-11-09 08:59:34 pm

Yep yep yep!

Quote from Paraxade:

Just a thought, if we're about knowledge now then obsoleted runs and their comments definitely need to be preserved somewhere.

Like Archive.org? It's gigantic and messy and slow, but I'm pretty sure it has practically every single run published on SDA.

As for the main point of this discussion, others have already said anything I could possibly think of.

Solairflaire

Edit history:

Solairflaire: 2012-11-09 09:04:53 pm

2012-11-09 09:04:14 pm

Quote from Paraxade:

Just a thought, if we're about knowledge now then obsoleted runs and their comments definitely need to be preserved somewhere.

Not actually that bad of an idea for SDA's direction in the future though.

They are, on Archive.org. It's just a pain to find them. A possible solution would be an SDA leaderboard that has runs for a game that were accepted by SDA with a link to the Archive.org page. Granted, it would be a huge undertaking to do that now.

Something that would help with the perceived "rift" between SDA and SRL would be having a link from each site to the other, preferably on the front page. Heck, even say, "Visit our Sister site ___".

edit: BlueGlass beat me to the Archive.org thing.

Paraxade

Edit history:

Paraxade: 2012-11-09 09:06:38 pm

2012-11-09 09:05:25 pm

Archive.org is missing a lot of run's comments and it's a huge mess to navigate. It's not good enough.

It's not necessary by any means, I just think it'd be a very good thing if we did go down the path of SDA being a knowledge repository.

Rakuen

2012-11-09 09:06:14 pm

Weegee Time

http://kb.speeddemosarchive.com/Historical_Records

It's over a year out of date now, but it's a still a really good start for listing out all the runs that have been on SDA.

Serris

2012-11-09 10:17:09 pm

A "general speedrunning news" box separated from the normal SDA updates at the top of the front page, which could be updated in a flash with just a link to new significant twitch/youtube videos + short comments, maybe even glitch discoveries for popular games, several times a week/day, accompanied by a forum thread where people can post potential links for these news, which then get picked up by the admins, sounds like a good idea to me. I don't think there is a go-to place for a quick overview/updates on all new speedrunning developments (including TASs) yet, so anyone interested in speedrunning would get into the habit of checking the front page regularly - even if they don't care (initially) about the normal SDA updates, they will notice those too. This way SDA can keep the focus on quality with the usual updates without fading into obscurity among this new speedrunning boom.

Miles

2012-11-09 10:23:39 pm

Quote from Darkwing Duck:

Someone, I think Miles, posted a pastebin showing the amount of time between verification and upload on SDA.

http://pastebin.com/p1QGhejq

The median is 79; not much better. Whenever the subject came up in the roundtable, the response was basically "a few runs have issues that cause them to be delayed"; I can understand for runs like Frogger and Operation C, but why is the average run taking around 80 days? And I saw it mentioned in the chat somewhere, but do we actually want 44 days to be the "best case minimum"?

UraniumAnchor

2012-11-09 10:42:28 pm

Not a walrus

Regarding historical pages/records, that's something I designed the backend to support eventually, but there's quite a few more important features on the list first.

And migrating all of that data over is going to be a monumental undertaking. Might have to crowd-source that one a bit. Tongue

As for posting delays, I suspect once the backlog clears out a bit that'll become less of an issue.

Lee

Edit history:

Lee: 2012-11-10 02:40:42 am

Lee: 2012-11-10 02:37:09 am

2012-11-10 02:35:46 am

Personally, my own issue is that there's a very large difference between bleeding edge and up to date, and SDA for sure isn't the former, nor does it try to be, but atm is having difficulty with the latter as well. When I come to SDA, do I expect the latest WR posted? No. But I do expect to find runs that at least reflect the current state of the game. People have heavily emphasized the verification process as a culprit, and I'll throw my vote there as well, but looking at it from a slightly more oblique angle.

The impact that the delays in verification have has been pretty clear to me. I'll reference Yoshi's Island, since that's the game I know the best (although I'm starting to grow lax in keeping up with changes in the game, thanks school). Trihex's old any% that he himself said was an embarrassment. Old 100% IL's. The absence of a 100% SS. Granted, this is not directly the fault of SDA. For a very long time the YI scene was not nearly as vibrant as it is now. However, now that the scene is there, for people who want to check out new runs/see the new content, it's very difficult for someone to do so. When Carl_Sagan first started learning, all he had to work with was trihex's old runs. I was in much of the same position. The issue I have with this is that even now, after an explosion of activity, anyone new who wants to get involved still is limited to the same options on SDA. Yes, they can go through twitch archives, but they also have to scrub through hours upon hours of resets. Maybe they want to learn, and the runner only adds educational audio commentary afterwards. To truly get involved, they must join our irc channel, which is all well and good, but that's not helping this site any. Twitch/streaming, although incredibly convenient and, imo, one of the best things to happen to speedrunning, does not satisfy all our needs. However, currently SDA is struggling a bit in meeting those unmet needs as well.

What SDA does offer me currently (and would love to see more of):
1. A place to exchange ideas with other people, and have those ideas be permanent, instead of having to look through irc logs.
2. A place for high quality videos (this is surprisingly important to me and I didn't realize how much I wanted it until I thought about it some more)
3. A place for audio commentary. This is great! I love being able to hear a runners thoughts, or laugh at some silly audio antics, without forcing the runner to lose concentration because he's more focused on explaining than on the game.
4. Of course, AGDQ. <3.

What I want from SDA (personal desire):
1. A place where I can go to to find quality videos of up to date material. Not exclusively World Record, although that certainly would qualify as up to date, but simply one that people would be proud to call a run. Some runners hate putting up a run, only to have it be beaten, and I fully understand that. However to me, that dislike stems less from knowing that your run was beaten, but rather that your run was the best, and it was obsoleted before anyone could recognize all the hard work you put into it, through no fault of your own, merely the length of the verification process.
2. A place where I can find good information, easily.

Some other thoughts, probably unreasonable, but still things I'd love to see:

Regarding game pages: I love integration. If everything could be on click away, I would be the happiest man alive. Some people have been voicing concerns that the forums may become obsolete. I think that the forums are still one of the best places for people to collaborate, but it'd be nice for it to have more visibility. To marry the two points together, I'd love if each game page had relevant links. Not just the runs themselves, but if the game has a thread (and most, if not all do), link to that thread. Someone has a WR? If there's a youtube of it, or a highlight on twitch, link to that. If it has a wiki, link to the wiki. Cross link to SRL Leaderboards. Make it easy on the visitor to both enjoy the run, find information, AND to participate in the community if they so desire.

I do realize that a huge part of what I'm suggesting, especially with the game page, requires a large amount of manpower. But I also think that the community is willing to step up in terms of doing a lot of that. People who are only authorized to edit game pages. Liquipedia/Team Liquid has its team of admins, but also groups of people whose only power role is to update records. For a site like that, up to date records is vital. No one on the admin team has the time to slog through all of that. The larger the community grows, the greater the need for people to step up and shoulder some of the burden. The race structure of SRL is quite similar, actually. It demands that there are responsible people around to record races when they finish. These people can't change the site, they have very little power outside of the few tasks they're entrusted with. But without them, SRL would flow a lot less smoothly.

Regarding my desire to want more information: People want to do this. People have already started wikis separate from this site to compile said information. The current wiki editing tools on SDA are a little clunky and not at all easy to use. I know we considered it when first starting the YI wiki, but decided against it for just that reason. If you provide people with the tools, they will use them.

In the end, I think most people ultimately want the same thing. We want to focus on these games that we're so passionate about. If you look at it through that lens, I think all of the arguments being presented in this thread are all quite reasonable.

Personally, my own issue is that there's a very large difference between bleeding edge and up to date, and SDA for sure isn't the former, nor does it try to be, but atm is having difficulty with the latter as well. When I come to SDA, do I expect the latest WR posted? No. But I do expect to find runs that at least reflect the current state of the game. People have heavily emphasized the verification process as a culprit, and I'll throw my vote there as well, but looking at it from a slightly more oblique angle.

The impact that the delays in verification have has been pretty clear to me. I'll reference Yoshi's Island, since that's the game I know the best (although I'm starting to grow lax in keeping up with changes in the game, thanks school). Trihex's old any% that he himself said was an embarrassment. Old 100% IL's. The absence of a 100% SS. Granted, this is not directly the fault of SDA. For a very long time the YI scene was not nearly as vibrant as it is now. However, now that the scene is there, for people who want to check out new runs/see the new content, it's very difficult for someone to do so. When Carl_Sagan first started learning, all he had to work with was trihex's old runs. I was in much of the same position. The issue I have with this is that even now, after an explosion of activity, anyone new who wants to get involved [i]still is limited to the same options on SDA.[/i] Yes, they can go through twitch archives, but they also have to scrub through hours upon hours of resets. Maybe they want to learn, and the runner only adds educational audio commentary afterwards. To truly get involved, they must join our irc channel, which is all well and good, but that's not helping this site any. Twitch/streaming, although incredibly convenient and, imo, one of the best things to happen to speedrunning, does not satisfy all our needs. However, currently SDA is struggling a bit in meeting those unmet needs as well.

What SDA does offer me currently (and would love to see more of): 
1. A place to exchange ideas with other people, and have those ideas be permanent, instead of having to look through irc logs. 
2. A place for high quality videos (this is surprisingly important to me and I didn't realize how much I wanted it until I thought about it some more)
3. A place for audio commentary. This is great! I love being able to hear a runners thoughts, or laugh at some silly audio antics, without forcing the runner to lose concentration because he's more focused on explaining than on the game. 
4. Of course, AGDQ. <3.

What I want from SDA (personal desire): 
1. A place where I can go to to find quality videos of up to date material. Not exclusively World Record, although that certainly would qualify as up to date, but simply one that people would be proud to call a run. Some runners hate putting up a run, only to have it be beaten, and I fully understand that. However to me, that dislike stems less from knowing that your run was beaten, but rather that your run was the best, and it was obsoleted before anyone could recognize all the hard work you put into it, through no fault of your own, merely the length of the verification process.
2. A place where I can find good information, easily.

Some other thoughts, probably unreasonable, but still things I'd love to see:

Regarding game pages: I love integration. If everything could be on click away, I would be the happiest man alive. Some people have been voicing concerns that the forums may become obsolete. I think that the forums are still one of the best places for people to collaborate, but it'd be nice for it to have more visibility. To marry the two points together, I'd love if each game page had relevant links. Not just the runs themselves, but if the game has a thread (and most, if not all do), link to that thread. Someone has a WR? If there's a youtube of it, or a highlight on twitch, link to that. If it has a wiki, link to the wiki. Cross link to SRL Leaderboards. Make it easy on the visitor to both enjoy the run, find information, AND to participate in the community if they so desire.

I do realize that a huge part of what I'm suggesting, especially with the game page, requires a large amount of manpower. But I also think that the community is willing to step up in terms of doing a lot of that. People who are only authorized to edit game pages. Liquipedia/Team Liquid has its team of admins, but also groups of people whose only power role is to update records. For a site like that, up to date records is vital.  No one on the admin team has the time to slog through all of that. The larger the community grows, the greater the need for people to step up and shoulder some of the burden. The race structure of SRL is quite similar, actually. It demands that there are responsible people around to record races when they finish. These people can't change the site, they have very little power outside of the few tasks they're entrusted with. But without them, SRL would flow a lot less smoothly.

Regarding my desire to want more information: People want to do this. People have already started wikis separate from this site to compile said information. The current wiki editing tools on SDA are a little clunky and not at all easy to use. I know we considered it when first starting the YI wiki, but decided against it for just that reason. If you provide people with the tools, they will use them.

In the end, I think most people ultimately want the same thing. We want to focus on these games that we're so passionate about. If you look at it through that lens, I think all of the arguments being presented in this thread are all quite reasonable.

RedArremer

Edit history:

RedArremer: 2012-11-10 05:31:05 am

RedArremer: 2012-11-10 05:30:22 am

2012-11-10 05:29:15 am

Since people mention records and all, how about some sort of news feed or so, that shows new records, and links to the Twitch/Youtube/whatever video?

I think that might be a good idea for a new section, and would ease up the rift between SDA putting videos online and people submitting them/reaching new times/etc.

SDA's video section would still have a purpose - good quality videos (in terms of video quality and run quality) - but there also would be a section that gets updated whenever a new record is reached in a game, and doesn't require a high quality video file to be submitted and verified - if the game gets a new record or so during the verification, there could be a blurb on the game's page, referring to the feed post.

The flagging system that was suggested earlier in this thread would also work very well with this - if a video submission is obviously cheated, then this would allow the SDA staff to remove the feed post.

Just some food for thought.

GalleonLogic

2012-11-10 05:45:23 am

One of the things that unless I have missed it in some of the longer comments that I haven't had a chance to read that I think has been overlooked is the point of the quality test that people are advised to do before submitting. My idea, and as with anything I say feel free to shoot it down, is to instead of waiting for a fully available high quality run to be made available, use a quality test post to gauge the expected quality of the video/ audio and a twitch recording to verify that no cheating has been performed in the run. The requirement then is that when the high quality video is available someone is tasked with Quality Assurance just needs to check that the quality is maintained (no desyncs etc). While this may not be perfect, it does allow for aspects of the verification process to be performed immediately or in the case of the Quality Test before hand the run is started/ completed.

On the side of the site's purpose, when I see a game that I don't know anything about running, SDA is the first place that I come. Even if the video on file is not the fastest time available, it is generally a very good starting point. In this case high quality run comments are extremely useful and something that I think TASvideos often does very well is giving details of the glitches used and how they work. If there isn't a run listed, then I search the forums for posts so I can learn about what has (if anything) been done in terms of speedrunning the game. From this standpoint alone I think SDA will always have some relevance, especially if those who are streaming can plug the site and the forums for even just this reason alone.

A final point on this is that was mentioned in a thread a few weeks ago (I think it was Essentia - sorry if I'm wrong), is that the KB is massively underused, a point that I full agree with. Given the move to streaming and racing, having an easily accessible and detailed knowledge base of tricks and routes would help those who are interested in picking up a game either to run or for racing. I know that trying to get that section up to date and keep it relevant would be tough, but I think it would be a great resource that might cut out searching through pages of forum posts (There must be some games with over 100 pages of posts).

One of the things that unless I have missed it in some of the longer comments that I haven't had a chance to read that I think has been overlooked is the point of the quality test that people are advised to do before submitting.  My idea, and as with anything I say feel free to shoot it down, is to instead of waiting for a fully available high quality run to be made available, use a quality test post to gauge the expected quality of the video/ audio and a twitch recording to verify that no cheating has been performed in the run.  The requirement then is that when the high quality video is available someone is tasked with Quality Assurance just needs to check that the quality is maintained (no desyncs etc).  While this may not be perfect, it does allow for aspects of the verification process to be performed immediately or in the case of the Quality Test before hand the run is started/ completed.

On the side of the site's purpose, when I see a game that I don't know anything about running, SDA is the first place that I come.  Even if the video on file is not the fastest time available, it is generally a very good starting point.  In this case high quality run comments are extremely useful and something that I think TASvideos often does very well is giving details of the glitches used and how they work.  If there isn't a run listed, then I search the forums for posts so I can learn about what has (if anything) been done in terms of speedrunning the game.  From this standpoint alone I think SDA will always have some relevance, especially if those who are streaming can plug the site and the forums for even just this reason alone.

A final point on this is that was mentioned in a thread a few weeks ago (I think it was Essentia - sorry if I'm wrong), is that the KB is massively underused, a point that I full agree with.  Given the move to streaming and racing, having an easily accessible and detailed knowledge base of tricks and routes would help those who are interested in picking up a game either to run or for racing.  I know that trying to get that section up to date and keep it relevant would be tough, but I think it would be a great resource that might cut out searching through pages of forum posts (There must be some games with over 100 pages of posts).

InsipidMuckyWater

Edit history:

InsipidMuckyWater: 2012-11-10 07:57:20 am

InsipidMuckyWater: 2012-11-10 07:56:44 am

InsipidMuckyWater: 2012-11-10 07:49:51 am

2012-11-10 07:47:42 am

Visit my profile to see my runs!

Quote from BlueGlass:

Like Archive.org? It's gigantic and messy and slow, but I'm pretty sure it has practically every single run published on SDA.

As for the main point of this discussion, others have already said anything I could possibly think of.

Yeah, I agree with Paraxade/UA on this. Past runs need to be way, way, WAY more visible than they are now through Archive and Historical Records. I eagerly await UA's future solution to this.

The point made about SDA being a niche is well put. I can just about guarantee you that if SDA were to ever disappear for whatever reason (god forbid), I think the speedrunning community would quickly regret the casualty.

Imagine the totality of speedrunning if there was no SDA. For that matter, imagine it without TASvideos too. Only live streaming remaining. Doesn't that completely change the nature of the speedrunning universe to you? It does to me. If there is some kind of rift between the streamers and SDA (I actually was totally unaware of one before this thread), I don't see what relevance this has to SDA, aside from some of the tragic bullying stories to which some have alluded (again being unaware of it before this thread, I have no idea what kind of bullying has supposedly taken place, though I presume it is genuine given the posts by lag.com and others).

Anyway, as I was saying, I think I would have no idea where I would go if I wanted to find "finished product" speedruns like the kind currently featured on SDA if SDA or TASvideos were not around. Twitch/SRL streaming is entertaining, but SDA sort of feels like the official release of a video game that previously was only in beta stage. [forgive the crude analogy - it's a placeholder for a better analogy, not meant to be a rip on stream-running] There will always be times, at least for me, when I want something official and certified, at least to some standard. I can see how more and more gamers might be attracted to the hobby of speedrunning as it pertains to acquiring audiences over the internet, but SDA is pretty much the only destination for me when it comes to a finished product.

To an extent, I suppose this somewhat contradicts my earlier post that SDA should be the forefront of the speedrunning conscience rather than merely a certification destination, but again I wouldn't have a problem with some kind of compromise situation for verification. If we can 'incorporate' live streaming somehow, at least in baby steps, that would probably be healthy for us, if nothing else (though please tell me the quality of the video and the ability to watch the run WITHOUT the live commentary from the streaming process, can both be preserved).

Normally, I'm usually one in favor of the status quo around here, but I do think the quicker turnaround as of late (MAJOR PROPS TO ADMINISTRATION BY THE WAY) has been great for the site, and I am also aware of the major hesitation expressed by the newest generations of runners.

[quote="BlueGlass"]Like Archive.org?  It's gigantic and messy and slow, but I'm pretty sure it has practically every single run published on SDA.

As for the main point of this discussion, others have already said anything I could possibly think of.[/quote]

Yeah, I agree with Paraxade/UA on this.  Past runs need to be way, way, WAY more visible than they are now through Archive and Historical Records.  I eagerly await UA's future solution to this.

The point made about SDA being a niche is well put.  I can just about guarantee you that if SDA were to ever disappear for whatever reason (god forbid), I think the speedrunning community would quickly regret the casualty.

Imagine the totality of speedrunning if there was no SDA.  For that matter, imagine it without TASvideos too.  Only live streaming remaining.  Doesn't that [i]completely[/i] change the nature of the speedrunning universe to you?  It does to me.  If there is some kind of rift between the streamers and SDA  (I actually was totally unaware of one before this thread), I don't see what relevance this has to SDA, aside from some of the tragic bullying stories to which some have alluded (again being unaware of it before this thread, I have no idea what kind of bullying has supposedly taken place, though I presume it is genuine given the posts by lag.com and others).

Anyway, as I was saying, I think I would have no idea where I would go if I wanted to find "finished product" speedruns like the kind currently featured on SDA if SDA or TASvideos were not around.  Twitch/SRL streaming is entertaining, but SDA sort of feels like the official release of a video game that previously was only in beta stage. [forgive the crude analogy - it's a placeholder for a better analogy, not meant to be a rip on stream-running] There will always be times, at least for me, when I want something [i]official[/i] and certified, at least to some standard.  I can see how more and more gamers might be attracted to the hobby of speedrunning as it pertains to acquiring audiences over the internet, but SDA is pretty much the only destination for me when it comes to a finished product.

To an extent, I suppose this somewhat contradicts my earlier post that SDA should be the forefront of the speedrunning conscience rather than merely a certification destination, but again I wouldn't have a problem with [i]some[/i] kind of compromise situation for verification.  If we can 'incorporate' live streaming somehow, at least in baby steps, that would probably be healthy for us, if nothing else (though please tell me the quality of the video and the ability to watch the run WITHOUT the live commentary from the streaming process, can both be preserved).

Normally, I'm usually one in favor of the status quo around here, but I do think the quicker turnaround as of late (MAJOR PROPS TO ADMINISTRATION BY THE WAY) has been great for the site, and I am also aware of the major hesitation expressed by the newest generations of runners.

ShadowWraith

2012-11-10 09:21:44 am

.

Quote from AlecK47:

On that note, I do have a potential suggestion. I may be wrong, but do the updaters/other admins have to go through all the final encodes after verification? If so, then that seems an ideal place for "open verification" in the sense that anyone could verify a video's quality, even if they know little to nothing about the game in question.

This already happens, and the turnaround time for this is lower than a week, since we have some very dedicated volunteers.

doicm

2012-11-10 09:53:14 am

Quote from RedArremer:

Since people mention records and all, how about some sort of news feed or so, that shows new records, and links to the Twitch/Youtube/whatever video?

As far as new records go, whether they be PB's or WR's, I refer to a twitter account called "Congratsio" that people can use to highlight their personal best records. This also has a specification as to whether or not it's a world record. I use it myself sometimes when I have a run that I like to highlight, and you are welcome to link a video as well.

Also, I really appreciate TSSB and what they do as far as highlighting world records. I honestly think that's one of the most awesome ideas to happen recently with speedrunning.

yoshifan

2012-11-10 12:42:13 pm

Quote from Tiberius:

First and foremost, an incredibly easy way (as far as I'm aware) of letting folk who care less about the video quality getting runs faster: why not leave the link to the verification copy in the posted verification thread? I know for sure that I for one would love to check out some runs that have been posted as an accept but are not yet actually on the site, just as an example Hydlide springs to mind. If there is a Twitch recording of the run a link to that in the verification thread would not be amiss either in my opinion. Add to this a link to the verification topic on the front page so that people can skip there quickly and easily and with almost no effort you have a bunch more SDA ACCEPTED runs that can be viewed, albeit in lesser quality.

Just wanted to point this out again, as a potentially easy, short-term alleviation of some of the issues here.

(Though, by "a link to the verification topic", do you mean the verification board? I think that would make more sense.)

[quote="Tiberius"]
First and foremost, an incredibly easy way (as far as I'm aware) of letting folk who care less about the video quality getting runs faster: why not leave the link to the verification copy in the posted verification thread? I know for sure that I for one would love to check out some runs that have been posted as an accept but are not yet actually on the site, just as an example Hydlide springs to mind. If there is a Twitch recording of the run a link to that in the verification thread would not be amiss either in my opinion. Add to this a link to the verification topic on the front page so that people can skip there quickly and easily and with almost no effort you have a bunch more [b]SDA ACCEPTED[/b] runs that can be viewed, albeit in lesser quality.[/quote]
Just wanted to point this out again, as a potentially easy, short-term alleviation of some of the issues here.

(Though, by "a link to the verification topic", do you mean the verification board?  I think that would make more sense.)

Marche_Fighter_Paladin

2012-11-10 02:03:21 pm

Claimh Happy

No, he means a link to that run's Verification Topic, which by his suggestion would have a DL link for a LQ vid of the run.

VorpalEdge

2012-11-10 02:17:13 pm

welcome to the machine

Leaderboards have never been something I've felt are really important. At SDA's inception, the prevailing attitude towards TG's leaderboards was basically lol, look at all these bad times. More recently, we've had cases where drama's occurred because of xbla leaderboards and stuff (iirc, the Rocket Knight topic and some resident evil topic or other). Some people have refused to submit because a leaderboard time beat their run (Greenalink on Shadow Complex until very recently), and I've always thought that was a tragedy. And personally, I've been 'running' the Super Metroid thread in the casual section and that hasn't really felt like I've been doing anyone a service.

Are leaderboards something a lot of people want?

There are a lot of people also talking about using twitch vods for submission in some form. What exactly do you envision them being used for? Verification? Final run download, to be posted? If to be posted, would you be alright with someone submitting with just a twitch vod or are they the LQ to a properly-encoded MQ/HQ? etc.

Where do you see speedrunning going over the next few years? What will get more important? What will become irrelevant? How do current SDA institutions fit into that?

--------

Question for Cosmo: the idea of a leaderboard honestly seems like a reversal from your previous stance. I was thinking about it, wondering if you were planning to integrate SRL into the site (i.e. finish a race, times update on leaderboard automatically if it's a pb), and then I remembered that hey, racebot used to have an RTA function but that got removed. What's changed?

I've definitely noticed that, once players get up to really high skill levels, they kinda stop racing a lot and start doing attempts a lot more. That's obviously a generalization but still.

Leaderboards have never been something I've felt are really important.  At SDA's inception, the prevailing attitude towards TG's leaderboards was basically lol, look at all these bad times.  More recently, we've had cases where drama's occurred because of xbla leaderboards and stuff (iirc, the Rocket Knight topic and some resident evil topic or other).  Some people have refused to submit because a leaderboard time beat their run (Greenalink on Shadow Complex until very recently), and I've always thought that was a tragedy.  And personally, I've been 'running' the Super Metroid thread in the casual section and that hasn't really felt like I've been doing anyone a service.

Are leaderboards something a lot of people want?

There are a lot of people also talking about using twitch vods for submission in some form.  What exactly do you envision them being used for?  Verification?  Final run download, to be posted?  If to be posted, would you be alright with someone submitting with just a twitch vod or are they the LQ to a properly-encoded MQ/HQ?  etc.

Where do you see speedrunning going over the next few years?  What will get more important?  What will become irrelevant?  How do current SDA institutions fit into that?

--------

Question for Cosmo: the idea of a leaderboard honestly seems like a reversal from your previous stance.  I was thinking about it, wondering if you were planning to integrate SRL into the site (i.e. finish a race, times update on leaderboard automatically if it's a pb), and then I remembered that hey, racebot used to have an RTA function but that got removed.  What's changed?

I've definitely noticed that, once players get up to really high skill levels, they kinda stop racing a lot and start doing attempts a lot more.  That's obviously a generalization but still.

Manocheese

2012-11-10 02:43:07 pm

Yes, a cucco riding the ground.

Leaderboards

The first topic in the roundtable video was leaderboards, so I'll start with that. Leaderboards have had great success on sites for individual games and series. See The Elite, The Sonic Center, and so on. One of the main draws of those sites is that you can compete without having the fastest time, encouraging more people to play. They can then work their way up the ranks and start setting records. Another benefit is that times can be posted immediately, so you can always see what the fastest (claimed) time is as long as people put it up. There are drawbacks to that kind of speed running site, though. Because anyone can post a time, cheated times can be put on the ranks. This is especially evident with times that aren't near the top of the ranks and are thus less likely to be scrutinized. The Elite has had problems with this. However, at least they require proof for times worth a certain number of points (i.e. times close to the fastest time on the ranks). But some leaderboards don't. On ZeldaSpeedRuns' well-hidden leaderboards for Ocarina of Time, the fastest time listed (as I write this) doesn't have a video. Leaderboards are susceptible to other problems too: some videos look so bad that they are almost unwatchable; some videos have no game audio (this seems very popular on Twitch); and videos are not verified.

None of these things are the direction SDA should go in. As I said above, leaderboards serve useful purposes. Currently, it's very difficult to answer the question "what is the fastest run?" for some games. It would be great to have a place where people can go to see what the fastest claims and the fastest videos are. But that place shouldn't be SDA.

The Role of SDA

So what should SDA be about? Speaking personally, when I look up a run on SDA, I want to see a finished product. That includes several things: a full run; a video that doesn't look like a dog ate it; intact game audio; the runner's comments; and oftentimes, audio commentary relevant to the run. Streaming doesn't fulfill those purposes. When you go to a stream on Twitch, you're likely to see failed attempts, irrelevant commentary, advertisements interrupting the video, clutter on the screen, a bad framerate, and so on. I'm not trying to knock streaming; I know there are a lot of people who like watching the process of speed running, talking in the chat, etc. This is the kind of thing some people want. It's not the same as what SDA offers, and not everyone wants both. That means that there are a lot of people who won't come to SDA no matter what we do. We shouldn't bend over backwards trying to appease them when they will never come here anyway.

Another role of SDA is as a source of information. If you want to learn about a game, you can watch the runs on it, read the comments, and read the forum threads. If there are obsoleted runs, you can find them on archive.org (side note: I agree that they should be on SDA itself, and I'm glad to hear that UraniumAnchor has done some work on it). If SDA didn't exist, none of that information would be preserved. If you wanted to learn about a game that wasn't fashionable to stream at the time, you would be out of luck. And even if the game were popular, it would be a lot harder to find information without SDA. Maybe there's a chat room for it somewhere, and maybe there's someone there who can answer your questions, and maybe they will take the time to do so. But it's far better to have a place where the information is kept for good, available to everyone, and carefully written, not just glossed over as briefly as possible in a chat room. In Nate's history topics, he says that that's one of the reasons he created Metroid2002. It's also why the Strategy Wiki should be used more than it is now.

There's also the fact that the forum is a great place to share knowledge about a game. People are really hurting the development of speed runs when they only share their ideas in chat rooms, instant messaging, or what have you. Anyone who isn't there when the discussion happens misses out. Even if the chat is logged, it's likely to be 99% garbage that no one wants to pick through on the off chance that there's something useful there. On a forum, discussions are visible to everyone, whether they were present at the time or not. The signal-to-noise ratio also tends to be higher, so it's easier for people to find information and get up to speed.

Misinformation about SDA

There's a lot of misinformation about SDA among people new to speed running. Unfortunately, even long-time runners have contributed to this phenomenon by spreading information that's downright false. For example, there's a myth that submitting to SDA is a very difficult and arduous process. The truth is that, while it's slightly harder than uploading to YouTube because you have to talk to a human being, it's still about as simple as possible. You just send some basic information about your run, then follow a few instructions to send the run itself. You don't even need to write real comments if you don't want to; see North & South, Super Metroid, and several Mega Man runs. Of course, detailed comments are better, but they're not required. Also, if you do want to submit detailed comments, you can initiate the submission process before you finish them.

There are also misconceptions about our play quality standards. Here's part of a recent post from The Elite:

Quote:

For instance, if you were to hit the glass strat in a 00A run, then all future 00A runs would have to hit that, because that strat is fastest. And even if someone beat that 00A time, if they didn't use the glass strat, SDA would not want the run... That's just how SDA operates.

This is so blatantly false that I won't even bother debunking it. That's not the point. The point is that wrong information like this is spread around because people believe whatever they hear. In many cases, the people giving or receiving the information have some irrational hatred of SDA, so they are especially likely to believe anything negative about it. As with the people who only want to watch streams, we shouldn't bend over backwards to appease these people. They've already made up their minds about SDA. Instead, let's try to prevent new people who are currently neutral about SDA from being poisoned by false information.

Open Verification

I'm still not sure exactly what this entails. The impression I got is as follows: As soon as the verification copy of a run is available, it's posted publicly. Then, for a time, anyone can post their opinion on it. At some point, an admin decides whether to accept or reject the run. If that's the system people are talking about, I question how how well it would work on SDA. First of all, it would lead to tons of opinions being posted by casual viewers. These opinions would be worse than useless; they could mislead admins to accept a bad run or reject a good one. Sure, the same people who would verify in the current system might post their opinions, but they could be drowned out by the majority. Also, let me point out something important: watching a run is not the same as verifying it. When you verify, you need to keep careful track of things like ammo count, visual inconsistencies, etc. You need to ask yourself, "if someone were to cheat by doing X, what trace would that leave?" Then check the run for those traces. A good example of this is the Legend of Zelda run that was recently discovered to be cheated. You might say that that's a failure of the verification system because the run went up, but remember that the run was posted a long time ago when verification was less rigorous. Thanks to the kinds of careful scrutiny that I described, the run was eventually caught. The kind of quick once-over that a lot of people think is sufficient would never catch this kind of thing. To address a related point: some people are saying that if a run is streamed, then it must be legitimate. But what's stopping someone with a little technical know-how from creating a cheated run, then playing it on their stream? Streaming is not the cure to all ills.

As far as the runs that sit in verification forever, what can we do? We can't just put them on the site with no verification. Open verification would do nothing to solve this problem; if someone hasn't played the game, then they can't verify under either system. Maybe interested members could do a little marathon to raise money for verification. Then if Obscure Game X for System Y has been waiting for a long time without enough verifiers, people who own System Y could volunteer to verify. The money from the marathon would be used to buy them the game. It's something to think about.

Time Between Verification and Posting

Everyone wants it to be lower, but most people don't really know what takes so long in the first place. I think people believe that the admins just slap some information into a template and they're done in ten minutes. Fortunately, Breakdown went a long way toward dispelling the fog around this subject in the roundtable video. And I just want to reiterate what the admins have said before: encoding videos is not a bottleneck. You can read more in this post by DJGrenola, though I'm not sure how up-to-date that information is. Maybe SDA 2.0 will solve some of these problems. In the mean time, I think posting links to the verification copies of runs after they are accepted is an excellent idea. A "Coming Soon" section to highlight these runs is also a good idea.

tl; dr

SDA is still relevant, but we can't please everyone.
Leaderboards are a good idea, but not for us.
Posting links to verification copies of runs after they are accepted is a good idea. Those runs could be highlighted in a "Coming Soon" section.
People should use the Strategy Wiki more.
Some sort of fundraiser to help get obscure games verified could help.

[b]Leaderboards[/b]

The first topic in the roundtable video was leaderboards, so I'll start with that. Leaderboards have had great success on sites for individual games and series. See [url=http://www.the-elite.net/]The Elite[/url], [url=http://www.soniccenter.org/]The Sonic Center[/url], and so on. One of the main draws of those sites is that you can compete without having the fastest time, encouraging more people to play. They can then work their way up the ranks and start setting records. Another benefit is that times can be posted immediately, so you can always see what the fastest (claimed) time is as long as people put it up. There are drawbacks to that kind of speed running site, though. Because anyone can post a time, cheated times can be put on the ranks. This is especially evident with times that aren't near the top of the ranks and are thus less likely to be scrutinized. The Elite has had problems with this. However, at least they require proof for times worth a certain number of points (i.e. times close to the fastest time on the ranks). But some leaderboards don't. On ZeldaSpeedRuns' well-hidden [url=http://zeldaspeedruns.com/leaderboards/oot/any]leaderboards[/url] for Ocarina of Time, the fastest time listed (as I write this) doesn't have a video. Leaderboards are susceptible to other problems too: some videos look so bad that they are almost unwatchable; some videos have no game audio (this seems very popular on Twitch); and videos are not verified.

None of these things are the direction SDA should go in. As I said above, leaderboards serve useful purposes. Currently, it's very difficult to answer the question "what is the fastest run?" for some games. It would be great to have a place where people can go to see what the fastest claims and the fastest videos are. But that place shouldn't be SDA.

[b]The Role of SDA[/b]

So what should SDA be about? Speaking personally, when I look up a run on SDA, I want to see a finished product. That includes several things: a full run; a video that doesn't look like a dog ate it; intact game audio; the runner's comments; and oftentimes, audio commentary relevant to the run. Streaming doesn't fulfill those purposes. When you go to a stream on Twitch, you're likely to see failed attempts, irrelevant commentary, advertisements interrupting the video, clutter on the screen, a bad framerate, and so on. I'm not trying to knock streaming; I know there are a lot of people who like watching the process of speed running, talking in the chat, etc. This is the kind of thing some people want. It's not the same as what SDA offers, and not everyone wants both. That means that there are a lot of people who won't come to SDA no matter what we do. We shouldn't bend over backwards trying to appease them when they will never come here anyway.

Another role of SDA is as a source of information. If you want to learn about a game, you can watch the runs on it, read the comments, and read the forum threads. If there are obsoleted runs, you can find them on archive.org (side note: I agree that they should be on SDA itself, and I'm glad to hear that UraniumAnchor has done some work on it). If SDA didn't exist, none of that information would be preserved. If you wanted to learn about a game that wasn't fashionable to stream at the time, you would be out of luck. And even if the game were popular, it would be a lot harder to find information without SDA. Maybe there's a chat room for it somewhere, and maybe there's someone there who can answer your questions, and maybe they will take the time to do so. But it's far better to have a place where the information is kept for good, available to everyone, and carefully written, not just glossed over as briefly as possible in a chat room. In Nate's [url=http://m2k2.taigaforum.com/post/history_of_metroid_2002_part_1_was_happy_birthday_m2k2.html[/url]history topics[/url], he says that that's one of the reasons he created Metroid2002. It's also why the Strategy Wiki should be used more than it is now.

There's also the fact that the forum is a great place to share knowledge about a game. People are really hurting the development of speed runs when they only share their ideas in chat rooms, instant messaging, or what have you. Anyone who isn't there when the discussion happens misses out. Even if the chat is logged, it's likely to be 99% garbage that no one wants to pick through on the off chance that there's something useful there. On a forum, discussions are visible to everyone, whether they were present at the time or not. The signal-to-noise ratio also tends to be higher, so it's easier for people to find information and get up to speed.

[b]Misinformation about SDA[/b]

There's a lot of misinformation about SDA among people new to speed running. Unfortunately, even long-time runners have contributed to this phenomenon by spreading information that's downright false. For example, there's a myth that submitting to SDA is a very difficult and arduous process. The truth is that, while it's slightly harder than uploading to YouTube because you have to talk to a human being, it's still about as simple as possible. You just send some basic information about your run, then follow a few instructions to send the run itself. You don't even need to write real comments if you don't want to; see [url=http://speeddemosarchive.com/NorthAndSouth.html#level2]North & South[/url], [url=http://speeddemosarchive.com/SuperMetroid.html#14]Super Metroid[/url], and [url=http://speeddemosarchive.com/MegaMan2.html#difficultzipglitches]several[/url] [url=http://speeddemosarchive.com/MegaMan3.html#SS]Mega Man[/url] [url=http://speeddemosarchive.com/MegaMan6.html]runs[/url]. Of course, detailed comments are better, but they're not required. Also, if you do want to submit detailed comments, you can initiate the submission process before you finish them.

There are also misconceptions about our play quality standards. Here's part of a recent post from The Elite:
[quote]For instance, if you were to hit the glass strat in a 00A run, then all future 00A runs would have to hit that, because that strat is fastest.  And even if someone beat that 00A time, if they didn't use the glass strat, SDA would not want the run...  That's just how SDA operates.[/quote]
This is so blatantly false that I won't even bother debunking it. That's not the point. The point is that wrong information like this is spread around because people believe whatever they hear. In many cases, the people giving or receiving the information have some irrational hatred of SDA, so they are especially likely to believe anything negative about it. As with the people who only want to watch streams, we shouldn't bend over backwards to appease these people. They've already made up their minds about SDA. Instead, let's try to prevent new people who are currently neutral about SDA from being poisoned by false information.

[b]Open Verification[/b]

I'm still not sure exactly what this entails. The impression I got is as follows: As soon as the verification copy of a run is available, it's posted publicly. Then, for a time, anyone can post their opinion on it. At some point, an admin decides whether to accept or reject the run. If that's the system people are talking about, I question how how well it would work on SDA. First of all, it would lead to tons of opinions being posted by casual viewers. These opinions would be worse than useless; they could mislead admins to accept a bad run or reject a good one. Sure, the same people who would verify in the current system might post their opinions, but they could be drowned out by the majority. Also, let me point out something important: watching a run is not the same as verifying it. When you verify, you need to keep careful track of things like ammo count, visual inconsistencies, etc. You need to ask yourself, "if someone were to cheat by doing X, what trace would that leave?" Then check the run for those traces. A good example of this is the Legend of Zelda run that was recently discovered to be cheated. You might say that that's a failure of the verification system because the run went up, but remember that the run was posted a long time ago when verification was less rigorous. Thanks to the kinds of careful scrutiny that I described, the run was eventually caught. The kind of quick once-over that a lot of people think is sufficient would never catch this kind of thing. To address a related point: some people are saying that if a run is streamed, then it must be legitimate. But what's stopping someone with a little technical know-how from creating a cheated run, then playing it on their stream? Streaming is not the cure to all ills.

As far as the runs that sit in verification forever, what can we do? We can't just put them on the site with no verification. Open verification would do nothing to solve this problem; if someone hasn't played the game, then they can't verify under either system. Maybe interested members could do a little marathon to raise money for verification. Then if Obscure Game X for System Y has been waiting for a long time without enough verifiers, people who own System Y could volunteer to verify. The money from the marathon would be used to buy them the game. It's something to think about.

[b]Time Between Verification and Posting[/b]

Everyone wants it to be lower, but most people don't really know what takes so long in the first place. I think people believe that the admins just slap some information into a template and they're done in ten minutes. Fortunately, Breakdown went a long way toward dispelling the fog around this subject in the roundtable video. And I just want to reiterate what the admins have said before: encoding videos is not a bottleneck. You can read more in [url=http://forum.speeddemosarchive.com/post/djgrenola_appreciation_thread15.html#djgrenola_appreciation_thread15]this post[/url] by DJGrenola, though I'm not sure how up-to-date that information is. Maybe SDA 2.0 will solve some of these problems. In the mean time, I think posting links to the verification copies of runs after they are accepted is an excellent idea. A "Coming Soon" section to highlight these runs is also a good idea.

[b]tl; dr[/b]
[ul][li]SDA is still relevant, but we can't please everyone.[/li]
[li]Leaderboards are a good idea, but not for us.[/li]
[li]Posting links to verification copies of runs after they are accepted is a good idea. Those runs could be highlighted in a "Coming Soon" section.[/li]
[li]People should use the Strategy Wiki more.[/li]
[li]Some sort of fundraiser to help get obscure games verified could help.[/li][/ul]

Worn_Traveler

2012-11-10 03:26:46 pm

Formerly known as Skullboy

Quick thing about leaderboards: If they are used, whose timing rules would we be going by? SRL or SDA? I don't think just sticking up times from one "group" into another "group" is going to be fair without things like the same % definitions and timing rules being used. SDA has someone who times the accepted runs (thanks IsraeliD). Do the people on SRL just use W-Split and stick with that? Are there multiple timing choices like at TASVideos? Without the same set of rules that apply to both groups, the field is not even and there won't be a fair comparison between an SDA verified run and a leaderboard time, legit or not.

I think that the misnomer of the arduous submission process may come from those who are less tech savvy (which inlcudes me). One either needs splitters and DVD recorders or capture cards, encoding abilities (thankfully we have Anri-Chan), the knowledge how to make a torrent, etc. Not everyone has the resources and items just hanging around and without the right technical knowledge, that can make the submission process seem arduous.

Quick thing about leaderboards: If they are used, whose timing rules would we be going by? SRL or SDA? I don't think just sticking up times from one "group" into another "group" is going to be fair without things like the same % definitions and timing rules being used. SDA has someone who times the accepted runs (thanks IsraeliD). Do the people on SRL just use W-Split and stick with that? Are there multiple timing choices like at TASVideos?  Without the same set of rules that apply to both groups, the field is not even and there won't be a fair comparison between an SDA verified run and a leaderboard time, legit or not.

I think that the misnomer of the arduous submission process may come from those who are less tech savvy (which inlcudes me). One either needs splitters and DVD recorders or capture cards, encoding abilities (thankfully we have Anri-Chan), the knowledge how to make a torrent, etc. Not everyone has the resources and items just hanging around and without the right technical knowledge, that can make the submission process seem arduous.

CMiller

2012-11-10 03:42:55 pm

Quote:

3) No easy way to download the video.

Can't speak for the first 2, but this one is pretty easy.

I use http://keepvid.com to grab youtube videos. There's ads and stuff, but adblock gets rid of them all. You can download any of the versions, including HD.

Aftermath

2012-11-10 03:56:35 pm

Quits halfway

I don't want any solution that lowers the video quality on SDA. Most streams don't even have the right resolution captured, in addition to low bitrates, framedrops, overlays, and such.

Echo what manocheese said on Open Verification even if it's not ideal now, making sure that SDA doesn't turn into TSSB's records page is not a bad thing. I'm a big fan of SDA being the final place for runs, there are other places, or even thr forum, for WIP runs, which serve a different purpose entirely. I don't think there needs to be any sort of merging.

SRL or Leaderboards are great when your game has lots of competition, but many games just have one runner for the entireity of the speedrunning process.

CosmoWright

2012-11-10 04:20:13 pm

Quote from VorpalEdge:

Leaderboards have never been something I've felt are really important. At SDA's inception, the prevailing attitude towards TG's leaderboards was basically lol, look at all these bad times.

I don't see what's wrong with bad times. I feel like the more people who can get involved the better, and that includes people who are slow or still learning.

Quote:

Question for Cosmo: the idea of a leaderboard honestly seems like a reversal from your previous stance. I was thinking about it, wondering if you were planning to integrate SRL into the site (i.e. finish a race, times update on leaderboard automatically if it's a pb), and then I remembered that hey, racebot used to have an RTA function but that got removed. What's changed?

don't know what you mean by reversal, but the RTA function was buggy, and it was sort of ridiculous to expect people to use an IRC bot for every attempt. also the timing of when people hit .done isn't entirely accurate, not enough to be auto-submitted I think.

as for what Manocheese was saying about having no video on a leaderboard, I've had a bunch of discussion with this and I'm sort of thinking that video proof would be good — most of the time a missing video is only because the runner was too lazy to highlight or upload their run, and that laziness should probably be discouraged.

Re: Timing
obviously it would be unfair to compare two runs that use a different timing method. a leaderboard would have to time all the runs on that board consistently. SDA timing seems to be the standard for a lot of games, although some communities have been branching out into their own timing methods (reset on super mario 64, file select in a lot of games, etc) and that would probably be the standard for those on a leaderboard because that's how those runs are already timed in the first place. I suppose "SRL Timing" tend to be the last input required to start the file -> loss of character control, which is nice because you don't generally need to go back and manually re-time, but of course this timing method isn't good across the board. mario 1 is a good example where SDA timing definitely the best due to the 21 frame rule (that said, there should probably be milliseconds for a run like Mario 1)

[quote="VorpalEdge"]Leaderboards have never been something I've felt are really important.  At SDA's inception, the prevailing attitude towards TG's leaderboards was basically lol, look at all these bad times.[/quote]

I don't see what's wrong with bad times. I feel like the more people who can get involved the better, and that includes people who are slow or still learning.

[quote]Question for Cosmo: the idea of a leaderboard honestly seems like a reversal from your previous stance.  I was thinking about it, wondering if you were planning to integrate SRL into the site (i.e. finish a race, times update on leaderboard automatically if it's a pb), and then I remembered that hey, racebot used to have an RTA function but that got removed.  What's changed?[/quote]

don't know what you mean by reversal, but the RTA function was buggy, and it was sort of ridiculous to expect people to use an IRC bot for every attempt. also the timing of when people hit .done isn't entirely accurate, not enough to be auto-submitted I think.

as for what Manocheese was saying about having no video on a leaderboard, I've had a bunch of discussion with this and I'm sort of thinking that video proof would be good — most of the time a missing video is only because the runner was too lazy to highlight or upload their run, and that laziness should probably be discouraged.

Re: Timing
obviously it would be unfair to compare two runs that use a different timing method. a leaderboard would have to time all the runs on that board consistently. SDA timing seems to be the standard for a lot of games, although some communities have been branching out into their own timing methods (reset on super mario 64, file select in a lot of games, etc) and that would probably be the standard for those on a leaderboard because that's how those runs are already timed in the first place. I suppose "SRL Timing" tend to be the last input required to start the file -> loss of character control, which is nice because you don't generally need to go back and manually re-time, but of course this timing method isn't good across the board. mario 1 is a good example where SDA timing definitely the best due to the 21 frame rule (that said, there should probably be milliseconds for a run like Mario 1)

AdamAK

2012-11-10 05:37:42 pm

Totally rad

It's quite often mentioned that the verification process is rigorous and that open verification might, to some extent, put an end to that. Is it really that rigorous and high-quality? Half the time I take a peek in a verification thread, the comments end up being "Great run. A/V is fine. Accept". Now, it's quite possible that runs are indeed great and that there is nothing to mention at all, but it sometimes seems a bit excessive, suggesting that we shouldn't even bother for some games or runners.