UraniumAnchor, you might of course be right that there can be cases that fall into the gray area. However, if it comes the day when communities are fighting over starting to work with sda, I think that'd also be a positive thing for this site.
presjpolk, as far as I know, sda is unique in that it's the only site that hosts videos. The other communities I've seen (such as the one I believe you're handling at redcandle) use twitch links and such. I'm not sure I understand exactly what you're trying to say, but I don't suggest to host twitch videos on sda. However, if the runner recorded the run while streaming and the video meets the sda-standards, then that's what would be submitted (and skip the normal verification process).
but it does open up a can of worms when it comes to the threshold for what's popular enough for the 'fast track'.
Just throwing out ideas here (I'm sure there's another can of worms in this, but let's see...).
How about if, once someone had submitted and had accepted X runs on SDA (where X = 1? 2? 3? 5?), their future submissions could be fast-tracked (i.e., just quickly check video quality and publish). While certainly not a perfect assumption, it's probably reasonably safe to presume that if a runner has undergone the whole verification process before (and passed), they have an idea of what's expected in terms of no-cheating and run quality.
I would be wary of anything that begins to rely on past performance for individuals. In general you shouldn't have to worry about cheating for people that have prior submissions, but the standards for play quality are always evolving and as such it should not be assumed that new videos will always meet the standard. I recall a verification I participated in a while back that was performed by a longtime runner with a lot of accepted runs, but the run under scrutiny ended up being rejected because it did not meet our collective idea of a polished run.
Don't get me wrong, I'm glad this discussion is happening because nobody likes the amount of time it takes from submission to publication. It's come down a lot in the last few months (mean age for a queue item has dropped from 112 days in mid-April to 58 days as of right now) but there's still a lot left that can be done to improve that.
I recall a verification I participated in a while back that was performed by a longtime runner with a lot of accepted runs, but the run under scrutiny ended up being rejected because it did not meet our collective idea of a polished run.
Of course As I said, my suggestion wasn't perfect. But, perhaps it's worth considering if reducing publication time would be worth the occasional (and presumably rare) not-as-polished run getting through. In fact, that could lead to it being obsoleted and replaced more quickly, too
So we started talking leaderboards, I guess that gave me the answer I was looking for.
ktwo is pretty much right, atleast some I know always record on the side when they broadcast live on twitch and any given run they make can become the new "wr". And they use the same set-up for their games as when they submitted. I know we don't like the term wr here but yeah, a better time then the current SDA run. And these games have an active community on SDA and so forth.
I wished there were a faster way for someone who got the sda run and the recently made an improvement to it to get that improvement up on sda. Right now they have to go through the same process, if the previous run was accepted to sda. If the current run was not accepted yet, then you are allowed to change the old run with the new one.
right now they have to go through the same process, if the previous run was accepted to sda. If the current run was not excepted, the you are allowed to change the old run ith the new one.
I'm not sure what part you're proposing we cut out here, since if the last run was recent enough we just re-use the same verifier set.
For example, the zelda 2 run was accepted, three days after he beat that one and now have to go through the same process again instead of just having it added as the better time.
but it does open up a can of worms when it comes to the threshold for what's popular enough for the 'fast track'.
Just throwing out ideas here (I'm sure there's another can of worms in this, but let's see...).
How about if, once someone had submitted and had accepted X runs on SDA (where X = 1? 2? 3? 5?), their future submissions could be fast-tracked (i.e., just quickly check video quality and publish). While certainly not a perfect assumption, it's probably reasonably safe to presume that if a runner has undergone the whole verification process before (and passed), they have an idea of what's expected in terms of no-cheating and run quality.
TSA is the gold example against this idea.
I will say that a growing idea in some communities, whether accurate or not, that SDA isn't relevant likely is fostered because the best LttP, OoT, SM, Yoshi's Island, amongst many other games that has a large speedrunning community times are vastly inferior to the best time.
Re: Marathons
My biggest distinction I've always wanted to have between the two GDQs is the focus. AGDQ is pretty much primarily focused on getting the most money. Most popular games with the most popular runners gets the most donations. It's "corporate," if you will. This guy has six games while this guy has none? Well, the guy is more popular and even though he doesn't run game six which is the same as the guy running the same game, He'll get it because he'll get more money
I've wanted SGDQ to be about the runners wanting to give back to the community. It's not about what will get the most donations. It's that these people who don't get to go to/play at AGDQ now have an opportunity. Yeah, he'll draw a quarter of the viewers and even a tenth of the money, but he gets to be a part of the community and feel like he's contributing to charity. I don't want SGDQ to become just AGDQ guys playing lesser known games with other guys filling the rest of the time. SGDQ would be the more "intimate" of the two.
That'd be my personal restriction on SGDQ. You play at AGDQ? You get a max 1 game or 1 hour, whichever is greater, at SGDQ. That restricts, but doesn't eliminate, runners from contributing at both events.
it's probably reasonably safe to presume that if a runner has undergone the whole verification process before (and passed), they have an idea of what's expected in terms of no-cheating and run quality.
Definitely not this. Average verification time is basically 2 weeks or less (usually less for more popular games, especially shorter ones). Just because we operate on the honor system doesn't mean we need to hand out free passes. The temptation would be too great for some people, and while we would eventually catch it, it's much worse to have to pull a cheated run down from the site than it is to discover a run is cheated during verification.
following on flip's post, unfortunately i can imagine that increasing competition at sda on a given game would also increase said temptation. this is a very difficult problem, nothing less than the balance between quality and quantity. it is good to have rational/calm discussion on it while successfully holding back the terrible frustration that inevitably stems from such a situation.
i think fast-track verification might better be based on some kind of tg-esque (yeah i said it) live verification requirement ... i think this is at the core of the previous proposal. certain people are well known. there is a crowd every time they stream. their setup is well known, their past and relationships with others are well known. their new record is well known. why shouldn't it go on sda as-is?
perhaps sda could designate "observers." i don't think they should be called "referees" ...
The new run still has to be verified, though. The onus is on the verifiers to be rapid about it.
It's also on the QC'ers to be rapid as well.
Those are the two bottlenecks that you can never quite remove. It takes time to review runs, both for performance and for quality. Sometimes everyone is on their A-game and a run can fly through the queue (see: The Big Push). Other times everyone gets bogged down with work and things take longer than expected. I'd like to think we hit a mid-point on average.
Quote from nate:
perhaps sda could designate "observers."
If we went that route, I think I would still like it if at least one person reviewed the run separately from the attempt. It'd have to be someone who can have a really quick turn-around time. A day or two is still a lot faster than a week or two.
Edit: So it just hit me in the shower that I'm not thinking big enough. Have the person familiar with the game do the QC as well for this kind of fast-tracking. You have to watch the video 3+ times anyway. Two birds with one stone.
I don't think we want to make things more complicated than they are and I think we shouldn't add a popularity contest aspect to things on the site (like people seem to consider the marathons are turning into?). Things like fast-tracking and such quickly falls into being a popularity contest and favoritism which is something I don't see bring any greater value compared to the amount of problems it would be adding. Suddenly you would have to have the right friends in the community, otherwise you'd be stuck with the "slow" ordinary queue. Not really a path that we should go down.
When it comes to cutting down the time it takes for a run to get up on the site, we already either have plans or are already working on making almost all of the neccessary steps much more automated.
nate's work on yua is to make it easier for people to self-encode their runs. The more people that self-encode their runs, the less time we have to spend on doing just that which mean that the time a run stays in the queue goes down. With an easier to use encoding program, the amount of runs that are self-encoded today that become held up due to encoding problems will also become fewer. Ultimately, the goal is for enough automation to pretty much have people submit their final encodes and immediately land in the quality assurance step without the need for any manual work.
We already have done some preparation work for getting automated game pages and news post updates which cuts out a whole lot of manual html editing which will also cut down the time a run stays in the queue.
We have already implemented an easier way for people to submit (the submission form) which cuts down some of the manual steps in getting a submission added for verification.
We have already implemented automated handling for how runners send in their videos which has cut down some of the manual steps in getting a submission to progress.
We have already implemented a notification system that cuts out some of the manual steps when a runner has to be contacted about something which also cuts down the time a run stays in the queue.
We have already set in motion a system that allows runs for more obscure games to get a fixed maximum wait time (public verification).
All in all, things are looking to be pretty good in the near future and the time an average submission spends in the queue just keeps on dropping (and will continue to do so without having to complicate things further).
Runners can already affect the amount of time their submissions spends in the verification step. All it takes is to make you and your run known to the community in some way (be it active discussion threads or a frequent streamer etc.). If you have multiple people involved in your run you could pretty much be able to get verifiers the same day you submit. We've had submissions that both got enough verifiers and were verified the same day the run was submitted! If they succeeded with managing that, you should also be able to. This is one part where you could say there's a popularity aspect to it. The thing is, affecting the popularity is entirely in the runners hand. If you are a popular and very well known streamer for a certain game, you shouldn't have any difficulty getting involved verifiers and should have no trouble getting things verified in just a couple of days.
Moooh makes great points. I'll chip something else in: some communities do talk down SDA, but in my observations sometimes the things said are obsolete are even wrong. Sometimes the things that are said reflect a different set of values and priorities.
Some communities don't care about high quality, verified runs. Some do. Some communities have best runners who were totally unequipped to make SDA-caliber runs, and they've had to be coached and walked throught he process. Heck, some top runners are just shy and don't want to subject themselves to the kind of attention of a site like SDA.
But these aren't reflections of SDA. These aren't flaws of SDA. SDA is not a World Records site. SDA makes no claims to be comprehensive.
So there's no need to *over* react to criticisms of SDA. SDA's verification speed has changed so much, but this change is new. Don't overcorrect beyond the continuing process of getting the process more efficient and effective. Don't bargain with yourself.
Hate to say it, but this perspective seems a little misguided. SGDQ originally started so that there could be a marathon for Americans in the west to attend because many of them couldn't make it to AGDQ. Because it ended up being held in Essentia's house, it turned out that there had to be somewhat of a cap on who could attend, which naturally led to anyone who actually made it being able to get on the schedule as they pleased. The AGDQ organizers also put less focus on SGDQ because they were busy setting up AGDQ constantly, which is one of the main reasons why the leadership is changing. The idea of both marathons has always been to do great work for charity, and SGDQ is now being given a better chance to grow since there will be people giving a better, more direct focus to it and a bigger venue.
While having SGDQ in someone's house was cool and did make it more intimate, with it growing and allowing more people to attend, that feeling will inevitably fade away a little bit. AGDQ is never going to have the same "homey" feel as the original CGDQ did, even if everyone gets along and loves each other. SGDQ is headed the same direction as far as this aspect goes, and there's no avoiding that and we shouldn't want to avoid that. This is a necessary change if we want to do the best job that we can for any given charity.
I would like to believe that people who go to both events do it because they want to give back to the community and help charity. People are sacrificing time and money to make a difference for both. Also, regardless of where you hold a marathon there are always going to be people who can only attend that event and no others. The AGDQ and SGDQ schedules both have several unique runners, and that's always going to be the case.
I'm also a little sad that having marathons in someone's house are gone from the 'GDQs; it took a while to adjust to the marathons never being like CGDQ again, and it'll probably take a bit to adjust to SGDQ never being quite the same again, but it's for the better and more people can make great contributions this way. This isn't to say that smaller, more intimate events shouldn't exist: there is plenty of room for great community marathons and there are speedrunner meetups that happen every year now in some places. I don't think any of this really needs to be discussed too heavily unless there's something I really missed the point on here; all of us who attended the smaller events in the past are just going to need to get used to the fact that the official marathons can't regress back to that now.
romscout, while your post was very well thought-out and made some important points, it failed to address one key thing that DW wrote (essentially, it counter-argued his entire post by counter-arguing against one part of it). Here is the part I had in mind:
Quote:
It's that these people who don't get to go to/play at AGDQ now have an opportunity. Yeah, he'll draw a quarter of the viewers and even a tenth of the money, but he gets to be a part of the community and feel like he's contributing to charity. I don't want SGDQ to become just AGDQ guys playing lesser known games with other guys filling the rest of the time. SGDQ would be the more "intimate" of the two.
This is a really important point that DW made. If the marathons continue just to grow and become more "corporate," what plans (as opposed to limited examples) are there for ensuring that more members of the community will be able to contribute by playing some of the lesser-known games or as some of the non-superstar members of the community (but still members of the community)?
And, I know that there are always a few games/runners that are new that are pointed to. But, there's also a feeling that the same runner playing the same Zelda game is in every marathon (this may go back to the openness and consultation involved in game/runner selection).
Maybe I'm missing something obvious, but I don't really feel like "superstars" has ever really been a problem. If you're good at a game that we think people will want to see, then why wouldn't we let you play it? Yes, other people might trump you because they're better, but I'm not really sure if there is anything we could reasonably do about that.
There have been comments by Mike in the game threads for marathons in the past that amount to (sorry, paraphrasing): "Unless Gamer X is coming to *GDQ, we won't be including Game Y. S/he is the only person worth watching run it."
That's just one example of the "superstar" issue, and it is making people feel rejected and alienated.
I'm sorry, UA; I'm not trying to be an ass to you. But dismissing an issue that people bring up repeatedly as something you don't see isn't helpful. In fact, it's the opposite.
This dismissing of issues was a complaint made to me (I don't know why me -- maybe I'm just the loudest asshole in the group) by multiple people over private channels about the previous roundtable. Please, it's important to a lot of people that there be time taken to reply to these issues (more than, "I don't see it").
The only time I can think of that being said was andrewg in regards to SMB1. And I'm pretty sure if somebody else reasonably close to as good as andrew that Mike would let them run it.
I don't think you're being an ass, I just don't remember this ever actually causing a problem, so I wanted some examples.
Counterpoint: cfox and gombill running SM64 (no offense to either intended).
It was in reference to your counterpoint that Mike made that claim. Basically it was because anybody who is not at the top for a game as highly publicized as SM64 ends up getting a lot of flak from the stream monsters etc despite the runner's own great skills in the game. "He's not Sig" has a longer reach than it should.
Okay, I won't put words in DW's mouth (I'm sure he can expand more on his original comment than I can). So, please take these as my words, and not his (though I'd love to hear more from him). To me, this all comes back to there being no clue among the community about how games and runners are selected for marathons. It feels like whether a game/runner is accepted or rejected from marathons is based on the day of the week on which it is proposed.
(The above comment, while harsh, I feel is actually true. The mood of the person accepting or rejecting; how many games have already been accepted; whether there's a game that's already been accepted that can, ex post facto, be cut (!!) to make room for the proposed game; etc.)
SDA marathons have been an amazing success. Heck, you're moving over $500,000 per year to charity! But businesses that move a fraction of that amount need to have processes. Rules. Clarity. These things are lacking here. So, when someone gets rejected, yet some other person attending their nth-straight marathon is running yet another game, the rejected person is going to see it as favouritism, an "old boys / superstars club," a corrupt process, etc.
Mostly going to focus on marathon stuff, but about the whole verification process and everything, maybe if the run is streamed live and it's an improvement, we can just use verification as more of a a/v check than a actual verification to help expedite the run? I don't want to base it on past reputation or performance, but I think there is some merit to the whole live streaming part.
Also it looks like some people have responded to poxnor with what I was going to say, so I'll probably be repeating some points.
RE: more open: Yeah that's one of things that I would try when I start the blog. Like I said earlier, a good ETA is after SGDQ since I'll be busy helping rom prepare SGDQ.
Chrno: That's a good proposed solution for the pc patching problem. The only problem is if the patch isn't legally available anymore.
Svenne: Pretty much anything in the what list is open to debate, yeah.
Aleck47: You make some really good points in your post, and it's worth considering. I know RE4 has done pretty well in past marathons too, and discussing new games and their viability is something I'd be open to in the future.
As much as a lot of newer games are derided for having too many cutscenes and qtes, there are definitely some worth being in a marathon that either don't fit that stereotype or have good tricks regardless.
DW: If I based everything on a popularity system then Tri-Hex would have run YI at AGDQ, not Carl. I'm pretty sure Tri-Hex gets 5x the viewers Carl does. I let Carl play because he was the best player coming to the marathon, not because he was the most popular. Also, Runnerguy ran OOT 100%, not Cosmo because he's an 100% specialist, and Cosmo (AFAIK) hasn't really dabbled much in the category.
Both events are about runners giving back to the community. It's not like the only way to give back to the community is to play a game for the marathon. Tasks such as donation reading, commentary, manning the tech station, are just as big of a contribution, if not bigger than playing a game at the marathon. Also, mind you that a lot of people come just to hang out with their friends they normally don't have a chance to hang out with.
If I threw in every single most popular and money-making game, then the list would be Zelda, Mario, Metroid, and Final Fantasy, without much room for anything else. Again, this is something I'd like to blog about because game choice and popularity is something that can cover many forum posts (or blog posts if you will)
I think a restriction system like that would make people withhold games from AGDQ only to propose them at SGDQ. If anything, lesser-known games might get crowded out at SGDQ because people are withholding games from AGDQ so they can have more than one game at SGDQ.
I mostly agree with Rom about the direction of SGDQ (and that is his area now that he's head organizer), but I agree with him that I think community marathons and meetups are the best way to fill this void, because there's no way to head back for the GDQs
Poxnor:
RE: Popularity Contest: Maybe I'm missing something, but the only game where I've really done the, "unless if X comes then game Y won't be played" is Super Mario 64 and Siglemic, because Siglemic is the best at SM64, and that game has come to the point where if someone who isn't Siglemic isn't playing the game, there is nonstop bitching.
That being said, that's pretty much the only game I can think of where one runner has become so big and the "face" of a game where pretty much no one else could run it at a marathon.
I also think you're confounding two issues here.
1. Letting in lesser-known games.
2. More popular players being allowed to play a game over a less popular player.
These two issues are generally separate, though there are a couple of cases where they can clash.
I won't lie, there are games that have been in every or almost every marathon because they are super popular, and they get good viewer reactions and donations, and this will probably continue. However, has the runner always been the same? In most cases, no. Just look at who has run Super Metroid in past marathons: Vorpaledge, Dessyreqt, UraniumAnchor (who probably won't play it again in a marathon, no offense UA), and Garrison. Newcomer Zoast is going to play the game for SGDQ.
Counter point to what UA said, Kosmicd12 is running Super Mario Bros. at SGDQ because he's on the same skill level as andrewg (he also has a 4:58, AFAIK).
tl;dr: Who is chosen for a game is less about popularity and more about skill level.
Okay in response to your new post. Yes, one of the purposes of the blog is to have more clarity on why I make decisions choosing games and runners. I can see having guidelines, but not hard and fast rules because I'm sure I have broken every rule at least 5 times by now.
Yes, one of the purposes of the blog is to have more clarity on why I make decisions choosing games and runners. I can see having guidelines, but not hard and fast rules because I'm sure I have broken every rule at least 5 times by now.
No (successful) selection process has hard and fast rules. That was the wrong word on my part; sorry for that. But guidelines is a big thing. And, seeing that you're not singlehandedly making choices at arbitrary times I think would go a long way (e.g., show the involvement of others; show the discussions -- e.g., in blog format; and choose all of the games that are accepted at once).