Big wave of public verifications coming

2015-02-20 04:51:34 pm

People have been submitting more and more esoteric games to SDA, which is awesome. We're extremely proud to be able to grow our game list into directions it hasn't gone before. Doing so requires the public verification process, which so far has worked well but has largely seen only a few participants (many of whom participate in nearly every public verification).

We're extremely grateful for the efforts of our public verifiers to date, but we need more! Right now there are 40+ runs ready to enter public verification. Until now, we've been gradually releasing these runs to public verification to ensure we don't have more than ten or so active at a time. Unfortunately, the rate of runs becoming ready for public verification is now exceeding that rate of consumption, and we have to ramp it up.

Now we're asking for your help. If you would be willing to participate in public verification, it will greatly help us. All you have to do is watch a speedrun and determine if it meets SDA standards for play quality, legitimacy, and AV quality. Even if it's a game you haven't run or played before, you can participate in verifying a run. (Though, of course, if you have run or played the game, that's a bonus!) You can verify as many or as few of the available runs as you like.

If you're willing to help, please just go to the "Public Verification" section of this forum and check out any of the runs that interests you. Your assistance here will be greatly appreciated.

Thread title:

People have been submitting more and more esoteric games to SDA, which is awesome. We're extremely proud to be able to grow our game list into directions it hasn't gone before. Doing so requires the public verification process, which so far has worked well but has largely seen only a few participants (many of whom participate in nearly every public verification).

We're extremely grateful for the efforts of our public verifiers to date, but we need more! Right now there are [b]40+ runs ready to enter public verification[/b]. Until now, we've been gradually releasing these runs to public verification to ensure we don't have more than ten or so active at a time. Unfortunately, the rate of runs becoming ready for public verification is now exceeding that rate of consumption, and we have to ramp it up.

Now we're asking for your help. If you would be willing to participate in public verification, it will greatly help us. [b]All you have to do is watch a speedrun and determine if it meets SDA standards for play quality, legitimacy, and AV quality.[/b] Even if it's a game you haven't run or played before, you can participate in verifying a run. (Though, of course, if you [i]have[/i] run or played the game, that's a bonus!) You can verify as many or as few of the available runs as you like.

If you're willing to help, please just go to the "Public Verification" section of this forum and check out any of the runs that interests you. Your assistance here will be greatly appreciated.

LotBlind

Edit history:

LotBlind: 2015-02-21 10:35:51 am

2015-02-21 10:33:05 am

Do you think it should be made into an unofficial rule that you can with good conscience wait for a week until you do anything (unless you actually know one of the games) just to make sure the efforts get placed in the titles that actually need it the most? That's what I've been doing.

In any case I want to use this opportunity to tell people it's actually been such fun doing these verifications: I've gotten to see so many games I otherwise would never have and there's something meaningful about it when you're keeping an eye out for big and small mistakes... I feel like a talent show judge: everyone's just there to entertain me, everyone's just there to try to win me over and avoid getting the buzzer. They're at your mercy, those hopeless crawling little...

AHEM! Sorry got carried away. Anyway it's rewarding to be able to find improvements for games you've never played Cheesy

EDIT: Another thing I do is when I see a game I know I'm going to verify, I post in the thread saying "will do this one, will take 2 weeks" or whatever I need to say which further reduces the redundancy and Vula will know when it's happening.

Do you think it should be made into an unofficial rule that you can with good conscience wait for a week until you do anything (unless you actually know one of the games) just to make sure the efforts get placed in the titles that actually need it the most? That's what I've been doing.

In any case I want to use this opportunity to tell people it's actually been such fun doing these verifications: I've gotten to see so many games I otherwise would never have and there's something meaningful about it when you're keeping an eye out for big and small mistakes... I feel like a talent show judge: everyone's just there to entertain me, everyone's just there to try to win me over and avoid getting the buzzer. They're at your mercy, those hopeless crawling little...

AHEM! Sorry got carried away. Anyway it's rewarding to be able to find improvements for games you've never played :D

EDIT: Another thing I do is when I see a game I know I'm going to verify, I post in the thread saying "will do this one, will take 2 weeks" or whatever I need to say which further reduces the redundancy and Vula will know when it's happening.

Samtastic

2015-02-21 10:58:02 am

Oddworld Speedrun Master!

I think a 60 day wait is a bit too long for finding verifiers. Maybe it should be brought down to 2 weeks or something before the run goes public.

Svenne

2015-02-21 11:43:23 am

Smörgåsbord

I sign up for Robin Hood: Prince of Thieves!

UraniumAnchor

2015-02-21 01:32:45 pm

Not a walrus

Quote from Samlaptop:

I think a 60 day wait is a bit too long for finding verifiers. Maybe it should be brought down to 2 weeks or something before the run goes public.

It was changed to 30 days a while back, but we're open for the possibility of tweaking it down more once the pile that already needs attention isn't so huge.

Elipsis

2015-02-21 01:33:54 pm

There would be less need for public verification if more people signed up for private verification. I do what I can, but there are a lot of games I'm totally unfamiliar with.

One suggestion I have for the admins - if I sign up to verify a run in private verification, and not enough others do so - it would be helpful to have a notification when that particular run goes into public. I think there was at least one instance when I signed up for a private verification, and then entirely missed the boat when it fell through to public.

UraniumAnchor

2015-02-21 01:36:28 pm

Not a walrus

Quote from Elipsis:

One suggestion I have for the admins - if I sign up to verify a run in private verification, and not enough others do so - it would be helpful to have a notification when that particular run goes into public. I think there was at least one instance when I signed up for a private verification, and then entirely missed the boat when it fell through to public.

I do plan on making this an option at some point in the near future, yeah. Alongside notifications for when a game that you want to "watch" gets a new submission.

LotBlind

2015-02-21 02:10:34 pm

Oh and like I've been saying, there should definitely be a front page post mentioning this: who's gonna come read it when it's here? The people who were already keeping an eye on verification. I'd like to think most of them already know about public verification as well.

Eternalspirit

2015-02-21 02:46:38 pm

I think if there was a list of runs up for verification on the front page, that might help drive it a bit more. A side bar kind of thing, or something.

Drifting Skies

2015-02-21 03:16:20 pm

steak Steak STEAK!!!

Is there any indication on how long it takes before an application for private verification is approved? I've been stuck in my offer as "pending" for almost a month now, and I don't know if that means that no one has gotten around to denying me yet, or what.

UraniumAnchor

2015-02-21 03:46:16 pm

Not a walrus

Pending in that case means that the run itself is pending, not that your offer is pending. It'll say "Pending Offer" if nobody's approved you yet.

Judgy

2015-02-25 01:38:05 am

Borderlands 2 Glitch Hunter/ router.

Im up for this as always depending upon the dates I currentlyhave no internet at my new apartment and will not have any untilBut like iyhe Sad

But as I say im up for it as I likr doing it Smiley

Fed981

2015-02-25 02:20:37 am

According to me there are too many runs that go to public verification. I think the waiting delay should be increased, or other criteria changed.
As an example, my first run of Assassin's Creed II had its 6 verifiers in less than a week, whereas the other one went to public verification... That's something I don't understand.

Drifting Skies

Edit history:

Drifting Skies: 2015-02-25 08:49:28 am

2015-02-25 08:40:22 am

steak Steak STEAK!!!

Quote from Fed981:

According to me there are too many runs that go to public verification. I think the waiting delay should be increased, or other criteria changed.
As an example, my first run of Assassin's Creed II had its 6 verifiers in less than a week, whereas the other one went to public verification... That's something I don't understand.

Seems like there's the complete opposite problem to me. Either the run gets enough verifiers within 4-5 days, or it languishes in "needs verifiers" forever until it gets to go to Public. Seems like two weeks would be a better cutoff, based on what I have observed.
By the way: Why 6 verifiers, anyway? It seems like a very arbitrary number.

presjpolk

2015-02-25 08:47:36 am

HELLO!

I don't agree at all with the implication of Fed981's comment, that public verification is somehow a problem that needs to be fixed.

Before public verification runs could literally sit a year or more. That is what's unacceptable, and why it's so great that SDA continues to improve this process.

UraniumAnchor

2015-02-25 10:19:39 am

Not a walrus

Quote from Fed981:

As an example, my first run of Assassin's Creed II had its 6 verifiers in less than a week, whereas the other one went to public verification... That's something I don't understand.

This, specifically, is something I hope to solve once I implement "past verifier notifications", for lack of a better term.

Basically you'll be able to sign up to receive PMs from the queue when somebody submits a run on a particular game so that you can go sign up.

I'll probably tie in "notify if a run you signed up for ends up in public verification" at the same time.

Eternalspirit

2015-02-25 02:56:53 pm

Quote from UraniumAnchor:

Quote from Fed981:

As an example, my first run of Assassin's Creed II had its 6 verifiers in less than a week, whereas the other one went to public verification... That's something I don't understand.

This, specifically, is something I hope to solve once I implement "past verifier notifications", for lack of a better term.

Basically you'll be able to sign up to receive PMs from the queue when somebody submits a run on a particular game so that you can go sign up.

I'll probably tie in "notify if a run you signed up for ends up in public verification" at the same time.

I'd even go one further and implement a way to allow people to opt-in to getting e-mailed as opposed to (or in addition to) getting PM'd. I often get PM notifications from SDA in my junk mail folder.

UraniumAnchor

Edit history:

UraniumAnchor: 2015-02-25 07:44:25 pm

2015-02-25 07:44:03 pm

Not a walrus

If PM notifications go into your junk folder it's likely anything the queue sent would also end up there since it's the same server sending the stuff out. PMs are easier to track in general because of the way the queue interfaces with Taiga. Not to say it'd never happen but PMs would be the priority.

Alternately, you might want to look into whitelisting anything from the SDA domain. Or get a better email host.

Eternalspirit

2015-02-25 10:03:01 pm

Quote from UraniumAnchor:

If PM notifications go into your junk folder it's likely anything the queue sent would also end up there since it's the same server sending the stuff out. PMs are easier to track in general because of the way the queue interfaces with Taiga. Not to say it'd never happen but PMs would be the priority.

Alternately, you might want to look into whitelisting anything from the SDA domain. Or get a better email host.

Is Google not a good e-mail host? Serious question, I really don't know what constitutes a "good" e-mail host.

Anyways, sounds like it would run into the same issue regardless, so probably meaningless extra work on your part. My apologies. Smiley

presjpolk

2015-02-25 10:04:17 pm

HELLO!

You get what you pay for.

Eternalspirit

2015-02-25 10:59:59 pm

Quote from presjpolk:

You get what you pay for.

True story. Smiley

UraniumAnchor

2015-02-26 01:43:20 am

Not a walrus

Hmm, I don't use GMail for my SDA notifications, but this is what I found in a google search of "gmail whitelist":

http://email.about.com/od/gmailtips/qt/et_whitelist.htm

Might give that a try.

Eternalspirit

2015-02-26 08:01:29 am

Quote from UraniumAnchor:

Hmm, I don't use GMail for my SDA notifications, but this is what I found in a google search of "gmail whitelist":

http://email.about.com/od/gmailtips/qt/et_whitelist.htm

Might give that a try.

Oh, it isn't something that bothers me that much since I regularly check my spam folder anyways. Thanks for that though!

LotBlind

2015-02-26 09:51:09 am

In hotmail it's as simple as clicking the "this isn't actually spam" button and it never ends up in the junk mail anymore.

UraniumAnchor

2015-03-04 12:26:44 am

Not a walrus

Posting it here because I just pushed two major changes relating to public verification:

1) Runners now have the option of allowing public verification immediately after we get the files. This doesn't guarantee anything, but once we get the pile of pending runs more manageable this should help speed things up in theory.
2) You can offer to verify runs that are marked as eligible for public verification, assuming they haven't already gone out. The reason for disallowing this in the first place (the public verification notice that runners would get) is no longer relevant, so I went ahead and changed it.