Calculating the Quality of a Speedrun

2005-12-30 05:07:42 am

I love YaBB 1G - SP1!

This is just something that I've thought about a little bit and wondered what others might think on the issue.

What I'm wondering is how one might mathematically rate the quality of a speedrun. I'm not looking for a Dead Poets' Society "calculate the perfection of the poem on the x-axis and blah blah blah" type of thing. I'm not looking to give 4:57 OoT a 9.7 in Mistakes, a 9.3 in Technical Moves, 9.4 in Enjoyability, etc.

My gut instinct was to use a ratio of some kind. Like, the ratio of the speedrun time to... something else. I'm not sure what the second measurement would be. Maybe, the average time it takes to complete the game the first time. Or maybe you'd use the time it takes once the player knows how the game was intended to be beaten.

What I was thinking about was Metroid Prime... Take the 1:04 and compare that to the ~20 hours to beat the game the first time. VERY impressive. Or you could compare it a player using the intended way to play, no tricks or sequence breaks, just a straight playthrough. Maybe 4 or 5 hours, I dunno. Those would be ratios of about 1:20 (or 5% of the first playthrough) and 1:5 (or 20% of the intended time) respectively. You could also think of Half-Life, where huge chunks of the game are skipped so you get a really low time compared to the first playthrough or the intended playthrough. The goal for a speedrun I guess is to get a lower ratio or percentage.

Kind of a weird thing to think about I suppose, but having a concrete way to measure the quality of a speedrun could help in discussion and in understanding WHY it is a quality speedrun at all.

What do you guys think? Is there any way to assign a quantitative value to the quality of a speedrun? If so, what is the best way to measure quality? Should such a measurement be pursued at all, or would the scientific mindset hamper enjoyment of speedruns by removing the "magic"? Or are speedruns inherently scientific (when it comes to planning routes and problem-solving to come up with tricks) and so a scientific measurement of speedrun quality would only make sense?

So many questions and so little time (the little time is because of college applications :'(). Discuss!

Thread title:

This is just something that I've thought about a little bit and wondered what others might think on the issue.

What I'm wondering is how one might mathematically rate the quality of a speedrun. I'm not looking for a Dead Poets' Society "calculate the perfection of the poem on the x-axis and blah blah blah" type of thing. I'm not looking to give 4:57 OoT a 9.7 in Mistakes, a 9.3 in Technical Moves, 9.4 in Enjoyability, etc.

My gut instinct was to use a ratio of some kind. Like, the ratio of the speedrun time to... something else. I'm not sure what the second measurement would be. Maybe, the average time it takes to complete the game the first time. Or maybe you'd use the time it takes once the player knows how the game was intended to be beaten.

What I was thinking about was Metroid Prime... Take the 1:04 and compare that to the ~20 hours to beat the game the first time. VERY impressive. Or you could compare it a player using the intended way to play, no tricks or sequence breaks, just a straight playthrough. Maybe 4 or 5 hours, I dunno. Those would be ratios of about 1:20 (or 5% of the first playthrough) and 1:5 (or 20% of the intended time) respectively. You could also think of Half-Life, where huge chunks of the game are skipped so you get a really low time compared to the first playthrough or the intended playthrough. The goal for a speedrun I guess is to get a lower ratio or percentage.

Kind of a weird thing to think about I suppose, but having a concrete way to measure the quality of a speedrun could help in discussion and in understanding WHY it is a quality speedrun at all.

What do you guys think? Is there any way to assign a quantitative value to the quality of a speedrun? If so, what is the best way to measure quality? Should such a measurement be pursued at all, or would the scientific mindset hamper enjoyment of speedruns by removing the "magic"? Or are speedruns inherently scientific (when it comes to planning routes and problem-solving to come up with tricks) and so a scientific measurement of speedrun quality would only make sense?

So many questions and so little time (the little time is because of college applications  :'(). Discuss!

illuminated

2005-12-30 05:10:32 am

I'd tell you. But I think you'd feel better if you found the answer yourself. Grin

Umbra

2005-12-30 05:12:55 am

I don't think it'd be necessary, since everyone’s times and the amount of shortcuts a game has. Even so, it'd take much more time and effort to find out the time it takes for a run, nevertheless all types of runs (i.e. 100%) The way I figure it, it should just be based on mistakes. Are there too many of them? If so, then the run isn't good enough. I understand what your getting at, but it'd be way to complicated to execute.

slYnki

2005-12-30 05:13:21 am

XXY

I know I'd have sex with a sub 18 Mario 64 run if I could :'(. And you have some interesting thoughts rolling here, LeCoureur. I have these kind of thoughts when thinking about the creation of skittles, or when I'm waiting for my toast to be toasted. To me personally, it'd be hard to rate a speedrun in a mathematical sense. Although, I'd give Chameleon Twist a 100%, but that's the exception.

I think I'd just stick with personal impressiveness, which is to pursue what you suggested, which is not rating them on a scale of 10. Just enjoying them for the kickass piece of material they are ;D.

illuminated

Edit history:

illuminated: 2005-12-30 05:15:55 am

2005-12-30 05:15:42 am

You'd have sex with a game? Huh?

Dragorn

2005-12-30 05:26:27 am

I thought it was just a surprise party!

I don't think that you can put a number on a piece of art. (i.e. most movie critics mainly rate films on the 4-star scale because they have to)

slYnki

2005-12-30 05:26:59 am

XXY

Quote:

You'd have sex with a game? Huh?

If I could ._.;

illuminated

2005-12-30 05:27:59 am

You shouldn't. Those pixles leave rashes.
(don't ask how I know...)

TSA

2005-12-30 05:31:32 am

Retired

The only thing I can see being worth the measurement is the % improvement over previous run, in regards to overall time of runs.

Until we know the ABSOLUTE fastest way to beat any game, it is impossible to gauge a run in regards to how good it really is overall. Once we have that info, we can do stats on mistakes, optimization, movements, etc...

But until we have that, it's pretty hard.

StrangenessDSS

Edit history:

StrangenessDSS: 2005-12-30 05:53:47 am

2005-12-30 05:42:31 am

Sleeping Terror

I have an equation for this!

N = R * fp * ne * fl * fi * fc * L

where:

N is the impressiveness of the speedrun

and

R is the number of people who have played the game
fp is the number of people who have played the game with the intention of completing it quickly
ne is the inverse of the number of segments used
fl is the number of people who have looked for new techniques to complete the game faster
fi is the number of people who have found such techniques
fc is the number of people who have actually tried making an speedrun of the game
L is the amount of time spent improving the speedrun beyond the record time

It's perfectly accurate! And it's scientific! Really!!!1!!11

jimsfriend

2005-12-30 05:48:28 am

Like A Fox

So if it took me 5 years to get good enough at a game to make the fastest speedrun, then it would be uber impressive.

StrangenessDSS

2005-12-30 05:54:02 am

Sleeping Terror

^ I changed that part a little.

bushwacker

2005-12-30 05:55:06 am

I love YaBB 1G - SP1!

Strangeness, come back when your equation has exponents and maybe some imaginary numbers. 8)

Yoshi348

Edit history:

Yoshi348: 2005-12-30 05:57:28 am

2005-12-30 05:57:11 am

dinosaur from the past

But then it wouldn't look like the Drake equation anymore...

Umbra

2005-12-30 05:58:39 am

Quote:

But then it wouldn't look like the Drake equation anymore...

Meh.

StrangenessDSS

2005-12-30 06:01:09 am

Sleeping Terror

LeCoureur: What? It's a perfectly good equation as it is! But if it makes you happier, you can add " * i^4" onto the end of it.

Yoshi: Yay, I was a bit worried it would fly over everyone's heads.

Umbra

2005-12-30 06:02:48 am

Who would have known someone would google the equation? Or did he already know about it beforehand?

spineshark

2005-12-30 06:51:31 am

新世紀進歩的羽扇子音楽

I'd guess he already knew it.

I got "3.3". Now I just need to figure out which speedrun that is...probably Marsh's Turok 2.

StrangenessDSS

2005-12-30 06:56:21 am

Sleeping Terror

I hope he knew it. It's not exactly esoteric.

P.S. Cait Sith > Red XIII

slYnki

Edit history:

slYnki: 2005-12-30 08:16:43 am

2005-12-30 06:57:10 am

XXY

That equation flew over my head ._.;

I'm pretty sure it took my pants too, but I can't be TOO certain of that. I'm getting the feeling I should have never responded in this thread @_@;

- 10,000th post Grin

Pweisger

2005-12-30 07:43:58 am

PwNzRd!

There's a reason why they don't give letter grades on works of art. The same applies here, in that it's all relative to the speed run watcher. I might give the 70-star Mario 64 run an A, while others might give it a B or less. The point I am trying to make is, it's tough as nails to pick turds out of your ass hair with toilet paper. It's like trying to get peanut butter out of shag carpet with a paper towel.

Radix

2005-12-30 07:51:49 am

I'm addicted to games

I give your post a B+

Mkt2015

2005-12-30 10:35:27 am

MGS for PS1 forever.

Being the logical person I am I will try and see if I can come up a way to figure out the quality of a speedrun.

I will use 2 genres as examples.

Platformers: roughly 10 hours gameplay

Rpgs: roughy 40 hrs (2d), roughly 60 hrs (3d)

First you would divide platformers by 3. 3 would be any platformer that is completed 100% therefore Mario 64 with 16 stars would not be included. There needn't be any differences for different difficulties since a player theoretically should be able to beat the game within the same time frame for any difficulty considering that on platformers and in speedrunning in general a person tries to avoid all fights. 3 and a half hours of gameplay should at the very least yield 5 minutes worth of mistakes. If it yields less than 5 it is good quality. More it is bad quality. If a platformer is 1 hour or less than it should yield 2 minutes of mistakes.

For 2d rpgs you would divide the time by 10 and for 3d rpgs you would divide it by 6. 2d would be 3 hrs and, like platformers, would yield at least 5 minutes worth of mistakes; I would also +/- 5 for sheer randomness. 3d rpgs would be 10 hours. 30 minutes worth of mistakes and +/- 10 for randomness.

Note: All platformers and rpgs are different therefore these figures are simply estimations.

I don't know who was the first to write this in a post and forgive me for not giving whom ever credit, but it was something like this (he didn't write all of this, but some of it):

3 hrs --> 5 minutes
1 hr --> 2-3 minutes
10 mins --> 30 secs.
1 min --> 2-3 secs.

Being the logical person I am I will try and see if I can come up a way to figure out the quality of a speedrun.

I will use 2 genres as examples.

Platformers: roughly 10 hours gameplay

Rpgs: roughy 40 hrs (2d), roughly 60 hrs (3d)

First you would divide platformers by 3. 3 would be any platformer that is completed 100% therefore Mario 64 with 16 stars would not be included. There needn't be any differences for different difficulties since a player theoretically should be able to beat the game within the same time frame for any difficulty considering that on platformers and in speedrunning in general a person tries to avoid all fights. 3 and a half hours of gameplay should at the very least yield 5 minutes worth of mistakes. If it yields less than 5 it is good quality. More it is bad quality. If a platformer is 1 hour or less than it should yield 2 minutes of mistakes.

For 2d rpgs you would divide the time by 10 and for 3d rpgs you would divide it by 6. 2d would be 3 hrs and, like platformers, would yield at least 5 minutes worth of mistakes; I would also +/- 5 for sheer randomness. 3d rpgs would be 10 hours. 30 minutes worth of mistakes and +/- 10 for randomness.

Note: All platformers and rpgs are different therefore these figures are simply estimations.

I don't know who was the first to write this in a post and forgive me for not giving whom ever credit, but it was something like this (he didn't write all of this, but some of it):

3 hrs --> 5 minutes
1 hr --> 2-3 minutes
10 mins --> 30 secs.
1 min --> 2-3 secs.

Lag.Com

2005-12-30 10:51:57 am

sda loyalist

Quote:

P.S. Cait Sith > Red XIII

Agreed.

I've thought about this myself actually, I haven't come up with a decent measurement, without bringing in the TAS time, which people seem to get rather iffy over. I reckon it would be nice to use the "TAS game with no tricks" time to base against, so that you can see where all the little bits and bobs giving you a good time come from.

Gorash

Edit history:

Gorash: 2005-12-30 12:18:14 pm

2005-12-30 12:10:16 pm

${$uid.$user}{' usertext'}

Hmm, I thought about it, and the result is:

Define the following attributes of a run, regardless of game:
- length [seconds]
- segments

- number of inputs

without directional input, like A,B,X,Y,R,L,Star,Select on SNES. A bitch to count but a good measure because:
- inputs/length [1/s] yields a measure how much control the player has to bring up in order to play the game. More input per time is more difficult than less. So lets name that measure degree of control.
- dimension of control - number of inputs the game uses

- the track&field rule. In my personal experience, games that use a lot of buttons are hard to control, and PC games with a whole keyboard filled are almost impossible to input as fast as SMB1, so lets take the number of inputs the player uses, and adjust the number of inputs by the log2, so we get the actual information the player transfers to the game:
- control - DegControl * log₂DimControl [1/s]

Keeping up control over a longer time is more difficult, so define:
- control per segment = Inputs/(Segments*length) * log₂DimControl [1/s]
A measure for better control over a longer time.

Now lets get game/run specific.
Ad hoc I can see a lot of properties to use, but most of them are highly subjective. What one can use however is:
- version of run

- assuming the first run is half decent, then every additional run seems to be decent too. I propose a 10% bonus for every run beyond the first.

Well, that's all I can think up at the moment. Feel free to smash it.

- Note that I did not include difficulty of the game. The way I see it, on speed, every game is difficult, and if it's still easy you're not going fast enough.
- I feel randomness is missing here. Something like the AoS ordeal of getting souls should deserve credit...

Hmm, I thought about it, and the result is:

Define the following attributes of a run, regardless of game:
- [i]length[/i] [seconds]
- [i]segments[/i] [#]
- [i]number of inputs[/i] [#] without directional input, like A,B,X,Y,R,L,Star,Select on SNES. A bitch to count but a good measure because:
- inputs/length [1/s] yields a measure how much control the player has to bring up in order to play the game. More input per time is more difficult than less. So lets name that measure [i]degree of control[/i]. 
- [i]dimension of control[/i] - number of inputs the game uses [#] - the track&field rule. In my personal experience, games that use a lot of buttons are hard to control, and PC games with a whole keyboard filled are almost impossible to input as fast as SMB1, so lets take the number of inputs the player uses, and adjust the number of inputs by the log2, so we get the actual information the player transfers to the game:
- [i]control[/i] - DegControl * log[sub]2[/sub]DimControl [1/s]

Keeping up control over a longer time is more difficult, so define: 
- [i]control per segment[/i] = Inputs/(Segments*length) * log[sub]2[/sub]DimControl [1/s]
  A measure for better control over a longer time.

Now lets get game/run specific.
Ad hoc I can see a lot of properties to use, but most of them are highly subjective. What one can use however is:
- version of run [#] - assuming the first run is half decent, then every additional run seems to be decent too. I propose a 10% bonus for every run beyond the first.

Well, that's all I can think up at the moment. Feel free to smash it.

- Note that I did not include difficulty of the game. The way I see it, on speed, every game is difficult, and if it's still easy you're not going fast enough.
- I feel randomness is missing here. Something like the AoS ordeal of getting souls should deserve credit...