guffaw
INTRODUCTION
I am now able to demonstrate one direction in which SDA could be taken in the future -- a dynamically-generated, database-driven front-end to the site's data and content. It is entirely my work, although inevitably it was inspired to some degree by Radix's original SDA architecture. Apparently lacking inspiration, I named it "sda2p". It incorporates some of the interface improvements that people have requested and suggested over the years, and a few improvements that people have not. While I am not sure to what degree this code will end up being used, I am demonstrating it here in the hope that productive things of some sort will happen.
A NOTE ABOUT THE DATA'S AGE
The data on which this demonstration is built is old. My records indicate that the cutoff date for the data was 18th February 2008. As a result, there will be many games and runs missing, and there will be many new runs not currently available via this system. Please do not post just to mention that an obsolete run is present, or a new run is missing. Since I performed some of my own edits on various pieces of text, and since there have obviously been edits to the main site since the sda2p data cutoff, there will also be some other differences between this and the main site. At this stage, I consider the somewhat out-of-date nature of this prototype's data to be unimportant, even advantageous: We do not want people using this as a substitute for the main site, since we need the google advertising revenue from the main site to keep SDA going. The data in the database at this point is more than sufficient for development purposes.
OTHER THINGS THAT DO NOT MATTER
Don't bother complaining about lack of original SDA colour scheme or fonts, or that the HTML doesn't validate against W3C standards. The top banner is something I threw together in 15 minutes in Inkscape when I got tired of writing code. Yes, the game pages lack pictures, and placeholders are used instead. None of these things matter at this stage.
PROJECT GOALS
There were a number of goals identified when nate and I spent some time in 2007 and 2008 thrashing out loose specifications for this software:
i) to improve the site's user interface;
ii) to automate and streamline the update process as much as possible;
iii) to remove the requirement for technically skilled staff to conduct day-to-day site operations;
iv) to incorporate mechanisms to deal with unusual conditions not currently handled well by SDA's systems.
There were also some other, secondary capabilities that were kept in mind during the design, but which have so far not been fully thought out:
v) to allow runners some degree of direct control over their own content;
vi) to keep track of obsolete runs and other old content, hopefully with the ability to roll back to this content should a current run be taken down for whatever reason.
NO UPDATE INTERFACE EXISTS YET
Although I was able to get 400 games' worth of data into the database, this was all achieved using extremely hacky tools that were written, used once, and then figuratively thrown away. Although I was able to automate some of this process, tasks such as category insertion had to be performed manually. There is, sadly, no web interface by which sda2p can currently be updated, and so its data is likely to remain largely unchanged for the time being. Writing such an interface would clearly be the next big challenge for this project; only at that stage could we consider replacing SDA with sda2p. However, the underlying database schema was very much designed to allow such an interface to be written -- it just needs to be done.
PLEASE HELP
I would welcome comments, discussion, and suggestions for improvement, particularly regarding general structure and layout. If you encounter any glaring errors not related to the age of the data (like a run missing some files), let's hear about those, too. As I mentioned, cosmetic flaws are not a concern at this stage. Most links should work, but some (particularly archive) may be broken. Please use your head when deciding what to report and what not to report.
GAME LIST
The logical place to start is the game list, although since this provides essentially the same functionality as exists now, it isn't terribly interesting. You will notice immediately that there are now two new ways of listing SDA's content, though -- the runner list, a full implementation of this popular feature request (yes, this one handles individual-level runs correctly); and the system list.
RUNNER LIST
The runner list displays and sorts runners surname first, which seemed to make sense. There are a few runners (listed at the end) who are identified in SDA's database purely by their handles. Since the decision seems to have been taken to allow runs in future from runners providing no real name, this may need to change if we start to get a lot of content from anonymous runners. While I'm on the subject of the runner list: I know that I have the two Wouter Jansens listed in the database as a single person. If someone could clarify which runs were done by which Wouter, I will get this fixed.
The runs-per-runner listing would also be capable of displaying a bio for runners that wished to write one, but while a field for this text does appear in the database, this feature is not currently implemented.
SYSTEM LIST
It is possible to filter games by released system via the game list, but this new implementation also knows on which systems runs were actually played, something of which I had always wished SDA would keep track. Even though there was nowhere at SDA to record it, I dutifully nagged Mike for this information for each and every run I posted, and recorded it at archive.org. This is how I happen to have this information for all runs from my tenure and older. Having said that, if you spot any glaring mistakes, please point them out.
There are a variety of ways in which runners, games and systems could be sorted, filtered and listed, so I would welcome discussion about which ways of manipulating these lists might be the most useful, both for experienced site users and for newcomers.
CATEGORIES AND METACATEGORIES
Trying to get the correct design for the category system was a difficult challenge. Run categories in sda2p are now distinct objects. Nearly all categories belong to one particular game, but there are a handful (100% and single-segment spring to mind) which are global and can be applied to any game. In addition, all categories have a "metacategory", which might also be referred to as a "supercategory" or a "category group". In essence, metacategories are "categories of categories". At present, the following metacategories are defined:
| Completion percentage
| Segmentation
| Game version
| Death abuse & Save Warping
| Game mode
| Path taken
| Result
| Character used
| Difficulty setting
| Glitches
| Scripts
So, "100%" and "low%" would be "Completion percentage" categories, as would "no items" in Contra; "Richter" or "Jill Valentine" would be "Character used" categories; "Good ending" and "Bad ending" would be "Result" categories. Some of these metacategories have overlap: "Path taken" and "Result" being the biggest offenders, so perhaps this list requires something of a rethink.
The rationale for grouping categories into metacategories may not immediately be obvious, but this design elegantly solves a number of problems:
i) We could add a feature to list runs by category; however, 374 categories are currently defined in sda2p's database, and presenting the user with a long list of terse, unhelpful category names like "Browny" would make very little sense out of context. Instead, users could search for runs by metacategory. There are a sensible number of metacategories, with sensible names. (This feature is not currently implemented, but that is beside the point.)
ii) Metacategories each have a priority code associated with them; that is, some metacategories are "stronger" than others. This means that sda2p can now automatically generate long multicategory filename strings like "eu_hard_100p_SS" in the correct order by itself. (Given the number of times we've screwed this up, it is a more useful feature than you might imagine.)
iii) Metacategories have a description associated with them. If there is no tooltip descriptive text provided for a category, then sda2p will fall back to displaying the description from the metacategory. Since virtually no categories in sda2p currently have a category description (I think "single-segment" is the only one that does), the category tooltips everywhere on the site currently display the description from the metacategory instead. (Of course, this default text can be replaced with a category-specific description for any category on the site). Needless to say, this is transparent to the end user.
iv) In addition to correctly assembling run filenames, the metacategory priority code also allows sda2p to automatically perform complicated game page layouts. Consider the legion of categories provided by Metroid Prime. The page layout engine is clever enough to know that the strongest category here is the game version, so it creates top-level game version bullet points for the two lists. The other categories, having lower priority, appear as part of the run description instead. Again, the metacategory priority is used to determine the order in which these categories appear on each line; it is also used to determine the order in which runs are listed on the page. Metacategory priorities are also key in determining how individual-level tables are formatted, but I'll come back to this.
GAME PAGES
I think that clicking on a time at the top of an SDA game page to take a visitor to the run comments, followed by another click on an apparently identical time to get to the run itself is confusing -- I will admit that in the very early days when I occasionally used to follow SDA, I would tend to get the runs from archive.org directly since this behaviour puzzled me so much. Clickable times are gone in sda2p, being replaced with the more cheesy but probably somewhat clearer "(go ->)" links. I'd appreciate people's thoughts on this, particularly if they can think of something better.
Another way in which sda2p's game pages have been made less confusing than SDA's is that run comments have now been moved to the run pages themselves. The game pages are now far more streamlined. They are also improved by having working links to runner and system pages.
Run completion times are all stored internally in milliseconds. They are formatted into human-readable times according to a time formatting code (i.e. a number) which may be different for each run; hence, run times are currently displayed in the old "whatever fits best" Radix fashion, but this could be changed site-wide with a few simple SQL queries to update the formatting codes.
Run dates are now displayed in textual format to avoid confusion between the various date formats used internationally, but again, this is easily changed. Via cookies, it should be possible to allow the user to pick his or her own date format.
Sda2p features full support for multiple runners working on a single run. Hexen II and Doom II demonstrate this. You can also see one form of auto-formatted IL table on the Doom II page, with the Wolfenstein secret exit category slotting itself intelligently into the table in the correct place.
Each game can have notes attached to it, which appear in a dedicated box under the game pictures; random example.
RUN PAGES
The flash player, the download links and the runner comments have now all been integrated onto a single page, which seemed to make the most sense. Now, as per nate's suggestion, you can watch a flash video and scroll through the runner comments (which appear in a scrollable div for flash-enabled runs) at the same time. The flash player allows mirror selection for the first time, and you can pick from either the primary or the secondary archive.org mirror. The flash player has its playlist mode enabled, and this allows you quickly to pick any segment of the run without leaving the page. I am not sure whether we could have this and the 2x/4x zoom currently offered by SDA at the same time, though.
The different video flavours now provide more information about themselves in those ubiquitous tooltips. Also displayed in the expandable download lists are the run length of each part, and the exact filesize in bytes. The download lists also display the segment descriptions for the few runs that have this data, such as Sparky's Metroid Prime 2: Echoes 100%. If there were any flash-enabled runs that had populated segment descriptions, these descriptions would appear in the flash player's part list too, but I don't think any such runs currently exist in sda2p's database. For some files, md5 of the file (allowing users to verify their downloads) are also displayed. This feature will probably not be used very much, but I have included it because I think it would be a good idea to record this information for all video files. It may also prove useful in diagnosing problems that visitors may have viewing downloaded runs. One note about md5: you may think that it would be trivial to fetch md5 for all SDA content by using archive.org's auto-generated md5 checksums. Unfortunately, though, I discovered a bug in archive such that if a file is replaced by a new file with the same filename, archive will fail to recompute the md5 of that file. Since we had to replace files on archive every time there was a video encoding screwup (certainly under my stewardship), archive.org's md5 cannot be relied upon, and no md5 is probably better than a possibly incorrect md5. So, only runs on which I had the opportunity to compute the md5 myself have this data at the current time.
Another useful innovation is that files that are not actually part of the run can now be attached to speed runs as "supplementary" parts. Examples of such files include ZIPs and RARs of save games and demo files; bloopers and credits segments; and image files of various types. LLCoolDave's Jedi Academy run demonstrates this feature admirably, having both a supplementary video part (bloopers/credits), and a supplementary ZIP file (game saves). Finally, these auxiliary files have an official resting place, rather than being stuffed awkwardly into the game page HTML. "Main parts" and "supplementary parts" may be somewhat goofy labels ... perhaps you can think of better ones. Also, I added all the auxiliary files I knew about, but if there are any such files missing from any runs, let me know.
BBCODE
Runner comments, like most blocks of text in sda2p (including the tooltips and game page introductions) are now stored internally not using HTML, but a BBcode variant which is converted into HTML by my own (hopefully quite fast) BBcode-parsing engine. This solves a few problems compared to the current static SDA situation. Firstly, it eliminates the site-wide issues we have at the moment with inconsistent and often W3C-invalid HTML markup. (No, sda2p's BBcode parser does not always produce W3C-valid HTML either, but that is a bug.) Secondly, it allows the look of the site to be standardised around a specific set of layout tags. Thirdly, by modifying or extending the parser, we can reformat SDA's content in various ways -- as XML or XHTML, as HTML, or as plain text. This would also be useful for creating a better-quality RSS feed. Fourthly, it eliminates the requirement for runners to send formatted content as HTML; most netizens these days must be familiar with BBcode, and a simple page can be provided to allow runners to preview their formatted comments before sending them in.
Not currently implemented, but definitely possible, is the ability to add tags to this parser that would allow simple linking to SDA objects like games, runs, categories, segments, video files, runners, and so on. If this were done intelligently, it would have various benefits such as simplifying writing news updates, providing a further means of dealing with unexpected future crises, and allowing graceful failure in the case of broken links to obsolete content.
OTHER COOL STUFF
marshmallow's 100% 6:31 Chrono Trigger run is missing its first 29 parts on SDA, since it branches off a now-obsolete previous any% run. There are various mechanisms to override the auto-generated URLs of sda2p runs, though, so my software handles this exception seamlessly.
The individual-level auto-formatting code is something of which I am quite proud. For example, sda2p's layout code is intelligent enough to lay out the IL table for Doom with the category that applies to all levels ("Ultra-violence") listed outside of the IL table, but it also realises that the "Secret exit" levels should be inlined into the table. This is made possible by some clever logic, although this logic is currently defeated by Max Payne 2: The Fall of Max Payne, where both categories should be outside the table (rather than NYM being inside). This can be fixed, although not trivially.
OTHER KNOWN ISSUES
To protect email addresses present in runner comments, the parser has been temporarily hacked to replace the '@' character with the '|' character.
There is no BitTorrent support yet. Opinions seem to differ at the moment on how this should be implemented -- as usual, individual-level runs and their categorisation are the problem.
The flash player does not provide a playlist for individual-level runs, displaying instead only the single level that was chosen by the visitor from the IL table. The reason for this is that it is not immediately obvious what to put in the playlist. Should it be all the IL levels for a particular game? What if some of them are in different categories? If the playlist is to contain multiple categories (i.e. duplicated levels), how should the categories and levels be ordered? Also, the run page for an IL level displays the IL comments from the runner who submitted that particular level, but the next level displayed by the flash player might be played by a different runner, and the wrong comments would then be displayed. It is not obvious to me how these problems should be solved, so any input would be helpful.
Some games with individual-level tables also require a ZIP file to be attached to an entire IL level set, specifically the SSBM Break the Targets ILs and Counter-Strike: Condition Zero. Again, I am not sure how this should be done. Files could be attached to IL categories, or to games, or a [vidfile=X] tag could be added to the parser as I suggested earlier (in which case a link could be placed into the game notes). Currently it is not possible to create files within sda2p that are not attached to a specific run, but that behaviour could be altered.
The system's speed could be improved quite a bit -- in particular, large (>10K) runner comments should have the BBcode-derived HTML cached in the database for future display.
TRIVIA
While early development was done on a PC, and later development on my MacBook, the last few weeks of work were hosted on a hacked Wii running Linux. The machine's modest CPU and even more modest RAM provided a good simulation of high load conditions on a web server, and in fact much of the database code was rewritten for better performance following this switch.
ABOUT THE AUTHOR
When not writing code, I may often be found knitting my own socks.
I am now able to demonstrate one direction in which SDA could be taken in the future -- a dynamically-generated, database-driven front-end to the site's data and content. It is entirely my work, although inevitably it was inspired to some degree by Radix's original SDA architecture. Apparently lacking inspiration, I named it "sda2p". It incorporates some of the interface improvements that people have requested and suggested over the years, and a few improvements that people have not. While I am not sure to what degree this code will end up being used, I am demonstrating it here in the hope that productive things of some sort will happen.
A NOTE ABOUT THE DATA'S AGE
The data on which this demonstration is built is old. My records indicate that the cutoff date for the data was 18th February 2008. As a result, there will be many games and runs missing, and there will be many new runs not currently available via this system. Please do not post just to mention that an obsolete run is present, or a new run is missing. Since I performed some of my own edits on various pieces of text, and since there have obviously been edits to the main site since the sda2p data cutoff, there will also be some other differences between this and the main site. At this stage, I consider the somewhat out-of-date nature of this prototype's data to be unimportant, even advantageous: We do not want people using this as a substitute for the main site, since we need the google advertising revenue from the main site to keep SDA going. The data in the database at this point is more than sufficient for development purposes.
OTHER THINGS THAT DO NOT MATTER
Don't bother complaining about lack of original SDA colour scheme or fonts, or that the HTML doesn't validate against W3C standards. The top banner is something I threw together in 15 minutes in Inkscape when I got tired of writing code. Yes, the game pages lack pictures, and placeholders are used instead. None of these things matter at this stage.
PROJECT GOALS
There were a number of goals identified when nate and I spent some time in 2007 and 2008 thrashing out loose specifications for this software:
i) to improve the site's user interface;
ii) to automate and streamline the update process as much as possible;
iii) to remove the requirement for technically skilled staff to conduct day-to-day site operations;
iv) to incorporate mechanisms to deal with unusual conditions not currently handled well by SDA's systems.
There were also some other, secondary capabilities that were kept in mind during the design, but which have so far not been fully thought out:
v) to allow runners some degree of direct control over their own content;
vi) to keep track of obsolete runs and other old content, hopefully with the ability to roll back to this content should a current run be taken down for whatever reason.
NO UPDATE INTERFACE EXISTS YET
Although I was able to get 400 games' worth of data into the database, this was all achieved using extremely hacky tools that were written, used once, and then figuratively thrown away. Although I was able to automate some of this process, tasks such as category insertion had to be performed manually. There is, sadly, no web interface by which sda2p can currently be updated, and so its data is likely to remain largely unchanged for the time being. Writing such an interface would clearly be the next big challenge for this project; only at that stage could we consider replacing SDA with sda2p. However, the underlying database schema was very much designed to allow such an interface to be written -- it just needs to be done.
PLEASE HELP
I would welcome comments, discussion, and suggestions for improvement, particularly regarding general structure and layout. If you encounter any glaring errors not related to the age of the data (like a run missing some files), let's hear about those, too. As I mentioned, cosmetic flaws are not a concern at this stage. Most links should work, but some (particularly archive) may be broken. Please use your head when deciding what to report and what not to report.
GAME LIST
The logical place to start is the game list, although since this provides essentially the same functionality as exists now, it isn't terribly interesting. You will notice immediately that there are now two new ways of listing SDA's content, though -- the runner list, a full implementation of this popular feature request (yes, this one handles individual-level runs correctly); and the system list.
RUNNER LIST
The runner list displays and sorts runners surname first, which seemed to make sense. There are a few runners (listed at the end) who are identified in SDA's database purely by their handles. Since the decision seems to have been taken to allow runs in future from runners providing no real name, this may need to change if we start to get a lot of content from anonymous runners. While I'm on the subject of the runner list: I know that I have the two Wouter Jansens listed in the database as a single person. If someone could clarify which runs were done by which Wouter, I will get this fixed.
The runs-per-runner listing would also be capable of displaying a bio for runners that wished to write one, but while a field for this text does appear in the database, this feature is not currently implemented.
SYSTEM LIST
It is possible to filter games by released system via the game list, but this new implementation also knows on which systems runs were actually played, something of which I had always wished SDA would keep track. Even though there was nowhere at SDA to record it, I dutifully nagged Mike for this information for each and every run I posted, and recorded it at archive.org. This is how I happen to have this information for all runs from my tenure and older. Having said that, if you spot any glaring mistakes, please point them out.
There are a variety of ways in which runners, games and systems could be sorted, filtered and listed, so I would welcome discussion about which ways of manipulating these lists might be the most useful, both for experienced site users and for newcomers.
CATEGORIES AND METACATEGORIES
Trying to get the correct design for the category system was a difficult challenge. Run categories in sda2p are now distinct objects. Nearly all categories belong to one particular game, but there are a handful (100% and single-segment spring to mind) which are global and can be applied to any game. In addition, all categories have a "metacategory", which might also be referred to as a "supercategory" or a "category group". In essence, metacategories are "categories of categories". At present, the following metacategories are defined:
| Completion percentage
| Segmentation
| Game version
| Death abuse & Save Warping
| Game mode
| Path taken
| Result
| Character used
| Difficulty setting
| Glitches
| Scripts
So, "100%" and "low%" would be "Completion percentage" categories, as would "no items" in Contra; "Richter" or "Jill Valentine" would be "Character used" categories; "Good ending" and "Bad ending" would be "Result" categories. Some of these metacategories have overlap: "Path taken" and "Result" being the biggest offenders, so perhaps this list requires something of a rethink.
The rationale for grouping categories into metacategories may not immediately be obvious, but this design elegantly solves a number of problems:
i) We could add a feature to list runs by category; however, 374 categories are currently defined in sda2p's database, and presenting the user with a long list of terse, unhelpful category names like "Browny" would make very little sense out of context. Instead, users could search for runs by metacategory. There are a sensible number of metacategories, with sensible names. (This feature is not currently implemented, but that is beside the point.)
ii) Metacategories each have a priority code associated with them; that is, some metacategories are "stronger" than others. This means that sda2p can now automatically generate long multicategory filename strings like "eu_hard_100p_SS" in the correct order by itself. (Given the number of times we've screwed this up, it is a more useful feature than you might imagine.)
iii) Metacategories have a description associated with them. If there is no tooltip descriptive text provided for a category, then sda2p will fall back to displaying the description from the metacategory. Since virtually no categories in sda2p currently have a category description (I think "single-segment" is the only one that does), the category tooltips everywhere on the site currently display the description from the metacategory instead. (Of course, this default text can be replaced with a category-specific description for any category on the site). Needless to say, this is transparent to the end user.
iv) In addition to correctly assembling run filenames, the metacategory priority code also allows sda2p to automatically perform complicated game page layouts. Consider the legion of categories provided by Metroid Prime. The page layout engine is clever enough to know that the strongest category here is the game version, so it creates top-level game version bullet points for the two lists. The other categories, having lower priority, appear as part of the run description instead. Again, the metacategory priority is used to determine the order in which these categories appear on each line; it is also used to determine the order in which runs are listed on the page. Metacategory priorities are also key in determining how individual-level tables are formatted, but I'll come back to this.
GAME PAGES
I think that clicking on a time at the top of an SDA game page to take a visitor to the run comments, followed by another click on an apparently identical time to get to the run itself is confusing -- I will admit that in the very early days when I occasionally used to follow SDA, I would tend to get the runs from archive.org directly since this behaviour puzzled me so much. Clickable times are gone in sda2p, being replaced with the more cheesy but probably somewhat clearer "(go ->)" links. I'd appreciate people's thoughts on this, particularly if they can think of something better.
Another way in which sda2p's game pages have been made less confusing than SDA's is that run comments have now been moved to the run pages themselves. The game pages are now far more streamlined. They are also improved by having working links to runner and system pages.
Run completion times are all stored internally in milliseconds. They are formatted into human-readable times according to a time formatting code (i.e. a number) which may be different for each run; hence, run times are currently displayed in the old "whatever fits best" Radix fashion, but this could be changed site-wide with a few simple SQL queries to update the formatting codes.
Run dates are now displayed in textual format to avoid confusion between the various date formats used internationally, but again, this is easily changed. Via cookies, it should be possible to allow the user to pick his or her own date format.
Sda2p features full support for multiple runners working on a single run. Hexen II and Doom II demonstrate this. You can also see one form of auto-formatted IL table on the Doom II page, with the Wolfenstein secret exit category slotting itself intelligently into the table in the correct place.
Each game can have notes attached to it, which appear in a dedicated box under the game pictures; random example.
RUN PAGES
The flash player, the download links and the runner comments have now all been integrated onto a single page, which seemed to make the most sense. Now, as per nate's suggestion, you can watch a flash video and scroll through the runner comments (which appear in a scrollable div for flash-enabled runs) at the same time. The flash player allows mirror selection for the first time, and you can pick from either the primary or the secondary archive.org mirror. The flash player has its playlist mode enabled, and this allows you quickly to pick any segment of the run without leaving the page. I am not sure whether we could have this and the 2x/4x zoom currently offered by SDA at the same time, though.
The different video flavours now provide more information about themselves in those ubiquitous tooltips. Also displayed in the expandable download lists are the run length of each part, and the exact filesize in bytes. The download lists also display the segment descriptions for the few runs that have this data, such as Sparky's Metroid Prime 2: Echoes 100%. If there were any flash-enabled runs that had populated segment descriptions, these descriptions would appear in the flash player's part list too, but I don't think any such runs currently exist in sda2p's database. For some files, md5 of the file (allowing users to verify their downloads) are also displayed. This feature will probably not be used very much, but I have included it because I think it would be a good idea to record this information for all video files. It may also prove useful in diagnosing problems that visitors may have viewing downloaded runs. One note about md5: you may think that it would be trivial to fetch md5 for all SDA content by using archive.org's auto-generated md5 checksums. Unfortunately, though, I discovered a bug in archive such that if a file is replaced by a new file with the same filename, archive will fail to recompute the md5 of that file. Since we had to replace files on archive every time there was a video encoding screwup (certainly under my stewardship), archive.org's md5 cannot be relied upon, and no md5 is probably better than a possibly incorrect md5. So, only runs on which I had the opportunity to compute the md5 myself have this data at the current time.
Another useful innovation is that files that are not actually part of the run can now be attached to speed runs as "supplementary" parts. Examples of such files include ZIPs and RARs of save games and demo files; bloopers and credits segments; and image files of various types. LLCoolDave's Jedi Academy run demonstrates this feature admirably, having both a supplementary video part (bloopers/credits), and a supplementary ZIP file (game saves). Finally, these auxiliary files have an official resting place, rather than being stuffed awkwardly into the game page HTML. "Main parts" and "supplementary parts" may be somewhat goofy labels ... perhaps you can think of better ones. Also, I added all the auxiliary files I knew about, but if there are any such files missing from any runs, let me know.
BBCODE
Runner comments, like most blocks of text in sda2p (including the tooltips and game page introductions) are now stored internally not using HTML, but a BBcode variant which is converted into HTML by my own (hopefully quite fast) BBcode-parsing engine. This solves a few problems compared to the current static SDA situation. Firstly, it eliminates the site-wide issues we have at the moment with inconsistent and often W3C-invalid HTML markup. (No, sda2p's BBcode parser does not always produce W3C-valid HTML either, but that is a bug.) Secondly, it allows the look of the site to be standardised around a specific set of layout tags. Thirdly, by modifying or extending the parser, we can reformat SDA's content in various ways -- as XML or XHTML, as HTML, or as plain text. This would also be useful for creating a better-quality RSS feed. Fourthly, it eliminates the requirement for runners to send formatted content as HTML; most netizens these days must be familiar with BBcode, and a simple page can be provided to allow runners to preview their formatted comments before sending them in.
Not currently implemented, but definitely possible, is the ability to add tags to this parser that would allow simple linking to SDA objects like games, runs, categories, segments, video files, runners, and so on. If this were done intelligently, it would have various benefits such as simplifying writing news updates, providing a further means of dealing with unexpected future crises, and allowing graceful failure in the case of broken links to obsolete content.
OTHER COOL STUFF
marshmallow's 100% 6:31 Chrono Trigger run is missing its first 29 parts on SDA, since it branches off a now-obsolete previous any% run. There are various mechanisms to override the auto-generated URLs of sda2p runs, though, so my software handles this exception seamlessly.
The individual-level auto-formatting code is something of which I am quite proud. For example, sda2p's layout code is intelligent enough to lay out the IL table for Doom with the category that applies to all levels ("Ultra-violence") listed outside of the IL table, but it also realises that the "Secret exit" levels should be inlined into the table. This is made possible by some clever logic, although this logic is currently defeated by Max Payne 2: The Fall of Max Payne, where both categories should be outside the table (rather than NYM being inside). This can be fixed, although not trivially.
OTHER KNOWN ISSUES
To protect email addresses present in runner comments, the parser has been temporarily hacked to replace the '@' character with the '|' character.
There is no BitTorrent support yet. Opinions seem to differ at the moment on how this should be implemented -- as usual, individual-level runs and their categorisation are the problem.
The flash player does not provide a playlist for individual-level runs, displaying instead only the single level that was chosen by the visitor from the IL table. The reason for this is that it is not immediately obvious what to put in the playlist. Should it be all the IL levels for a particular game? What if some of them are in different categories? If the playlist is to contain multiple categories (i.e. duplicated levels), how should the categories and levels be ordered? Also, the run page for an IL level displays the IL comments from the runner who submitted that particular level, but the next level displayed by the flash player might be played by a different runner, and the wrong comments would then be displayed. It is not obvious to me how these problems should be solved, so any input would be helpful.
Some games with individual-level tables also require a ZIP file to be attached to an entire IL level set, specifically the SSBM Break the Targets ILs and Counter-Strike: Condition Zero. Again, I am not sure how this should be done. Files could be attached to IL categories, or to games, or a [vidfile=X] tag could be added to the parser as I suggested earlier (in which case a link could be placed into the game notes). Currently it is not possible to create files within sda2p that are not attached to a specific run, but that behaviour could be altered.
The system's speed could be improved quite a bit -- in particular, large (>10K) runner comments should have the BBcode-derived HTML cached in the database for future display.
TRIVIA
While early development was done on a PC, and later development on my MacBook, the last few weeks of work were hosted on a hacked Wii running Linux. The machine's modest CPU and even more modest RAM provided a good simulation of high load conditions on a web server, and in fact much of the database code was rewritten for better performance following this switch.
ABOUT THE AUTHOR
When not writing code, I may often be found knitting my own socks.
Thread title:
