Username:
B
I
U
S
"
url
img
#
code
sup
sub
font
size
color
smiley
embarassed
thumbsup
happy
Huh?
Angry
Roll Eyes
Undecided
Lips Sealed
Kiss
Cry
Grin
Wink
Tongue
Shocked
Cheesy
Smiley
Sad
<- 1,  2,  3,  4,  5,  6,  7 ->
--
--
List results:
Search options:
Use \ before commas in usernames
InsipidMuckyWater
InsipidMuckyWater
Edit history:
InsipidMuckyWater: 2011-04-30 08:39:29 pm
2011-04-30 08:37:35 pm
Visit my profile to see my runs!
When I try to use mkverge, I get this: "Error: The track number 2 from the file 'C:\Users\adrian\Desktop\gh\GrapplingHook_hard_Level03_01224.mp4' cannot be appended to the track number 2 from the file 'C:\Users\adrian\Desktop\gh\GrapplingHook_hard_Level02_02022.mp4'. The track parameters do not match."  I tried appending all at once and appending each separately, same thing.
InsipidMuckyWater
InsipidMuckyWater
Edit history:
InsipidMuckyWater: 2011-04-30 09:42:29 pm
InsipidMuckyWater: 2011-04-30 09:39:46 pm
InsipidMuckyWater: 2011-04-30 09:39:22 pm
2011-04-30 09:38:03 pm
Visit my profile to see my runs!
Riddle me this:  After the Grappling Hook fiasco (from the desynch from using avidemux, that is), I checked out the other run I had already appended with avidemux: Metro 2033.  It's a 2 hour, 45 minute run over like 40 segments. 

No desynch.

Jumped to the end, watched a few segments, and it was fine.  So uh ... how did THAT happen?
Exo
2011-05-01 02:33:34 am
Sandbagging
@IMW: The audiotrack for those 2 segments is encoded at different bitrates.
InsipidMuckyWater
InsipidMuckyWater
Edit history:
InsipidMuckyWater: 2011-05-01 12:29:28 pm
2011-05-01 12:27:34 pm
Visit my profile to see my runs!
Why would they be encoded at different bit rates?  (Remember, I'm an idiot, so I have no clue)  And secondly, if that is the reason why, then basically even MKVERGE isn't a solution, assuming I'm fine with mkv in the first place  ..?

Finally, is that why I CAN append Metro 2033 using avidemux but I can't append Grappling Hook in avidemux?  If the answer is yes, is there a way we can encode runs to be of the Metro 2033 type from now on instead of being like Grappling Hook?  It's amazing to me that a 20 minute run can't be appended while 2-3 hour run can.  I'd love to know what explains this.  (If it's the bitrates, why do some runs have varying bitrates and not others?)
Exo
2011-05-01 12:35:27 pm
Sandbagging
@IMW: The individual-level-runs were recorded by different runners who didn't use unified settings ergo the audio is recorded with different bitrates.
That has got nothing to do with the fact that avidemux is appending metro2033 correctly.
Metro2033 is probably being concatenated correctly because the audio and video tracks match up perfectly for whatever reason.
InsipidMuckyWater
2011-05-01 12:36:52 pm
Visit my profile to see my runs!
Okay, so maybe runs that were run and encoded by a single person are more likely to have matching tracks.  Well, it's at least something for me to go on...
Mystery
2011-05-01 12:39:27 pm
If tracks aren't encoded with the same settings, then no program in the world can append them properly without re-encoding.
If the tracks match, then mkvmerge SEEMS to be a good solution. Not perfect, but it does seem to work better than mp4box.
Poxnor
Poxnor
Edit history:
Poxnor: 2011-05-01 02:49:14 pm
2011-05-01 02:30:15 pm
Moo! Flap! Hug!
Quote from Poxnor:
The approach my program uses is as follows.  For each segment except the last, it:
- Trims the segment to the length of the shortest track; then
- Adjusts the final video sample in the video track to create a smooth edge.
The final segment is untouched (so if there's no audio commentary during the credits, the credits won't be cut).

If that approach won't work, then the program fundamentally will not work.  If it will work, however, well...then...yay Smiley

It links against mp4v2, is written in C, and works nicely in OS X / *NIX.  I'm not a Windows programmer, and I have no experience in Windows; but, if I post the code, I'm sure someone in this thread will know how to port it and post a Windows executable.

Attached.  It works perfectly with the Wario Wii run that Flip suggested.  Pathological corner cases could be designed that would break this approach, but I imagine that it should work in general.
InsipidMuckyWater
InsipidMuckyWater
Edit history:
InsipidMuckyWater: 2011-05-01 03:21:30 pm
2011-05-01 03:20:44 pm
Visit my profile to see my runs!
*anxiously awaiting the .exe of this*

By the way, for those who live and breath IMW news, I've just been re-encoding compilation runs into unified h.264 mp4 before appending them in avidemux, and that has been working (my favorite user-friendly converters are Oxelon and Super C, but if you have better suggestions, let me know).  Obviously though, I'd love to see a simpler process, but at least compilation runs are generally less common than runs performed and encoded by a single person.
Poxnor
2011-05-01 07:39:44 pm
Moo! Flap! Hug!
Just to give a quick overview of how this program actually worked on the Wario Land Wii run, here's its output:
Code:
% time mp4Join WarioLand5_12718_part*.mp4 WarioLandJoined.mp4
Creating new file WarioLandJoined.mp4

Preparing to append WarioLand5_12718_part1.mp4
    Audio track 1 duration: 0:07:53.067
    Audio track 2 duration: 0:07:49.345
    Video track 1 duration: 0:07:53.006
Plan for audio track 1:
    Will copy 22001 / 22175 samples
    Audio and video will be synchronized
Plan for audio track 2:
    Will copy 20213 / 20213 samples
    Audio and video will be synchronized
Plan for video track 1:
    Will copy 14067 / 14176 samples
    Last video sample duration: 0.033 seconds -> 0.019 seconds
Copying audio track 1/1 (100.00%)
Copying audio track 2/2 (100.00%)
Copying video track 1/1 (100.00%)
Copying chapter titles...no chapters found...done!

Preparing to append WarioLand5_12718_part2.mp4
    Audio track 1 duration: 0:15:20.043
    Audio track 2 duration: 0:15:18.047
    Video track 1 duration: 0:15:19.986
Plan for audio track 1:
    Will copy 43033 / 43127 samples
    Audio and video will be synchronized
Plan for audio track 2:
    Will copy 39537 / 39537 samples
    Audio will be 9.723 milliseconds behind video
Plan for video track 1:
    Will copy 27514 / 27572 samples
    Last video sample duration: 0.033 seconds -> 0.020 seconds
Copying audio track 1/1 (100.00%)
Copying audio track 2/2 (100.00%)
Copying video track 1/1 (100.00%)
Copying chapter titles...no chapters found...done!

Preparing to append WarioLand5_12718_part3.mp4
    Audio track 1 duration: 0:16:54.955
    Audio track 2 duration: 0:16:50.184
    Video track 1 duration: 0:16:54.881
Plan for audio track 1:
    Will copy 47352 / 47576 samples
    Audio and video will be synchronized
Plan for audio track 2:
    Will copy 43505 / 43505 samples
    Audio will be 0.290 milliseconds ahead of video
Plan for video track 1:
    Will copy 30275 / 30416 samples
    Last video sample duration: 0.033 seconds -> 0.034 seconds
Copying audio track 1/1 (100.00%)
Copying audio track 2/2 (100.00%)
Copying video track 1/1 (100.00%)
Copying chapter titles...no chapters found...done!

Preparing to append WarioLand5_12718_part4.mp4
    Audio track 1 duration: 0:15:49.056
    Audio track 2 duration: 0:15:44.843
    Video track 1 duration: 0:15:48.981
Plan for audio track 1:
    Will copy 44290 / 44487 samples
    Audio and video will be synchronized
Plan for audio track 2:
    Will copy 40691 / 40691 samples
    Audio will be 8.417 milliseconds ahead of video
Plan for video track 1:
    Will copy 28317 / 28441 samples
    Last video sample duration: 0.033 seconds -> 0.043 seconds
Copying audio track 1/1 (100.00%)
Copying audio track 2/2 (100.00%)
Copying video track 1/1 (100.00%)
Copying chapter titles...no chapters found...done!

Preparing to append WarioLand5_12718_part5.mp4
    Audio track 1 duration: 0:18:04.523
    Audio track 2 duration: 0:18:00.076
    Video track 1 duration: 0:18:04.450
Plan for audio track 1:
    Will copy 50628 / 50837 samples
    Audio and video will be synchronized
Plan for audio track 2:
    Will copy 46515 / 46515 samples
    Audio will be 1.741 milliseconds behind video
Plan for video track 1:
    Will copy 32370 / 32501 samples
    Last video sample duration: 0.033 seconds -> 0.018 seconds
Copying audio track 1/1 (100.00%)
Copying audio track 2/2 (100.00%)
Copying video track 1/1 (100.00%)
Copying chapter titles...no chapters found...done!

Preparing to append WarioLand5_12718_part6.mp4
    Audio track 1 duration: 0:15:13.685
    Audio track 2 duration: 0:15:12.057
    Video track 1 duration: 0:15:13.613
Plan for audio track 1:
    Will copy 42753 / 42829 samples
    Audio and video will be synchronized
Plan for audio track 2:
    Will copy 39279 / 39279 samples
    Audio will be 10.449 milliseconds ahead of video
Plan for video track 1:
    Will copy 27335 / 27381 samples
    Last video sample duration: 0.033 seconds -> 0.020 seconds
Copying audio track 1/1 (100.00%)
Copying audio track 2/2 (100.00%)
Copying video track 1/1 (100.00%)
Copying chapter titles...no chapters found...done!

Preparing to append WarioLand5_12718_part7.mp4
    Audio track 1 duration: 0:08:17.600
    Audio track 2 duration: 0:04:04.669
    Video track 1 duration: 0:08:17.530
Plan for audio track 1:
    Will copy 23325 / 23325 samples
    Audio and video will be synchronized
Plan for audio track 2:
    Will copy 10537 / 10537 samples
    Audio will be 3.048 milliseconds ahead of video
Plan for video track 1:
    Will copy 14911 / 14911 samples
    Last video sample duration unmodified
Copying audio track 1/1 (100.00%)
Copying audio track 2/2 (100.00%)
Copying video track 1/1 (100.00%)
Copying chapter titles...no chapters found...done!

Writing file to disk...done!
Optimizing file layout...done!

real	0m7.883s
user	0m4.788s
sys	0m1.806s

So, audio track 1 was always perfectly synchronized with the video in the final, merged video.  The worst desynchronization of the commentary track (audio track 2) in the final, merged video was during Segment 6: there, the commentary stream was desynchronized from the video by 10.449 milliseconds (less than a frame).  The entire merging process took 7.883 seconds to run on my iMac.

Cheers
--Poxnor
Mystery
2011-05-02 01:16:44 pm
Does your source depend on some external library?
I can't find

#include <mp4v2/mp4v2.h>

Also, I hope it is written in ISO C? No Linux-specific calls?
It would be nice if it was. I could then compile a Windows binary easily.
Poxnor
2011-05-02 01:19:17 pm
Yeah, it depends on the SVN version of mp4v2. See the readme.

It works on both OS X and Linux, so there's nothing Linux-specific. Whether there are Posix things in there, I can't recall.

Cheers
--Ryan
Mystery
Mystery
Edit history:
Mystery: 2011-05-02 01:27:59 pm
Mystery: 2011-05-02 01:23:01 pm
2011-05-02 01:22:29 pm
Yeah, I just realized.
But the code seems to rely heavily on non-ISO C stuff.
What's worse--it uses C99...
nate
2011-05-02 01:29:49 pm
what are you trying to compile it with? just use mingw ...
Mystery
Mystery
Edit history:
Mystery: 2011-05-04 07:35:36 am
2011-05-02 01:31:48 pm
Don't have it. Can't be bothered. I don't use C myself, so I really don't have any need for it myself.
I'll give someone else the honor of porting it.

EDIT:
Gave it another try.
It ... JUST... DOES... NOT... WORK!
Bloody Linux and its make, configure, and whatever else crap.

Have fun trying to port this. It's Linux through and through. Not portable.
Also, it uses non-standard C stuff. Shame on Poxnor. Non-standard is bad.
honorableJay
2011-05-06 06:05:56 pm
The Dork Knight himself.
I might not be a coder, but I was thinking of how to make an auto-appending system work with the website. From what I can tell, it would take Nate a whole heap of work to update the entire website to support this type of system that may or may not appeal to everybody. A good possibility would be to use a Greasemonkey script to add in the extra functions on the games pages without having to overhaul the site (along with making testing for future site updates hopefully easier). The idea of making the user do all of the grunt work to get the segments appended together is fine for the power users who frequent SDA, but for the average person who likes to watch the runs AND wants to get 1 large file it's more of an annoyance than a convenience (which is the point). I'm thinking a browser plug-in to handle the actual merging with little to no user input would be another good possibility.

Ideally the Greasemonkey script would add a few extra buttons next to the main download link with a pop-up/expand info window showing the details of the files (amount to download, available qualities, etc). Once the user chooses which quality, the plug-in then takes over and starts downloading the files in order. Once the downloads are comleted, they get merged into 1 large file. If the user doesn't want to merge the files, the Greasemonkey script can just show the normal download links.

A few things I can think of that would be a problem:
1) Apparently there are a lot of videos that are having sync issues, with the biggest culprit being audio encoded at different bit-rates/khz rates. Other videos might have a problem being merged if the resolutions/framerates don't match. This could be avoided if the file details (res, bit-rates, amount of video/audio tracks, etc) can be accessed in advance, whether by having the plug-in grab the file headers in the background after a page is loaded or if they're embedded into the webpage as comments (that the browser would ignore, but the plug-in wouldn't). If the files have conflicting settings that would cause av desyncs the user could then be warned before advancing, always having the option of simply forcing the merge anyway.

2) Hard drive space for each run would effectively be doubled on the user's end with this system. A quick fix would be to have the excess segments deleted once the merge is completed successfully, but this would be a major headache if the merge was successful with major desyncs.

3) If a file fails to download or doesn't download properly. There are a few workarounds, the first being that the plug-in keeps track of which files it wasn't able to download and alerts the user along with trying multiple times to fetch the file before completely giving up. For files that don't download properly, the plug-in could also have the ability to check the MD5 checksum of the file and compare it with a value stored on the page page (again embedded in comments that the browser ignores). If the checksums don't match up, alert the user and attempt to grab the file again (checking if it needs to resume the download or start over).

4) It's too hard to code properly. To solve this one, said coder should be locked in a room and told to write out IMW's full-name 1000 times until his fingers bleed..........that'll teach him to complain.......  Smiley

I don't expect anyone to actually take up a project like this, just throwing in my own 2 cents.
Poxnor
2011-05-06 06:38:56 pm
Moo! Flap! Hug!
Quote from Mystery:
Have fun trying to port this. It's Linux through and through. Not portable.

Given that I use OS X as my primary system, I have to tell you that you're wrong.  I wrote, tested, and used this code under OS X.  The fact that I tested it in a Linux VM was done as a courtesy, to help make sure that the code runs as broadly as possible.

Quote from Mystery:
Also, it uses non-standard C stuff. Shame on Poxnor. Non-standard is bad.

Wow, seriously?  Well, please accept my most profound apologies for taking the time to write and contribute working code.

Perhaps if you told me what "non-standard" stuff I'm doing, I could help in the porting effort.  Instead, you've lambasted me for writing code that is supposedly Linux-specific, despite the fact that it wasn't even written in Linux.
dballin
2011-05-06 08:40:37 pm
Balls jerky
Welcome to dealing with Mystery. Good luck have fun.
Poxnor
2011-05-06 08:43:15 pm
Moo! Flap! Hug!
Minor change in the code.  I fixed some truly bizarre behaviour that I was seeing when playing merged files with audio commentary in Quicktime.  Instead of playing the default track, it was playing the commentary track.  There were some flag that I had never heard of before that turned out to be the key to fixing that, heh.  New version attached (sorry, still OS X / UNIX only).
Attachment:
MP4Utils-2.0.1.tgz
Poxnor
2011-05-06 11:37:41 pm
Moo! Flap! Hug!
Quote from honorableJay:
I don't expect anyone to actually take up a project like this, just throwing in my own 2 cents.

It's an awesome idea, if anyone has to drive to write it Smiley  I know that I don't personally, but it's a really good idea that would make everyone happy.
Mystery
Mystery
Edit history:
Mystery: 2011-05-07 05:18:35 am
Mystery: 2011-05-07 05:18:19 am
Mystery: 2011-05-07 02:38:28 am
2011-05-07 02:37:20 am
Quote from Poxnor:
Given that I use OS X as my primary system, I have to tell you that you're wrong.  I wrote, tested, and used this code under OS X.  The fact that I tested it in a Linux VM was done as a courtesy, to help make sure that the code runs as broadly as possible.

Wow, seriously?  Well, please accept my most profound apologies for taking the time to write and contribute working code.

Perhaps if you told me what "non-standard" stuff I'm doing, I could help in the porting effort.  Instead, you've lambasted me for writing code that is supposedly Linux-specific, despite the fact that it wasn't even written in Linux.

Sorry, accept my sincerest apologies. I was quite frustrated.
The non-standard stuff should be obvious if you compile with -pendantic and -std:c99.
For one, variadic macros are non-standard.

Oh, and your printf format-specifier is also non-standard for printing 64-bit integers. C99 has a standard specifier. I believe it's %lli or something.

Quote from Poxnor:
Quote from honorableJay:
I don't expect anyone to actually take up a project like this, just throwing in my own 2 cents.

It's an awesome idea, if anyone has to drive to write it Smiley  I know that I don't personally, but it's a really good idea that would make everyone happy.

Although, first things first. We need a reliable way of appending segments without desync.
InsipidMuckyWater
InsipidMuckyWater
Edit history:
InsipidMuckyWater: 2011-05-07 06:21:54 am
InsipidMuckyWater: 2011-05-07 06:21:36 am
2011-05-07 06:21:01 am
Visit my profile to see my runs!
Avidemux works for everything encoded under the same settings, so that'd cover about everything that was performed by a single runner.  Compilation runs like Grappling Hook result in desynch, but those kind of runs are the exception, anyway.  So maybe the greasemonkey idea is enough to get us started if we just put off worrying about compilation runs / IL tables for another day.
honorableJay
2011-05-07 01:09:56 pm
The Dork Knight himself.
This might be helpful for passing the merging to Avidemux. It shows all of the command line options that can be passed along with examples of how to do quick container switches (shows mp4 to avi, but I'm sure others could be useful too). There's also a tutorial for scripting which could also come in handy.

I was thinking that for runs done with multiple users/settings the script would have to flag which segments are different and offer some option to have the offending parts re-encoded before merging. Since the audio is less time intensive it could be possible to script a joblist that would re-encode the audio streams, re-mux them into their respective segments, then merge once everything is done. The only real limitation I've seen with Avidemux is that it doesn't fully support multiple audio streams. You can have multiple streams, but the second track must be CBR mp3/ac3 and the output file has to be avi dual-audio. Hopefully that shouldn't be too much of a burden.
Poxnor
2011-05-07 02:44:00 pm
Moo! Flap! Hug!
Wow, how did I completely miss it in this thread that Avidemux works for videos that share the same encoding?  My apologies for not reading more closely.  I wasn't trying to duplicate effort for a solved problem.  If Avidemux works properly, then there's probably no reason for me to port my code to Windows.  If I'm wrong, and there is some reason why my approach would be wanted, let me know and I'll see if I can compile it in Cygwin to run in a MinGW environment.
Mystery
2011-05-07 02:46:34 pm
What about the "looping audio" issue that was mentioned?