Saving videos and re-encoding at a later date

2012-12-29 05:15:00 pm

I'm starting to use Amarec to record PC games. I use Amarec to get a lossless Lagarith/PCM AVI recording, and then I can convert that to a lossy H.264/AAC MP4 video with a much smaller filesize.

Now, when I record console games on DVD, I'm used to keeping the DVD files since they're not awfully big (4.38 GB for 2 hours of SP quality = 36.5 MB per minute). By keeping the DVD files around, I can later re-encode the video if I want to. For example, I can make an MP4 with different quality settings, or I can make a route comparison video containing clips from old DVD runs as well as newly recorded runs.

However, when recording PC games, the original uncompressed AVI is on the order of 500 MB per minute, which is too big to keep on my hard drive(s). I want to keep a smaller re-encodable video.

I'd like *one* of the following two solutions:

(1) Be able to re-encode directly from the H.264 MP4 file (with good, but not necessarily super good quality). I've read the Avisynth article on loading MPEG-4 AVC files (if I understand right, this is what H.264 MP4 is), but I can't seem to find the DGAVCIndex program anywhere, and I'm reading that FFmpegSource may have caveats with frame accuracy. I also know Avidemux can open H.264 MP4 and has scripting, but after an initial look at it, it seemed like the Avidemux scripting documentation was a bit lacking (not documenting every function, often not documenting function parameters clearly, etc.).

(2) Save a version in an intermediate format that's smaller than uncompressed AVI, and easier to re-encode from than H.264 MP4. Something that has as good of a quality to size ratio as possible. I know of XviD, since SDA used to use it, but I don't know if anything better has surfaced in the past 5 or so years.

Basically, I think I have several options and I'm not sure if any one of them stands out. I also feel like I might be missing out on some better options, as I'm having trouble finding good resources on video encoding. Can anyone give advice on what would be the best options (giving a fairly good quality to size ratio, possible to automate encoding tasks, and not too difficult to figure out)?

Thread title:

I'm starting to use Amarec to record PC games.  I use Amarec to get a lossless Lagarith/PCM AVI recording, and then I can convert that to a lossy H.264/AAC MP4 video with a much smaller filesize.

Now, when I record console games on DVD, I'm used to keeping the DVD files since they're not awfully big (4.38 GB for 2 hours of SP quality = 36.5 MB per minute).  By keeping the DVD files around, I can later re-encode the video if I want to.  For example, I can make an MP4 with different quality settings, or I can make a route comparison video containing clips from old DVD runs as well as newly recorded runs.

However, when recording PC games, the original uncompressed AVI is on the order of 500 MB per minute, which is too big to keep on my hard drive(s).  I want to keep a smaller re-encodable video.

I'd like *one* of the following two solutions:

(1) Be able to re-encode directly from the H.264 MP4 file (with good, but not necessarily super good quality).  I've read the [url=http://avisynth.org/mediawiki/FAQ_loading_clips#How_do_I_load_MP4.2FMKV.2FM2TS.2FEVO_into_AviSynth.3F]Avisynth article on loading MPEG-4 AVC files[/url] (if I understand right, this is what H.264 MP4 is), but I can't seem to find the DGAVCIndex program anywhere, and I'm reading that [url=http://avisynth.org/mediawiki/FFmpegSource]FFmpegSource[/url] may have caveats with frame accuracy.  I also know Avidemux can open H.264 MP4 and has scripting, but after an initial look at it, it seemed like [url=http://www.avidemux.org/admWiki/doku.php?id=using:scripting]the Avidemux scripting documentation[/url] was a bit lacking (not documenting every function, often not documenting function parameters clearly, etc.).

(2) Save a version in an intermediate format that's smaller than uncompressed AVI, and easier to re-encode from than H.264 MP4.  Something that has as good of a quality to size ratio as possible.  I know of XviD, since SDA used to use it, but I don't know if anything better has surfaced in the past 5 or so years.

Basically, I think I have several options and I'm not sure if any one of them stands out.  I also feel like I might be missing out on some better options, as I'm having trouble finding good resources on video encoding.  Can anyone give advice on what would be the best options (giving a fairly good quality to size ratio, possible to automate encoding tasks, and not too difficult to figure out)?

nate

2012-12-29 10:55:11 pm

not my cup of tea, but if it were, i might like quantizer 1 xvid avi. i hate using mp4 as a source for the reasons you note. i know some people use lossless h.264 though. maybe you could even look at lossless h.264 in avi. i've never used that but it could be good.

xvid vfw codec is here. i use it with vdub. example setup is pictured below.

btw, just google dgavcdec if you want to download it. it kind of sucks though. slow and buggy too. sometimes i end up dumping hd ts files using mencoder instead because it just can't handle the input and it's crazy slow decoding.

Attachment:

Screen shot 2012-12-29 at 11.53.30 PM.png

Sir VG

2012-12-29 11:01:36 pm

Fucking Weeaboo

My HD recorder basically uses lossless H.264 AVI (well, newer firmwares have switched to MP4 instead but I have purposely avoided it). The quality is amazing, the size is decent, but the fact that anri-chan and VirtualDub work with it flawlessly is the best part of it all. That's my suggestion, though my recorder is a bit different than encoding on a PC, but I'd suggest looking into it, since H.264 and MP4 are not synonymous (thank god).

yoshifan

2012-12-29 11:20:28 pm

Ah, I hadn't heard of H.264 in AVI, interesting. I think I'll look at that then, and if it doesn't work out for whatever reason, I'll use XviD (appreciate the screenshot, nate, as I wasn't sure what settings I'd use). Thanks. Smiley

Also, I found DGAVCDec pretty easily, but it was DGAVCIndex that I couldn't find. I'm most likely just going to avoid that route, though.

nate

2012-12-29 11:46:29 pm

are they not the same package?

yoshifan

2012-12-30 12:31:05 am

Oh, uh, they are. Wow, guess I didn't take the hint from that "DGAVCIndex.exe" in the package I downloaded, lol.

ballofsnow

2012-12-30 10:28:40 am

DGAVCDec is included in latest Anri, but it won't load your typical H.264 mp4 file. It's meant for m2ts files, and probably other similar formats. It says it can index H.264 in raw format, so you can give that a try.

yoshifan

Edit history:

yoshifan: 2013-01-02 04:48:07 pm

2013-01-02 04:43:41 pm

Thanks for the info, ballofsnow. I tried that and was able to index and decode a raw .264 video with DGAVCIndex and DGAVCDec, respectively. So that should work well as long as I have a raw .264 video.

So I haven't explored all my options yet, but I have the following two methods working:

(1) H.264/x264 video as raw .264, and LAME MP3 audio as .mp3.

- Video encode: in MeGUI, select "RAWAVC" for the video file format, and select the x264 options the same as the SDA HQ/IQ/etc. profile, except select the bitrate as high as needed (or use quantizer mode and set the quantizer as low as needed).
- Audio encode: set the LAME MP3 bitrate as high as needed.
- Keep the unmuxed video and audio files separately.
- Video decode: run the .264 through DGAVCIndex to get a .dga file, then load that .dga file with DGAVCDec.
- Audio decode: get NicAudio and use the function NicMPG123Source.

Code:

# For AVC source, indexed with DGAVCIndex.
LoadPlugin("C:\ ... \anrichan3.3\dgavcdec109\DGAVCDecode.dll")

# For NicAudio.
# http://nicaudio.codeplex.com/
LoadPlugin("C:\ ... \NicAudio_206\NicAudio.dll")


# Video.
# Remember: the .dga points to a .264 file.  If you move the .264 file, you
# must modify the .dga file to look at the new path.
# Otherwise, you get "AVCSource: Could not open one of the input files"
V = AVCSource("C:\ ... \6-kinoko-encode.dga")

# Audio.
A = NicMPG123Source("C:\ ... \6-kinoko-encode.mp3")

# Muxed.
AudioDub(V, A)

ConvertToYV12()

(2) XviD video + LAME MP3 audio as .avi.

- Video/audio encode: use XviD for video, LAME MP3 for audio.
- Mux them together into an .avi. No need to keep the video and audio separately.
- Video/audio decode: use AviSource and specify XviD as the FourCC.

Code:

AviSource("C:\ ... \6-kinoko-encode.avi", fourCC="XVID")

ConvertToYV12()

---

I have some test data comparing the above.
See the attached screenshots_combined.png for a screenshot comparison, where:

- The .264 was encoded with IQ bitrate (5000 kbps). This resulted in 6.62 MB video for a 1 minute clip.
- The XviD .avi was encoded with quantizer 1.0. This resulted in 55.8 MB video for the same 1 minute clip (the .avi was 57.8 MB, the .mp3 from method (1) was 2.00 MB).
- All three .mp4 re-encodes were done with HQ video bitrate (2048 kbps).

Alternatively, see the attached screenshots.zip of individual images, since it's much easier to compare tiny differences with a photo viewer that scrolls through individual images.

I find that all the images except uncompressed have a slight bit of fuzziness, but the XviD .avi has one other thing: faint vertical lines on the cave walls. Not sure why that is, but that's one thing that makes me lean in favor toward the .264 encode method.

---

As for exploring other options, here are things I haven't managed to figure out yet:

(a) What's the best codec for H.264 in AVI? Or for that matter, what's a codec that can do so? I have a feeling there are some obvious answers, but I couldn't find anything when searching around.

(b) How to decode AAC audio? I tried DirectShowSource, and it gave an error (think it was "The source filter for this file could not be loaded").

Still, I think for my purposes, one of the two working methods above will suffice. Especially since I'm looking mostly at route comparison videos, which require frame accuracy but not super high quality video. So for now I'll go with (1), which is saving .264 and .mp3.

Thanks for the info, ballofsnow.  I tried that and was able to index and decode a raw .264 video with DGAVCIndex and DGAVCDec, respectively.  So that should work well as long as I have a raw .264 video.

So I haven't explored all my options yet, but I have the following two methods working:

(1) H.264/x264 video as raw .264, and LAME MP3 audio as .mp3.

- Video encode: in MeGUI, select "RAWAVC" for the video file format, and select the x264 options the same as the SDA HQ/IQ/etc. profile, except select the bitrate as high as needed (or use quantizer mode and set the quantizer as low as needed).
- Audio encode: set the LAME MP3 bitrate as high as needed.
- Keep the unmuxed video and audio files separately.
- Video decode: run the .264 through DGAVCIndex to get a .dga file, then load that .dga file with DGAVCDec.
- Audio decode: get NicAudio and use the function NicMPG123Source.

[code]# For AVC source, indexed with DGAVCIndex.
LoadPlugin("C:\ ... \anrichan3.3\dgavcdec109\DGAVCDecode.dll")

# For NicAudio.
# http://nicaudio.codeplex.com/
LoadPlugin("C:\ ... \NicAudio_206\NicAudio.dll")

# Video.
# Remember: the .dga points to a .264 file.  If you move the .264 file, you
# must modify the .dga file to look at the new path.
# Otherwise, you get "AVCSource: Could not open one of the input files"
V = AVCSource("C:\ ... \6-kinoko-encode.dga")

# Audio.
A = NicMPG123Source("C:\ ... \6-kinoko-encode.mp3")

# Muxed.
AudioDub(V, A)

ConvertToYV12()
[/code]

(2) XviD video + LAME MP3 audio as .avi.

- Video/audio encode: use XviD for video, LAME MP3 for audio.
- Mux them together into an .avi.  No need to keep the video and audio separately.
- Video/audio decode: use AviSource and specify XviD as the FourCC.

[code]AviSource("C:\ ... \6-kinoko-encode.avi", fourCC="XVID")

ConvertToYV12()[/code]

---

I have some test data comparing the above.
See the attached [i]screenshots_combined.png[/i] for a screenshot comparison, where:

- The .264 was encoded with IQ bitrate (5000 kbps).  This resulted in 6.62 MB video for a 1 minute clip.
- The XviD .avi was encoded with quantizer 1.0.  This resulted in 55.8 MB video for the same 1 minute clip (the .avi was 57.8 MB, the .mp3 from method (1) was 2.00 MB).
- All three .mp4 re-encodes were done with HQ video bitrate (2048 kbps).

Alternatively, see the attached [i]screenshots.zip[/i] of individual images, since it's much easier to compare tiny differences with a photo viewer that scrolls through individual images.

I find that all the images except uncompressed have a slight bit of fuzziness, but the XviD .avi has one other thing: faint vertical lines on the cave walls.  Not sure why that is, but that's one thing that makes me lean in favor toward the .264 encode method.

---

As for exploring other options, here are things I haven't managed to figure out yet:

(a) What's the best codec for H.264 in AVI?  Or for that matter, what's a codec that can do so?  I have a feeling there are some obvious answers, but I couldn't find anything when searching around.

(b) How to decode AAC audio?  I tried DirectShowSource, and it gave an error (think it was "The source filter for this file could not be loaded").

Still, I think for my purposes, one of the two working methods above will suffice.  Especially since I'm looking mostly at route comparison videos, which require frame accuracy but not super high quality video.  So for now I'll go with (1), which is saving .264 and .mp3.

Attachments:

screenshots_combined.png

screenshots.zip

nate

2013-01-02 06:30:30 pm

very interesting. love it when other people find out stuff for me. wonder if you're seeing blocking in the xvid that h.264 is immune to. sounds like the bitrate is way better for h.264 too which isn't surprising ... just helps if you have hardware decoding support otherwise sloooow.

ballofsnow

Edit history:

ballofsnow: 2013-01-02 08:51:12 pm

ballofsnow: 2013-01-02 08:47:32 pm

ballofsnow: 2013-01-02 08:45:08 pm

ballofsnow: 2013-01-02 08:33:40 pm

ballofsnow: 2013-01-02 08:22:05 pm

ballofsnow: 2013-01-02 08:09:57 pm

ballofsnow: 2013-01-02 08:09:05 pm

2013-01-02 08:08:14 pm

Quote from yoshifan:

I find that all the images except uncompressed have a slight bit of fuzziness,

Are you recording in rgb? You lose color sharpness when going down to yv12. It's most noticeable in the type of video you're recording, as opposed to some 3d fps game for example. Ideally, for an intermediate file you want to try to keep the original colorspace. This is hard to do with RGB when you start messing around with filters (in avisynth) that operate in yuy2. Anri version latest uses yuy2 on anything intermediary, with yv12 on a final, final, encode. I don't know if it's possible to keep yuy2 and feed into dgavcdec. I'll give it a try, see if I come up with anything.

edit- nope, yv12 only. Anyway, I think it's fine for your current needs. But I would rethink how you want to do this when getting down to submission quality.

edit2- Ok, skip avisynth since you're not really using any filters. Feed your avi file directly into x264.exe, generate your intermediate mp4, later on feed your intermediate mp4 as input directly into x264.exe and create a final mp4. I'll have some recommended command line soon...

edit1million- Try something like this in a command window with x264.exe. Just have to figure out the easiest way to take care of the audio:

Code:

x264.exe --profile high422 --output-csp i422 --preset veryslow --qp 10 -o intermediate.mp4 input.avi

and at some point

x264.exe (blablabla whatever is appropriate for a final encode) -o final.mp4 intermediate.mp4

I highly recommend using -qp as opposed to 2pass and crf on the intermediate. 2pass is... 2 passes so it's longer to encode, and a low action video file will end up looking way better than a high action video file; you want consistency. And not crf, because crf will lower the quality on high motion, raise the quality on low motion - this is perfectly fine on the final encode, and is in fact what happens on 2pass anyway. Anri uses crf or 2pass on final encodes.

If you want to use 444 color space.. give it a try. Use a faster preset if you want on the intermediate.

So no raw, no dgavcdec, maybe not avisynth or at least very little, and you can easily watch the intermediate.

For x264 settings, just run x264.exe --fullhelp > textfile.txt. Dumping it into a text file for easier reading.

[quote=yoshifan]I find that all the images except uncompressed have a slight bit of fuzziness,[/quote]
Are you recording in rgb? You lose color sharpness when going down to yv12. It's most noticeable in the type of video you're recording, as opposed to some 3d fps game for example. Ideally, for an intermediate file you want to try to keep the original colorspace. This is hard to do with RGB when you start messing around with filters (in avisynth) that operate in yuy2. Anri version latest uses yuy2 on anything intermediary, with yv12 on a final, [i]final[/i], encode. I don't know if it's possible to keep yuy2 and feed into dgavcdec. I'll give it a try, see if I come up with anything.

edit- nope, yv12 only. Anyway, I think it's fine for your current needs. But I would rethink how you want to do this when getting down to submission quality.

edit2- Ok, skip avisynth since you're not really using any filters. Feed your avi file directly into x264.exe, generate your intermediate mp4, later on feed your intermediate mp4 as [i]input[/i] directly into x264.exe and create a final mp4. I'll have some recommended command line soon...

edit1million- Try something like this in a command window with [url=http://x264.nl/]x264.exe[/url]. Just have to figure out the easiest way to take care of the audio:

[code]x264.exe --profile high422 --output-csp i422 --preset veryslow --qp 10 -o intermediate.mp4 input.avi

and at some point

x264.exe (blablabla whatever is appropriate for a final encode) -o final.mp4 intermediate.mp4[/code]
I highly recommend using -qp as opposed to 2pass and crf on the intermediate. 2pass is... 2 passes so it's longer to encode, and a low action video file will end up looking way better than a high action video file; you want consistency. And not crf, because crf will lower the quality on high motion, raise the quality on low motion - this is perfectly fine on the final encode, and is in fact what happens on 2pass anyway. Anri uses crf or 2pass on final encodes.

If you want to use 444 color space.. give it a try. Use a faster preset if you want on the intermediate.

So no raw, no dgavcdec, maybe not avisynth or at least very little, and you can easily watch the intermediate.

For x264 settings, just run x264.exe --fullhelp > textfile.txt. Dumping it into a text file for easier reading.

ballofsnow

Edit history:

ballofsnow: 2013-01-20 10:34:19 am

ballofsnow: 2013-01-03 02:38:35 pm

ballofsnow: 2013-01-03 02:26:46 pm

ballofsnow: 2013-01-03 02:23:37 pm

ballofsnow: 2013-01-03 02:19:03 pm

ballofsnow: 2013-01-03 02:18:27 pm

2013-01-03 02:17:37 pm

Try this.

Set up a folder with necessary tools:

FFMS2.avsi --> http://code.google.com/p/ffmpegsource/downloads/detail?name=ffms-2.17.7z
ffms2.dll --> included above
ffmsindex.exe --> included above
MP4Box.exe --> Just grab from the anri folder
libgpac.dll --> Just grab from the anri folder
wavi.exe --> Just grab from the anri folder
neroAacEnc.exe --> I grabbed the latest, might not matter. Generate email here to get the download button.
x264.exe --> http://x264.nl/. I grabbed the 32bit 8bit-depth.

encode.bat -> don't use this one. See later posts.

Code:

@ECHO OFF
CLS

SET input=capture.avi
SET output=intermediatefile

REM	Generate intermediate video mp4. I suggest --qp (quantizer mode) and not 2pass nor crf modes.
REM	Change the color values if needed. i420 (yv12) on an intermediate might not be good..
x264.exe --profile high444 --output-csp i444 --preset medium --qp 10 -o "%output%_video.mp4" "%input%"

REM	Extract audio. Can try instead to feed an avs to nero directly if you want. From experience, nero
REM	seems to work better with direct access to an audio file.
wavi.exe "%input%" "%output%_audio.wav"

REM	Encode to AAC with nero.
neroAacEnc.exe -q 1 -lc -if "%output%_audio.wav" -of "%output%_audio.mp4"

REM	Mux it, MUX it up, MUX IT UP! WOOO
MP4Box.exe -tmp . -new -add "%output%_video.mp4" -add "%output%_audio.mp4" "%output%.mp4"

REM	Generate ffmpegsource avisynth files for later final encode. The avs file -should- be able to load
REM	into Anri-chan when you choose avi as source.
ECHO>"%output%.avs" loadplugin("ffms2.dll")
ECHO>>"%output%.avs" import("FFMS2.avsi")
ECHO>>"%output%.avs" FFIndex("%output%.mp4")
ECHO>>"%output%.avs" AudioDub(FFVideoSource("%output%.mp4"), FFAudioSource("%output%.mp4"))

REM	Clean up
DEL "%output%_audio.mp4" "%output%_audio.wav" "%output%_video.mp4"


ECHO.
ECHO.
PAUSE

Change whatever makes sense. It's a bit ugly with everything in one folder. Let me know if it works.

Try this.

Set up a folder with necessary tools:

FFMS2.avsi  --> [url]http://code.google.com/p/ffmpegsource/downloads/detail?name=ffms-2.17.7z[/url]
ffms2.dll  --> included above
ffmsindex.exe  --> included above
MP4Box.exe  --> Just grab from the anri folder
libgpac.dll  --> Just grab from the anri folder
wavi.exe   --> Just grab from the anri folder
neroAacEnc.exe --> I grabbed the [url=http://www.nero.com/enu/company/about-nero/nero-aac-codec.php]latest[/url], might not matter. Generate email [url=http://10minutemail.com/10MinuteMail/index.html]here[/url] to get the download button.
x264.exe --> [url]http://x264.nl/[/url]. I grabbed the 32bit 8bit-depth.

encode.bat  -> don't use this one. See later posts.
[code]@ECHO OFF
CLS

SET input=capture.avi
SET output=intermediatefile

REM	Generate intermediate video mp4. I suggest --qp (quantizer mode) and not 2pass nor crf modes.
REM	Change the color values if needed. i420 (yv12) on an intermediate might not be good..
x264.exe --profile high444 --output-csp i444 --preset medium --qp 10 -o "%output%_video.mp4" "%input%"

REM	Extract audio. Can try instead to feed an avs to nero directly if you want. From experience, nero
REM	seems to work better with direct access to an audio file.
wavi.exe "%input%" "%output%_audio.wav"

REM	Encode to AAC with nero.
neroAacEnc.exe -q 1 -lc -if "%output%_audio.wav" -of "%output%_audio.mp4"

REM	Mux it, MUX it up, MUX IT UP! WOOO
MP4Box.exe -tmp . -new -add "%output%_video.mp4" -add "%output%_audio.mp4" "%output%.mp4"

REM	Generate ffmpegsource avisynth files for later final encode. The avs file -should- be able to load
REM	into Anri-chan when you choose avi as source.
ECHO>"%output%.avs" loadplugin("ffms2.dll")
ECHO>>"%output%.avs" import("FFMS2.avsi")
ECHO>>"%output%.avs" FFIndex("%output%.mp4")
ECHO>>"%output%.avs" AudioDub(FFVideoSource("%output%.mp4"), FFAudioSource("%output%.mp4"))

REM	Clean up
DEL "%output%_audio.mp4" "%output%_audio.wav" "%output%_video.mp4"

ECHO.
ECHO.
PAUSE[/code]

Change whatever makes sense. It's a bit ugly with everything in one folder. Let me know if it works.

yoshifan

2013-01-05 03:04:03 am

Wow, thanks for all the detail! I'll definitely try this (and post new screenshots) when I get the chance; a bit occupied with AGDQ stuff for now, though, so it might not be for a little while.

yoshifan

2013-01-19 07:30:31 pm

Okay, finally spent some time on this again. Sorry for the weird timing.

---

I made two changes to the batch file to make it work for me:

(1)
The following line:

Code:

wavi.exe "%input%" "%output%_audio.wav"

Caused the following error:

Code:

Error: Could not find PCM audio track in "C:\ ... \input-video.avi". (However, the file contains an audio track of another format.)

According to VLC's Tools -> Media Information, the audio format is "PCM S16 LE (araw)". Not sure what's the difference between this and the PCM that wavi is expecting.

I made Avisynth load the .avi instead, and it worked; not sure if it would cause any quality loss though:

Code:

ECHO>source.avs AviSource("%input%")
wavi.exe source.avs "%output%_audio.wav"

(2)
When I gave the .avs to Anri-chan, it said something like "Error: Probably not a supported AVI file". This didn't affect Anri's ability to encode the video; it encoded just fine, at least as long as I didn't use the trim frame ranges step.

When I said yes to the "do you want to trim off unwanted content" question, I couldn't seek through the video in VirtualDub effectively. If I advanced frames sequentially from the start, it would work, but if I tried to jump ahead to some later frame by clicking on the frame timeline, the frame's picture wouldn't show, and hitting the Play button would bring up "Avisynth read error: DirectShowSource: The video graph failed to restart after seeking. Status = 0x80004005". Also, if I just specified trim frame ranges anyway (valid ones) and started the encode, then the encode would stop very early, and the encode's console output would contain the same Avisynth read error.

So I didn't use the Anri-chan trim frames step. Instead, I added a Trim statement to the end of the .avs that is given to Anri-chan.

Code:

ECHO>>"%output%.avs" Trim(1460,2160)

Again, not sure if this causes any quality loss compared to trimming frames with Anri-chan.

---

Other notes:

a. For recording this game, I use AmaRecCo, which does screen capture. AmaRecLite (which does DirectX capture) also works, but my computer can't run it as well as AmaRecCo; I'd need to sacrifice more resolution, framerate, etc. to make AmaRecLite run as smoothly as AmaRecCo.

b. The game resolution is 640x480, but I have AmaRecCo resize it to 480x360 while recording. Again, this is to help recording performance; if I save the video at full resolution, then I have to lower the framerate to keep the recording smooth.

c. I recorded in 50 fps. At full resolution (640x480), the highest recording fps my computer could take was about 30. Full resolution looks really good, but I really didn't like how this game looked in 30 fps, so I compromised some resolution for a better framerate.

d. Yes, I'm recording in RGB (VLC's Tools -> Media Information says the uncompressed video's decoded format is "24 bits RGB"). As mentioned in my first post, I used Lagarith, which I heard was relatively gentle on the CPU compared to other codecs. I'm not really sure what options I have to make the recording's color format different, like YV12 (I'd be interested in hearing both CPU intensive and non CPU intensive options for this, in case I get a new computer someday soon).

e. I'm not thinking about SDA submission at this time. It's nice to maximize my video quality given the resources I have, though.

---

So yeah, the results. I'll just attach the videos to this post, as they're not too big.

- intermediate_encode.mp4 (23.7 MB, 1:03) is the intermediate file that I could save and re-encode from, analogous to DVD .vob files for console games.

- 6-Kinoko_test.mp4 (0.97 MB, 0:14), 6-Kinoko_test_HQ.mp4 (1.90 MB, 0:14), 6-Kinoko_test_IQ.mp4 (2.22 MB, 0:14), and 6-Kinoko_test_XQ.mp4 (2.22 MB, 0:14) are the medium, high, insane, and X-treme quality encodes from Anri-chan. The videos are trimmed to about 14 seconds, to make them reasonable enough filesizes to attach to this post. The input of the Anri-chan encode is the .avs file generated from the Windows batch file above (which in turn uses intermediate_encode.mp4 as a basis).

My observations:

I can't really notice much it the way of quality issues or oddities, besides that the vertical lines on the cave walls are kind of visible in all of the encodes attached to this post, including the intermediate_encode.mp4. Well, the medium quality final encode doesn't have the vertical lines, but that's just because medium quality is (expectedly) fuzzy all around. Not sure what to make of the vertical lines in general; it's not like they're even that noticeable other than in stillshots, anyway.

---

Please let me know if I missed any significant oddities in the final videos, as I'm not too experienced in looking for video artifacts.

Overall, though, I'm feeling all right with using this process (or something similar) to save a minimally-compressed .mp4 and use that for future encodes. Thanks, ballofsnow!

Okay, finally spent some time on this again.  Sorry for the weird timing.

---

I made two changes to the batch file to make it work for me:

(1)
The following line:
[code]wavi.exe "%input%" "%output%_audio.wav"[/code]
Caused the following error:
[code]Error: Could not find PCM audio track in "C:\ ... \input-video.avi". (However, the file contains an audio track of another format.)[/code]
According to VLC's Tools -> Media Information, the audio format is "PCM S16 LE (araw)".  Not sure what's the difference between this and the PCM that wavi is expecting.

I made Avisynth load the .avi instead, and it worked; not sure if it would cause any quality loss though:
[code]ECHO>source.avs AviSource("%input%")
wavi.exe source.avs "%output%_audio.wav"[/code]

(2)
When I gave the .avs to Anri-chan, it said something like "Error: Probably not a supported AVI file".  This didn't affect Anri's ability to encode the video; it encoded just fine, at least as long as I didn't use the trim frame ranges step.

When I said yes to the "do you want to trim off unwanted content" question, I couldn't seek through the video in VirtualDub effectively.  If I advanced frames sequentially from the start, it would work, but if I tried to jump ahead to some later frame by clicking on the frame timeline, the frame's picture wouldn't show, and hitting the Play button would bring up "Avisynth read error: DirectShowSource: The video graph failed to restart after seeking. Status = 0x80004005".  Also, if I just specified trim frame ranges anyway (valid ones) and started the encode, then the encode would stop very early, and the encode's console output would contain the same Avisynth read error.

So I didn't use the Anri-chan trim frames step.  Instead, I added a Trim statement to the end of the .avs that is given to Anri-chan.
[code]ECHO>>"%output%.avs" Trim(1460,2160)[/code]
Again, not sure if this causes any quality loss compared to trimming frames with Anri-chan.

---

Other notes:

a. For recording this game, I use AmaRecCo, which does screen capture.  AmaRecLite (which does DirectX capture) also works, but my computer can't run it as well as AmaRecCo; I'd need to sacrifice more resolution, framerate, etc. to make AmaRecLite run as smoothly as AmaRecCo.

b. The game resolution is 640x480, but I have AmaRecCo resize it to 480x360 while recording.  Again, this is to help recording performance; if I save the video at full resolution, then I have to lower the framerate to keep the recording smooth.

c. I recorded in 50 fps.  At full resolution (640x480), the highest recording fps my computer could take was about 30.  Full resolution looks really good, but I really didn't like how this game looked in 30 fps, so I compromised some resolution for a better framerate.

d. Yes, I'm recording in RGB (VLC's Tools -> Media Information says the uncompressed video's decoded format is "24 bits RGB").  As mentioned in my first post, I used Lagarith, which I heard was relatively gentle on the CPU compared to other codecs.  I'm not really sure what options I have to make the recording's color format different, like YV12 (I'd be interested in hearing both CPU intensive and non CPU intensive options for this, in case I get a new computer someday soon).

e. I'm not thinking about SDA submission at this time.  It's nice to maximize my video quality given the resources I have, though.

---

So yeah, the results.  I'll just attach the videos to this post, as they're not too big.

- intermediate_encode.mp4 (23.7 MB, 1:03) is the intermediate file that I could save and re-encode from, analogous to DVD .vob files for console games.

- 6-Kinoko_test.mp4 (0.97 MB, 0:14), 6-Kinoko_test_HQ.mp4 (1.90 MB, 0:14), 6-Kinoko_test_IQ.mp4 (2.22 MB, 0:14), and 6-Kinoko_test_XQ.mp4 (2.22 MB, 0:14) are the medium, high, insane, and X-treme quality encodes from Anri-chan.  The videos are trimmed to about 14 seconds, to make them reasonable enough filesizes to attach to this post.  The input of the Anri-chan encode is the .avs file generated from the Windows batch file above (which in turn uses intermediate_encode.mp4 as a basis).

My observations:

I can't really notice much it the way of quality issues or oddities, besides that the vertical lines on the cave walls are kind of visible in all of the encodes attached to this post, including the intermediate_encode.mp4.  Well, the medium quality final encode doesn't have the vertical lines, but that's just because medium quality is (expectedly) fuzzy all around.  Not sure what to make of the vertical lines in general; it's not like they're even that noticeable other than in stillshots, anyway.

---

Please let me know if I missed any significant oddities in the final videos, as I'm not too experienced in looking for video artifacts.

Overall, though, I'm feeling all right with using this process (or something similar) to save a minimally-compressed .mp4 and use that for future encodes.  Thanks, ballofsnow!

Attachments:

intermediate_encode.mp4

Mystery

2013-01-20 08:40:09 am

There is also huffyuv, which I believe is easier on the CPU than lagarith. Plus it allows you to encode in RGB. It's also completely lossless. I don't know if lossless video is small enough for you, though?
There is also FLAC for lossless audio, if you want, though you could just use uncompressed audio since it doesn't take much space...
For extracting audio from files, ffmpeg might be a good tool. I don't recall how to use it, though. Sorry.

ballofsnow

Edit history:

ballofsnow: 2013-01-20 10:05:51 am

ballofsnow: 2013-01-20 10:04:47 am

ballofsnow: 2013-01-20 08:54:55 am

ballofsnow: 2013-01-20 08:45:17 am

2013-01-20 08:42:31 am

Quote from yoshifan:

(2)
When I gave the .avs to Anri-chan, it said something like "Error: Probably not a supported AVI file". This didn't affect Anri's ability to encode the video; it encoded just fine, at least as long as I didn't use the trim frame ranges step.

When I said yes to the "do you want to trim off unwanted content" question, I couldn't seek through the video in VirtualDub effectively. If I advanced frames sequentially from the start, it would work, but if I tried to jump ahead to some later frame by clicking on the frame timeline, the frame's picture wouldn't show, and hitting the Play button would bring up "Avisynth read error: DirectShowSource: The video graph failed to restart after seeking. Status = 0x80004005". Also, if I just specified trim frame ranges anyway (valid ones) and started the encode, then the encode would stop very early, and the encode's console output would contain the same Avisynth read error.

So I didn't use the Anri-chan trim frames step. Instead, I added a Trim statement to the end of the .avs that is given to Anri-chan.

Code:

ECHO>>"%output%.avs" Trim(1460,2160)

Again, not sure if this causes any quality loss compared to trimming frames with Anri-chan.

So basically Anri needs to be updated to have better support for loading H.264/AAC as source. Right now, even with the avisource/avs trick, Anri doesn't find a "valid" fourcc, i.e. it doesn't recognize it so it switches to directshowsource() instead of avisource(). If you get through the source file load, the next screen is for game properties.. wait there, open the newly created _source.avs and change directshowsource([file].avs) to import([file].avs). Then continue and it should work better this way, won't be perfect though.

edit-

Not being satisfied, thought about it some more. The idea is to help ffmpegsource in its decoding of video, so maybe simplify the intermediate file. My first hunch was to turn off b-frames with the trade-off of larger file size.

Test 1 was at medium preset, default b-frame settings. Intermediate file size of 68.2 MB. When running through Anri using import() and specifying 4 trim ranges, the encode just fails:
avs [error]: FFVideoSource: Insanity detected: decoder returned an empty frame occurred while reading frame 2

Test 2 was at medium with --bframes 0. Intermediate file size of 80.2 MB. When running through Anri using import() and specifying 4 trim ranges, the encode actually finishes. The videos play back fine.

Conclusion?: ffmpegsource doesn't play nice with b-frames when seeking??

The encode.bat script again, with b frames off:

Code:

@ECHO OFF
CLS

SET input=capture.avi
SET output=intermediatefile

REM	Generate intermediate video mp4. I suggest --qp (quantizer mode) and not 2pass nor crf modes.
REM	Change the color values if needed. i420 (yv12) on an intermediate might not be good..
x264.exe --profile high444 --output-csp i444 --preset medium --bframes 0 --qp 10 -o "%output%_video.mp4" "%input%"

REM	Extract audio. Can try instead to feed an avs to nero directly if you want. From experience, nero
REM	seems to work better with direct access to an audio file.
wavi.exe "%input%" "%output%_audio.wav"

REM	Encode to AAC with nero.
neroAacEnc.exe -q 1 -lc -if "%output%_audio.wav" -of "%output%_audio.mp4"

REM	Mux it, MUX it up, MUX IT UP! WOOO
MP4Box.exe -tmp . -new -add "%output%_video.mp4" -add "%output%_audio.mp4" "%output%.mp4"

REM	Generate ffmpegsource avisynth files for later final encode. The avs file -should- be able to load
REM	into Anri-chan when you choose avi as source.
ECHO>"%output%.avs" loadplugin("ffms2.dll")
ECHO>>"%output%.avs" import("FFMS2.avsi")
ECHO>>"%output%.avs" FFIndex("%output%.mp4")
ECHO>>"%output%.avs" AudioDub(FFVideoSource("%output%.mp4"), FFAudioSource("%output%.mp4"))

REM	Clean up
DEL "%output%_audio.mp4" "%output%_audio.wav" "%output%_video.mp4"


ECHO.
ECHO.
PAUSE

[quote="yoshifan"]
(2)
When I gave the .avs to Anri-chan, it said something like "Error: Probably not a supported AVI file".  This didn't affect Anri's ability to encode the video; it encoded just fine, at least as long as I didn't use the trim frame ranges step.

When I said yes to the "do you want to trim off unwanted content" question, I couldn't seek through the video in VirtualDub effectively.  If I advanced frames sequentially from the start, it would work, but if I tried to jump ahead to some later frame by clicking on the frame timeline, the frame's picture wouldn't show, and hitting the Play button would bring up "Avisynth read error: DirectShowSource: The video graph failed to restart after seeking. Status = 0x80004005".  Also, if I just specified trim frame ranges anyway (valid ones) and started the encode, then the encode would stop very early, and the encode's console output would contain the same Avisynth read error.

So I didn't use the Anri-chan trim frames step.  Instead, I added a Trim statement to the end of the .avs that is given to Anri-chan.
[code]ECHO>>"%output%.avs" Trim(1460,2160)[/code]
Again, not sure if this causes any quality loss compared to trimming frames with Anri-chan.
[/quote]
So basically Anri needs to be updated to have better support for loading H.264/AAC as source. Right now, even with the avisource/avs trick, Anri doesn't find a "valid" fourcc, i.e. it doesn't recognize it so it switches to directshowsource() instead of avisource(). If you get through the source file load, the next screen is for game properties.. wait there, open the newly created _source.avs and change directshowsource([file].avs) to import([file].avs). Then continue and it should work better this way, won't be perfect though.

edit-

Not being satisfied, thought about it some more. The idea is to help ffmpegsource in its decoding of video, so maybe simplify the intermediate file. My first hunch was to turn off b-frames with the trade-off of larger file size.

Test 1 was at medium preset, default b-frame settings. Intermediate file size of 68.2 MB. When running through Anri using import() and specifying 4 trim ranges, the encode just fails:
avs [error]: FFVideoSource: Insanity detected: decoder returned an empty frame occurred while reading frame 2

Test 2 was at medium with --bframes 0. Intermediate file size of 80.2 MB. When running through Anri using import() and specifying 4 trim ranges, the encode actually finishes. The videos play back fine.

Conclusion?: ffmpegsource doesn't play nice with b-frames when seeking??

The encode.bat script again, with b frames off:
[code]
@ECHO OFF
CLS

SET input=capture.avi
SET output=intermediatefile

REM	Generate intermediate video mp4. I suggest --qp (quantizer mode) and not 2pass nor crf modes.
REM	Change the color values if needed. i420 (yv12) on an intermediate might not be good..
x264.exe --profile high444 --output-csp i444 --preset medium --bframes 0 --qp 10 -o "%output%_video.mp4" "%input%"

REM	Extract audio. Can try instead to feed an avs to nero directly if you want. From experience, nero
REM	seems to work better with direct access to an audio file.
wavi.exe "%input%" "%output%_audio.wav"

REM	Encode to AAC with nero.
neroAacEnc.exe -q 1 -lc -if "%output%_audio.wav" -of "%output%_audio.mp4"

REM	Mux it, MUX it up, MUX IT UP! WOOO
MP4Box.exe -tmp . -new -add "%output%_video.mp4" -add "%output%_audio.mp4" "%output%.mp4"

REM	Generate ffmpegsource avisynth files for later final encode. The avs file -should- be able to load
REM	into Anri-chan when you choose avi as source.
ECHO>"%output%.avs" loadplugin("ffms2.dll")
ECHO>>"%output%.avs" import("FFMS2.avsi")
ECHO>>"%output%.avs" FFIndex("%output%.mp4")
ECHO>>"%output%.avs" AudioDub(FFVideoSource("%output%.mp4"), FFAudioSource("%output%.mp4"))

REM	Clean up
DEL "%output%_audio.mp4" "%output%_audio.wav" "%output%_video.mp4"

ECHO.
ECHO.
PAUSE
[/code]

nate

2013-01-20 10:14:08 am

doesn't surprise me too much ... seeking in ffmpeg is a clusterfuck. do you have the latest version of ffmpegsource? what about whatever dlls it needs, libavcodec.dll or whatever?

ballofsnow

2013-01-20 10:15:38 am

Still testing on the same VM with http://code.google.com/p/ffmpegsource/downloads/detail?name=ffms-2.17.7z posted above. I don't recall any issues about libavcodec.dll.

nate

2013-01-20 10:16:35 am

can you upload that test file that it failed to seek in? i'm curious to see whether my implementation can do it.

ballofsnow

Edit history:

ballofsnow: 2013-01-20 10:37:15 am

ballofsnow: 2013-01-20 10:27:19 am

2013-01-20 10:18:19 am

Dropbox link, let me know when finished downloading so I can remove it.
~~https://...intermediatefile.mp4~~ - medium, default b-frames.

Trim range 100-200, 300-400, 500-600, etc. Just fails..

nate

2013-01-20 10:34:32 am

ok, got it.

nate

2013-01-20 10:42:42 am

black picture in quacktime 7. black picture in quacktime x. my program can decode the video and seek albeit slowly. thanks; this has been helpful.

ballofsnow

Edit history:

ballofsnow: 2013-01-20 10:48:07 am

2013-01-20 10:47:08 am

Quote from nate:

black picture in quacktime 7. black picture in quacktime x.

Keep in mind the color space of the intermediate file. Trying to keep it as 444 or 422 depending on what the source is.

nate

2013-01-20 10:48:29 am

ah, so that's why.

ballofsnow

2013-01-20 12:09:39 pm

I'm going to patch in a 4th load option for avs files which will use import(). It won't be an implementation of ffmpegsource, just an option to load avs files where the burden is on the user to provide a working file. Will post it here whenever I'm done with it (no eta sorry).

My thoughts on ffmpegsource is that we can't yet recommend it for general usage, because inevitably someone will try to load a b-framed source file, but I think it can be good for a situation such as this topic where you're starting off with an avi source file, and creating the intermediate yourself and can thus control whether b-frames are on or not.

yoshifan

2013-01-20 01:18:12 pm

Hmm, I tried encoding the intermediate with --b-frames 0. The resulting .mp4 is 27.4 MB (the one with b-frames was 23.7 MB), so the filesize change is indeed acceptable. However, I get the same error as before (Avisynth read error: DirectShowSource: The video graph failed to restart after seeking. Status = 0x80004005) when trying to frame-seek or trim the intermediate file in Anri-chan.

This fixes the frame-seeking and trim perfectly, though:

Quote from ballofsnow:

... If you get through the source file load, the next screen is for game properties.. wait there, open the newly created _source.avs and change directshowsource([file].avs) to import([file].avs). Then continue and it should work better this way, won't be perfect though.

I've attached my b-frame-less intermediate encode to this post. Could one of you see if you can get frame seeking / trimming to work with it, without the directshowsource -> import trick?

Attachment:

intermediate_encode.mp4