Virtualdub/Avisynth file limit?

YoukaiDragoon

Edit history:

YoukaiDragoon: 2011-04-28 02:04:43 pm

YoukaiDragoon: 2011-04-28 01:18:31 pm

2011-04-28 01:17:24 pm

Is there some sort of limit to how many files or frames can be opened/concatenated in VirtualDub? I'm trying to get my frame numbers for trimming on my segments and it seems I always get an error (access violation) if I try to do too many files at once (somewhere between 20 and 30 segments).

EDIT: I'm pretty sure it is. If I remove filenames from the PROJECT_Source.avs file it will start working. Can anyone confirm the limit of segments/video length? Or is there a way to increase it?

Thread title:

nate

2011-04-28 02:07:42 pm

snow knows way more about this than me. it's been discussed extensively in the past. you might try searching the tech support forum if you haven't already. afaik there was never a good solution found but there were some workarounds.

YoukaiDragoon

2011-04-28 02:09:36 pm

I did a quick search on the forum and didn't find any conclusive answers. It's ok though I can still figure out my trims by just doing it 10 segments at a time and doing the math for frame numbers. It's just going to be a pain in the butt. I hope the viewers appreciate all the work I'm doing to improve their viewing experience; and same them over 30 minutes of their day in loading screens Smiley

ballofsnow

2011-04-28 03:07:44 pm

Yep, no good solution yet. I may revisit ffmpegsource and see if that works.

Anri uses a workaround when it detects too many files. It basically encodes new master files in batches of 20, then uses those as the source files for the rest of the project. But... it's a workaround.. and from the feedback in this forum apparently doesn't work so well, though I think there were times that the Fraps codec was to blame.

Did you actually try using Anri from the start? I'm curious whether it would fail on the amount of videos you have.

YoukaiDragoon

2011-04-28 03:15:17 pm

I have a 99 segment run that is 3:04 of video total. They are .m2ts files. I got "bmn-anrichan-3". I put all 99 segments in anri and it indexed them all correctly but I can't open the temp_PROJECT_trim.avs in VirtualDub because of an error. If I alter my PROJECT_Source.avs to only have 20 files (i.e. temporarily delete the rest) then it will open.

So I will just figure out the frame numbers in groups of 10 segments and make sure I keep track of the total number of frames in each group so I can do the math to get actual frame numbers on all of the groups after the first one.

YoukaiDragoon

Edit history:

YoukaiDragoon: 2011-04-28 08:59:51 pm

2011-04-28 08:59:07 pm

ARG! I spent hours fiddling with my files to get the damn frame points (280 of them) and do the math so I had start and end frames for my whole run and the encoding has the same issue that the other thing had. I wanted to supply my run encoded as a single trimmed file with audio commentary.

What are my options? Are there things I can try to work around this issue? If not do I have to encode the run as a few different files? I suppose 7 trimmed files would be more friendly to watch than 99 individual segments.

bimanc

2011-04-29 12:02:57 am

I'km not sure it works, but in the past when I had the same issue, I had to encode an uncommpressed RGB from the original files, which then were okay for anri. However that's a lot of work, and insane hard drive space.

gia

2011-04-29 12:31:24 am

I love YaBB 1G - SP1!

same, I believe its not about the number of files, but ram, you can try to increase your ram SetMemoryMax(3000)
its like it tried to decompress the data on the fly, but keep it on memory. When you go 100% uncompressed it processes on the fly and drops the data immediately, or perhaps a memory leak

This happens to me with fraps videos if one of the videos isn't on the same codec (ie uncompressed file).

MUGG

2011-04-29 03:31:39 am

TASer

Without having read through the thread, I can say that there is a limit as to how many times you can use AviSource.
See the bottom of this page: http://avisynth.org/mediawiki/AviSource

YoukaiDragoon

Edit history:

YoukaiDragoon: 2011-04-29 09:13:34 am

2011-04-29 09:12:42 am

Quote from MUGG:

Without having read through the thread, I can say that there is a limit as to how many times you can use AviSource.
See the bottom of this page: http://avisynth.org/mediawiki/AviSource

Good catch. With 99 segments I'm sure I'm well above that; and that would explain why it pukes on my just above 20 files (each segment is actually two calls in the source.avs)

I think I will just encode it in segments and then concatenate them. Does anyone know any good software that can do that without re-encoding? A quick google search turned up "yamb" http://yamb.unite-video.com/features.html but I cant' find anything about if it's any good or not.

nate

2011-04-29 09:43:47 am

you can't append mp4 afaik, certainly not that many of them without getting hella audio desync. looks like you're going to have to buy a hard drive and encode nmf to feed to anri. if you use lagarith yv12 it won't be insane, maybe less than 500 gig.

YoukaiDragoon

2011-04-29 09:50:23 am

I'm not sure I follow you Nate. What is NMF short for? Is there some tutorial on how to do what you are saying?

ballofsnow

Edit history:

ballofsnow: 2011-04-29 10:40:16 am

2011-04-29 10:39:04 am

New Master File. Meaning you create a new master/source file(s), hopefully with a lossless codec so you don't lose quality.

Some more stuff: http://speeddemosarchive.com/kb/Glossary_of_terms

btw, regarding appending, Exo's post from The SDAppend Project topic is interesting. My post.

YoukaiDragoon

Edit history:

YoukaiDragoon: 2011-04-29 11:33:17 am

YoukaiDragoon: 2011-04-29 11:21:52 am

YoukaiDragoon: 2011-04-29 11:21:16 am

YoukaiDragoon: 2011-04-29 11:20:29 am

2011-04-29 10:59:43 am

I don't understand how that will help. Would the new master file be all of the segments together as one file?

I will do a test with adidemux and see if it affects the quality.

If I decide to go the NMF route:
1) What program would I use do do that and what format will my file end up as? I am starting with .m2ts files.
2) My .m2ts files are 16GB, you are saying that will bloat to 500GB if I create a NMF?
3) My source files are 60fps but the game is only 30fps. When I create NMF's can I reduce the framerate as well?

EDIT: I just tested with two of my shortest segments. Avidemux concatenated them in about half a second so it's obviously pasting them together and not re-encoding them. The quality after looked as good as before so I am just going to encode my run in multiple segments and then concatenate them. If someone could answer my NMF questions for future refrence though I would appreciate it.

Thanks to everyone that has been helping me recently I know it's probably annoying having then new guy ask so many rookie questions.

I don't understand how that will help. Would the new master file be all of the segments together as one file?

I will do a test with adidemux and see if it affects the quality.

If I decide to go the NMF route:
1) What program would I use do do that and what format will my file end up as? I am starting with .m2ts files.
2) My .m2ts files are 16GB, you are saying that will bloat to 500GB if I create a NMF?
3) My source files are 60fps but the game is only 30fps. When I create NMF's can I reduce the framerate as well?

EDIT: I just tested with two of my shortest segments. Avidemux concatenated them in about half a second so it's obviously pasting them together and not re-encoding them. The quality after looked as good as before so I am just going to encode my run in multiple segments and then concatenate them. If someone could answer my NMF questions for future refrence though I would appreciate it.

Thanks to everyone that has been helping me recently I know it's probably annoying having then new guy ask so many rookie questions.

nate

2011-04-29 02:50:55 pm

yeah, that's cool that avidemux supports m2ts. i guess if supports vob then it supports that so now i look dumb. very interesting solution to this problem.

YoukaiDragoon

Edit history:

YoukaiDragoon: 2011-04-29 04:19:05 pm

YoukaiDragoon: 2011-04-29 03:50:51 pm

2011-04-29 03:28:56 pm

I didn't test it with the .m2ts. I used two of my shortest segments (because they would render quick) and made LQ and merged the resulting files.

Just tested the .m2ts files though and when I opened it it said "This looks like mpeg. Do you want to index it?" I said yes. For the options I chose copy and copy and saved it as and mp4. The resulting file studdered though; I'm guessing the B-frame issue it mentioned.

I just realized if I make my clips and the string them all together at the end I won't have a station ID on the video. So I need to make a NMF. Is there a tutorial somewhere about how to do that?

EDIT: --BINGO! Finally found something that works. TSSplitter http://www.ffprojects.net/tssplitter/tssplitter.htm did what I needed it to. Based on the resulting file size of my test the joined file is an exact copy of my other files just as one single segment. Happy days.

nate

2011-04-30 09:54:08 am

thanks for the info. no need to worry about no statid at the end. that's just an anti-copying thing. it's not a requirement to have one at all.

YoukaiDragoon

Edit history:

YoukaiDragoon: 2011-04-30 10:02:15 am

2011-04-30 10:01:29 am

Just in case it wasn't obvious from my previous post; TTSplitter was able to join my .M2TS files so I am able to encode it from a single file without creating an NMF (unless you consider the joined file my NMF). This will also allow me to add the state ID to the file. Happy days.

Also once I get all of my videos rendered what is the preferred method of adding the commentary? Can the current version of anri add a commentary track without re-encoding or will I need to use something like avidemux?

nate

2011-04-30 10:10:53 am

Quote from YoukaiDragoon:

Can the current version of anri add a commentary track without re-encoding

yes.

ballofsnow

2011-04-30 11:21:21 am

Nice find! I'll note this in the anri thread as possible nmf method for mpeg videos.