Thanks to One River media's codec site, its a great resource.
Introduction
On CML-POST mailing list I decided to respond to Jack Kelly about using a lossless compression profile with the new open source x264 codec. He was suggesting that it might be a good archival format as it is very powerful in reducing file size potentially.
I downloaded Staxrip and all the required parts and proceeded to setup an encoder. I decided to conduct the tests with a high compressed copy of a television show I happened to have on my laptop. This was a bad idea and latter I conducted a better test using 902 JPG's captured with my Nikon D50 in medium resolution at fine compression using a 50mm f1.8 lens.
Initial test
An Xvid download of a show I happened to have on my laptop at the time was the initial source for the first test. I opened it in Virtualdub and took a short segment (little over 1200 frames) and output it to uncompressed 8 bit RGB avi file. This is part of the mistake, originally the Xvid was in YV12 which means it had a reduced colour space which I didn't notice.
I setup Staxrip 0.9.9.2 to encode to CQ-lossless profile which was fine and it encoded well. However I was unable to decompress this to an uncompressed RGB avi because of issues with direct show codecs. Later I resolved this with a beta version of virtual dub, but in order to do the test's I had to use VLC media play and take several screenshots and compare then in photoshop.
In photoshop using difference layers it showed all pixels black for three frames I tested and I could not find a single pixel that was any other value than 0 R 0 G 0 B so I decided it was truly lossless. It gave almost a 10:1 compression ratio which was unheard of so it seemed like a big deal.
Tim Sassoon debated that it must be visually lossless and not truely lossless, I freely admitted that it was indeed a bad test and that I was going to try and get some better footage.
Big Test
Out comes my Nikon D50 in continuous shooting mode at 3fps set to medium and a while latter I had a rather funny video of poor focus to sharp transitions and a test that I think is very hard on a lossless codec like x264. I think this because the frames change so dramatically that intraframe techniques would be fairly useless on these frames.
The initial output file was 8.54GB and the initial compressed file was 1.07 GB which meant that this looked like a really efficient codec. So I got out adobe after effects and decided to do more than just a single frame difference.
This showed that it was obviously not a truly lossless codec - Or so I though. most values were off by a single value in each channel but a lot were off. So I posted to CML my findings but then it struck me, In my Avisynth script I had to convert to YV12.... what was this, why was it needed, did it change the video.... Sure enough this converted the video to 4:2:0. I was comparing a 4:4:4 video that was truly 4:4:4 to a 4:2:0 and expecting them to be the same, not likely. So I proceeded to setup some other tests with a reduced resolution
New Test
Alright.... so here is the files and what happened to prove x264 is lossless but only in 4:2:0 mode
My source file was 902 Jpg files, converted to a 2256x1504 uncompressed 8bit RGB avi file, that is then resized down to 720x480 a(8 bit 4:4:4 RGB avi) - That file is D50-SD-uncompressed-420.avi. I took this file and using staxrip encoder it output the D50-SD-x264-RAW.mp4, but in order to do that it made a raw file called D50-SD-x264-RAW_encoderOutput.264. File D50-SD-x264-RAW.avs is the avisynth script staxrip used to put into the command line encoder (x264.exe in another directory). D50-SD-x264-RAW_staxrip.log is just a log file and I have no idea why it created the xxx.rip file.
So I created "Copy of D50-SD-x264-RAW.avs" and modified it to read the following:
directshowsource("Z:\x264 testing\SD test\yv12\D50-SD-x264-raw.mp4")
And thats it for inside that.
I opened the file "Copy of D50-SD-x264-RAW.avs" in VirtualDub-1.6.2 (in the folder on the left) and output the file D50-SD-x264-420.avi. Notice the files are almost exactly one frame in the difference - This is a problem for archiving I think! I output 4:4:4 8bit RGB, because afterfx didn't like YV12.
I then opened D50-SD-Uncompressed-420.avi in VirtualDub-1.6.2 (it may have been 1.6.16, i just needed 1.6.2 because its the only one I could find to support direct show codecs and set colour back to 4:4:4 and output.
Click the picture for the afterfx setup before turning on the layer and the one below for after
So it looks lossless except for some issues on the side. I'm still trying to figure out what the issue with that is.
I had to sync the videos by a single frame and this seems like a consistent problem and I think its due to virtual dub. When I did Jpeg200 testing I had to sync by two frames because QuickTime was outputting the first frame 3 times for some reason.
How the test is done
Black pixels are unchanged, white are changed, this is on the origial 2256x1504 when I didn't realize that the colour space x264 is encoding was 4:2:0 so this is comparing 4:2:0 lossless with 4:4:4 uncompressed