New video codec based on texture compression

When it comes to media playback, at ArKaos we always fight to get the best out of current computer configuration.

When you are in the show industry and try to find out the best way to compress your content it’s still a little bit of black magic. While some codec are able to compress video very well they are heavy to handle for the machines, even more, the codec that does the best job at compressing while keeping a good quality such as H264 are very bad when you need to scratch your media.

The best codec for users that need to interact a lot with the content should allow to play forward and backward easily and should allow to jump into the content quickly.

Recently, I’ve heard about experiments with a file format designed by graphic card manufacturers and based on texture compression DXT1 DXT3 or DXT5 and I was wondering how hard it would be to add support for such a format to QuickTime through a new codec. Being programing for the QuickTime API since it’s version 1.0 beta I was considering the challenge fun and interesting.

I started by resuscitating an old sample code from 1999 found on the Apple web site and after a few hours of coding and testing I had an ArKaos codec generating video files that could be played back by QuickTime.

ArKaos codec in the QuickTime player

Then I jumped on the DXT texture compression problem and chose to use the squish library to handle the job. A few more hours later and my experimental codec was working with texture compression.

The codec when used by QuickTime at that point was not efficient because the texture decompression was done on the CPU, so there’s no advantage of using the GPU to handle that texture format.

To measure the speed up I used an experimental player I am working on, and patched ffmpeg to handle my new codec. This allowed me to leave the texture data untouched and pass it to the GPU via the OpenGL extensions that handle texture compression.

When this was done I could start enjoying the show and make some performance tests; in the table below the file size is in MB and CPU load is taken from running top in the console:
 

If you are curious about frame quality here is a crop from an original frame in Apple intermediate :

Original frame - Apple intermediate

Here is the same cropped frame using Dxt1 with texture compression:

HD frame - Dxt1 compressed

And here is the same cropped frame with Dxt5 compression:

Same frame with Dxt5 compression

So the results of these tests are that pure Dxt1 is the best format when it comes to CPU load, Dxt5 is better if you focus on quality but it generates files that are twice bigger. Actually, using Dxt1 I could have played 7 HD loops at the same time with my laptop … I mean, if hard drive’s data rate would’ve allowed it!

The problem with pure Dxt5 or even Dxt1 files is that they are huge, about 4 times bigger than photo jpeg or H264 encoded files. So my workaround was to use data compression on the CPU, since it’s almost unused!

To conclude, it looks like using Dxt1 with texture compression is the best solution to optimize both CPU and disk usage with a file size that is almost the same as Photo jpeg and a little bit bigger than H264. And the codec is still twice faster than any other codec I tested when using the CPU to decompress the texture data from disk.

On my venerable old MacBook pro I can’t smoothly play a loop in Full HD and even at 1280*720 my machine is still struggling. Thanks to Dxt1 with texture compression I can play two HD layers and three 720p layers.

Now I know you guys would like to play with this new toy so here’s a .dmg file with the test player I have used, along with a set of video loops. Beware that this dmg file is for Mac OS X x86 only (no Power PC): and here’s the link

You can be sure that we will bundle that new experimental codec in the next update of GrandVJ andMediaMaster, so everyone can enjoy it!