I’d like to offer congratulations to all the Xiph folks, but especially Monty, Ralph and Tim on the theora-1.1alpha1 release from the Thusnelda encoder branch.
To try it out, I transcoded a short (1m40s) 720p trailer video from H.264+AAC to Theora+Vorbis, at libtheora quality setting of 32, using the 1.0 theora encoder, and again with 1.1. It’s not a very rigorous experiment, but enlightening nonetheless.
Both encoders produced output of comparable visual quality. With the 1.0 theora encoder the output file was 24383125 bytes. Of that, the video portion is about 1.835 Mbit/s. With libtheora 1.1alpha1, the same quality is reached with only 19088009 bytes (video at 1.416 Mbit/s)!
That’s a pretty easy 20-ish% compression improvement for no loss in quality. As an added bonus, the transcoding time dropped from 2m10s to 1m52s. That’s not quite real-time for this frame size in either case, but an impressive step closer.
There’s some degradation from the visual quality of the original file. That’s to be expected when taking a 5.61 Mbit/s video stream down to less than a third of the original size – even on the same codec.
For comparison, here’s a random frame from the output of each encoder, along with the same frame from the original file:
Sweeeet. Well done, guys!
Only one of the three screenshot links works.
Yeah – I was having a bit of trouble with the images and the limited bandwidth this machine lives on.
I moved them to another server, and they should all be working now
X-Men?
I only skimmed over the text and immediately loaded the images to compare them. I thought “hm, there’s a noticable difference between H.264 and the Theora samples, but I don’t see a difference betweeen the Theora samples”. After reading the text it’s pretty obvious why that is so 😉 I looked quite a bit and didn’t find any visible differences between the Theora samples. And with 20% savings that’s pretty cool!
It is indeed X-Men! (X-Men 3, to be exact)
It’s worth pointing out that the encoder for the original (H.264) is pretty much irrelevant as it’s merely the input. Ideally a lossless input would be used but most people don’t have lossless video lying around.
If you wanted to compare H.264 perfomance with Theora you’d need to re-encode with H.264 and match output quality/bitrates and lots of other things that would probably be too much bother.
Bottom line, these screenshots are not a valid Theora/Thusnelda vs. H.264 quality comparison. The original post doesn’t claim they are, but the 4th comment could be read that way and there’s enough Theora misinformation about as it is.
They are however a nice quick and dirty demonstration of Theora progress, so cheers for that.
Hi,
there is a noticeable difference in the sky. teh 1.1 version is more blurred, in the 1.0 version you can see more detail (not too much, but still very noticeable)
there is quite a noticeable difference between the two theora files to me at least, the clouds in 1.1 are far blockier than in 1.0
That’s great news for me. I convert H.264 to ogg because the quality is the same (that I see on my screen) and the filesize is significantly smaller.
The Theora 1.1 output looks worse to me. Look at clouds, particularly the wisps in the top right of the shot. They’ve become quite blocky. Considering the two Theora frames are encoded at the same bitrate, I’d go so far as to claim it’s “significantly” worse. There also seems to be detail loss on the building. It’s harder to quantify, but it looks softer.
To my eye, the quality of the 1.1 encode is noticeably worse than the 1.0 encode. See the image below showing an example – a segment of the blue sky from the right side of the image. The 1.1 encode is much more blocky here, with clear visible horizontal lines.
http://codebeard.homelinux.com/tmp/2009/03/theora-sky-cmp.png
Could I get a link to refutations of commonly-heard FUD about theora? There’s a Microsoftie who likes to talk smack about theora, but I don’t (yet) do hardcore video stuff, so I’m reliant upon my news sources to learn about it. Unfortunately, my news sources aren’t that good. 🙁
The clouds in the second theora shot look more “grainy” than the first theora shot.
Yeah, it’s a general problem that most Theora examples have been produced by transcoding H.264, which means the Theora examples are guaranteed to look worse…
Maybe the best way to demonstrate to the detractors that Thusnelda is a far better encoder is to ramp up the quality of the encoding to match the output size with the theora-1.0 encoder.
Also would love to see someone grab the Big Buck Bunny source pngs and encode them into the three formats for comparison.
So… The above is an inadvisable test methodology.
As others here have pointed out, the sky in the ‘1.1’ sample is less clear. This is NOT, however, a flaw in 1.1: It’s producing that output with a 20% lower bitrate. I, personally, find it impressive that its holding up that well at the lower rate.
The quality knob in Theora is a unitless relative value (you could think of it as a ‘sharpness knob’ with differing meaning version to version), it has no meaning across versions. It’s a knob you turn up for more quality (and bitrate) and down for less. It might be that 40 in 1.1 produces worse quality than 35 1.0, but so long as 1.1 does it with lower bitrate it still may be fair to call 1.1 the better encoder.
What all this means is that the images here are not really comparable. What you should do is adjust the settings until both have the same bitrate. Then gauge quality. The one with the better quality wins.
Unfortunately this method is weak too— all these encoders are variable bitrate for a frame. So an overall better encoder may have just used fewer bits on any given frame. If you happen to pick that frame for your comparison you’re going to get incorrect results. A good test would need to involve many images.
…and even that isn’t enough: These are moving images, still comparisons are not always representative.
It was mentioned above, but I should repeat it: Using low bitrate compressed input makes for bad test cases. The decompressed input file is full of quantization noise which may not be visible to you, but which the encoder now has to waste bits attempting to approximate. The behaviour ends up being very sensitive to the source material. Compressed inputs are a reality, true, but nothing is going to handle low bitrate compressed inputs especially well, so they don’t make for good general compression comparison points.
Just to reiterate what was said in the original post, because some commenters missed it, the two Theora shots are not at the same bitrate.
Given that fact, for those that feel that the Thusnelda frame looks worse (or even just part of the frame looks worse), note that it’s possible for this frame to actually be slightly worse and for this to still represent an improvement in the codec. Note the words from the original post: “output of comparable visual quality”, “a random frame” and “not a very rigorous experiment”.
The changes in Thusnedla have sound theory behind them and despite having to work within certain fixed limits of the format the developers are confident that Theora has the potential of being competitive with MPEG4-class codecs. I guess we just have to wait and see if there’s enough improvement to move the general geek attitude towards Theora from “total apathy” to “grudging acceptance”.
For the record, I wasn’t saying that the 1.1 encode being slightly lower quality somehow invalidated the main point. Obviously it’s quite an achievement to get such similar results with very different bitrates.
The author said “that’s a pretty easy 20-ish% compression improvement for no loss in quality.” I was just suggesting that maybe it is only 15-ish% for the same quality – even that would be a huge achievement.
Building off of comment 17 and others, why not re-post with a cleaner example? (Starting from lossless input, use clips rather than stills, provide more captioning detail, etc.)
Another interesting point is managed-bitrate vs quality mode.
Theora (and Vorbis, for that matter) can either target a ‘quality’ and allow the bitrate to swing wildly depending on the complexity of the source material; or it can target a particular bitrate where the codec works hard to achieve a specific bitrate within a specific time window (comparable to the players buffer).
The former is what you use when you are interested in smaller sizes for bulk transport as it gives the best quality for a given file size. The latter is what you need to use for streaming since it will not allow a suddenly complex scene to leave the user stuck paused waiting for re-buffering.
Theora 1.0 managed bitrate mode was simply broken, it frequently missed its bitrate target (failing far in both directions) and the quality was much worse than it should have been at whatever bitrate it ended up with. The 1.1 alpha does much better in this regard: it is accurate to the target rate and delivers much better quality.
This can, paradoxically, lead to 1.1a giving worse quality for a given requested bitrate as 1.0 may have overshot the request and ended up giving better quality but at a bitrate higher than the set limit. This isn’t a flaw in 1.1a; if you didn’t really care about the bitrate limit you shouldn’t have been using bitrate managed mode.
Significant about all this is that multiple people comparing Theora may come to dramatically different conclusions about how much of an improvement 1.1a is: People comparing quality mode are likely to conclude that it is a clear but moderate improvement, while people comparing bitrate managed mode are more likely to decide that it is an enormous improvement… unless they hit a case where the old encoder was seriously overshooting, in which case they may decide that 1.1a sucks.
Would it be possible to show an H.264 version of the image at the same bitrate as the Theora 1.1 version? This would help to dispel most of the confusion.
Comparing to H.264 wasn’t the point – I wanted to see what would happen if I dropped theora 1.1alpha1 in place of theora-1.0 and encoded something.
The H.264 show is only to give an idea of which detail the 2 encoders *could* preserve.
[b]@ dave[b]
What do you mean by these 2 things:
[i]
dave says:
March 28th, 2009 at 3:54 am
It’s worth pointing out that the encoder for the original (H.264) is pretty much irrelevant as it’s merely the input. Ideally a lossless input would be used but most people don’t have lossless video lying around.
[/i]
[i] If you wanted to compare H.264 perfomance with Theora you’d need to re-encode with H.264 and match output quality/bitrates and lots of other things that would probably be too much bother. [/i]
Raw video.
As far as input video available in raw format…
Sita Sings the Blues (awesome movie, btw)
is available in raw format. It might be interesting source of raw video for encoding testing. It is animation, so it would be a different kinda of test source.
http://www.sitasingstheblues.com/wiki/index.php?title=SitaSites