As you might have noticed, my main hobby is videographing lately. Having researched the market, I think I can offer some useful advice regarding where the camcorder market is going and hopefully save you a few bucks.

The idea is this: if you already have an old camcorder, hold on to it. If you just want “a” camcorder, get the lowest-end DV Canon one, which is usually selling at around $230. If you want to buy an HD camcorder wait 2-3 more years. The reason I am suggesting this is because the HD camcorder landscape is going to rapidly change in the next few years and so it’s wise to wait for this change to happen and then buy. We are currently living in a transitional stage.

The “tape”-based DV/HDV camcorders are going the way of the dodo. That much is a fact, no matter what the fanboys will tell you. The new standard is AVCHD and all major companies are going for it (including Canon which is keeping mum so far about it). It is a more convenient format for the user, as only SD or CF cards are needed instead of big, boring, last-century tapes. Additionally, because AVCHD uses USB 2.0, it will be the final strike against Firewire (main reason why Macs always come with firewire is video support).

AVCHD does not yet produce the same quality video as MPEG-2 does for HDV camcorders, but eventually h.264 encoders will be perfected. However, it’s more difficult for NLEs to have support for h.264 editing rather than MPEG-2 with large GOPs. So far, only few NLEs have mastered AVCHD, while most either crash sooner or later with it, or they only support h.264 flavors from specific brands. Maturity will come there too, just not yet though as h.264 is a bitch to decode properly, let alone edit it. You will also need a very beefy PC and 4 GB RAM to work with full progressive HD — specifications that are top of the line today but will be common ground in a few years. Finally, the future AVCHD camcorders will record in full HD 1920×1080 progressive format, while the current HDV standard is limited to 1440×1080-anamorphic and interlaced. In fact, they already started doing so, check this Panasonic model that was announced today.

So, to get an HD camcorder, make sure it’s in the format that will rule in the future and also make sure that you will get it in a time that the market is ready for it. Currently, the market does not even support fully HDV, let alone AVCHD. But the right time will come, just be patient and hold off any purchases regarding HD. Of course, this kind of advice is for people who buy 1 camcorder every 7-10 years. If you are a prosumer who changes gear every 2-3 years, this advice won’t matter much.

A detailed comparison between AVCHD and HDV can be found here.

I had watched most episodes on Sci-Fi Channel, but recently I started re-watching it from the beginning using the DVD version. Galactica is “ok”, but not outstanding. Many things just don’t make sense. What I like on “Lost” is the amount of realism it brings with its characters and situations, even if Lost is also a scifi/fantasy series. But Battlestar Galactica’s writers are just not as careful with continuity and simple logic. In detail:

* The technology is very uneven. They have technology to do “jumps” in space and create big ships, but other, more basic technology, is absent. Heck, they don’t even have iPods. It’s like the writers cut back on daily technology on purpose just so they make their situation more dramatic. But by doing so, they made it less realistic.

* First the Cylons are killing billions(?) of humans without mercy, and then they cut back on their killing for no good reason other than advancing the plot.

* Why are there no robots among people? “Good” robots that is. I mean, it was not that robots were completely banned after the Cylon Wars. Or at least, we were not told so.

* They still haven’t cured cancer, not even myopia.

* It does not make sense to be running away from Caprica for months and yet, when Starbuck had to go back she got there in an instant. If jumping so far was so fast, galactic exploration would have been a common practice and possibly Earth would have already been found.

* From the moment they got into “New Caprica”, they still wanted to find “Earth”. Why? What’s wrong with New Caprica and even some other habitable planets they visited before? As long as Cylons could not find them, there was no reason to continue the search for Earth specifically, but to find a habitable planet and plant their asses there.

* Cylon detection among humans should have been easy as a pie. And yet, it’s not, again for plot reasons.

* Cylons can download to a new body, but for some weird reason they have no connection to the hive mind? Even if it’s not possible to communicate peer-to-peer, it should be possible to communicate with the “server” if a spaceship is close. It is stupid to not be optimized to do that. More over, their political structure doesn’t make enough sense either.

* Shaky cameras suck balls. Get a tripod. It costs less than $500 for the kind of cameras they use.

* The worst of all: RELIGION. All the super-natural religious crap, are just that: crap. It feels like even their political decisions are based on religious grounds which is at least laughable — at least for humans with a level of technology.

Other than that, Battlestar Galactica is not a bad show. But I think it evolved to be cult show because there is nothing similar on TV. If only the writers were paying more attention to detail.

Why the heck LCD TV manufacturers are using 1366×768 (wide-XGA) instead of 1280×720 (720p)? The first resolution is the 16:9 version of 1024×768 and it might make SOME sense for computer monitors. But instead, the majority of LCD TVs meant for living rooms are using that resolution for their TVs instead of the real 720p one. The only explanation I have is that it might be cheaper to do so this way because creating a wide-XGA might be easier for existing factories than trying to create something totally different. Problem with this though is that the pixels are not square and so there is loss of quality when viewing HD on these TVs. And we are talking about 95% of the LCD 1080i TVs out there come with that resolution.

So I connected the 32″ HDTV on the PC. Apparently the graphics card can only do two monitors at the time, even if there are 3 connectors on the card. So I have to choose profiles all the time between my two DVI monitors or the primary DVI and the HDTV. When the HDTV is on, the second monitor goes OFF.

I don’t know if this is a limitation of nVidia’s hardware or the OEM who designed the card went cheap on this and didn’t include the extra hardware required to have all 3 monitors work at the same time (JBQ thinks is the latter). I know that under some circumstances ATi can do 3 monitors at a time (two PC and one TV connections), but for nVidia the best option I have heard is to get a GeForce 6200 PCI card and use it as a second graphics card. Good and somewhat inexpensive work around, but I am out of any PCI and PCI-e slots (one goes to my USB card and one to my Audigy HD sound card).

Kinda sucks to be hitting limitations like this one. I mean, come on, that was a $200 graphics card.

The adventures of me trying to create a *proper* 1080p video to playback on PS3/XBox are continuing. Apparently the output of any lossless .avi codec from Vegas is washed out. It just doesn’t look good. Somehow when exporting in .avi changes the contrast to high & saturation to low.

Enter Avidemux2. I wouldn’t try it, but someone mentioned it on a forum so I gave its Windows version a try. So, I tried exporting my clip from Vegas using .avi uncompressed and Avidemux2 will render it with the wrong colors. You have to use a special filter to get the right colors… Lagarith, Cineform lossless codecs aren’t supported at all. Huffyuv with its default mode has wrong colors too, but as I said it’s fixable with an extra step. I was ready to delete the app, when I saw the “mplayer eq2″ filter that let’s you adjust the contrast and saturation and bring it back to normal levels. So, I decided to give it another try, as this plugin did fix the visual problem.

However, then, the de-interlacing won’t work properly. Again, I would get ghosting when trying to de-interlace the Huffyuv YUY2 file just like I originally got with ffmpeg, although the problem does not happen when trying to de-interlace mpeg2 streams instead of .avi. The application comes with about 6 different de-interlacers, and all of them had the exact same problem. On ffmpeg the undocumented “trick” was to put the -deinterlace argument before the “input file” argument, but avidemux does not use ffmpeg as far as I know, and yet, it still has the same bug. Changing the order of the filters didn’t help either. Even after ignoring the de-interlacing problems and exporting and creating a 1080p x264 file, the resulted mp4-avc/faac file wouldn’t playback with Quicktime Pro, but only with VLC. Moreover, for some weird reason too, all x264-created files are very slow to decode, while Quicktime’s created h.264 files are not so.

So basically, I am still stuck about all this. There is not a single free or low-cost app out there that can receive as input an HDV interlaced file in a lossless codec and transform it to 1920×1080 by stretching it, de-interlacing properly, keeping the right visual look, and exporting a proper h.264 file. There is always something that just doesn’t work well in the process. I know that Premiere Pro can do all that properly btw, but I can’t justify $1000 just so I can export videos for the PS3/XBox. Having thought about the problem a bit more, I think that the only real solution to this mess is for Vegas to add h.264 support and do the job properly from beginning to end with enough flexibility. I should not even be trying to export lossless files to do the job with external utilities and hitting the one incompatibility or inflexibility after the other. Sony Vegas should add support for the Sony PS3. It’s that simple.

These guys have bandwidth to burn… ABC has just made live (only 3 episodes so far) their new HD player. It is using the On2 Flash codec at 2mbps at the 1280×720/24p resolution and it doesn’t sweat on our Comcast connection. It looks fantastic and makes you feel that you already live in the future, but you will need a really fast machine to play it back properly. My PC, a 3 Ghz P4 HT, drops about 4-5 frames per second, so don’t even think about of using the HD version without at least a CoreDuo or above… Other than that, it looks great, it has more interesting content and better quality than Joost, too bad that it’s restricted to US IP addresses though.

I received a new HDTV today, 32″, 1080i, HDMI. It won’t be used as a “home TV” though, but rather as a testing ground for my video editing station — in other words, for computer-related purposes only. I will be writing a review about it soon.

Additionally, Tiffen is sending over 6 filters for the HV20. I love playing with filters and provide “before” and “after” clips/images. I am still deciding if I want to actually buy 52mm telephoto/macro/wide-angle lenses or 43mm. The HV20 is 43mm, but all my filters are on the more popular 52mm size now, so I think it makes more sense to try to buy stuff at that size.

There are two mp3 players on the plate too, and an upcoming cellphone review for August. I am supposed to pick which phone model I want to review next, I just don’t know what I should pick. I have pretty much everything in my collection except an HTC, a Blackberry and a Treo, but because there are cost restrictions about these review devices that I pick, I can’t really have these devices (except if I pay extra). I don’t blame the retailers of course, it doesn’t make sense for them to send over too expensive devices. I will have to decide between some less-expensive Nokia and Sony Ericsson models and most Motorola, Samsung and LG models.

I hope they had more BenQ/Siemens phones though. I don’t have any of these in my collection. It’s funny too, because I have only one phone that uses my husband’s browser! And that browser version is 3 years old. BenQ/Siemens usually ships with more recent 7.x Openwave browser versions, but these phones are hard to come by here in the States.

Now that I can export to HD, here is a quick quality test. It seems to me that quality at the same bitrate between WMV and h.264 is similar, although WMV features more of the original colors. I tried giving more kbps to h.264 but it didn’t got much better, while mpeg2 at 25mbps (that bitrate is the standard for .m2t HDV files) has really nice punchy colors and detail. Mpeg2 and wmv were exported by Vegas, while x.264 via ffmpeg on 2 passes:

ffmpeg.exe -deinterlace -y -i “lossless.avi” -threads 2 -an -pass 1 -f mp4 -title “My JBQ” -vcodec h264 -level 41 -refs 2 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -partb8x8 1 -me full -subq 6 -brdo 1 -me_range 21 -s 1920×1080 -r 30000/1001 -b 9216k -bt 9216k -bufsize 20000k -maxrate 25000k -g 300 -coder 1 -acodec aac -ac 2 -ab 128k “1080p-30.mp4″

* I am sorry for the language below, but I am very frustrated right now *

I f*cking hate software. It never, ever, ever, EVER works as it should be. After 7 hours trying to find an acceptable solution to this, I am so frustrated right now that I could smash whatever is in front of me.

So, as you remember from my yesterday’s blog post, I am trying to export 1080p for use with the Xbox360 and the PS3, from interlaced HDV videos. The only utility that came close but not quite, is SUPER. SUPER is just not as flexible as it should be, because it only allows for 29.97 fps encoding, while my camera also supports 24 fps too. Everything else seems to be ok with that app.

My previous efforts included using the freeware XViD4PSP and the very popular MediaCoder which failed miserably in many levels: from encoding errors, to wrong aspect ratios., to crashes (they all use ffmpeg, mencoder, x264 btw). So, I went the commercial route. I tried ArcSoft’s MediaConverter 2.0 which came with my hardware h.264 encoder stick, but it failed miserably with HD resolutions. I downloaded the new version of MediaConverter, 2.5 but this doesn’t get the aspect ratios correctly either (it doesn’t “stretch”, even if the “stretch” option is checked), neither it de-interlaces. Another commercial utility I tried which was possibly riddled with spyware but I had no choice but to try it, MP4Converter, wouldn’t encode at all, plus it wouldn’t go more than 3 Mbps which is too low for 1080p.

Want to go with QuickTime Pro? Slow-ass h.264 encoder (worse of all in terms of encoding speed so don’t ask me to buy FinalCutPro and a new Mac either), with no de-interlacing support and with keyframe problems (visible “ticks” every second or so in the video no matter what keyframe settings you set), as of with versions from since 6 months ago onwards. Apple stills sleeps over the issue.

PS3Video9 you said? Terrible interface aside, it won’t de-interlace, no matter if you add (as suggested) the -deinterlace ffmpeg option in the CLI. It’s like ffmpeg, to which PS3Video9 is a front-end, doesn’t take it into account at all. FFmpeg “writes your request in its balls”, as we say in Greece very accurately. Either that, or its de-interlacing sucks balls.

Overall, from my experience with at least 7 shitty utilities which for some reason are very popular, their main problem seems to be flexibility and open mind when they develop their software. They just don’t think too much about the problem they are trying to solve. If your video is already 4:3, good luck trying to stretch it. If it is widescreen, non-interlaced (which that would mean that this would possibly be a pirated movie/show), you will be in better luck. But if you have your OWN footage from your own HD camcorder (in other words, if you want to convert legal home videos), you are f*cked. It seems that people are creating encoding front-ends for pirated DivX movies rather than interlaced home footage (that is usually anamorphic too).

So, mediocre software, all over the place. I have not seen a SINGLE application that is able to “understand” a 1440×1080 HDV interlaced file (with the anamorphic flag set *or* not), and be able to transform it to 1920x1080p widescreen at the requested industry standard fps settings or 10 Mbps bitrate. There is ALWAYS something missing from each app, or something not working right. ALWAYS.

God damn it, I f*cking hate software. If this problem is not solved within 1-2 months, I will sell my camcorder. There is no f*cking point getting sunburns while shooting under the f*cking sun if I can’t enjoy my f*cking footage as I should be in a f*cking 1080p TV. SUPER does most of what I need except providing an option for 24fps, but thing is that I want to shoot 24fps with my camcorder and it’s shit like that that keep me from doing it. I am extremely frustrated right now. Things just don’t work properly. I have tried everything and this ordeal makes me have ZERO tolerance for software. > : (

Update: Finally, some half-assed progress on PS3Video9. Add this to your C:\Program Files\Red Kawa\Video Converter\Profiles\Video\PS3.xml .

<profile name="PS3-Eugenia" encoder="PS3 1-Pass - FFmpeg MI" position="0" duration="0" vcodec="MPEG-4 AVC" vprofile="Main Profile" vlevel="Level 4.1" vmode="ABR" vbitrate="8192" vquality="0" vquantizer="0" vwidth="1920" vheight="1080" vaspect="Original" vframerate="29.97" varenabled="true" varmaxres="1920x1080" varmod16="false" vartotal="false" vafenabled="true" vcroptop="0" vcropbottom="0" vcropleft="0" vcropright="0" vpadtop="0" vpadbottom="0" vpadleft="0" vpadright="0" vcli=" -deinterlace " vavsenabled="true" vavsautofps="false" vavsmanualfps="0" vavsconvertfps="false" vbufsize="20000" vminrate="0" vmaxrate="25000" vkeyint="300" vthreads="1" vcabac="true" acodec="AAC-LC" amode="ABR" abitrate="128" achannels="2" asamplerate="0" avol="100" acli="" />

Now, in order for ffmpeg to de-interlace, your source video file MUST be in the YUY2 colorspace. So, as per the tutorial yesterday, install the Huffyuv lossless codec, change its settings as described on one of the last paragraphs to use the YUY2 colorspace, and then use that exported .avi file to re-encode to 1080p using my profile above. Apparently, FFmpeg is extremely picky about YUY2 (which is not of the highest-quality btw) and it won’t even support that in some other lossless codecs that use that same colorspace (e.g. Lagarith). Not sure if the resulted file will playback on XBox360 (I will need testing), but it should play ok on the PS3. Anyways, at last, something that kinda works, I can definitely live with the YUY2 restriction as long as everything else works ok. It wasn’t easy to figure this out btw. It took 8 hours of trial and error.

Update 2: I talked too early. Apparently ffmpeg’s deinterlacer SUCKS. This leaves me with NO application that can create good 1080p h.264 videos out of an HDV interlaced/anamorphic source with reasonable fps/bitrate/aspect-ratio-resize flexibility. I talked to JBQ about it and he agrees that the features I need to work flawlessly are pretty basic and he said that what’s wrong is the lack of process that all these developers have. There were no product requirements ever specified when they had the “cool idea” to start developing an encoder front-end.

It’s official: Software sucks.

Update 3: Apparently, in order to de-interlace correctly with FFmpeg, the -deinterlace CLI option must be the FIRST argument in the ffmpeg command line! After I did that, I got a correct de-interlaced output! I hope the PS3Video9 developer fixes this.

I also downloaded and checked Adobe Premiere Pro CS3, only to find out that the trial version has no HD support (stupid Adobe, how can I test my hardware and my exporting options now?). However, by simulating the resolutions it seems that Premiere is able to create a correct HD h.264 output.

Update 4: So, to get it right, use ffmpeg from the command line. Install PS3Video9, just don’t use its UI (at least until the deinterlace bug is fixed). Copy your Huffyuv YUY2 interlaced .avi file on C:\Program Files\Red Kawa\Video Converter\Tools\FFmpeg\ and navigate there with a DOS command prompt. I use the follow commands for my projects, which should be relevant to all people with HDV and DV *NTSC widescreen* camcorder footage who want their videos to play nice on their EDTV/HDTV via the PS3 or the XBox360.

1080/30p (for 24 fps use “8192″ bitrate instead of “9216″ below):
ffmpeg.exe -deinterlace -i "C:\Program Files\Red Kawa\Video Converter\Tools\FFmpeg\XXXX.avi" -y -f mp4 -title "XXXXX" -vcodec h264 -level 41 -s 1920x1080 -r 30000/1001 -b 9216k -bt 9216k -bufsize 20000k -maxrate 25000k -g 300 -coder 1 -acodec aac -ac 2 -ab 128k "C:\Program Files\Red Kawa\Video Converter\Tools\FFmpeg\XXXX.mp4"

720/30p:
ffmpeg.exe -deinterlace -i "C:\Program Files\Red Kawa\Video Converter\Tools\FFmpeg\XXXX.avi" -y -f mp4 -title "XXXXX" -vcodec h264 -level 41 -s 1280x720 -r 30000/1001 -b 4096k -bt 4096k -bufsize 15000k -maxrate 16000k -g 300 -coder 1 -acodec aac -ac 2 -ab 128k "C:\Program Files\Red Kawa\Video Converter\Tools\FFmpeg\XXXX.mp4"

480p/30p:
ffmpeg.exe -deinterlace -i "C:\Program Files\Red Kawa\Video Converter\Tools\FFmpeg\XXXX.avi" -y -f mp4 -title "XXXXX" -vcodec h264 -level 41 -s 852x480 -r 30000/1001 -b 2048k -bt 2048k -bufsize 10000k -maxrate 11000k -g 300 -coder 1 -acodec aac -ac 2 -ab 128k "C:\Program Files\Red Kawa\Video Converter\Tools\FFmpeg\XXXX.mp4"

For 23.976 fps instead of 29.97 fps, change the “-r 30000/1001″ to “-r 24000/1001″.

The most cost-effective way to play your own HDV files as HD videos on your TV is either the Sony PS3 (or XBox360) or the Ziova CS505. With the latest firmware the PS3 supports playback of h.264 files up to 1080/30p. The Ziova CS505 goes up to 1080i for some filetypes, and 720p for others (no h.264 support btw). So basically, if your TV is 1080i and you don’t have lots of money, go for the Ziova product. If you have a 1080p TV, you have more money and more time, and you would find the Blu-Ray and gaming abilities useful, get the PS3.

I have ruled out the following ways to playback your personal HD files on your TV:
– A Media PC: can be more expensive than the PS3 before its specs can handle 1080p, takes too much space and if you run it with Linux instead of Windows it can be difficult to configure properly. It can be a flexible solution with lots of extra features, but it can also be a nightmare. Decide with care.
– AppleTV: Good price, nice interface, but it goes only up to 720/24p, and most HDV camcorders grab 30fps, not 24. AppleTV would be THE device to go, if and only if, Apple upgrades it to support full HD resolutions one day.
– HDV Camcorder: you will have to use tapes all the time and deal with 2-hour battery life each time. Yuck.
– HD-DVD/Blu-Ray burner and separate TV player: way too expensive, buggy burner software up to this day.
– DivX-enabled upscaling DVD players: *Most* of these models only have NTSC resolution DivX support, not HD. Don’t fall into the trap of their “1080p upscaling” marketing because that doesn’t usually apply for your own mpeg-2/4 files.

Now, here is how you could export your HDV home videos to HD, from let’s say, the $130 consumer NLE Sony Vegas Movie Studio 8 Platinum (VMS8-P), which is the best deal for the money compared to other consumer NLEs in terms of feature-set. VMS8-P is not the easiest to operate and it has its own quirks and bugs, but overall, I found it to be the best bang for the buck if you need flexibility and power. Most consumer NLEs don’t allow you to save on anything above 1080i-HDV (that’s 1440×1080/60i) and that’s ok for Ziova (export either as WMV or MP4 or MP2 in 1080i or 720p, Ziova should be able to handle it) but that’s not as optimal for the PS3 which is able to output 1080p progressively, so we will need to pull a few tricks and re-encode from 1080i-HDV resolution to true 1080p without loss of quality.

First thing to do is install the Huffyuv lossless codec and MediaCoder. The Lagarith codec would have been my personal choice instead of Huffyuv, but it is not supported by most freeware encoder front-ends, so we are going with Huffyuv for this tutorial. Edit your HDV video clips as you normally would on VMS8-P. When your video editing is finished, select “render as” from the “File” menu, select the .avi method and the “1080/60i-intermediate” template. Click “Custom”. From the “Video” tab there is a “Video Format” drop-down menu. From there, select the Huffyuv lossless codec. Encode your finished work using these settings and a huge.avi file will be produced. It is not uncompressed, but because it’s lossless, the filesize will be big (about 2 GB per minute of video length).

Open MediaCoder, load the newly created .avi file, and from the “Extensions” menu select the “Game Consoles / PSP”. Select “yes” to revert settings, select either the default MPEG4 or AVC versions (note: the AVC HD versions produced by MediaCoder don’t playback on Quicktime/VLC so you might want to go with the default mpeg-4 encoding option to be on the safe side), and then close the popup PSP window that MediaCoder has opened. From the “Video” tab at the bottom of MediaCoder’s main window change the bitrate to 8192 Kbps or more. From the “Picture” tab on the Resize option type 1920×1080, select your source frame rate (usually 29.970 on NTSC), Aspect Ratio 16:9. Then, click on “Effects” button below and from the “De-Interlace” option select “Linear Blend” (not sure if it’s the best option) and then click “ok”. Select 128 Kbps from the Audio tabs on the right by changing the “target quality” to “streaming CBR”. Then, start encoding. Depending on the speed of your PC and how lengthy your video was, you should be having a 1080p file now.

Instead of using MediaCoder, you could use XviD4PSP, which is more well-tested against the PS3. However, in order to force that utility to de-interlace, you first will have to click “configure” on the Huffyuv lossless codec setup under Vegas’ .avi export dialog, and tell it to use YUV2 instead of its default. Then, you export normally from Vegas and you load the .avi file on XviD4PSP. *If* it has black bars left and right of the image (if it shows up with wrong aspect ratio) then crop it by 240 pixels on the left and 240 pixels on the right. Then on the next screen you select the PS3 preset format, and you adjust the bitrate to 8192 Mbps (or more), resolution 1920×1080, Aspect Ratio 16:9. On the “Advanced” tab click on the “De-Interlace” checkbox and then click “ok” and then “Start” to encode. That resulting HD .mp4 file is compatible with the PS3, Quicktime and VLC.

You can burn a CD or DVD with these files, as normal files in the optical drive filesystem (not as DVD-video), or copy them to PS3’s hard drive or a USB stick or via networking, and then enjoy them on your PS3 and HDTV.

Update: Using the second method described above, I have created this 5 MB test 1080p file for testing with the PS3 or XBox360, shot with my HV20. Let me know if it works on your console (if you have the latest firmware installed).

Update 2: It seems that only the SUPER encoder will create valid –and with fewer headaches– 1080p files for the XBox360 and PS3. Works well with avi source files using Huffyuv btw, but it will only accept 29.97 fps as output, no matter if your source video is 24fps — which makes it useless for people with Canon HV20 cameras who shot in 24f mode.

Update 3: Read the Update on this blog post for the best way yet to convert interlaced HDV files to PS3/XBox360.