Archive for June 29th, 2008

Basic Canon HV20/HV30 knowledge

There are a lot of new HV users who get information overload on their brains about basic knowledge regarding these cameras. Greek fellow Socrates asked me for a basic tutorial, so here it is. Here’s a non-scientific, as simply-written as possible, explanation article:

HDV 1080i: The HV series are using the HDV mpeg2 format to record (rather than the competing AVCHD mpeg4 format). HDV does not record in full 1080p, it records in 1440×1080 with aspect ratio 1.333 (yes, that’s still widescreen HD, and the HV picture quality makes up for the loss of pixels).

HDMI capture: There is a way to hack around the 1440×1080 recording limitation and record at higher quality at full 1920×1080 (and without mpeg2 visual artifacts), by using the HDMI output port instead of firewire to capture. You basically use a PCI card with an HDMI input port to capture. It costs a few hundred dollars, but it’s simply impractical for normal users to do capture like this as it requires a full PC to be in the shooting site at all times, so please stop dreaming about it.

Aspect ratios: As I said above, the HV series record in aspect ratio 16:9, and pixel aspect ratio 1.3333. This means that the horizontal resolution, which is 1440, when multiplied with 1.3333, it gives us 1920 pixels wide, which is the right HD resolution (and that’s why it still looks widescreen even if M2T format reports a 4:3 pixel count resolution at 1440×1080). More info on aspect ratios here. Aspect ratios are an advanced topic that confuse most users.

M2T: the file extension that the HV series use when capturing from tape when using a non-Apple editor (Apple’s editors tend to re-encode during importing on another format and not edit natively in M2T). M2T is an mpeg2 type format, at 25mbps bitrate. When viewing the video in 1:1 size on a large monitor, some mpeg2 artifacts are visible if looked carefully.

NTSC: The Japan, US and Canada (and a few other countries) TV format. NTSC uses 60 frames per second. The NTSC version of the HV cameras record by default in 60i, meaning 60 interlaced frames per second.

PAL: The rest of the world’s TV format. PAL has quite a few iterations and versions depending on the country, but most of them use 50 frames per second. The PAL version of the HV cameras record by default in 50i, meaning 50 interlaced frames per second. Explanation of what a frame is in the first place, here.

Recording formats: The NTSC HV20 cameras records in 60i (default), PF24 and the HV30 adds also the ability to record in PF30. The PAL HV cameras do 50i (default) and PF25. The PF24 mode (also known as 24p), only available on NTSC HV cameras, is not true 24p, because of compatibility problems with the HDV standard that didn’t have provisions for 24p support. So Canon created a hack to go around the HDV limitation: record in 24p but with pulldown addition (extra frames in a stream that “fools” the camera and the video editor that the video is really 60i and not 24p). To get the true 24p out of this mess and edit as true 24p (otherwise you will get ghosting artifacts), you need to perform a pulldown removal. PF30 and PF25 users should not need to do anything extra other than telling their video editor that these are progressive formats (so on Sony Vegas for example, you use the supplied 1080i HDV template for your region, but you change the field order to read “none/progressive”, and deinterlacing to “none”). As for the default 50i and 60i recording modes these are the most common kinds and all HD-enabled video editors support them. And to make it clear: PAL HV users do NOT need to remove pulldown, only NTSC users who have switched their cameras to record in PF24 need to do that (and only in a few cases, read the link I provided about it).

Interlacing: The HV series record in an interlaced format by default, meaning, that it has some ugly horizontal lines by default. Unless you are burning a DVD or you are writing back to the camera or you are re-saving for archiving purposes back to the same M2T format, you need to de-interlace your video (in other words, to make it progressive). TV-viewed footage doesn’t require to be de-interlaced/progressive, as TVs can actually do de-interlacing on their own, during playback. Check this picture to see how the first image which is interlaced with ugly lines looks like, while the rest two pictures using two de-interlacing methods to make the footage “progressive”. Remember: there is no perfect de-interlacing algorithm (that’s why we have so many of them, each one thinking they can do better than the other one — long story). That’s why in the future always strive for a camera that can also record in true progressive mode, like the HV30 can with its PF30 mode. The PAL HV cameras also record in true progressive at 25p, but the NTSC HV20 does not record in true progressive, as the PF24 (24p) mode has some of its frames in interlaced format (only 18 out of the 24 frames are progressive).

In which mode should you record: Generally speaking, use the defaults: 50i and 60i. These modes while interlaced, they offer the best frame rate, which means smoother movements. So it’s ideal for sports and casual life events that end up on a DVD. But if you are going to export for the web/PC only and not DVDs, or you are doing artistic videos or real movies, then you can start considering 24p, 25p or 30p, which are frame rates that real movies use. However, these modes are not very smooth in motion, so you must be very careful how fast you move your camera.

Firewire: To read the video from the tape and transfer it to your computer, you need to have a “firewire” port (also known as IEEE1394). Most PCs don’t have a Firewire port, so you must make sure that you either have one, or that you can buy a cheap PCI firewire card to add such a port. All Macs have a firewire port. You must also buy a cable, as the HV box doesn’t come with a firewire cable. You can use your video editor to capture the M2T video from the tape, or the freeware HDVSplit, or if you are also need to remove pulldown in case you recorded in PF24, you can use Cineform’s HDLink tool which does both jobs in one step (costs $200 and up, there’s a special offer for HV users in particular as the company is HV-friendly because they recognize the huge hobbyist filmmaking community around these cameras).

Editing speed: You need a minimum of a Pentium 4 class CPU at 2.8 Ghz for PCs, or a dual 1.2 Ghz G4 or above for Macs. 2 GBs of RAM or more (users with 1 GB of RAM will find their operating system swapping sooner or later and things will get really slow). Get a big hard drive (for best performance get two hard drives, one holds the OS, editor and temp files while the other one holds all the captured footage). XP is usually recommended over Vista as many of the utilties used for the freeware pulldown removal method work best with XP. One last thing: don’t confuse a “pentium 4 CPU at 2.8 Ghz” with a “Core2Duo 2.8 Ghz”. You see, even a CoreDuo at 1.6 Ghz will be faster than a Pentium 4 at 3.2 Ghz! So it’s not the Ghz you should be looking at, but the generation of your CPU. Generally speaking, any PC sold 2006 onwards is good enough for HDV editing speed-wise. Just make sure you got enough RAM and a high resolution monitor (over 1280×1024).

Cinemode: When switching the camera to manual mode, you have the ability to use the “cinemode”. Cinemode is a collection of settings that Canon put together (shutter speed, color, gamma, sharpness, aperture etc), to make the look of your video to look more like the expensive film cinema cameras. This mode, especially when used with the “neutral” color setting, is not meant to simply be used as is. It’s meant to be color-graded during editing. The color grading behaves better with the Cinemode/neutral settings ON.

35mm adapters: One interesting way to extend your HV camera (apart from the Canon offered add-on microphones, telephoto and wide-angle lenses), you can also get a 35mm adapter. When using this adapter with photography lenses, you can get a “shallower depth of field” than the HV cameras can provide out of the box. Shallow depth of field means that you can focus to a person or object, and have the background being all blurred out in order to look more “cinematic” or “artistic”. That’s the main difference in the “look” between big Hollywood movies and cheap home video or cheap TV series. Visual example with a stock HV20 here, and with a 35mm adapter and lens here. A 35mm adapter with a lens costs anywhere from $200 (non-vibrating version) to $1300 (vibrating version with achromat addon).

Exporting formats: If you are exporting back to tape, export in the same M2T format that the HV cameras capture (1440×1080, interlaced). If you are exporting for DVD, export with the supplied templates your video editor supplies for DVD. If you are exporting for archiving reasons or to exchange footage with another person, export in a lossless codec that doesn’t lose quality when encoding (also known as “intermediate codecs”). If you are exporting for Vimeo, youtube, 720p, 1080p or other “casual viewing” reason (also known as lossy “delivery codecs”), export in one of these ways (it must be de-interlaced, usually at resolutions 1280×720 or 1920×1080 with pixel aspect ratio 1.000). The difference between intermediate and delivery codecs is explained here. Remember, when using a delivery codec, each time you re-encode the resulted file, you lose quality. So always have as few “jumps” from format to format to ensure best quality as possible. For best results edit directly the M2T format (or Cineform AVI if you removed pulldown for PF24), and after editing export once in a lossless intermediate codec for your archiving reasons, and once in a lossy delivery codec for casual future viewing.

Finally: General advices on how to shoot for best results, here.

Email me directly or IM me if you need more information or explanation.

From Windows to the Balcony OS

NYTimes wrote an article discussing the possibility of Microsoft writing a brand new OS that is not Windows-based.

First things first:
1. I am a huge proponent of backwards compatibility because in my agenda, the user comes first.
2. Writing a new, truly modern and revolutionary, OS will take Microsoft 10 years (including its maturitization time). Drivers will be scarce, adoption will be a pain (I know the drill, we’ve lost a green card because of it with BeOS).

Despite these huge undertakings and problems, I don’t think Windows has any life on its knees anymore. It’s a 25-30 years old architecture, patched all over. Vista sucked big time — at least in the UI level. I believe that Microsoft has only two options:

1. Use these new ideas they have in their R&D dpt, for a future modern OS and start writing a new OS as soon as possible, in PARALLEL to upcoming Windows 7. All previous versions of Windows should run through virtualization, first releases should include a Windows 7 virtualization version for free.
2. Forget the OS business. Keep supporting Windows after Windows 7 for 3-4 years, and then keep supporting it for specific PCs only, for a fee, for another 20-30 years (like Sun does for Solaris). Move to different business.

I wrote the other day that Gnome should do the same: re-invent the wheel. Not because Gnome 2.22 is bad, but because when you see the bigger picture, with Linux, X11 and all that shit that fuck us users for 15 years now, I believe there’s a need for a clean slate for Linux’s desktop in general too. I just saw Gnome as the project which will bring that change, rather than Gnome itself needing that change (I hope I am making some sense).

Same way with Windows, it’s just so much craft in there, that’s just painful. Either take the big decision and put billions into the development of a new modern OS, or get out of the OS business and just keep lightly supporting the last released OS.

If Microsoft won’t do that, then their OS division will die a SLOW death. It won’t be pretty for any project manager in that division.

UPDATE: Interesting.