And so Canon announced officially their new line today. I don’t have much gripe on their AVCHD line because while not as good as the HV20 in several levels, I didn’t expect much either. But their new “Vixia HV30”, is a FUCKING JOKE. It is basically the same as the HV20, but in a black coat. Ok, ok, there are a few more differences, but they are not huge:

The HV30 allows for 30p recording.
The HV30 has a VIVID LCD screen.
The HV30 has a new kind of zoom button.
The HV30 has version 28 connection kit (HV20 has version 24).
The HV30 allows for longer record times with the new battery.

But it it still uses 1440×1080. And they don’t seem to offer true 24p either, it’s still PF24 on tape, and its 30p feature is wrapped inside a 60i stream too, it’s not real 30p. So much for great new features.

So basically, the “HV30” is nothing but “HV20.1”. It’s a joke. Canon cashed in like crazy with the HV20, one of the most successful camcorders ever, and instead of offering a camcorder that truly improves on the HV20, they laugh at us with these no-real-improvement product. It’s like they were afraid to add any of new real feature to not spoil their success. Or, they just got greedy and instead of releasing a new firmware version they created a new model.

As for their new AVCHD line of flash-based camcorders, I must say that I won’t touch them. They are dumbed down compared to the HV20/HV30 in many ways, including a smaller lens/filter thread, and their 1/3.2″ CMOS instead of 1/2.7 of the HV-series means almost no background blur!!! And we still don’t know for sure if these camcorders support real 24p without pulldown or not either. The only advantage they seem to have is that one of their modes allows full 1920×1080 recording (not sure until we see a review).

Honestly, in terms of “progress” over their previous line up, Panasonic’s new high-end consumer 24p/progressive camcorder seems much more “progressed” compared to their previous models, than any of these Canon cameras. Canon really disappointed me today. It’s like they don’t want the HV20 owners to upgrade. How idiotic is that?

I think I will wait for the RED MINI pocket camera and be done with that shit.

Many HD cameras shoot 24p these days, and while some consumer-grade video editors can deal with 24p, there aren’t many cheap DVD authoring applications that support 24p. For example, ‘Sony Vegas Platinum 8’ unofficially supports 24p timelines, but its accompanied ‘DVD Architect Studio’ application doesn’t. If you are shooting in 24p, it’s better to edit and burn a real 24p DVD and let the TV add pulldown during playback, rather than adding pulldown during the DVD authoring process. Besides, there are some new playback devices in the market these days that can output true 24p to 24p Sony TVs, without adding pulldown, so these cases while rare, can benefit from pure 24p DVDs. Here’s how to go around DVD authoring limitations and produce a 24p NTSC DVD:

1. Download and install the latest “nightly by clsid” ffdshow build.

2. Capture your footage the way you usually do. If your camera does not capture as true progressive in the 23.976 frame rate, but instead it uses something like Canon’s PF24 or some of Sony variants, you must remove pulldown (methods for the HV20 here, or for Canon’s AVCHD line here).

3. Then, bring your footage into your video editor, just make sure your editor does support true 24p editing (e.g. VMS Platinum). In the “File”/”Project Properties” select 1440×1080 size, frame rate of 23.976 (type it if there’s no such option), progressive field order, aspect ratio of 1.3333, rendering quality “best” and “none” for de-interlacing method. Then, edit as usual.

4. When your editing is done, you export in an intermediate format. Click “File”, “Render As”, select the “avi” type and its “HDV 720-25p intermediate” template. Then, click “custom”. In the “video” tab of the dialog that pops up select “23.976 (IVTC Film)” for frame rate, and then from the video format menu select the “ffdshow video codec”. Click “configure”. From the newly created dialog select the “encoder” tab, and from the encoder menu select “Lossless JPEG” and “YV12” for its Colorspace. Click “Ok” to close that dialog. Click “Ok” to close the other dialog too. Then, hit “Save” to start encoding the .avi file.

5. Install the latest “DVD Flick” version (as I write this, the latest beta version can be found here) and then load the application (regularly check for new versions of this app). Click “Project Settings” and go to its “Video” tab. There, select “NTSC-film” as target format. From the “Burning” tab you can instruct the application to burn a disc at the end of your authoring, or just create the DVD/.iso files without burning. Then, load your LJpeg .avi file(s) you exported from your video editor into DVD Flick. Read the DVD Flick manual to learn how to author DVDs with this application. It is a simple application to use, but it doesn’t have enough templates and beautifications. Save often too, as the application is not super-stable either, but it’s the best we got to do this job for free. After you are done authoring, you can click the “Create DVD” icon to burn or just create an .iso file. Enjoy!

Some notes on this method:

* I don’t use 1440×1080 to export via the video editor on step #4 because the kinds of .avi files that VMS produces are not recognized as widescreen by other applications. This results in DVD Flick having vertical letterbox bars, and that’s not what we want. Besides, downscaling first to 720p and then to 480p is not very lossy to make you worry about it.

* If your 24p camera is a DV one instead of HD, export in 874×480 (progressive, 23.976 frame rate, aspect ratio 1.000), instead of the suggested 720p resolution on step #4 (use the same codec as suggested though). If your camera is the DVX-100 export in 848×480.

* I am using LJpeg instead of the Huffyuv codec in this tutorial because the mode of Huffyuv that produces smaller files uses a colorspace that DVD Flick does not support. As for the Lagarith lossless codec, or Cineform, they are not supported by DVD Flick either, so your best bet is LJpeg (the FFv1 ffdshow codec could be a workable idea, but it’s slower to encode that LJpeg).

Samsung and Panasonic have announced their new camcorder line up, and Canon’s already have been leaked too. The new trend is flash storage and full progressive 1080/30p recording. Panasonic goes the extra mile to include true progressive 1080/24p too, while Samsung can capture 300fps for 10 seconds which yields very “clean” slow motion. We know no tech specs about the new Canon cameras yet, but we know that they went flash-storage too. Sony didn’t offer major new ideas/features in their new lineup.

I am very happy to at last see true progressive recording, no more wacky aspect ratios, no more interlaced crap for consumers (50% of them don’t know how to de-interlace and then they export interlaced to sites like vimeo and so their videos end up looking like crap), and of course 24p.

However, Panasonic did not go the extra mile to compete with the HV20, and Samsung didn’t seem to even try either. Just because you can shoot progressively and 24p does not make your camcorder a better device than the HV20. Not only because AVCHD is still not as good as HDV in quality, or because 3CCDs are so small in these consumer cameras that yield essentially no background blur compared to 1/2.7 CMOS of the HV20, but also because there is no manual focus ring, no filter threads, no manual controls in hardware buttons.

This reminds me a lot of the efforts of Samsung and LG to compete with the iPhone last year. Instead of actually replicating all the existing iPhone features plus adding more new features, they just kept their old software feature-set and just gave their device a big ass touchscreen. Well, that’s not enough to compete with the iPhone, and in the same way, Panasonic’s solution is not a better deal for the hobbyist/indie filmmaker than the HV20 is.

Personally, I will still wait for this. If Canon won’t deliver that, no one will.

Update: This Casio digital camera looked very promising for a moment, but then I read that it has no image stabilization, not conversion lens support, and not enough background blur (same as in the HV20). This is why followers are not leaders in the market. Because they make a lot of buzz about a single feature that no one else has, but they forget to take care of the basic stuff. Stupid Casio.

I just watched a special broadcast of the “Terminator” series that FOX will air next week. It sucks. It feels cheap, boring, and direction was bad. It felt a lot like the “Bionic Woman” actually. Avoid.

So today it marks the day that Blu-Ray has won for good. Warner, the largest distributor of movies, has decided to go exclusively with Blu-Ray, while so far they supported both formats. The shift will be huge towards Blu-Ray, and coupled with the fact that Blu-Ray was already winning even without Warner’s help, should put these stupid BD-vs-HDDVD wars to rest. If you didn’t know which player to buy so far and you waited for a sure winner, BD it is.

Update: This seems to suck for Toshiba, they just canceled their CES conference, and I smell a lawsuit.

Canon’s consumer HD cameras have 24p support, only that it’s not a… truly true 24p. It’s PF24, which incorporates both progressive and interlaced frames in a 60i stream. To get the pure 24p stream out of it, you need to do a “pulldown removal” (aka “inverse telecine”). PAL users do not need to do this because their PAL cameras don’t have a PF24 mode — this burden is only for NTSC users who manually chose to shoot in the PF24 mode. If your final output is the TV (e.g. via DVD or the camcorder itself), and as long as you never de-interlace that footage during editing/exporting, then you don’t need to remove pulldown. But if your output is not a home DVD/TV but the web, or a PC, or other non-TV viewing device, then you better get to work and remove pulldown because your footage will have an ugly “ghosting” effect whenever there is motion on the screen. Here are pulldown-removed images before & after, showcasing the problem and its fix. Here’s another example showing the problem.

There are several ways to remove PF24 pulldown for the Canon HDV cameras, but their AVCHD cameras (e.g. HG10, HR10) did not enjoy the same fate, until now. If you are using the latest versions of After Effects or Final Cut Pro, then you can use the tutorials linked to remove pulldown (although I have not confirmed that these tutorials will work for AVCHD formats as well). On the PC side, the cheapest, fastest and cleanest way to do it, would be to purchase the Cineform NeoSCENE tool for $129. Highly recommended.

However, there is another way, a freeware method, as long as you already have a video editor that supports the AVCHD format, DirectShow/VfW and 24p (e.g. Sony Vegas/VMS, AE/Premiere, Avid etc). Please note that Premiere Elements, Pinnacle, Magix Movie Edit Pro and Ulead don’t support true 24p so avoid these for 24p editing. Premiere Elements can do 24p editing only after installing a 24p Premiere Pro template file from Adobe’s site. The following method is an involved procedure to follow and it requires some very basic usage of the DOS prompt (found in Windows’ Accessories program menu as “MS-DOS”), but if you have no money and you need it done, it should work fine as long as your clips are not too long (tested with HG10’s PF24 files). Longer clips (over 3 minutes or so) might crash the mencoder utility used below, or create A/V sync issues. If you really need this done the proper way, use AE/FCP, or buy Cineform NeoSCENE.

Method 0 (for Vegas users only)
Read about it here. Come back to this article to use another method if this one is not good enough for you.

Method 1

1. Install Avisynth.
2. Download and unzip the TIVTC v1.05 plugin. Inside that zipped folder, you will find a file called TIVTC.dll. Drag-n-drop that .dll file on the C:\Program Files\AviSynth 2.5\plugins\ folder. Vista/XP 64bit uses a different path.
3. Unzip ffmpeg somewhere.
4. Install Avid’s DNxHD PC codecs at the bottom of this page, (or Huffyuv).
5. Download and install the latest ffdshow version.
6. Unzip the Batch Intermediate Creator somewhere (thanks goes to Khaver, read discussion in that link).).
7. Load its ffmpeg-bic.exe application and fill up what it needs to know, like in my screenshot here.
8. Click the “ffdshow video decoder configuration” and make sure the “deinterlacing” checkbox is checked. Also, make sure that ffdshow is the default .m2ts/.mts decoder in your system (you can set the h.264 codec in the ffdshow configuration panel to “ffmpeg-mt”). Otherwise, it won’t work correctly.
9. Add some PF24 files, and start the process. This will create either Avid DNxHD .mov files, or Huffyuv .avi files at 23.976 fps, progressive. I would suggest you go with the Avid DNxHD codec.

The setup as shown in the screenshot also creates proxy files — smaller versions of these big 24p intermediate files that are too slow to edit. Here’s how to use proxy files with Vegas. It is possible to tweak that proxy editing tutorial for other 24p-capable editors too. You can uncheck the proxy-related checkboxes to not create any proxy files.

This method works with *some* AVCHD formats from some manufacturers. Others crash. You will have to try to see if it will work with your AVCHD format or not. If they don’t work, use method 2.

Method 2

1. Download the plain version of Mplayer for Windows and unzip it somewhere that you can find back easily. Inside that unzipped mplayer folder, create a folder named “input”, and a folder named “output”. This action only needs to be done once for your system.

2. Download the Huffyuv lossless codec. Unzip the .zip file on your desktop, right click on the huffuyv.inf file and select “Install” (note: Vista might not give you that option, in which case manual registration of the codec’s DLL file must happen via the DOS prompt, google it). After about 15 seconds, you can delete these files from your desktop, as the Huffyuv lossless codec is now installed on your system. Open a DOS prompt and navigate to the mplayer folder (e.g. “cd C:\Mplayer\”, without the quotes). There, run this command: vfw2menc -f huff -d huffyuv.dll -s settings.mcf and on the new dialog that pops up select: “Predict median (best)” from the first drop down menu, and “<-- Convert to YUY2" from the second drop down menu, while leaving unchecked the other options. Then click "ok" to discard the Huffyuv configuration dialog. This action only needs to be done once for your system. 3. Download the Lagarith lossless codec. Unzip the .zip file on your desktop, right click on the lagarith.inf file and select “Install” (note: Vista might not give you that option, in which case manual registration of the codec’s DLL file must happen via the DOS prompt, google it). After about 15 seconds, you can delete these files from your desktop, as the Lagarith lossless codec is now installed on your system. Open a DOS prompt and navigate to the mplayer folder (if it’s not open already at that folder). There, run this command: vfw2menc -f laga -d lagarith.dll -s settings.mcf and on the new dialog that pops up select: “YUY2” from the “Mode” menu, and “Use Multithreading” if your CPU is hyperthreaded/multi-CPU or it has more than one Core (leave unchecked if you don’t know). Then click “ok” to discard the Lagarith configuration dialog. This action only needs to be done once for your system. Note: Do not change to YUY2 if you are using Adobe’s editors, as they don’t understand that format.

4. I will work with Vegas on this step because that’s what I use, but it should be similar for Premiere/AE/Avid. Open Vegas and load your PF24 .m2ts or .mts AVCHD files in the “Project media” bin. Make sure that these files were indeed recorded as PF24 (don’t mix 60i and PF24-recorded files), and that your hard drives are formatted in NTFS. Then open the “Project Properties” dialog and use the HDV 1080/60i template from the drop down menu. Now you have two options, either pull all your scenes at once in the timeline to export, or export them one by one. The first method does not require your attendance, but it will produce one large file that will have no scene detection, while the second method will produce lots of per-scene files, but it will require that you be in front of your computer quite often to export the scenes one by one. Think hard of what your needs are, and make your choice. I would personally recommend you export the scenes one by one mostly because this way you can avoid the possibility of the audio and video getting out of sync (plus, MEncoder is known to not handle huge files too well either).

5. Once the clip(s) are on the timeline, hit “File” and “Render As”. Select .avi for the “Save as type”, and the “HDV 1080-60i intermediate” template. Then, hit “Custom”. On the first tab select “Best” quality. On the video tab leave everything as is except the video format, select there “Huffyuv v2.1.1”. Then, hit “Configure” and a new dialog pops up. There, make sure that “Predict median (best)” from the first drop down menu, and “<-- Convert to YUY2" from the second drop down menu are selected (leave unchecked the rest of the options there). Then click "ok" to take that new dialog away. Leave unchanged the "Audio" tab, and then hit "ok" in the "custom template" dialog. Then, you must give a name to your .avi file. If you chose the "one big file with no scene detection" method give it any name you want (as long as there are no spaces), but if you chose to export the scenes separately then use the same filenames as in their original AVCHD files, but with the .avi extension this time, and with no spaces in the filename. Export the files in the "input" folder you created inside the Mplayer folder. 6. After all the exporting is done, open a DOS prompt, navigate to the Mplayer folder that has the mencoder.exe file in it, and then run the following command, substituting each time the right input/output filenames with your own video filenames: mencoder input/INPUT_FILENAME_001.avi -aspect 16:9 -fps 30000/1001 -oac pcm -vf detc,scale=1440:1080 -ofps 24000/1001 -ovc vfw -xvfwopts codec=lagarith.dll -o output/OUTPUT_FILENAME_001.avi
If you are proficient with DOS scripting, you can even write a script that will automatically read all the files on the “input” folder and export them one by one without requiring your attendance. Phoric wrote such a script and sent it to us (save it down and put it in the mplayer folder before you execute it). Thanks Phoric!

7. After MEncoder is done re-encoding all your files, you can choose to delete or move away the .m2t files on the “input” folder (so they don’t mix with video files you will place there in the future), while all the .avi files found on the “output” folder are now ready to edit! Windows Media Player won’t recognize these files as widescreen, but don’t worry, they are widescreen: Vegas will recognize them as such (if not, just set manually the clips’ aspect ratio to 1.3333). Create a new project in Vegas (or any other 24p-enabled video editor), select again the HDV 1080/60i template, but change the following this time: Field Order should be “progressive scan”, frame rate should be “23.976” (type it if it’s not in the list), rendering quality “Best”, and de-interlacing method “none”. Then, edit as usual and enjoy true 24p!

NOTE: Trevor Marshall suggests an alternative method (not tested by me, might have an audio sync issue).

1. Download, unzip and drop the MEncoder files for Mac OS X on a folder of your choice, e.g. on one that’s called “mplayer”. Create two folders in that folder, one called “input” and one called “output”. This action only needs to be done once for your system.

2. Use a Quicktime-enabled application (any will do, as long as it has access to Quicktime’s exporting dialogs) to export your AVCHD files as MJPEG/PhotoJPEG inside the “input” folder. MJPEG is not exactly a lossless codec, but it’s the only high quality near-lossless codec that comes out of Quicktime that MEncoder will support.

3. By using OSX’s Terminal.app, navigate to the mplayer folder (it requires basic understanding of Terminal usage), and then use the following MEncoder command line to remove pulldown and export in Huffyuv, like this:
./mencoder input/INPUT_FILENAME.mov -aspect 16:9 -fps 30000/1001 -oac pcm -vf detc,scale=1440:1080 -ofps 24000/1001 -ovc lavc -lavcopts vcodec=huffyuv:pred=0:format=422P:vstrict=-1:aspect=16/9 -o output/OUTPUT_FILENAME.avi
You need to manually replace the filenames for INPUT_FILENAME and OUTPUT_FILENAME for each of your files.

4. After all the encodings are done for all your files, you can safely delete the files inside the “input” folder. Your final, true 24p files to be used with editing can be found on the “output” folder.

5. Install the “Perian” utility which will enable Quicktime-enabled video editors to read and edit these created Huffyuv .avi files. This action only needs to be done once for your system.

If you are proficient with Bash scripting or AppleScript, you can write a script that will automatically read all the files on the “input” folder and export them one by one using MEncoder without requiring your attendance. If you write such a script, please send it to me to incorporate it in this tutorial and help others. Please note that iMovie and Final Cut Express don’t support true 24p editing, so if these are the only video editors you got, then either don’t bother removing pulldown, or buy a PC to do the job (it will be cheaper to buy a fast PC with 2 GB of RAM and Vegas Platinum and use that as a dedicated video station, than buying Final Cut Studio or After Effects for the Mac).

There is no way to apply this method under Linux because MEncoder crashes when you try to read an AVCHD file, while ffmpeg has a known bug in the reading of such files that results the video to export twice as long as it should. As of this writing, no Linux library or application can read AVCHD files properly.

1. Admittedly, this is a slow way of doing things. For 1 hour of footage, it will take between 8 and 10 hours to complete the various exports, depending on the speed of your computer. If time is money for you, buy Cineform’s NeoSCENE. If not, let it run overnight. A trick you can do to speed up the various exports is to have the input and output files on different hard drives. For example, when exporting the AVCHD files in the beginning, have the AVCHD files on one drive, and export into the “input” folder of MEncoder on another drive. Then, while using MEncoder, encode the “output” files back to the first hard drive again. This can save up to an estimated 15% of time in the overall process because the hard drive’s head won’t jump like crazy back and forth to service both the input file reader and the exporting encoder, but instead each hard drive will dedicate itself to a single action.

2. You will need a lot of storage. We are talking about 180 GBs per 1 hour of footage, because there are two stages of Huffyuv/Lagarith encodings going on. After the export to the “output” folder is done, you can delete the files found on the “input” folder, in order to save some storage space. Always keep around the original AVCHD files though.

3. Editing Huffyuv or Lagarith files can be pretty slow, because their filesizes are so large. You can choose to edit them directly and be done with it, or export again in Cineform 1080/24p/progressive through Vegas/VMS’s .avi custom codec dialog, and then edit these Cineform files instead. It’s pretty fast to re-encode in Cineform, its format is fast to edit too, and because all these codecs are visually lossless you won’t see much of quality degradation after all these exports. Another way to deal with this problem is to use proxy files (faster way). But I leave this decision to you as to in which format you want to edit with, because exporting and re-exporting until you reach a fast codec can be nerve-wrecking.

4. Vegas Platinum 7/8 won’t export in full 1920×1080 (limited to 1440×1080 export size), so you will need to use either the “SUPER” or “Handbrake” freeware utilities, to go around the problem and export in full 1080/24p. Vegas Platinum 9 and Pro don’t have this limitation. If you are ultimately exporting for Vimeo’s glorious 720p HD web service, check my tutorials here.

FIC upgraded the hardware of OpenMoko, and some nice upgrades are in. However, they still didn’t go for a quad-band phone, with EDGE. It’s still triband with plain GPRS. These guys really need to get a clue. Just because in Asia EDGE is not very popular doesn’t mean that it’s not in the rest of the world — their focus is the whole world anyway.

Click the picture to see before and after color grading shots from my jellyfish video.

With the recent release of v2.4, Avidemux could have been one of the few apps that could support pulldown removal for PF24. I have been emailing its developers the past 6 months for over 10 times to fix two important bugs, but they ignored me. The bugs are:
1. If you use audio shifting of -222msec while you use pulldown removal filters for the video, the audio ends up being 220% longer than the video.
2. The h.264/AAC .mp4 files created by Avidemux don’t playback on Quicktime.

For the first bug, they ignored me because obviously it’s simply a hard engineering problem. They just don’t seem to want to work hard. They probably have a fucked up architecture, and they don’t want to touch it and potentially make it worse.

For the second bug, they replied “it’s Quicktime’s problem”, failing to understand that iTunes/QT has 80% of the market and it does not matter whose problem it is. If you can’t playback your videos what’s the fucking point of using Avidemux to encode them? The irony is that they could easily fix that on their end, they just don’t want to (apparently the bug is in their implementation of the MP4 container).

These open source guys really need to get a clue sometimes. I like and endorse the open source ideals for the benefit of the user, it’s just that I HATE its development model.

I have come to realize that I am not an artist. By “artist” I mean the ability to take something ugly, boring or not important and make it look attractive, abstract or just “different”. I can’t do that. For example, look at Fabian’s video here, he’s obviously an artist. I could never create something like it. If someone gave me a storyboard and mockups, I could easily put such a video together at the *technical* level. I know how to recreate it, but I can’t initiate it.

I think the reason why I am not an artist is because I am a hard harsh realist. If something doesn’t make “logical sense” (visually or not), I just don’t use that footage. I discard it. And yet, that same footage can be a work of art at the hands of an artist. Although not everything is lost: I believe that I can put documentaries or other realistic material together pretty well, and I believe I can go as far as creating a presentable music video clip. Maybe even a simple short movie. But creating truly “artsy” stuff, that, I can’t do.

Gnothi seauton.