Video RAW

Caitlin sent me an interesting website showing off the Andromeda Video Acquisition System.

Basically this system purports to take a 3-CCD DV camera and turn it into a high-definition RAW video stream. Note that this only works with the Panasonic AG-DVX100, which is a DV camera.

The AG-DVX100?

The AG-DVX100 is probably a camera you haven’t heard of. The reason is because a few years back the video world split into two camps: JVC and Sony felt that professional videographers would want high definition video and Canon, Panasonic and the rest felt that they would want a “film-like” 24 frames/sec (instead of 30 that they currently did) and invested in that.

High Definition video and the ensuing “HDV” standard—that allowed you to put high definition video onto a DV tape with a few compromises—won out and 24p cameras such as the AG-DVX100 were quickly replaced with the Panasonic AG-HVX200 which could output DVCPRO HD data and more framerates than you can shake a stick at.

DVCPRO HD is a little-to-no compromise high definition video standard with one major weakness: the bandwidth is too large to fit on DV tape. This means that if you own a AG-HVX200 you can only record HD to flash memory or a hard drive out. I’ll give you a funny use-case you can do with the AG-HVX200: you can record HD video and have it down-sampled to DV in-camera for writing to tape.

Essentially you are recording DV which is inevitably what many “high definition wedding videographers” who use this camera are doing. They’re selling it as high-defintion coverage, but the person is getting DV recording. What is the purpose? Once you are in DV, there is no way to recover the high definition recording.

DV recording sucks because the DV codec was invented in the early 90’s. To understand the consequence of this, my Palm Tungsten T|X handheld has about 5 times the processing power of a top of the line desktop PC of that era. Because of the limitations of computing power in consumer electronics of the time period, DV is a straight 5:1 down-sample of the incoming video stream, producing artifacts that are even visible to a casual observer. This is one of the major reasons why real high-definition or HDV recording will look better on your DVD than DV recording, even though the nominal resolution of DVD is no better than DV.

It is sad that an otherwise great camera is being abused by professional videographers in this manner. Sadly, the AG-AVC200 is a ENG “sound byte” camera or a indy filmmaker camera, not a wedding videographer or documentary camera. This is why Caitlin doesn’t use this model for wedding work.

How do you get high-definition from DV?

If the resolution of DV is 640×480 (or 640×372 in 16:9) and the resolution of HD is 1280×720 (or 1920×1080i if you ignore that the pixels aren’t usually square and the video is interlaced so it’s really usually 1280×540 at twice the frame rate), how is it possible for a DV sensor to make high-definition video?

Well they are doing it with a variation with the way your digital camera works and the same way the AG-HVX200 works.

Huh?

Well you know that if you have a 6 megapixel sensor, you can get a 6 megapixel photograph from it. But did you realize that in front of each sensor is a filter that causes it to only record red, green, or blue? Since you need a red, green, and blue data to create a color photograph, your real resolution is 2 megapixels of color, not 6!

How does it work? Well your camera uses the fact that the pixels are offset for you to extrapolate their effective color. The math to do this works very well and you end up with a 6 megapixel shot. But doing this math can introduce visual artifacts that are visible in your photograph: color moiré on fine textures being the most annoying one.

In a 3-CCD video camera, a prism exists where a mirror (film or digital SLR photo camera) or spinning mirror (film cinema camera) would exist. This sends red, green, and blue in different directions. So you have 3 sensors recording the data independently.

Now a nice little quirk of the Panasonic AG-DVX100 (and leveraged in the AG-HVX200) is that these sensors are spatially offset from each other. So you can combine to generate a much larger resolution image than the sensor would normally allow.

In the case of the AG-HVX200, this allowed you to extrapolate up to 1920×1080 image using a 960×540 sensor. This was done because Panasonic had invested resources into a variably timed CCD sensor (for 24p video) instead of, like Sony and JVC, investing resources in a higher resolution one. (If you are wondering about the uber-expensive rip-off known as the Canon XL-H1 can exist when Canon was in the 24p camp the answer is they are using a Sony sensor and then driving it with a Canon processor—having been caught with betting on the wrong horse, they too had to find an interesting compromise to keep up.)

Well it looks like these guys over at Reel|Stream have figured that if you can do that trick with the HVX200, then why not turn the sensor of an DVX100 into a 1540×990 camera? That’s better than HD for progressive frames and can up-sampled to 1920×1080 video.

Cool, huh?

Okay, so what is the catch

There are two and they’re pretty fun. The first is this upgrade is $3000. This puts the camera at $6000 street price. But before you pee your pants, remember that high definition cameras of this quality cost around $6000. This sort of stuff isn’t for the light of wallet.

The other one is that you have to bypass all 4:2:0 (or 4:1:1) sub-sampling¹ and the 5:1 DV encoding by piping the stuff through a USB 2.0 port somewhere. Where? A computer. This means you need to tether this to a computer to record and process the video.

This picture says it all:

See the Mac mini and the 12V battery to power it? That’s the real price you pay. Now once you do this, it makes perfect sense to drop in a barn doors/matte box, a follow focus, a Nikkor mount adapter, and a 7″ LCD monitor.

Yeah, I don’t do videography but even I am a bit jealous. The guy has Nikkor lenses on it.

Of course, since it’s still a 1/3″ sensor, I’m not that jealous. Video will never approach film in quality until they use APS-C or full-frame sensors to get the depth of field that, more than 24 frames per second, is synonymous with elegant “cinematic look.” Besides, I already have severe equipment envy.

“What’s in her bag (unpacked)” by tychay
(Nikon D70, Nikkor 12-24mm f/4G, UV) f/4 at 1/1250 second, iso 200, 12mm (19mm)

Doing Panasonic one-better

Realizing that people won’t purchase this thing with the Panasonic AG-HVX200 out there doing this for the same price, Reel|Stream had to do Panasonic one better.

Well since you are bypassing the image processing in camera the Andromeda is essentially the first “video RAW” camera. This means you can take the RAW 12-bit sample data and combine it with your own special de-mosaicing algorithm (they call it “sculptorHD” but it’s basically just a variation of a de-mosaicing algorithm that you would do in RAW processing) to create an image with up to 4 more bits (16x) the dynamic range (in reality it is actually 2 more bits (4x) the dynamic range).

Hence this quote on their website

“…the unmodified camera has on the order of 7 stops latitude, but the Andromeda image shows nearly 9.5 stops. Pretty cool!”
Adam Wilt

I agree completely, it’s not for me, but that is pretty damn cool.