Super8 signal processing

[carllooper]

There is some beautiful work by VideoFred at the link he provided. Absoultely gorgeous. And Frank's machine vision rig is great work. I considered quite a few machine vision cameras for a while but they were either out of my price range or the resolution was too small for the requirements of the project (SR).

The new batch of digital 'consumer' camera's, while not as sturdy as machine vision cameras, fit the bill. Although I'm not looking forward to the day the camera drops dead from exhaustion - but an hour's worth of Super 8 will be more than I need for the current research and dev project.

The camera trigger problem was something I considered during the design phase, prior to obtaining the camera. I was on the verge of committing to adaption of a cable release to computer control when a collegue mentioned some guys in Melbourne Australia doing stop-motion work using the Canon EOS SDK. That was the first time I had heard of it. Perfect. The camera can be triggered by the computer in sync with the frame advance system.

And frames can be transferred to the computer at the time of capture rather than filling up the camera's memory card.

That was the clincher for Canon. Nikon also has an SDK, but the fact that someone had used the Canon SDK with results was enough of a confidence boost for me to take that direction. My expertise (as far as it goes) is in digital image algorithms - both synthesis (CGI) and analysis (machine vision), and computer programming in general.

Now it may be the case that once this project is done I commit to a machine vision camera of the appropriate resolution based on the alternative cost of plugging in a new camera after every hour of film scanned.

[carllooper]

Sensor Noise

I thought might address this issue in more detail.

There are a number of variables involved in sensor noise, many of which cancel out during multiple exposure/long exposure and/or SR processing.

But there is a source of noise called "dark image noise" which is a fixed pattern of noise visible when "exposing a lens cap" (so to speak). When denoising is turned off this pattern is more pronounced. This noise doesn't cancel out during stationary multiple exposure/long exposure.

A solution to dark image noise is two fold.

The first is simply "dark image subtraction" wherein an exposure of the dark image is taken and then used in subsequent image processing to subtract that pattern. Otherwise known as "error correction".

But the dark image noise means some pixels can over-expose (clip) while others can under-expose. And the subtraction approach can't recover the information lost to clipping or undersampling.

So the second part of the solution involves a multiple exposure design wherein the same frame of film is captured in different locations with respect to the sensor array, enabling statistical recovery of the signal falling on the sensor pixels.

[peaceman]

This can be fixed with a double or even triple exposure of every film frame. And then a smart software merging of the frames

I love that. HDR for S8 scanning

Indeed I mixed two things into one sentence there: The 8-bit limit of avisynth has nothing to do with the hume gamut of eg. E100D or V50D.
Its getting a little bit off Carl's SR topic, but I would love to see ICC profiles of recent film stock. I wonder why kodak does not provide this.

[miles&jules]

Hi Peaceman
We used the shutter. I actually just bought a raynox super macro202 lens to go on our hdv cam. I'll give that a go on the bellows. The nikon does lens look great. But Ill have to check again for any distortion but I never noticed any.

Miles

[carllooper]

The irony is not lost regarding our analog (S8)/digital hybrid processing efforts.

There is an irony of sorts. On the one hand there is a championing of the Super8 format and on the other, a championing of digital systems.

As digital cameras like the Red One, and it's successors, attempt to look more like film, here we are trying to get film to ... look more like digital?

Well not quite. We're trying to get film (or in particular Super8) to look more like "reality". And so too are digital cameras. Cameras like the new Epic HDR are inspired by how film reproduces reality: Dynamic range and motion blur.

One of the hurdles that digital cameras face as they evolve into cameras like the Epic HDR, quite apart from more complex maintaince issues, is the vast ammount of information they need to process in "real-time" - on the day, so to speak. Typically compression algorithms. Although I'm not sure about the Epic. It may be storing raw data.

But by comparison with most digital video cameras the hybrid system we're developing here is relatively simple -and quite cheap. The digital stage occurs back in the studio rather than on location. We use the relative simplicity of film during acquistion (on location) with the digital work being done at a much more leisurly pace in post. It means we can test, apply and update very expensive algorithms (computationally speaking) because there is no need for realtime implementations of such (other than eventually improving workflow).

Carl

[aj]

Not to spoil the mood but possibly as a goal set to match:

http://vimeo.com/14317782
Shown and discussed before but likely not all have watched it

[carllooper]

Some nice work there, from a transfer and cinematography point of view.

Even without SR, Super8 transfers can look great these days. So one can imagine when SR is factored in how much better (technically speaking) it can look.

Of course, in some works, such as the one shown, SR may be considered unnecessary. The signal may be considered looking just fine, as is.

Carl

[carllooper]

This is slightly off-topic but can be considered contextual information.

One of my favourite film critcs is Andre Bazin, writing in the 40s. He was somewhat malligned during the post-modern era, and easy to misunderstand from a late twentieth century point of view. He was a champion of what was called (for want of a better word?) "realism".

I often ressurect Bazin and ask - what would he think of any film work I'm doing. Would he approve? Would he diapprove? Usually I can't get a definitive answer but the process of trying is nevertheless productive.

Bazin didn't like special effects. Not because they were necessaily badly done (think the 40s) but simply because, no matter how convincing they might be they were still special effects. Bazin was a lover of the photographic image.

Is SR a special effect - in the Bazinian sense?

As I work on SR I like to keep this question close at hand. When the first phase of the S8 SR is done I'd like to write something about it in the context of Bazin.

I'll end on a quote from Bazin:

"No matter how fuzzy, distorted, or discoloured, no matter how lacking in documentary value the image may be, it shares, by virtue of the very process of it's becoming, the being of the model of which it is the reproduction; it is the model." - Andre Bazin. The Ontology of the Photographic Image. What is Cinema. Volume 1.

[peaceman]

Isn't the eye (or better: the brain) doing some kind of "SR" perception/illusoin as well?
I imagine that when I am sitting in a dark room and seeing a projection on a huge screen, recognizing all silver grains and not being disturbed by a single compression artifact -- isn't that what makes me perceive so much more crisp sharpness and detail than there actually is in each frame on its own?

That I think should be the actual goal -- to achieve a digital copy that matches the quality of a decent projection.

Boy it is amazing how much data they store on these tiny 50ft rolls since more than 50 years now. How DARE we think digital is superior?

[carllooper]

Isn't the eye (or better: the brain) doing some kind of "SR" perception/illusoin as well?

Yes that's right. If you've ever compared a single frame of film to the motion picture from which it was plucked, the motion picture looks higher definition. That's because it is higher definition. And the brain can decode that. The definition isn't in the frame. It is in the empty space between the frames (so to speak).

I imagine that when I am sitting in a dark room and seeing a projection on a huge screen, recognizing all silver grains and not being disturbed by a single compression artifact -- isn't that what makes me perceive so much more crisp sharpness and detail than there actually is in each frame on its own?

If watching an actual film projected, the grain itself is in high definition - very sharp (because it is very sharp - in B&W at least). But this is different from the signal component. For example you can use high defintion grain to create the illusion that a lower definition signal was sharper than it was. But that is an illusion.

SR is not an illusion of definition as such. It is the reconstruction of actual emperical information - information that was distributed across a number of frames - but otherwise difficult to see in any one single frame.

That I think should be the actual goal -- to achieve a digital copy that matches the quality of a decent projection.

That is the first goal - simply to reproduce the Super 8 at the highest definition possible, because our brain, despite the grain, is capable of decoding the high definition signal within the film. It does a limited form of SR by itself.

The second goal is to switch off the grain (SR) because that will make the higher definition signal so much easier to appreciate. Our brain might perform decent SR but it also can't help but see the grain.

Not that grain is necessarily a bad thing. I kind of like it. But only if it's the real thing. The idea of fake grain can annoy me.

Boy it is amazing how much
data they store on these tiny 50ft rolls since more than 50 years now. How DARE we think digital is superior?

[VideoFred]

SR is not an illusion of definition as such. It is the reconstruction of actual emperical information - information that was distributed across a number of frames - but otherwise difficult to see in any one single frame.

This is exactly what I'm doing with my Avisynth filmscript. By averaging several frames, you make detail , spread over several frames, visible on every frame. Detail that otherwise would be lost in the grain. I hereby officially claim that the digital version often looks better than the original. But I'm talking about old, dirty , shaking and sometimes very grainy 8mm film now.

Not that grain is necessarily a bad thing. I kind of like it. But only if it's the real thing. The idea of fake grain can annoy me.

The Great Grain Discussion !

I like grain if it is not to heavy. On old 8mm film it is often clustered. The E64T grain was to heavy for my taste, but the color was wonderful. 100D has less grain but I do not like the color. But I think the interpretation of grain is something personal.

Fred.

[etimh]

Not that grain is necessarily a bad thing. I kind of like it. But only if it's the real thing. The idea of fake grain can annoy me.

But I think the interpretation of grain is something personal.

http://www.youtube.com/watch?v=0bXNGmgUg2w

Tim

[carllooper]

This is exactly what I'm doing with my Avisynth filmscript. By averaging several frames, you make detail , spread over several frames, visible on every frame. Detail that otherwise would be lost in the grain. I hereby officially claim that the digital version often looks better than the original. But I'm talking about old, dirty , shaking and sometimes very grainy 8mm film now.

From an information theory point of view, film projectors can be regarded as the traditional "decoding" half of what might be called the "film codec". The film camera is the "encoder" part. And film is the data storage medium.

Digital transfer of the data encoded in the film, plus the application of SR algorithms, such as VideoFred's, can be thought of as an improved "decoder" for the film codec - that restores more of the encoded information than the traditional decoder (a film projector) does (or did).

That's the cool thing about SR. It is capable of bringing into focus information that was otherwise diffcult to appreciate in the originally projected film.

[carllooper]

http://www.youtube.com/watch?v=0bXNGmgUg2w

Tim

I saw this on cinematography.com.

It is very slightly humourous - but the subject can touch a nerve - and I don't know if the the clip fully appreciates the issues it touches on.

Fake grain is often used/abused to match shots otherwise done on different mediums (or having undergone different processes). One tweaks the grain/noise of one shot to match another.

But just doing this, as a matter of course, is mindless.

A great movie to see is Woody Allen's "Zelig". Shots from throughout history have Woody Allen's character composited into the shots. Particularly hilarious is when he appears at the Nuremberg Rally in shots from Reifenstahls famous work.

The shots from different epochs are not matched (homogenised). Indeed their differences are respected. But the new shots (of Woody Allens character) are exactly matched (in terms of grain etc) to the shots in which he appears. It's technically quite stunning.

In art one often appreciates the physical material in which something is wrought, as much as anything else. The grain of film can be regarded in the same way, as speaking to the physical manner in which the work was created.

In this context fake grain becomes quite stupid, because it is stuffing around with our ability to appreciate the work on this level.

In Zelig there is no problem in this respect because the very concept of physical appreciation of the medium is obviously understood. We know the shots are from the archive and the more Woody Allen looks like he really is in the shot the more we appreciate the technical hurdles that would have been required to achieve that. There is an appreciateion of the medium's physical nature in the superb way it is handled.

There is love there.

But in other works, grain can be regarded as just noise. It is not there because the filmmaker necessarily wants it there. It is there due to various limits imposed on the filmmaker. One makes do with the limits. It is not necessarily about particularly enjoying those limits but working within them.

And the audience can appreciate the limits within which the filmmaker works. But if the filmmaker is just faking such limits then that appreciation goes out the door.

For example, digital video makers don't particularly enjoy having to compress their data. That is just a limit within which they have to work. But a better compressor can alleviate that limitation.

Likewise SR is about alleviating particular limits.

Now sometimes one deliberately choses to work within a particular limit - even if one doesn't have to do so. This is not faking a limit. It is about a self imposed limit. Limitations can be a good thing because they can force you to think in ways you may not have, were the limits not there.

So there are a lot of angles one can start to see on this sort of thing.

In Roland Bathes "Camera Lucidia" he coins a term called the "punctum". The punctum is that apsect of a photograph that the photographer doesn't put there, indeed they can not put there - because it is defined in that way.

The punctum can't be imposed by the artist. It is that happy accident. Syncronicity. It is symbolised by the figure in the bushes that appears in Antonionio's film "Blow Up". A photographer (a charcter in the film) discovers the figure back in the lab. He didn't see the figure when taking the photograph. He didn't set out to photograph the figure. But it is in the image nevertheless.

I like to think of the punctum as that which SR attempts to bring into view.

Carl

[VideoFred]

But in other works, grain can be regarded as just noise. It is not there because the filmmaker necessarily wants it there. It is there due to various limits imposed on the filmmaker. One makes do with the limits. It is not necessarily about particularly enjoying those limits but working within them.

We should not forget that grain is actualy an unwanted side effect of the film system. Later, it has been used for artistic reasons.

I have done some tests with fake grain. If you remove heavy grain from an 8mm film and replace it with fine fake grain, it looks like 16mm.

After all, it's all in the mind.

Fred.