Optical Timecode for 16mm

[carllooper]

Am working on printing timecode to the optical sound track area of 16mm.

I'm going to use this standard: http://en.wikipedia.org/wiki/Linear_timecode
http://en.wikipedia.org/wiki/SMPTE_timecode

This requires the soundtrack encode 80 bits per frame. Visually it would look like closely spaced horizontal lines printed to the film's optical sound track area. During projection this signal will go out on the projector's audio out line to the microphone input on a laptop, where software decodes the signal back into timecode, to regulate playback of the actual soundtrack.

To print the timecode on the film am going to computer generate it on a computer screen, and photograph it a frame at a time using a Super16 camera. An alternative is to get hold of an Auricon and generate the timecode as an audio signal to feed to the Auricon's galvanometer. But doing it the other way appeals to my sense of hybrid creativity.

The bandwidth of optical sound on 16mm is about 6KHz from memory - which may have more to do with legacy galvanometers than the film itself. In terms of analog audio that's not that great (although certainly it has a compelling analog quality). It is however, more than enough to carry 80 bits/frame timecode reliably.

The software for regulating the actual soundtrack has already been done - this was done for a previous system I built last year where I had a rotary encoder on a projector shaft, counting frames, and feeding that to the software. The hassle with that system was ensuring film was on the right frame at startup. And that it required a specially modified projector. That said, it worked perfectly - frame accurate. Mishaps occured but were explainable. The first mishap was a USB port misbehaving (fixed when using a second port). The second mishap was the rotary encoder coming loose after the projector spent some time in air freight. For all other screenings it operated perfectly. The new system will be far more robust - no mechanical mods that might suffer stress and otherwise limit screenings to a special projector, ie. will work with any 16mm projector that supports optical sound; won't require any fussy startup procedure; will keep sync no matter what damage occurs to a print; uses an ordinary audio lead between projector's audio out and a laptop's microphone in; and so on.

While of obvious use for synronising sound, it can be used to syncronise other things as well. An interesting experiment might be to have a digital projector working in parallel with the film projector, in which real time animation interacts in sync with the film. Could drive automated musical instruments in sync with the film. But as much as I enjoy expanded cinema (automated or otherwise) I have a special affinity for traditional film screening setups: single rectangular image on the screen, with speakers either side, and comfortable chairs in front of such.

In terms of Structural Film Theory it's not necessary that such a system requires or implies that a soundtrack will be "in sync" with the film in the traditional sense: lip sync, diegetic sound. I've found it's the case that almost always any unsynced sound/music played with a film will automagically syncronise with a film - or the film will automagically syncronise with the sound. It's quite disturbing (in a good creative sense) - it's just one of those really strange things that occurs far more often than not. The more apparently random the relation established between film and sound, the more it will decide to operate in a way that defies any concept of randomness: all sorts of strange syncronicities will emerge with really strange powers and projecting all sorts of strange and wonderful concepts. Time in particular goes out of joint. Cause and effect swap their traditional roles. Some sort of voodoo is always at work. It's a reality of some sort. A ghostly or spooky reality. Or an illusion. Whatever you call it doesn't really matter (I reckon). But I'd suggest a sync system provides, counter-intuitively, the ability to actually "de-sync" or counter-act this kind of natural syncronicity - if that is something one is compelled to do. Or just as equally, to elaborate it (as I'd inevitably do). As for how Structural Film Theory would read it, I don't know. It depends entirely on what such theory assumes is an illusion on the one hand, and/or a reality on the other. I find such things are never that obvious. I find its structural films themselves (or any film for that matter) that resolve (if only in part) such problems, either in conjunction with a theory, or quite independently of such.

C

[Nicholas Kovats]

Great idea, Carl! Why specifically Super 16? Is the TC to be exposed across the entire width horizontally? If so how is this horizontal optical record of the LTC transposed to a 16mm optical track as an 80 bit LTC? Your optical printer? Reduction lenses?

Typical 16mm projector are wild. Speeds vary. In essence are the "digital" laptops, smartphones, iPods, etc..."chasing" the optical LTC playback? Self correcting?

Great example of film as a low density optical storage medium! There are papers that describe the use of 35mm film as a very high density storage medium utilizing the entire frame. 1 micron photo-chemical particles to encode binary data. A single "encoded" 35mm frame is serious competition for multiple LTO back up tapes! And archival if split into 3-way RGB panchromatic film records. I will have a look for that paper once again.

One thing that struck me from the paper was that an optical low band pass filter? decodes the 35mm frame. The encoding method was some sort of grey scale matrix. Very clever. But it got me thinking. What if the upper color frequencies were recorded as a standard film image "multiplexed" with the high density low frequency encoder? Sort of a tech fantasy whereby "traditional" film would blow current digital image metadata out of the water.

[Nicholas Kovats]

I found the 2009 patent application as orginally submitted by James Arthur Fancher and Jeff James Rosica as per Thomson Licensing also known as Technicolor! Brilliant use of optical gray scales and "off-the-shelf" encoding schemes, i.e.
https://www.google.ca/patents/US2009019 ... CDwQ6AEwBA

"Recording data on motion picture film:

As described earlier, previous techniques of recording data on film operate in a binary manner —any particular area of film is either exposed (black and opaque in the negative film image) or not exposed (clear and transparent in the negative film image). While transition zones (areas of partial transparency) between exposed and nonexposed areas may exist in these previous techniques, these transition zones are not used for storing data.

In contrast, the inventive concept takes advantage of the fact that film is capable of reproducing a wide range of gray levels (differing degrees of transparency) with good accuracy to increase the quantity of data that may be stored on a given area of film. In particular, the inventive concept applies the idea of “symbols” from digital transmission systems to storing data on film."

Fancher and Rosica's earlier 2004 version of the same patent applivation, i.e. https://www.google.ca/patents/WO2004114 ... CCwQ6AEwAg


Their colleague, Markus Loeffler, submitted a variation in 2012.
https://www.google.ca/patents/WO2013137 ... CCUQ6AEwAQ

"Data archival on film

Archival of information and particularly binary data remains, vitally important to many entities, including government and industry. The archival period will typically dictate the archival medium. For example, short-term storage of binary data for months or even a few years can occur using magnetic or optical media. However, after a few years, magnetic and optical media can degrade, making reliable retrieval of data difficult. Optical film, when stored under appropriate conditions, can last for several hundred years, making optical film a useful medium for long-term data archival."


Inventors Mark Alan Zurbuchen and Charles Thomas Hoskinson of the The Aerospace Corporation submitted this interesting alternative in a 2009 patent application which mentions "bit rot". One of my favorite concepts when the digital purists start spouting the longterm benefits of digital practive and "archival" methods ("... just keep copying forever!"), i.e.
https://www.google.ca/patents/US8085304 ... CFYQ6AEwCA

"Photographic silver emulsion-based digital archival storage:

Embodiments described herein utilize an analog archival medium (such as a silver-emulsion film) to preserve digital data in a manner that effectively eliminates the problem of “bit rot” for long-term archival or storage purposes. More specifically, an article of photographic media is employed. In an example embodiment, the article of media includes a silver-emulsion based light-sensitive material which can survive for millennia in dry ambient conditions and in the absence of UV light. The invention also eliminates the problem of “format rot” through the possible incorporation of a human-readable description of the encoding scheme and provides for non-obsoleting digital archiving technologies"

[carllooper]

Thanks Nicholas - that's great information.

Have been working on similar ideas but not for data storage/retrieval per se. But could certainly use it for that. Basically it's a film projector but where the film is recorded on the film as a hologram. During projection, lasers and lenses (an analog system) are used to decode the hologram on the fly, for a laser projected image on the screen. Have only tested the idea mathematically and in computer simulations. It comes out of some research I was doing in analog computing and quantum teleportation for an otherwise technologically traditional film, where the technique is for certain special effects (or special defects) that would be done. The hologram is digitally computed (as an interference pattern) and printed onto film off a computer screen. This film (with an interference pattern) is then optically printed onto another film, but using the said lasers and lenses (an analog decoding system), rendering a 'decoded' image on the print. In addition to this (computing the hologram) is also transferring analog material into the digital domain using the same principle, where the film is acquired using lasers and lenses - arriving in the digital domain as a hologram. It remains as a hologram in the digital domain using algorithms operating in the requisite fourier domain, with the signal printed back to film in the same way, ie. as a hologram (as an interference pattern). Digitally it never exists as a traditional digital image (other than for preview purposes). It is decoded back into a conventional analog image, through the laser/lens setup (an analog decoder). The idea is to keep the digital side from introducing traditional digital artifacts in the processing pipeline, while still taking advantage of digital processes. Its a way of preserving (to a greater degree) characteristics of the analog signal that would otherwise be lost when wanting to do any digital processing of an analog signal. Its sort of a quasi-quantum computer system exploiting the quantum mechanical properties of light to perform coding/decoding. A lot of this goes back more than half a century to analog computing techniques used in military surveillance during World War II.

Regarding LTC (and previous system): yes the sound track follows the film in the same way an analog soundtrack would. If the projector runs slow, so to would the sound. I'm perfectly fine with this. The system assumes the projector is working fine, ie. that the drive belts are the right tension for the standard projector rate. But I've found, even if the projector isn't perfectly tuned, the audio is quite acceptable. And insofar as the film soundtrack is prepared digitally, various techniques can be used to mitigate for variations in pitch across different projectors, eg. for music one might use cue points for start and stop, rather than have it frame locked for the duration, ie. in much the same way one might play some music when screening an otherwise silent film (wild sync). And for the adventurous, all sorts of real time adaptions can be built into the sound. Indeed the sound track could be different every time you played the film, working according to some artificial intelligence design.

Have done some experiments in projector speed regulation. A system that worked fine was using a digitally controlled relay to interrupt the power supply at a dynamic correction rate using feedback from a sensor on the shutter blades. Of course, this only works if the projector was otherwise running faster than you require. Was able to keep two 16mm projectors running in frame accurate sync indefinitely (where the normally faster one was slaved to the slower one). The idea for this was courtesy of Richard Tuohey for a work involving two 16mm projectors in frame lock sync.

C

[MovieStuff]

Regarding LTC (and previous system): yes the sound track follows the film in the same way an analog soundtrack would. If the projector runs slow, so to would the sound. I'm perfectly fine with this. The system assumes the projector is working fine, ie. that the drive belts are the right tension for the standard projector rate. But I've found, even if the projector isn't perfectly tuned, the audio is quite acceptable.

C

Fun project! One thing to remember is that acceptable "sync" is relative. In my youth, I did what many fledgling film makers did, which was to try and wild sync a tape player to a projector. It would eventually drift but, amazingly, it would stay in relative sync that looked right often minutes at a time. And, of course, even though the projector was obviously the wild card, in terms of fluctuating speed, the audio sounded perfect during the short time of relative sync. Also, before starting this business of building telecine units, I worked on film restorations for a distributor of old movies. Many of the 16mm and 35mm film prints' soundtracks were amazingly out of sync to the picture, some by as much as a couple of frames, and you never notice it on screen when watching. There is a threshold of "too much" out of sync, of course, and once you cross that threshold you can never get the viewer back in the game. But, in general, audiences will never notice being a frame or so out of sync.

What I learned from this is that absolute sync isn't necessary and, sometimes, not even desirable if you are attempting to chase an unstable projector that may exhibit a considerable degree of "wow". The height of irony would be to have a perfect digital audio track that is trashed by the mechanical shortcomings of a film projector.

My suggestion?

Don't use the time code to chase the projector. Instead, use the time code to "trigger" sections of the audio track to play in wild sync for maybe only a minute at a time. Better yet, don't even bother mixing the final audio down to a single audio track. Just keep the various sound elements all separate and use the time code to trigger each element independently. Even with a projector running grossly fast or slow, any sound element would stay in sync for the few seconds required.

A gunshot happens at X point in the time code. A door slams at another point. Rain starts on a specific cue. Even lip sync would work this way. He speaks at X point in the time code, she replies at X point, he counters at X point, etc. The music rises at the appropriate time in full digital fidelity, unaffected by the projector's mechanical flaws. After all, it's a fair presumption that all these elements - voice, music, sound effects - are all present on the timeline as individual pieces of an audio jigsaw puzzle on individual tracks. Think of them as bullets loaded in a gun waiting to be fired at exactly a precise point in time. They all play at the right time and mix together to create the final sound experience. Even if you preferred to mix down to a single track, you could judiciously pick silent areas to bridge one 60 second or 30 second section to another.

Anyway, you get the idea. The point is that you will always have your PC playing the audio so you might as well take full advantage of digital audio fidelity while maintaining relative sync with the picture. After all, the audience will notice fluctuations in the audio more than they would notice being a frame or two out of sync. ;)

My two cents....

Roger

[carllooper]

Don't use the time code to chase the projector. Instead, use the time code to "trigger" sections of the audio track to play in wild sync for maybe only a minute at a time. Better yet, don't even bother mixing the final audio down to a single audio track. Just keep the various sound elements all separate and use the time code to trigger each element independently. Even with a projector running grossly fast or slow, any sound element would stay in sync for the few seconds required.

Thanks Roger - fully agree. That's the basic methodology I've been using - just using the timecode as a cue system for the sounds. The whole soundtrack is stored as individual sounds, each being invoked at it's nominated time. For sounds that do require sync across an extended period (such as some extended lip sync dialogue) the way the system works is quite fine - the system is always monitoring the average rate of the projector (number of frames over a number of seconds) and so for any sounds that require it, it is able to start those sounds playing at that average rate, ie. the sound is not slaved to the projector at every microsecond - the projector is allowed to vary all over the shop, as it doesn't have any direct influence on the sound playback - rather, for any sounds that require it, the sound stays in sync by virtue of being started on the correct frame cue, and playing at the average rate the projector is otherwise playing, (at the time the sound was started).

Basically the sound track becomes the digital equivalent of those piano players, in the silent film era, who provided a real time accompaniment for an otherwise silent film - improvising sounds between and on cues. The sound preparation effectively remains the way you might prepare it in an NLE. Instead of rendering the sound out from the NLE to some audio format (mp3 etc), the screening is akin to playing the sound directly from your NLE, where the timecode from the film just drives the cues for each sound, rather than the sound itself as such.

C

[Aussiebill]

I appreciate this board relates to TC for 16mm but would it be possible to put optical TC on Super8 gauge, ideally between the sprocket holes or if that is insufficient, by reserving a horizontal area between frames?
I think expanding the vision frame width into the area previously reserved for sound track is desirable (As done for SuperDuper8 / Max8). If the area between frames was utilised I expect frame height would be reduced making for an even wider frame ratio. The question would then arise how much would the vision frame be impacted to accommodate viable optical TC (if it were at all possible).
I imagine a camera such as the Logmar Super 8 with a capacity to simultaneously record matching TC on film and the on board digital audio media. Then I imagine a compatible TC film projector completing the synchronised film projection experience.
Any comment appreciated. Bill.