Is an HD Super 8 transfer overkill?

[carllooper]

ps. my only reservation, which is not a great one, is that if a low end HD transfer really was no better than a high end SD transfer then one can't simultaneously propose that the very same low end HD transfer might be better than the very same high end SD transfer.

Of course. Which is why I never said that. I said the low end HD transfer might still look better on an HD display than the high end SD transfer because the live upscaling issues of SD>HD are avoided by having an original HD scan. Again, the entire context is about comparing SD and HD content on an HD viewing system. Otherwise, this entire discussion is moot.

Roger

I guess it all depends on what is meant by "better" and whether that is any different from "look better". Can we not say that something looks better because it is better, ie. according to whatever criteria one has established (be it sharpness, colour, information, etc).

So how might an HD scan, that was no better than an SD scan look better if it wasn't better?

The only way I was able to make any sense of this was to reinterpret the "no better" as "no worse". I don't otherwise know how to read it.

How can an HD scan look better if it isn't better?

Or to put it another way, if it looks better, then what does "no better" mean? Surely it means "no worse".

[MovieStuff]

ps. my only reservation, which is not a great one, is that if a low end HD transfer really was no better than a high end SD transfer then one can't simultaneously propose that the very same low end HD transfer might be better than the very same high end SD transfer.

Of course. Which is why I never said that. I said the low end HD transfer might still look better on an HD display than the high end SD transfer because the live upscaling issues of SD>HD are avoided by having an original HD scan. Again, the entire context is about comparing SD and HD content on an HD viewing system. Otherwise, this entire discussion is moot.

Roger

I guess it all depends on what is meant by "better" and whether that is any different from "look better". Can we not say that something looks better because it is better, ie. according to whatever criteria one has established (be it sharpness, colour, information, etc).

So how might an HD scan, that was no better than an SD scan look better if it wasn't better?

The only way I was able to make any sense of this was to reinterpret the "no better" as "no worse". I don't otherwise know how to read it.

How can an HD scan look better if it isn't better?

Or to put it another way, if it looks better, then what does "no better" mean?

Good lord. Is there anyone here reading this that did NOT understand what I wrote?

Carl, the entire context of this particular part of the discussion is about what the SD or HD content LOOKS LIKE on an HD display. If the SD video is uprezzed "live" to HD then the SD is subject to the whim and degradation of the uprezzing process. This could make a really great SD transfer not look as good as a low end HD transfer viewed on the same HD system. Thus, the low end HD transfer may LOOK -as in APPEAR- as in -SEEM- to be better than the high end SD transfer.

Are we good?

Roger

[carllooper]

Good lord. Is there anyone here reading this that did NOT understand what I wrote?

Carl, the entire context of this particular part of the discussion is about what the SD or HD content LOOKS LIKE on an HD display. If the SD video is uprezzed "live" to HD then the SD is subject to the whim and degradation of the uprezzing process. This could make a really great SD transfer not look as good as a low end HD transfer viewed on the same HD system. Thus, the low end HD transfer may LOOK -as in APPEAR- as in -SEEM- to be better than the high end SD transfer.

I understand the argument. It's a counter-factual one, but one that seems to want to remain counter-factual.

Certainly if the HD looks better (appears better, seems better) one need not therefore conclude it actually is better.

One could maintain the idea that it was just an illusion, that in reality, it was no better.

The only problem for me, (if only me) is that I know an HD scan can only look better than an SD>HD scan, if it actually is better.

Other than that we're good.

Carl

[Uppsala BildTeknik]

Certainly if the HD looks better (appears better, seems better) one need not therefore conclude it actually is better.

One could maintain the idea that it was just an illusion, that in reality, it was no better.

This is almost like a joke... Or are you trying to play the role of a forumtroll, for real?

You cannot seriously really mean that you are now including illusions as arguements in your theories? 8O

[carllooper]

Certainly if the HD looks better (appears better, seems better) one need not therefore conclude it actually is better.

One could maintain the idea that it was just an illusion, that in reality, it was no better.

This is almost like a joke... Or are you trying to play the role of a forumtroll, for real?

You cannot seriously really mean that you are now including illusions as arguements in your theories? 8O

Dear Kent,

if you read the posts, rather than just skipping through the words for those that will give you a headache, you'll see that I'm mounting a criticism against treating the HD scan as an illusion. I'm not your enemy.

Carl

[carllooper]

This is how I'd set up a scenario that was consistent with Roger's otherwise well thought out argument. In the same vein as Roger this is purely for discussion sake, the purpose of which is to show how a counterfactual position can work even without necessarily knowing why, but that knowing why must necessarily put into question the original counterfactual position.

As in Roger's scenario we'll propose a "low end HD transfer system" and a "high end SD transfer system" where the former is "no better" than the latter. The crucial point here is the meaning of "no better". I originally reinterpreted this as "no worse" in order to make sense of it. But Roger insists we treat this as "no better" so lets try and do that.

How do we construct a scenario in which a HD system is no better than an SD system?

One way might be for the HD system to have a bad lens. There may be other scenarios that could work just as well or better, but this is the one that immediately comes to mind. Now for this scenario to work we could require that the signal going to the HD array be no better than the signal otherwise captured by the SD sensor array. We can treat the signal falling on the HD sensor as sufficiently soft as to be "no better" than an otherwise clear signal beig captured on the SD system. The SD system undersamples it's clear signal by the exact same amount as the lens of the HD system softens it's signal. What this means in practice remains to be seen but we try to be as faithful as we can to the possible meaning of "no better".

This scenario, it seems to me, is consistent with Roger's scenario and what he is suggesting. We have a low end HD system encoding a soft signal and a high end SD system undersampling a clear signal. And that to the best of our ability we try to match what might be meant by an HD system "no better" than the SD one.

Now lets further assume, for the sake of discussion, that the SD system is upscaled on a cheap upscaler. We get a "pixellated" or "mosaic" type of image after upscaling.

We note, as Roger clearly argues, that the HD result looks better. And there ends Roger's argument. And it's a good argument.

The crucial point, for me, is the characterisation of the HD system as "no better". What I'd argue is that the HD system is better. It is better because although it is sampling a signal supposedly no better than the undersampled SD system captured, it is precisely because it is oversampling its signal with respect to the SD system, that it's better. Or the other way of looking at is that the SD system is worse because it is undersampling it's signal, and would therefore require expensive upscaling algorithms to match the HD result.

In other words the proposition of an HD system no better than an SD>HD system is the illusion. It is a counterfactual proposition but it's one that when approximated in the real world, as close as possible, the HD comes out looking better. But it comes out looking better because it actually is better. Because unlike the counterfactual proposition it is not an illusion. The counterfactual proposition (the illusion) can only ever be approximated.

But despite this, Roger's argument remains a good argument. It does not depend on the attributes of the film being scanned (as mine did) and shows that even when the technical differences are made as small as you like (approaching but never reaching the counterfactual proposition) the HD scan will always be, at the very least, no worse than the best possible SD>HD. And due to screens using cheap upscalers, would be invariably better.


Carl

[carllooper]

Perhaps it's time for a summary of the recent debate between myself, Roger and Kent:

1. My argument is that Super8 contains sufficient additional information to warrant an HD transfer over SD, and that this can be demonstrated by appropriate analysis of a single HD scan.

2. Roger's argument is that even without taking any position on the above, there are good reasons to do HD scanning anyway, due to variability (and associated costs) in what SD to HD upscaling algorithms (hardware, pipelines, or workflows) can otherwise accomplish.

3. Kent's underdeveloped theory is that my methods are a joke, but otherwise agrees with Roger and I, that HD scanning is better than SD>HD scanning, and can demonstrate that with real life examples.

On my side I don't think there are any more outstanding issues or ideas to address, unless Kent has done some additional work on his theory regarding my methods.

Carl

[Uppsala BildTeknik]

1. My argument is that Super8 contains sufficient additional information to warrant an HD transfer over SD, and that this can be demonstrated by appropriate analysis of a single HD scan.

We all agree that super8 hold more information than can be captured with a SD transfer. But you are the only one who refuses to accept that in order to compare something you call "SD"... You actually need to do a SD transfer. Otherwise you are not comparing any SD transfer to anything, you are not comparing real life results.

It is by doing these theoretical calculations you probably have come to the conclusion that 8mm film should be transferred with at least 4K and 16mm with more than 24K. Whatever on earth real life transfers you might find at 24K.

2. Roger's argument is that even without taking any position on the above, there are good reasons to do HD scanning anyway, due to variability (and associated costs) in what SD to HD upscaling algorithms (hardware, pipelines, or workflows) can otherwise accomplish.

Yes, of course real life upscaling needs to be included in the comparison. If not, you are not getting real life results.

3. Kent's underdeveloped theory is that my methods are a joke, but otherwise agrees with Roger and I, that HD scanning is better than SD>HD scanning, and can demonstrate that with real life examples.

I don´t think I said your methods were a joke (perhaps that your explanations are, if you are/were including illusions). All I am saying is that your test is flawed, because you never got any SD transfer to compare with to begin with.

You are comparing a image up/downscaled to different resolutions. It is not the same, and will not give the same result, as comparing two individual actual real life transfers. Comparing a HD transfer to a SD transfer will give you real life results.

Comparing one transfer up/downscaled will show you how up/downscaling a image works and what results the scaling will give you. It will not show you how a SD transfer would hold up against a HD transfer. That is just a fact of life.

[carllooper]

Yes, of course real life upscaling needs to be included in the comparison. If not, you are not getting real life results.

Well then, I must be getting real life results because I use the exact same real life scalers as used in all real life digital capture and digital display devices, from the cheapest scalers to the most expensive. In other words, if I can compare a very large range of real life results (as I do) surely my results must be more real life than the results one gets from testing only two real life systems.

The scalers used in real life digital display devices are no more real (and no less theoretical) than the scalers used in Photoshop, or the scalers used in my own software. They all have their origin in theory, in particular a category called "interpolation". It is from such theory that all scalers have their origin, and are written up as software, or otherwise hardcoded in hardware. There aren't that many scaling algorithms. You could count the most important ones on one hand.

By definition, an HD scan will have more definition than an SD scan. Because that is how those words are defined. So you don't even need to do any tests to know this.

Both HD and SD scans are an approximation of the film being scanned. What is at issue is not whether HD scans are intrinsically better (higher definition) than an SD scan. We already know they are. They are so by definition. We don't need to do any tests to confirm that. But a test will confirm it. It's just that it's unnecessary. A little thought can be used instead.

What I'm exploring is not the already defined difference between HD and SD, (or SD to HD scaling). Because I'm not interested in that. I already know it is different. It is different by definition. HD is higher defintion, by definition. And as Roger elaborated, since SD would require upscaling to match HD (if that), HD is already in a better place by virtue of just thinking this anyway. No tests required. Indeed it is Roger's thought experiment that effectively thinks through this very thing.

BUT what I'm exploring is the difference between the film and whatever scanning resolution you care to nominate (or I can achieve within my budget). I'm interested in the properties of the film, not the properties of the scanning system. I already know the properties of the scanning system. Digital properties are well defined. That is what makes them attractive. The properties of film are not as well defined because, for one thing, we have only recently had the ability to digitally measure it's properties with high definition systems and to analyse those properties with better digital techniques. But we've always had the ability to just look at the film with our eyes and brain anyway. Digital analysis is just a way of theorising how our eye/brain might otherwise do it, and to do it in a more formal and rigourous way.

Now all digital scans, irregardless of resolution, are approximations of the film. And the film is itself an approximation of the image otherwise mediated by the film. So it's not about which scanning resolution will be better than which (the highest definition achievable will be the best). Rather it is about at which resolution will a digital approximation be no worse than the film and it's approximation.

Carl

[supa8]

You're just contradicting yourself

Comparing

scanning resolution

between HD and SD means that you do one scan in SD and one scan in HD.
Then you compare them (HD scan scaled down to SD vs Sd scan _ SD scaled up to HD resolution vs HD scan).

Whatever looks best is worth going for.

But surely, you know the answer to that by now...

[carllooper]

You're just contradicting yourself.

That's dead easy to say but it's a lot harder to say how. If you can't show how this claim might be true then it's just an empty claim.

But I remain open to being shown otherwise.

Perhaps you mean why I'm using scalers at all if HD is better than SD by definition?

Well that's the whole point. The issue, as I've mentioned, is not the difference between HD and SD, because that is already a known. It is numerically defined. You don't need any tests to test whether a 640 x 480 sensor is a 640 x 480 sensor because if it wasn't such, you wouldn't be calling it a 640 x 480 sensor in the first place (unless you're lying to yourself or being lied to).

What is not known (to me at least) is at what resolution the scan of the FILM is no longer any better than an upscaled lower definition scan of that same FILM. And you then use that found resolution. You don't use the upscaled lower definition scan.

The FILM should hold an image of a resolution chart or something similar otherwise the above iteration won't ever terminate, precisely because, by definition, a higher resolution scan will always be higher definition (or no worse) than a lower definition scan (whether the low def was upscaled well or not).

Now I've only tested Super8 up to 2.5K with a resolution chart, and it was still better than an upscaled lower definition scan (across all possible lower definitions). But the reason I'm exploring extremely high definition scans, 12K+ (not yet with resolution charts) is more about exploring the signal/noise relationship at those definitions rather than the resolution of the signal. I'm not at all suggesting Super8 requires such excessive definitions, but at these excessive definitions I'm finding useful information that helps me understand the structure of the film and how it might aid in enhancing things like dynamic range of a scan. It is information that is not available at lower definition scans. And I'll be doing resolution tests at these excessive definitions as well, in due course, precisely because the 2.5K Super8 test did not show me a resolution limit.

I'm not out to prove anything I already know. I'm out to discover things I don't already know.

And use this information to build a good transfer system for the film I'm making.

And like everyone else(?), it is also for me, about what LOOKS BETTER. It's not just that (as I've argued elsewhere) but I'm certainly right into the LOOK as well. But my particular position on that is that if something looks better, it is because something about it IS better. So I just want to know what that something is. If I know why or how something IS better (or can be better), then the result of implementing that knowledge will LOOK BETTER. That's the theory. And so far it's working for me. I'm getting results that LOOK BETTER. Even if my theory is completely wrong or contradictory. But if it is wrong or contradictory I'd still like to know, because perhaps I can get something that LOOKS EVEN BETTER.

Carl

[Uppsala BildTeknik]

Well then, I must be getting real life results because I use the exact same real life scalers as used in all real life digital capture and digital display devices, from the cheapest scalers to the most expensive.

Theproblemis that your source material is not a true SD transfer that is upscaled to HD. Your source material is not "true SD" it is downscaled SD-version of a HD transfer. There is a difference, you know.

In other words, if I can compare a very large range of real life results (as I do)...

But all you are comparing is one transfer originated from a HD-machine scaled to different resolutions.
If you want to compare a SD transfer to a HD transfer: Get a SD transfer, get a HD transfer, then compare them.
If you want to compare a 24K transfer to a 1080p transfer: Get a 24K transfer done, get a 1080p transfer done, then compare them.

It is quite simple and fully logical, and the only way you would get "real life results".

Might I ask you where you are getting the 24K done, to what format and what the cost and storage requirements are per hour?

[carllooper]

The problem is that your source material is not a true SD transfer that is upscaled to HD. Your source material is not "true SD" it is downscaled SD-version of a HD transfer. There is a difference, you know.

My source is a true SD transfer. Always has been. Always will be. Is an SD transfer of Super8 any more true SD than an SD transfer of 35mm? No, because the result is always SD irregardless of the definition of the source. So it doesn't matter whether I'm transferring the original film to SD or the HD scan to SD. The result is always the same definition: true SD.

In other words, if I can compare a very large range of real life results (as I do)...

But all you are comparing is one transfer originated from a HD-machine scaled to different resolutions.

Yes. That's right. It works because an HD scan already has exactly the same information in it as an SD scan, plus the additional information afforded by the additional number of sensor results in the HD scan.

Might I ask you where you are getting the 24K done, to what format and what the cost and storage requirements are per hour?

I've built my own scanning system and continuing to improve it. So far it has cost me about $2000. The storage requirements are what you would expect for large files. This is a research and development project. Also, as I mentioned when I mentioned scanning film at this resolution, I'm only scanning a small window on the film. The scanning resolution of 24K you mention is what I said the scanning resolution would be if I were scanning the whole frame. But I'm only ever scanning and storing a 4K window on that. I said that originally and I'm saying it again here lest there be any misconceptions.

Carl

[supa8]

Ok this conversation is going nowhere.
Carlooper. Fair play to ya. Good luck.
Just beware not to overkill yourself with these flawed tests though

[Uppsala BildTeknik]

No, because the result is always SD irregardless of the definition of the source.

If all you are comparing is scaling of images back and forth then fine. It has nothing to do with neither a SD transfer nor a HD transfer.

You can play with still images all you want. But if you are going to compare a SD transfer, get a SD transfer done on a SD machine.

...I'm only scanning a small window on the film. The scanning resolution of 24K you mention is what I said the scanning resolution would be if I were scanning the whole frame.

So once again all you have are theoretical scaling data. No real life transfers, no real life results and no real life conclusions.

Transfer one hour to 24K, transfer the same hour to 4K. Compare results, figure out how you will use the material, if you have use for the extra details and if it is worth the hassle of dealing with a 24K file.

My guess is that you won´t ever even end up doing a real 24K transfer. So what is the point of the whole "24K test" then?