Avisynth RemoveDirt not working

[ThePhoenix]

Hi all

I am new to this game, with much to learn, so I would be grateful is any responses to my queries are made in very simple terms :?

I have been building a frame-by-frame capture telecine machine on and off for a couple of years and now have it working to my satisfaction. I have been aware of the Avisynth and the highly regarded scripts that various talented individuals have made available for download and have always planned to be using one of this to both clean up my footage and do frame rate conversion.

Here’s what I do:

Capture my individual frames.
Batch process them in Photoshop, colour adjust, rotate, contrast, noise and trim to 720 x 576
Import into Sony Vegas to output as an avi at 16.667 fps - not educated myself well yet with an understanding of the
various containers and codecs, but the file I create is Uncompressed and Progressive scan.
I am then using a frame blending script by Freddie Van de Putee (forum.doom9.org/showthread.php?t=144271), within Virtualdubmod.

And herein lies my problem: the filters within this script all seem to be doing their stuff, with the exception of the one that I was really looking forward to seeing good results from, namely the RemoveDirt filter, which is invoked by RemoveDirtMC.avs This seems to have no effect at all.

I can step through my footage frame by frame and see dust appear on a single frame and disappear on the next, so I don’t think it’s anything to do with any interlacing across multiple frames that has been introduced without my knowledge. Also, the script as downloaded has the cleaning parameter set to 40, (I think) with some narrative suggesting that this should be incremented for dirtier footage. No details of the magnitude to which this number can be extended, so I’ve tried numbers as large as 10,000, with absolutely no difference in output.

Does anyone know what I may be missing or can suggest what I could try?

Thanks in advance.

[RCBasher]

Check you are returning the right option. Fred's scripts tend to have different returned clips depending on your preferences for processing..e.g. with or without denoising.

I have my scripts organised into functions, you could try this in a cleaning only script and see if it works for you.

Frank

Code: Select all

#============================================================
#PLUGINS SECTION
#------------------------------------------------------------
#these need to be enabled if not in the Avisynth PlugIns folder
#pathname(s) will need to be added if these are not in the same folder as this script

#Loadplugin("removegrain.dll")		#required by function RemoveDirt
#LoadPlugin("MVTools2.dll")		#required by functions RemoveDirt and RemoveNoise
#Loadplugin("warpsharp.dll")		#required by function RemoveNoise
#loadplugin("RemoveDirtS.dll")		#required by function RemoveDirt
#============================================================



#Open the source file and apply the cleaning function of this script
#------------------------------------------------------------
#set the file to be processed here
MyClip="V:\Films\Scans\film5701.avi"	#enter full path to file between the quotes

#open the file and convert it if necessary to a colour format the filters will understand
InputClip =avisource(MyClip).converttoyv12()

#Clean the clip and return it
return CleanClip(InputClip)
#All done
#------------------------------------------------------------


#CLEANING FUNCTIONS
#------------------------------------------------------------
function CleanClip(clip input)
{
  #uncomment the next line to return the clip unprocessed
#  return input

  #call RemoveDirt, apply result to a call to RemoveNoise and return the result
  return RemoveNoise(RemoveDirt(input))
}

function RemoveDirt(clip input)
{
  #parameters which user may usefully tweak if required
  dirt_strength=23                           #sets amount of dirt removal (big spots)

  # create super clip with hierarchical scaled frame data
  super1=input.MSuper(pel=2)

  #use the super clip to generate forward and reverse vector data
  bvec = MAnalyse(super1,isb=false, blksize=8,overlap=0, delta=1, truemotion=true)
  fvec = MAnalyse(super1,isb=true, blksize=8,overlap=0, delta=1, truemotion=true)

  #create motion estimated forward and reverse versions of the current frame
  backw = MFlow(input,super1,bvec)
  forw  = MFlow(input,super1,fvec)

  #combine into a three-frame clip
  clp=interleave(backw,input,forw)

  #apply heavy cleansing 
  clensed=clp.Clense(grey=false, cache=4)

  #create an alternate clip for the RestoreMotionBlocks filter
  alt=clp.RemoveGrain(2)	#mode=2 retains fine detail

  #create the cleaned clip
  clp1=RestoreMotionBlocks(clensed,clp,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,dmode=2,debug=false,noise=dirt_strength,noisy=4, grey=false)

  #and just return the wanted frame
  return clp1.SelectEvery(3,1)
}

function RemoveNoise(clip input)
{
  #parameters which user may usefully tweak if required
  PRE_sharpness= 60
  PRE_radius= 3
  denoising_strength=600

  # create super clip with hierarchical scaled frame data
  super2 = input.MSuper(pel=2)

  #use the super clip to generate forward, forward+1, reverse and reverse-1 motion vector data
  bvec1 = MAnalyse(super2, isb = true,  delta = 1, blksize=16, overlap=8)
  fvec1 = MAnalyse(super2, isb = false, delta = 1, blksize=16, overlap=8)
  bvec2 = MAnalyse(super2, isb = true,  delta = 2, blksize=16, overlap=8)
  fvec2 = MAnalyse(super2, isb = false, delta = 2, blksize=16, overlap=8)
  
  #filter the noise/grain and apply compensating sharpening
  return input.MDegrain2(super2, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength).unsharpmask(PRE_sharpness,PRE_radius,0)
}

[ThePhoenix]

Thank you so much for this advice, I am so frustrated with this, spending many hours and not really knowing what to try.

I have implemented your script, but am getting the failure "There is no function named MSuper". At least I'm getting an error to work on now.

Just off to trawl the net to see what I can find.

I'm very grateful of the help, thanky you.

[RCBasher]

Make sure you have MVTools2 available. http://avisynth.org.ru/mvtools/mvtools2.html

[ThePhoenix]

Off to check that now. Cheers.

[ThePhoenix]

Not doing something right here as the whole thing falls in a heap now.

I need to go to work now, but will continue this evening. I will report in with progress then.

Thanks again.

[ThePhoenix]

Back on the case now. This seems to be where I am:

I have Avisynth 2.5 running under Windows 7
Multithreading is running MT.dll
I have no Mvtools dll's in the Avisynth plugins folder
In the folder that has my scripts I have both Mvtools. and Mvtools2.dll
Mvtools.dll is dated 08/04/2006 and is 644KB

When I execute the script provided by RCBasher with this configuration I get "There is no function named Msuper"
If I replace the Mvtools2.dll above with the one downloaded from the RCBasher link a few posts earlier (19/09/2011 and 612KB) I get the same message.
If I copy the original Mvtools2.dll into the Avisynth Plugins folder I get "There is no function named Clense"
If I copy the latest Mvtools2.dll into the Avisynth Plugins folder VirtualDub reports a terminal error, and shuts down.

Any thoughts anyone?

Thank you.

[RCBasher]

I have no Mvtools dll's in the Avisynth plugins folder
In the folder that has my scripts I have both Mvtools. and Mvtools2.dll

Given this placement of the plugins, did you uncomment the references to the plugins in my code as per the instructions in the PLUGINS section? Basically, you need to remove the # at the begining of each line:

Code: Select all

#============================================================
#PLUGINS SECTION
#------------------------------------------------------------
#these need to be enabled if not in the Avisynth PlugIns folder
#pathname(s) will need to be added if these are not in the same folder as this script

Loadplugin("removegrain.dll")      #required by function RemoveDirt
LoadPlugin("MVTools2.dll")      #required by functions RemoveDirt and RemoveNoise
Loadplugin("warpsharp.dll")      #required by function RemoveNoise
loadplugin("RemoveDirtS.dll")      #required by function RemoveDirt
#============================================================

[ThePhoenix]

I am so sorry, I didn't notice the instruction in your script. :oops:

I have uncommented as instructed, downloaded the RemoveDirtS.dll and now it runs, but still no different, as far as I can see.

Would it be too cheeky for me to request that I send you a few seconds of my footage for you to check that I haven't encoded it in some weird way? Or can you think of anything else I may have wrong?

I just know that I am going to be very embarrassed when the solution is found!

[RCBasher]

Would it be too cheeky for me to request that I send you a few seconds of my footage for you to check that I haven't encoded it in some weird way?

Upload it somewhere with public access (Dropbox?) and send me a link (via PM if your prefer)

Frank

[RCBasher]

Ok, I have your clip and tried it. It does do something, but I agree it is not a lot!

Change the line:

Code: Select all

return CleanClip(InputClip)

to:

Code: Select all

return stackhorizontal(InputClip,CleanClip(InputClip))

you will then get a side by side comparison.

I think the problem is the source clips is...how can I put it without being rude...erm...not very good ;) Sorry, but that's the truth of it I think. The script looks for frame scene changes so as not to respond to them. I believe what might be happening is that the frames are so grainy and noisy that it thinks each frame is a scene change. My guess anyway.

How have you made the captures, they look very low res for the supposed frame size.

Frank

[ThePhoenix]

Your absolutely right!

I just re-encoded my my original frames, without putting them through any pre-processing first (sharpening, colour and contrast adjustments), ran them through your script and the result is amazing; every piece of dirt vanishes, along with the grain.

Thank you so much for your time and experience.

[RCBasher]

Good, glad you now have it sorted.

Credit for the script function goes to those before me...most recently VideoFred. I just rewote into a function and added a few comments to make it easier to use.

Frank