Changing compressionType of TIFF files memory consumption

Hi,

I'm using the trail version 3.0.0 to see if Aspose.Imaging match our needs.
We have around 1800 TIF files that vary in size from 500 KB to 130 MB and they vary in compressiontype too.
We need to run through all the files and set their compressiontype before saving them to a new file.
This Works quite well for small amounts of files of small size however when the load gets bigger the memory consumption is eating all memory available and things get very slow obviously.
I realize that Aspose caches data to gain performance but as the caching eats all memory performance dies...
I'm currenty running my C# console application on a server which does nothing but run my job, it's got 32GB RAM which are used after running 10 files or so and then it just lingers at 98-99% memory consumption files are getting handled but it's VERY slow! :(
Now I've tried to set the various settings of the Aspose.Image.Cache but sadly to no avail...

Any adwise?

Thanks in advance
Michael

Hi Michael,

Thank you for your inquiry.

In order to improve the performance, you may set the cache option to store intermediate data on local disk. For more details, please follow the link Aspose.Imaging Improves Performance with Customizable Cache. Secondly, you should try to dispose the instance immediately after it is out of scope. In this way you will get memory free.

Further, in order to investigate the issue we need a sample application from your side that present complete scenario on the situation and sample files/images after bench marking the memory consumption against each sample. We will recreate the scenario at our end and will update you accordingly about our finds.

Hope the above information helps. In case of any issues, need further clearance please be sure to let us know, we will be glad to assist you.

Hi Ikram,

Thanks for the quick reply. :)

I've acctually been looking at the example you are linking to, but no dice. :|

I can't send you the files 'cus it's confidential data :o) My code looks like this as you'll notice it's build from examples on this site :) :

private static void FixTiffFile(string sourceFilePath)
{
try
{
FileInfo fi = new FileInfo(sourceFilePath);
string fileName = fi.Name;
// By default the cache folder is set to the local temp directory.
// You can specify a different cache folder from the default this way:
Cache.CacheFolder = @"Temp";

// Auto mode is flexible and efficient
Cache.CacheType = CacheType.Auto;

// The default cache max value is 0, which means that there is no upper limit
Cache.MaxDiskSpaceForCache = 0;
Cache.MaxMemoryForCache = 1073741824; // 1 gigabyte

// We do not recommend that you change the following property because it may greatly affect performance
Cache.ExactReallocateOnly = false;

//Load input Tiff image
using (TiffImage source = (TiffImage)Aspose.Imaging.Image.Load(sourceFilePath))
{
//Create an instance of TiffOptions for resultant Tiff image
TiffOptions options = new TiffOptions(TiffExpectedFormat.TiffJpegYCBCR);

options.Source = new FileCreateSource(string.Format(@"FixedTifFiles\{0}", fileName), false);

using (TiffImage result = (TiffImage)Aspose.Imaging.Image.Create(options, source.Width, source.Height))
{
foreach (TiffFrame frame in source.Frames)
{
//Create a copy of the frame for further processing
TiffFrame copiedFrame = TiffFrame.CopyFrame(frame);
var compressionType = copiedFrame.FrameOptions.Compression;
switch (compressionType)
{
case TiffCompressions.Jbig:
copiedFrame.FrameOptions.Compression = TiffCompressions.Jpeg;
copiedFrame.FrameOptions.BitsPerSample = new ushort[] { 8, 8, 8 };
copiedFrame.FrameOptions.Photometric = TiffPhotometrics.Ycbcr;
break;
case TiffCompressions.Jp2000:
copiedFrame.FrameOptions.Compression = TiffCompressions.Jpeg;
copiedFrame.FrameOptions.BitsPerSample = new ushort[] { 8, 8, 8 };
copiedFrame.FrameOptions.Photometric = TiffPhotometrics.Ycbcr;
break;
case TiffCompressions.Jpeg:
copiedFrame.FrameOptions.Compression = TiffCompressions.Jpeg;
copiedFrame.FrameOptions.BitsPerSample = new ushort[] { 8, 8, 8 };
copiedFrame.FrameOptions.Photometric = TiffPhotometrics.Ycbcr;
break;
case TiffCompressions.Lzw:
copiedFrame.FrameOptions.Compression = TiffCompressions.Lzw;
copiedFrame.FrameOptions.BitsPerSample = new ushort[] { 8 };
copiedFrame.FrameOptions.Photometric = TiffPhotometrics.Rgb;
copiedFrame.FrameOptions.Predictor = TiffPredictor.Horizontal;
break;
case TiffCompressions.Ojpeg:
copiedFrame.FrameOptions.Compression = TiffCompressions.Jpeg;
copiedFrame.FrameOptions.BitsPerSample = new ushort[] { 8, 8, 8 };
copiedFrame.FrameOptions.Photometric = TiffPhotometrics.Ycbcr;
break;
default:
break;
}
//Add copied frame to the destination image
result.AddFrame(copiedFrame);

}
//The first frame is created by default, we may just remove it as it is empty
//But it is not possible to remove the empty frame unless there are more than one frames
if (result.Frames.Length > 1)
{
//Set next frame as Active
result.ActiveFrame = result.Frames[1];
//Remove the first frame
result.RemoveFrame(0);
}

// Save the image with changes
result.Save();
}
}
}
catch (Exception ex)
{
Console.WriteLine("FixTiffFile: " + ex.Message);
Console.WriteLine("FixTiffFile: " + ex.StackTrace);
Console.ReadKey();
}
}

Hi Michael,

Thank you for the sample code.

Please, visit the link End User License Agreement. If the thread is marked as private, we are bound not to disclose any information or data of the customer that he shares with us. Please, mark the thread as private and feel safe to send us your information or data. I am again suggesting you to share a sample application that present complete scenario of the situation and sample files/images after bench marking the memory consumption against each sample. We will recreate the scenario at our end and will update you accordingly about our finds.

Looking forward for your valuable response.

No matter what your End User License Agreement states, I will get fired for sending this information anywhere outside our organisation... :| So That's not gonna happen, sorry.
But couldn't you just try to reproduce with a bunch of TIF files of various sizes and see what happens?

Anyways:
Now I've made a workaround that restarts the application for every 20 files, this seems to make sure the ressources are released and things run in an acceptable pace.

/Michael

Hi Michael,

It is good to know that you have a workaround for the said problems and you are carrying out the process. We are working on our end on the issue. We will update you accordingly. We are sorry for the inconvenience caused.

Hi Michael,

Please check the following piece of code. I have made two significant changes in order to improve the memory consumption.

  • The code now uses only the disk for storing the intermediate data. The option (Cache.CacheType = CacheType.CacheOnDiskOnly) will effect the performance in terms of execution time a bit, however, the memory usage will be decreased significantly.
  • Instead of iterating over the Tiff frames and changing the compression for each frame, the following code changes the compression of the Tiff image as a whole where decision is being made on the compression of active Tiff frame. If this solution suits your need, you can try it with CacheType.Auto as well.


Please give the updated code a try against your actual batch of file and let us know of your feedback.

C#

DirectoryInfo di = new DirectoryInfo(folderPath);
foreach (FileInfo fi in di.EnumerateFiles())
{
string fileName = fi.Name;
// By default the cache folder is set to the local temp directory.
// You can specify a different cache folder from the default this way:
Cache.CacheFolder = “D:/Temp”;

// Auto mode is flexible and efficient
Cache.CacheType = CacheType.Auto;

// The default cache max value is 0, which means that there is no upper limit
Cache.MaxDiskSpaceForCache = 0;
Cache.MaxMemoryForCache = 1073741824; // 1 gigabyte

// We do not recommend that you change the following property because it may greatly affect performance
Cache.ExactReallocateOnly = false;

//Load input Tiff image
using (TiffImage source = (TiffImage)Aspose.Imaging.Image.Load(fi.FullName))
{
TiffOptions saveOptions = null;
var compressionType = source.Frames[0].FrameOptions.Compression;
switch (compressionType)
{
case TiffCompressions.Jbig:
case TiffCompressions.Jp2000:
case TiffCompressions.Jpeg:
case TiffCompressions.Ojpeg:
saveOptions = new Aspose.Imaging.ImageOptions.TiffOptions(TiffExpectedFormat.TiffJpegRGB);
break;
case TiffCompressions.Lzw:
default:
saveOptions = new Aspose.Imaging.ImageOptions.TiffOptions(TiffExpectedFormat.TiffLzwBW);
break;
}
source.Save(folderPath + “out” + fi.Name, saveOptions);
}
}

Hi,

Thanks, the code here seems to work without too much memory consumption, that's great! However the resolution of created tiff is 96 dpi and not 200 dpi as the original, but that's a minor challenge... :)

Thanks again
Michael

Hi Michael,


It is good to know that you are observing improvement in the memory usage with code segment provided earlier. Regarding your recent concerns, although the code is not altering the resolution of the image, however, it is best to coupe with this problem as follow.

C#

DirectoryInfo di = new DirectoryInfo(folderPath);
foreach (FileInfo fi in di.EnumerateFiles())
{
//Load input Tiff image
using (TiffImage source = (TiffImage)Aspose.Imaging.Image.Load(fi.FullName))
{
var verticalResolution = source.VerticalResolution;
var horizontalResolution = source.HorizontalResolution;
ResolutionSetting resolution = new ResolutionSetting(horizontalResolution, verticalResolution);

TiffOptions saveOptions = null;
var compressionType = source.Frames[0].FrameOptions.Compression;
switch (compressionType)
{
case TiffCompressions.Jbig:
case TiffCompressions.Jp2000:
case TiffCompressions.Jpeg:
case TiffCompressions.Ojpeg:
saveOptions = new Aspose.Imaging.ImageOptions.TiffOptions(TiffExpectedFormat.TiffJpegRGB);
break;
case TiffCompressions.Lzw:
default:
saveOptions = new Aspose.Imaging.ImageOptions.TiffOptions(TiffExpectedFormat.TiffLzwBW);
break;
}
saveOptions.ResolutionSettings = resolution;
source.Save(folderPath + “out” + fi.Name, saveOptions);
}
}