Pdf.concatenate uses too much memory

When concatenating a large number of PDFs, the memory foot print of the process grows extremely large. In a test, I am concatenating 1000 pages of roughly 10K each in size resulting in a 11MB file (byte array returned by method). The process memory goes up by over 350MB while in the concatenate method and doesn’t go back down after the method until my program terminates.

We are planning on utilizing this for 50,000 plus pages, but in a test of just 10,000 pages (11K each) saw the memory foot print go in excess of 3gig.

We tried first creating several 100 page byte arrays and then having them concatenate, but the problem seems to be just with the total number of pages, not necessarily the number of concatenations.

Hi,

Can you please try using the following code snippet to avoid excessive memory utilization?


using Microsoft.Win32;


using System.Runtime.InteropServices;


public class MemoryManagement


{


[DllImportAttribute(“kernel32.dll”, EntryPoint = “SetProcessWorkingSetSize”, ExactSpelling = true, CharSet = CharSet.Ansi, SetLastError = true)]


private static extern int SetProcessWorkingSetSize(IntPtr process, int minimumWorkingSetSize, int maximumWorkingSetSize);


public static void FlushMemory()


{



GC.Collect();


GC.WaitForPendingFinalizers();


if (Environment.OSVersion.Platform == PlatformID.Win32NT)


{


SetProcessWorkingSetSize(System.Diagnostics.Process.GetCurrentProcess().Handle, -1, -1);


}


}


}

I hope this helps. If it doesn't resolve your issue, then please do let us know.

We're sorry for the inconvenience.
Regards,

I did add that code and assumed you would want me to call it after the concatenate call. It does immediately free the memory it used, but it still uses a tremendous amount of memory to do its process. 1,000 page document took 350megabytes of extra memory to do the concatenate to result in an 10 meg file. I’m trying to scale this to say a 50,000 page file.

Is it normal and needed for the concatenate to need so much memory to do its thing?

Hi,

I’m afraid to inform you that Concatenate method sometimes consumes extensive memory while processing some of the files. Can you please share the sample file you’re having problem with? We’ll test it at our end and try to see if we can improve the process and reduce the memory utilization.

We’re looking forward to help you out.
Regards,

I’ll upload the 1 page pdf I am testing with. I am simply trying to take this one pdf (for testing) and append it a whole bunch of times (like 1,000 or 10,000).

here is the one page pdf sample…it was created using your pdf.kit product.

Hi,

Thank you for sharing the sample file. We’ll look into the issue at our end and update you accordingly.

We’re sorry for the inconvenience.
Regards,

Hi,

We have logged the issues as PDFKITNET-13206 and PDFKITNET-13208, for Append and Concatenate methods respectively, in our issue tracking system. Our team will look into these issues and you’ll be updated via this forum thread once we improve the memory utilization by append and concatenate methods.


We’re sorry for the inconvenience.
Regards,

Is there an estimated time frame for a patch so we can plan accordingly?

Hi,

I’m very sorry to inform you that we’re unable to share the ETA for this issue at the moment. Our team is investigating the issue and we’ll be able to give you some idea in a week or two. We’ll update you with the estimate as we get some idea.

We appreciate your patience.
Regards,

I possibly have some relevant information. We had a similar memory issue (unrelated to this problem) where the garbage collector was not doing its job timely. Every time we used the DataTable.NewRow() function call, our memory foot print went up. As we were calling this perhaps a million times (testing), we saw our memory balloon. We changed our logic to reuse the row we already had with a .Clear() type function and our memory footprint scaled to only 15Mb vs almost 3 gig.

Hi,

Thank you very much for sharing this useful information. I have updated our development team with the information.

Regards,

The issues you have found earlier (filed as 13208) have been fixed in this update.


This message was posted using Notification2Forum from Downloads module by aspose.notifier.

The issues you have found earlier (filed as 13206) have been fixed in this update.


This message was posted using Notification2Forum from Downloads module by aspose.notifier.
(1)