Problems with closing memory streams after using Concatonate in static method

Aspose.PDF Product Family
1

I use a static method to merge my documents into a single PDF document but cannot close any of my streams within the method. Attempting to use the subsequently produced document gives a stream closed error.

The items to merge parameter contains a collection of rendered Word and Cells documents. They are then converted to PDF file specification memory streams using a helper function appropriate to their rendered type (ConvertRenderObjectToFileSpecMemoryStream()).
I have commented out the close stream lines because any use of the resultant PDF document caused the Stream Closed error when they are present. Once commented out, the PDF document works when used elsewhere in my project.
I added the optimise() and save() calls and even the CloseConcatonatedStreams property in an attempt to try and force the document to finish using the streams so they could be closed but without success.

What should I do to make this work, tidy up and prevent possible memory leaks?

For reference, I am using the latest copy of Aspose.PDF v6.3.0.0

Thanks
Jon
        /// 

/// Merges the render docs to PDF.

/// 

/// 

public static Aspose.Pdf.Document MergeRenderDocsToPDF(Dictionary<string, object> itemsToMerge)

{

        <span style="color:#2b91af;">PdfFileEditor</span> pdfEditor = <span style="color:blue;">new</span> <span style="color:#2b91af;">PdfFileEditor</span>();
        Aspose.Pdf.<span style="color:#2b91af;">Document</span> returnItem = <span style="color:blue;">null</span>;

        <span style="color:green;">// Set the merge array to how many items are in the group</span>
        <span style="color:#2b91af;">MemoryStream</span>[] itemLists = <span style="color:blue;">new</span> <span style="color:#2b91af;">MemoryStream</span>[itemsToMerge.Keys.Count];
        <span style="color:#2b91af;">MemoryStream</span> finalDoc = <span style="color:blue;">new</span> <span style="color:#2b91af;">MemoryStream</span>();
        <span style="color:blue;">int</span> i = 0;

        <span style="color:green;">// Go through each group item in order and merge the data</span>
        <span style="color:blue;">foreach</span> (<span style="color:blue;">string</span> key <span style="color:blue;">in</span> itemsToMerge.Keys)
        {
            <span style="color:blue;">object</span> renderedDoc = itemsToMerge[key];

            <span style="color:blue;">if</span> (renderedDoc != <span style="color:blue;">null</span>)
                itemLists[i] = ConvertRenderObjectToFileSpecMemoryStream(renderedDoc, key);

            i += 1;
        }
        pdfEditor.CloseConcatenatedStreams = <span style="color:blue;">true</span>;
        <span style="color:blue;">if</span> (i > 1)
            pdfEditor.Concatenate(itemLists, finalDoc);
        <span style="color:blue;">else</span>
            finalDoc = itemLists[0];
        
        returnItem = <span style="color:blue;">new</span> Aspose.Pdf.<span style="color:#2b91af;">Document</span>(finalDoc);
        returnItem.Optimize();
        returnItem.Save();
        
        <span style="color:green;">// Tidy up the streams by closing them to prevent memory leaks</span>
        <span style="color:green;">//for (int j = 0; j < itemLists.Length; j++)</span>
        <span style="color:green;">//    itemLists[j].Close();</span>
        <span style="color:green;">//itemLists = null;</span>

        <span style="color:green;">// HACK not closing the stream because otherwise PDF file fails due to open stream.... need to test this for memory leak......</span>
        <span style="color:green;">//finalDoc.Close();</span>
        <span style="color:green;">//finalDoc = null;</span>

        <span style="color:blue;">return</span> returnItem;
    }
2

Hi Jon,

Thank you for the sample code. I tried using your code to test your issue but due to missing variable values and methods, I was unable to replicate the scenario at my end. Please create a sample application to show the issue. This will help us figure out the issue and reply back to you soon.

Thank You & Best Regards,

3

Please find attached a zip file containing a sample application that replicates this. There is a variable inside the code called 

closeStreamsAfterUsing=true / false
which you can change to test with and without closing. Also included are screen shots of red gate’s 
memory profiler and the console to show how the streams are left in memory even after the document has been set to 
null with no way of closing them.
To set up the project create a folder called temp and add the “out.doc” file to it then run the code.  
I have added pauses so you can take memory snapshots.  Press the G key to move onto the next stage.
Hopefully you should now have enough information to be able to replicate it your end…

Thanks
Jon

4

I have tried to tidy the code further so use this function to do the merge as it closes additional streams which are definitely not needed however the memory is still retained:-


        /// 

/// Merges the render docs to PDF.

/// 

/// 

public static Aspose.Pdf.Document MergeRenderDocsToPDF(string[] itemsToMerge, bool closeSteamsAfterUsing)

{

        <span style="color:#2b91af;">PdfFileEditor</span> pdfEditor = <span style="color:blue;">new</span> <span style="color:#2b91af;">PdfFileEditor</span>();
        Aspose.Pdf.<span style="color:#2b91af;">Document</span> returnItem = <span style="color:blue;">null</span>;

        <span style="color:green;">// Set the merge array to how many items are in the group</span>
        <span style="color:#2b91af;">MemoryStream</span>[] itemLists = <span style="color:blue;">new</span> <span style="color:#2b91af;">MemoryStream</span>[itemsToMerge.Length];
        <span style="color:#2b91af;">MemoryStream</span> finalDoc = <span style="color:blue;">new</span> <span style="color:#2b91af;">MemoryStream</span>();
        <span style="color:blue;">int</span> i = 0;

        <span style="color:green;">// Go through each group item in order and merge the data</span>
        <span style="color:blue;">for</span> (<span style="color:blue;">int</span> j = 0; i < itemsToMerge.Length; j++)
        {
            <span style="color:#2b91af;">Document</span> docWord = <span style="color:blue;">new</span> <span style="color:#2b91af;">Document</span>(itemsToMerge[j]);
            itemLists[i] = ConvertWordDocToFileSpecificationMemoryStream(docWord, itemsToMerge[j] + j.ToString());

            i += 1;
        }
        
        <span style="color:blue;">if</span> (i > 1)
            pdfEditor.Concatenate(itemLists, finalDoc);
        <span style="color:blue;">else</span>
            finalDoc = itemLists[0];

        returnItem = <span style="color:blue;">new</span> Aspose.Pdf.<span style="color:#2b91af;">Document</span>(finalDoc);
        returnItem.Optimize();
        returnItem.Save();

        pdfEditor.CloseConcatenatedStreams = <span style="color:blue;">true</span>;

        <span style="color:green;">// Tidy up the streams by closing them to prevent memory leaks</span>
        <span style="color:blue;">for</span> (<span style="color:blue;">int</span> j = 0; j < itemLists.Length; j++)
            itemLists[j].Close();

        itemLists = <span style="color:blue;">null</span>;

        <span style="color:blue;">if</span> (closeSteamsAfterUsing)
        {
            <span style="color:green;">// HACK not closing the stream because otherwise PDF file fails due to open stream.... need to test this for memory leak......</span>
            finalDoc.Close();
            finalDoc = <span style="color:blue;">null</span>;
        }

        <span style="color:blue;">return</span> returnItem;
    }</pre></div>
5

Hi Jon,

Thank you for the sample code. I am able to reproduce the issue with your shared application and I have registered your issue in our issue tracking system with issue id:PDFNEWNET-31567. Our development team will further look into this issue and I will update you as soon as I get any feedback from their side.

Sorry for the inconvenience,

6

Great, I look forward to hearing from you shortly. This is really important for me so I would appreciate feedback as soon as possible.

Regards
Jon
7

Just so you know, the expected functionality from my point of view should be that if I instantiate a document from a stream, the stream can be closed immediately afterwards if I choose to.

I shouldn’t have to call a Save() method or set a parameter to allow closing of the streams - that should all be managed entirely by me. The document object should therefore be in a usable state from first instantiation.
Hope this helps
Thanks
Jon
8

Hi there, Is there any progress on this (and have your developers managed to reproduce it too) as combining documents is a key part of the application we are writing and I need a rough ETA to give my boss for when we could consider being able to release our application.

regards
Jon
9

Hi Jon,

Our development team is still working hard to get this issue resolved and we hope to get this issue fixed, shortly. Please be patient and spare us little time. We are really sorry for this inconvenience.

10

ok, cool. As mentioned this is quite important to us so because it affects when we can go live so if you have any hotfixes that may resolve it I will be happy to help test… :slight_smile:

11

Hi there,

Any news or rough ETA on a fix for this as we want to put our application live shortly and need to know whether we will need to code in a #hack to cope with this leak?

thanks
Jon
12

Hi Jon,<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />

Thank you for your patience.

Our development team has looked into your reported issue and following is there feedback which will hopefully help you in resolving your issue.

Streams created in your code should be closed in the code as well. CloseConcatenatedStreams closes all streams (including output stream) and should be set to false as it will close the output stream as well. This way all the data will be lost before the save process is executed.

Please see the following sample scenario:

//function to load files to memory streams

private MemoryStream[] GetMemoryStreams(string[] files)

{

MemoryStream[] result = new MemoryStream[files.Length];

for(int i = 0; i < files.Length; i++)

{

result[i] = new MemoryStream();

byte[] buffer = new byte[1024];

int read;

FileStream fs = new FileStream(TestSettings.GetInputFile(files[i]), FileMode.Open);

while ((read = fs.Read(buffer, 0, buffer.Length)) > 0)

{

result[i].Write(buffer, 0, read);

}

fs.Close();

}

return result;

}

//get sources memory streams.

Stream[] streams = GetMemoryStreams(new string[]

{ "PdfWithSeveralPages.pdf",

"PdfWithImages2.pdf",

"PdfWithText.pdf" }

);

//create output memory stream.

MemoryStream outStream = new MemoryStream();

PdfFileEditor fe = new PdfFileEditor();

//We must not set CloseConcatenatedStreams to true.

//If ConcatenatedStreams is set to true source streams and output stream will be closed.

//(So, if output stream would be for example FileStream it will be written and closed).

//But we can't allow this because we use MemoryStream which must be accessible after Concatenate.

//That's why we will close streams manually.

fe.CloseConcatenatedStreams = false;

fe.Concatenate(streams, outStream);

//We can close all source streams here.

foreach (Stream stream in streams)

{

stream.Close();

}

Document doc = new Document(outStream);

//We can't close outStream here because it contains data used by Document.

//Closing outStream here (before Document.Save call) would cause "Stream is closed" error.

doc.Save(TestSettings.GetOutputFile("outfile.pdf"));

//After working with document and saving it we can close outStream.

outStream.Close();

Please make the changes in your code as mentioned above and let us know if this resolves the issue at your end.

Thank You & Best Regards,

13

Hi there,

Thanks for your reply. Actually, it was only by accident that my example wasn’t closing the streams (I had the close streams property after the save instead of before it)
Moving it to before it in order to test your comments completely negates the whole point of the concatenate function. If it closes ALL of the streams INCLUDING the stream it is concatenating to then that has to be a bug - either that or a very pointless property/method! This was easy enough to test with the sample sent.
As it is, leaving CloseConcatonatedStreams at false as was originally the case still doesn’t work even though I am calling the Save() function. I would expect this to finalise the document and leave me free to close my stream when I want. Really the stream should be finished with and the document in a working state as soon as I have called:-
 returnItem = new Aspose.Pdf.Document(myMemoryStream);
My code just to remind you is as follows (itemLists is an array of streams):- 
            pdfEditor.Concatenate(itemLists, finalDoc);

returnItem = new Aspose.Pdf.Document(finalDoc);

returnItem.Optimize();

returnItem.Save();

I am already calling the Optimize() and Save() functions just to pretend I am working with the document as you suggest. I cannot save the document to a physical file because I need it to be a memory stream as the document is stored in a database and potentially sent back to the user in an HTTP request so putting it to a physical file and reloading it is not practical or a valid solution.
Hope this helps with moving to a practical solution

Thanks
Jon
14

btw (I did re-run all my tests using the latest version just in case you had fixed it by accident since there were several concatenate fixes in the last release)

15

Hi there, any further progress with this or a rough eta?

16

Hi, has now been several weeks waiting for a reply on this. Slight mistype in my previous post, I want a complete in memory document back not a memory stream. Calling the Save() function should provide this and free up my streams.


Is there an ETA of when this will be fixed/put in the queue?

Thanks
17

Hi Jon,<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />

Please accept our apologies on a delayed response.

I have requested our development team to share further details and ETA regarding your issue. I will update you as soon as I get a response from our development team.

Sorry for the inconvenience,

18

<?xml:namespace prefix = o ns = “urn:schemas-microsoft-com:office:office” /><o:p>

Hi Jon,

Sorry for the delay in response.

We have tested a simple example where concatenated file is saved into a memory stream and memory stream is still usable after the concatenation and you can manually close the stream when required. Please see the below sample and let us know if you come up with the same results or your still face the problem of not able to close the stream manually.

PdfFileEditor pfe = new PdfFileEditor();

MemoryStream result = new MemoryStream();

pfe.CloseConcatenatedStreams = false;

//concatenated document is saved into result memory stream

pfe.Concatenate(

new Stream[]

{

new FileStream(@"D:\AP Data\testing.pdf", FileMode.Open),

new FileStream(@"D:\AP Data\Test_014.pdf", FileMode.Open)

},

result);

Document doc = new Document(result);

//we can make with the document anything, for example add new page

doc.Pages.Add();

//save the document into memory stream

doc.Save();

//memory stream is not closed and we can use it again

Document doc2 = new Document(result);

Console.WriteLine(doc2.Pages.Count);

JonBaggaley:
I want a complete in memory document back not a memory stream. Calling the Save() function should provide this and free up my streams.

I am not clear about your point above. Please further elaborate on that to give us a better idea.

Sorry for the inconvenience,

19

Hi there,

If you look at my sample, you will see that I am calling a static helper function. This is because I want to create, consume and close all the streams before passing back a completed document which has no reliance on streams being in continued existence (which I don’t want them to be because it is a function call).

To reiterate, calling Save() should put the document into a completed state. If I then choose to close or do something else with the stream used to create it, there should be NO impact on the created document which can now be passed around safely to other procedures and functions.
Please examine my sample and turn the “close stream” variable on and off then watch it in a memory profiler such as red-gate’s to see the leak that is currently created…
Hope this helps
20

The issues you have found earlier (filed as 31567) have been fixed in this update.

This message was posted using Notification2Forum from Downloads module by aspose.notifier.
(2)