Free Support Forum - aspose.com

Outofmemory issue while converting large DWG file (121 MB)


#1

I am receiving outofmemory issue while converting DWG file and retrieving the text from this file. I am passing file as stream but some how inside the method Aspose.Cad.Image.Load() it is loading whole file into memory. I suppose this is causing the problem. Can you please confirm the solution to this problem. There is something wrong happening in this method.

Thanks! and hope to get solution soon…


#2

@nilesh.pasalkar,

I have observed your comments. Can you please share sample project along with source files to reproduce issue so that i may further investigate to help you out.


#3

I am using this code to process cad file…

    private void ProcessCADFile(string sSourceDWGFile, ref StringBuilder sbDWGText)
    {
        try
        {
            //Optional, below 3 lines can be commented : license setting of Aspose CAD
            using (FileStream stream = File.OpenRead(sSourceDWGFile))
            {
                if (Image.CanLoad(stream))
                {
                    CadImage cadImage = Image.Load(stream) as CadImage;
                    //{
                        // Search for text in the entities section
                        foreach (var entity in cadImage.Entities)
                        {
                            IterateCADNodes(entity, ref sbDWGText);
                        }

                        // Search for text in the block section 
                        foreach (CadBlockEntity blockEntity in cadImage.BlockEntities.Values)
                        {
                            foreach (var entity in blockEntity.Entities)
                            {
                                IterateCADNodes(entity, ref sbDWGText);
                            }
                        }
                        
                    //}
                }
                
            }
        }
        catch (Exception ex)
        {
            Trace.WriteLine(ex);
        }
    }

#4

Sample Autocad file you can download any file more than 100 MB on net.


#5

One thing I don’t understand is why it is using around 10GB memory size to load file specially only 100MB of size? and if we are passing stream still if internally it is loading whole file into memory then what is purpose of having overloaded method for stream?


#6

Sending source project is not possible at this is production code which I am not allow to share to someone. But I am sure your development team can create simple sample project for this, it is just matter of loading DWG file more than 100 MB using Aspose.Cad.Image.Load() method.


#7

@nilesh.pasalkar,

I have observed your comments. Can you please share the file that is reproducing the issue on your end so that we may log the issue if reproduced with that and resolve on the basis of that.


#8

File I am right now working on is the customer file, which I definitely can’t share with you. You can search any large file for more than 100 MB in size online and use it?


#9

@nilesh.pasalkar,

You are trying to load a huge file. I suggest you to please try using x64 based application on your end which give you enhanced program memory to avoid OutofMemory Exception. You can change the project settings to x64 rather than default x86. If there is still an issue then please feel free to share with us.


#10

Yes Sir,

I am using 64 bit application. my problem is for 102 MB of file it utilizing 10 GB of memory and then not releasing memory immediately after using Dispose method due to which I am after sometime running out of memory. This is my problem. Out of memory issue I have solved now, by turning application in to 64 bit. There is another help I need from you about how to release the utilized memory after disposing the CadImage object.

Thanks,


#11

@nilesh.pasalkar,

It’s good to know that exception issue is gone on your end after changing to x64 bit application. For issue related to memory not getting released on disposing the CadImage, an issue with ID CADNET-760 has been created in our issue tracking system to further investigate and resolve the issue.


#12

@nilesh.pasalkar,

We have done some investigation on largest file found in our test set, and had successfully freed memory after loading it.

Dispose(), by spec, should release unmanaged resources. In our case we do release file or stream the image has been loaded from. It is often expected that Dispose() fully clears the object and makes it unusable, but spec doesn’t dictate that and we don’t do that, as such, all loaded and processed data remains in memory.

To unload that data and free memory, one just has to null all references to loaded image and call garbage collection. Here is an example.

        Image img = null;
        try
        {
            img = Image.Load(filename);
            //do work here
        }
        finally
        {
            if(img!=null)
                img.Dispose();
            img = null;
        }

        GC.Collect(3); //mark objects in all heaps and trigger finalizers
        GC.WaitForPendingFinalizers();
        GC.Collect(3);//actual cleanup should happen now.

There is a nuance with using() statement. It holds a hidden reference to variable it declares, and it is not cleared until one returns from method where using() is used. As such, only call garbage collection after return from this method, like in the following example:

    void ProcessImageAndClean(string filename)
    {
        ImageProcessingMethod(filename);

        GC.Collect(3);
        GC.WaitForPendingFinalizers();
        GC.Collect(3);
    }


    void ImageProcessingMethod(string filename)
    {
        using (Image img = Image.Load(filename))
        {
            //do your work here
        }
    }

#13

I tried with that too but it is not releasing memory. Please see the following file which is giving me this problem the file size is 102 MB aprox but while loading into memory it is utilizing more than 10 GB of memory, why is this happening could you please explain to me. How can we tackle this problem.

Please download file from this link https://we.tl/t-enpchlNaXc


#14

@nilesh.pasalkar,

Thank you for sharing the feedback. I have associated the shared information in our issue tracking system and will get back to you as soon as possible. I also request you to please don’t remove the file from download link till the issue gets resolved.


#15

Any news on this?


#16

@nilesh.pasalkar,

We have investigated the issue in detail on our end. We have run tests with your file on a machine with more RAM and managed to successfully free memory, but there was a nuance. Regardless of whether using() construct was used, memory could only be freed after returning from method where image was loaded. This seems to be a behavior of framework with x64 mode and more available RAM, as in x86 mode behavior was just as I described previously.

Thus, in case without using() following code should be used:

void ProcessImageAndClean(string filename)
{
ImageProcessingMethod(filename);
//There should now be no references to image, as they are all contained in ImageProcessingMethod, and memory can be reclaimed
GC.Collect(3);
GC.WaitForPendingFinalizers();
GC.Collect(3);
}

    // This method must not return Image. 
    void ImageProcessingMethod(string filename)
    {
        Image img = null;
        try
        {
            img = Image.Load(filename);
            //do work here
        }
        finally
        {
            if(img!=null)
                img.Dispose();
            img = null;
        }
    }

And if using() us used then use following code (same as previously posted):

void ProcessImageAndClean(string filename)
{
ImageProcessingMethod(filename);

        GC.Collect(3);
        GC.WaitForPendingFinalizers();
        GC.Collect(3);
    }


    void ImageProcessingMethod(string filename)
    {
        using (Image img = Image.Load(filename))
        {
            //do your work here
        }
    }

Please note that there must be no references to image outside of ImageProcessingMethod. If this still won’t succeed, we’ll probably need to see a sample project used on your end, as there might be something else holding a reference.

As for why a 100Mb DWG file takes 10Gb of memory, Aspose.CAD’s primary role is to export a whole graphic representation of a CAD file to a vector image format, thus it reads and unpacks the whole file with all the objects, regardless of their size in the final image.


#17

Thanks for reply from your end, I will check making these changes in my code and lets see if that will help.
I will come back to you on this…


#18

@nilesh.pasalkar,

Sure, please feel free to contact us any time.