How to deal with huge attachment like cloud attachments without OOM

[schukka]

We have a requirement where need to get the cloud attachment and add as an attachment to MailMessage, below is the sample code, but with this we are running into OOM issue. Is there any way to add huge attachments?
Sample code:

        EmlLoadOptions loadOptions = new EmlLoadOptions();
        loadOptions.setPreserveTnefAttachments(true);
        loadOptions.setPreserveEmbeddedMessageFormat(true);

		MailMessage mailMessage = MailMessage.load(inputStream, getEmlLoadOptions());
		
		InputStream cloudAttachmentStream = restClient.getCloudFile(exAttachment.getSourceUrl()); // restClient.getCloudFile will get the cloud attachment data as a inputstream
		
		Attachment attachment = new Attachment(cloudAttachmentStream, fileName); //This is giving OOM, we have cloud attachment of size 5gb (even for 500mb we get same issue).

		Error:
		java.lang.OutOfMemoryError: Java heap space
		at com.aspose.email.system.io.MemoryStream.setCapacity(Unknown Source) ~[aspose-email-23.4-jdk16.jar:23.4]
		at com.aspose.email.system.io.MemoryStream.b(Unknown Source) ~[aspose-email-23.4-jdk16.jar:23.4]
		at com.aspose.email.system.io.MemoryStream.write(Unknown Source) ~[aspose-email-23.4-jdk16.jar:23.4]
		at com.aspose.email.zbfq.a(SourceFile:105) ~[aspose-email-23.4-jdk16.jar:23.4]
		at com.aspose.email.zbfq.a(SourceFile:84) ~[aspose-email-23.4-jdk16.jar:23.4]
		at com.aspose.email.AttachmentBase.a(SourceFile:551) ~[aspose-email-23.4-jdk16.jar:23.4]
		at com.aspose.email.AttachmentBase.<init>(SourceFile:110) ~[aspose-email-23.4-jdk16.jar:23.4]
		at com.aspose.email.Attachment.<init>(SourceFile:79) ~[aspose-email-23.4-jdk16.jar:23.4]

[Professionalize.Discourse]

@schukka

Handling large attachments, such as those exceeding several hundred megabytes or even gigabytes, can lead to memory issues when using the Aspose.Email library. The error you are encountering, java.lang.OutOfMemoryError: Java heap space, indicates that the application is running out of memory while trying to process the attachment.

Here are some strategies to deal with large attachments without running into out-of-memory (OOM) issues:

1. Use Reference Attachments: Instead of attaching the file directly to the email, consider using a reference attachment. This method allows you to include a link to the file rather than the file itself, which can significantly reduce memory usage. You can find more information on how to implement reference attachments in the Aspose documentation here.

2. File Sharing Services: For very large files, it may be more efficient to upload the file to a cloud storage service and then share a download link in the email. This approach avoids the need to handle large files directly in your application, thus preventing memory issues.

3. Increase Java Heap Size: If you must attach large files directly, consider increasing the Java heap size allocated to your application. This can be done by setting the -Xmx parameter when starting your Java application, for example:

   java -Xmx2048m -jar your-application.jar
   

   This command increases the maximum heap size to 2048 MB. Adjust the value based on your system’s capabilities.

4. Stream the Attachment: If you are fetching the attachment from a cloud service, ensure that you are streaming the data efficiently. Instead of loading the entire file into memory, process it in chunks if possible.

By implementing these strategies, you should be able to manage large attachments more effectively and avoid running into memory issues.

Sources:
[1]: Adding a reference attachment

[schukka]

@Professionalize.Discourse

ing the data efficiently. Instead of loading the entire file into memory, process it in chunks if possible.

We have requirement to load cloud attachment as a attachment, not as a reference link. So is there any way that com.aspose.email.Attachment will not load the entire attachment inside memory, but only process when we do msgMessage.save(fileOutputStream, getMsgSaveOptions());

[margarita.samodurova]

@schukka
We have opened the following new ticket(s) in our internal issue tracking system and will deliver their fixes according to the terms mentioned in Free Support Policies.

Issue ID(s): EMAILJAVA-35326

You can obtain Paid Support Services if you need support on a priority basis, along with the direct access to our Paid Support management team.

[schukka]

I see EMAILJAVA-35326 status is resolved, what is the resolution step where can i find it?

[sergey.vivsiuk]

Hello @schukka,

A separate API method was created for adding an attachment as a stream.

MapiMessage msg...
// add streamed attachment (lazy loading)
msg.getAttachments().add("test.bin", inputStream);
// save message with streamed attachment
msg.save("output");

The changes will be available in the next release 24.11, which will be released on December 6th.

[schukka]

Hi,

With this ( msg.getAttachments().add(“test.bin”, inputStream); ) we are not able to get the mime type of the attachment, we are using MapiAttachment.getMimeTag() to get the mime type, is there any way we can set this property, if so, based on the attachment type how to set this property? else can you provide the fix with this property set?

Thanks.

[schukka]

@sergey.vivsiuk

.save("output");

With this fix, now, we are getting attachment with empty content and mime type is missing (zero content attachments).

[margarita.samodurova]

Hello @schukka,

Thank you for sharing the details.
To investigate further, we recommend providing us with the following information:

• A sample attachment file that reproduces the issue.
• The output message file showing the problem.

To set the MimeTag, you can use the setProperty method as shown below:

attach.setProperty(KnownPropertyList.ATTACH_MIME_TAG, "image/jpeg");  

This approach allows you to explicitly define the MIME type for your attachment.

[schukka]

@margarita.samodurova

his approach allows you to explicitly define the MIME type for your attachment.

Hi,

We are attaching the attachment dynamically to the email, you can take any file and convert it into input stream and use the code you provided as a fix to attach it to the email, and save it to message file, when we open the msg file attachment is shown with no content.
For you reference attaching the attachment which is image file in our case and out message file here.
aspose_attachment.zip (37.2 KB)

Thanks.

[margarita.samodurova]

Hello @schukka,
We have opened the following new investigation ticket in our internal issue tracking system and will deliver their fixes according to the terms mentioned in Free Support Policies.

Issue ID(s): EMAILJAVA-35335

You can obtain Paid Support Services if you need support on a priority basis, along with direct access to our Paid Support management team.

[sergey.vivsiuk]

Hello @schukka

You can add a property to the new attachment as demonstrated in the example below

MapiMessage msg...
// add streamed attachment (lazy loading)
msg.getAttachments().add("test.bin", inputStream);
// set attachment property
msg.getAttachments().get(0).setProperty(KnownPropertyList.ATTACH_MIME_TAG, "image/jpeg");
// save message with streamed attachment
msg.save("output");