Java Heap Space errors with large emails

Aspose.Email Product Family
1

Aspose Support Team,

We continue to see intermittent issues with loading large emails or emails with large attachments (sample would be attached if the zip file could be uploaded but multiple attempts to upload the file as a 7Zip have failed). We’re using Java Aspose version 23.9 and attempting to load the contents of this email - approximately 1.1GB - an email with a large mp4 attachment. Here’s an example (below) of what we’re doing in the context of our code. The OOM errors occur when using a Linux server with JVM max memory at 4GB. When the max memory is increased to 6GB, the file load just seems to hang or not respond. Two questions from this:

  1. If you tell us to increase the amount of memory to handle files of this type, is there a way to estimate the amount of memory that’s required for a 1GB file for example? Do we need 4x, 8x, etc. amount of memory to handle a file of this size?

  2. Is there a way to change the load options somehow so that a file of this size can be handled more efficiently with regards to memory usage, i.e., maybe using disk instead of memory? If not, can you suggest any other workarounds besides increasing the machine’s available memory?

“StackTrace”:"java.lang.OutOfMemoryError: Java heap space\n\t
at com.aspose.email.system.io.MemoryStream.setCapacity(Unknown Source)\n\t
at com.aspose.email.system.io.MemoryStream.b(Unknown Source)\n\t
at com.aspose.email.system.io.MemoryStream.write(Unknown Source)\n\t
at com.aspose.email.zaqw.d(SourceFile:653)\n\t
at com.aspose.email.zaqw.a(SourceFile:344)\n\t
at com.aspose.email.zaqq.a(SourceFile:308)\n\t
at com.aspose.email.zaqq.a(SourceFile:336)\n\t
at com.aspose.email.zaqq.b(SourceFile:174)\n\t
at com.aspose.email.MailMessage.b(SourceFile:1596)\n\t
at com.aspose.email.MailMessage.a(SourceFile:1577)\n\t
at com.aspose.email.MailMessage.c(SourceFile:1491)\n\t
at com.aspose.email.MailMessage.load(SourceFile:1436)\n\t

2

Hello @jmuth,

Email communication doesn’t involve large-sized attachments. Typically, attachments should not exceed a size of several tens of megabytes.
Handling very large attachments can be quite resource-intensive, and Aspose.Email library is encountering memory limits.
For attachments this large, it might be worth considering an alternative method of sharing, such as providing a download link via a file-sharing service rather than attaching the file directly to the email.

To transfer larger files, reference attachment is used. A reference attachment is a type of attachment that includes a link or a reference to a file or item, rather than including the file or item itself in the email message.

3

Thanks for your quick response, Margarita.

Unfortunately, however, this data originates from outside our company so we need the ability to load our client’s data that has emails with large attachments like this. It’s not like we have a choice in the matter on how the email is constructed.

Please provide other suggestions on how to handle this type of email using the Aspose libraries. Thank you.

4

Hello @jmuth,

Unfortunately, there is nothing we can do in this regard. Email formats are not intended for large data transport. Aspose.Email was designed with these restrictions taken into account.
Anyway, we can discuss with developers what options may be added to solve the issue.
Please share more details about your case (what classes are used, where do you load messages from, how do you handle emails, sample code, etc.).

Thank you.

5

Margarita,

Thanks for clarifying but not sure that’s going to help us handle emails of this type. I didn’t include a code snippet because I thought it was evident from the stack trace but our use case is about as simple as you might imagine:

String filePath = “path_to_large_email_file.msg”;
MsgLoadOptions mlo = new MsgLoadOptions();
mlo.setKeepOriginalEmailAddresses(false);
MailMessage mailMessage = MailMessage.load(filePath, mlo); <===== this is where the error occurs

Please let me know if you need anything else. Thank you.

6

@jmuth,

Please clarify how you further use the loaded message?
Are you changing any properties?
Are you saving it in any other format?
Are you just reading the properties?
Are you processing attachments?
Also, you use the MailMessage class. Have you tried using the MapiMessage class?

I apologize for asking so many questions. We need to know the details to try resolving the problem.

7

Hi, Margarita.

Here are answers to your follow-up questions:

  1. We are NOT changing properties
  2. We are NOT saving the email in another format
  3. We ARE reading the metadata properties of the email
  4. We would attempt to process the email’s attachments if we could get it to load but never make it far enough to do that
  5. We see similar issues when using either the MailMessage or the MapiMessage classes.

Please let me know if you need any more information. Thank you.

8

Hello @jmuth,

Thank you for the answers.
We’ll discuss this case with the developers and let you know a solution as soon as possible.

9

@jmuth ,

Could you please provide us with the message file you are having problem with?
Upload the file to some cloud service like GoogleDrive and share a public link to the file.
We’ll make this topic private so that no outsiders can see your link.

Thank you.

10

Hi @margarita.samodurova
This file was shared with one of your co-workers under a paid support ticket - 176968

Thanks.

11

Hello @aweech,

I got it, thanks for the info.

12

Hello @aweech,

We are working on optimization methods for a large attachments.
Also, аfter processing the message, you can release resources using the following methods:

msg.close()

or

msg.dispose()

or

try (MapiMessage msg = MapiMessage.load("large_attachment.msg")) {
...
}

Code sample:

public final void test()
{
    for (int i = 0; i < 10; i++) {
        System.out.println(i + " iteration");
        try (MapiMessage msg = MapiMessage.load("large_attachment.msg")) {
            System.out.println(msg.getSubject());
            System.out.println(msg.getAttachments().get(0).getSize());
        }
    }
}
13

@jmuth
We have opened the following new ticket(s) in our internal issue tracking system and will deliver their fixes according to the terms mentioned in Free Support Policies.

Issue ID(s): EMAILJAVA-35231

You can obtain Paid Support Services if you need support on a priority basis, along with the direct access to our Paid Support management team.

14

Thank you for the update - I believe this has also been linked to a paid support ticket.

15

Hello @jmuth ,

Yes, it has also been linked to a paid support ticket.

16

Hello @jmuth,

You can load a message with MsgLoadOptions.LeaveOpen option to optimize memory usage:

MsgLoadOptions lo = new MsgLoadOptions();
lo.setLeaveOpen(true);
try (MapiMessage m = MapiMessage.load("large_attachment.msg", lo)) {
    System.out.println(m.getAttachments().get(0).getFileName());
    System.out.println(m.getAttachments().get(0).getDisplayName());
    m.getAttachments().get(0).save("att.dat");
}
17

Thank you, Sergey, we’ll give it a try.

18

Hello @jmuth ,

After you try, please spare a minute to share your feedback.

19

So with the sample file we provided, although the file has an “msg” file extension, internally the file is an “eml” and needs to be loaded using EmlLoadOptions. Since there does not appear to be an equivalent “setLeaveOpen” property for EmlLoadOptions, this suggestion may prove helpful for some actual “msg” files but doesn’t help in the case of our sample file which is an “eml”.

Thank you for investigating. We are continuing to look at this ourselves.

20

Hello @jmuth ,

Thank you for your feedback. We will consider this issue.