Huge PST file when there were lots of addresses and slow

Aspose.Email Product Family
1

I reported it in a different thread. Will report it here with a sample code which can reproduce the issue.
1. 1000 emails, each has 3000 addresses
2. The total size of 1000 emails is 50MB
3. The size of PST is: 1.08GB, 20 times larger than original size
4. It took 19 minutes to add the messages to PST which is too slow

So issues
1. PST file is 20 times larger than original emails
2. 19 minutes to add 1000 small messages (50KB each) which is too slow

Attached is the source code. Just run it with parameters:
–pstFile /tmp/lotsofaddr.pst --count 1000 --outputDir /tmp/outdir


2

Hi Curtis,


Thank you for posting your inquiry.

1. We tested this issue at our end using the latest version of Aspose.Email for Java 5.8.0 and our findings are that the PST size is far less than the email files created. It seems you are comparing the PST size with the original 1000 EML files that indeed is very small as compared to the PST. But please note that its the MSG files and not the EMLS that are added to a PST. If you save the MapiMessage to disc, you will not that each message size is on average 5 MB and the combined size of 1000 such messages is 5 GB, which is far greater than the PST size as you reported. We are afraid that this increase in size from MailMessage to MapiMessage is inevitable and can’t be avoided at this stage.

2. With respect to speed, we modified your sample file a little in a way that we didn’t save the EMLs to disc (to avoid I/O operations) and also stopped adding the messages to PST. In this case where there were no disc saving and PST operations, the application took 905 seconds (15 minutes). This shows that maximum time is consumed by the message creation process in this case and 1/4th of the total time is taken by the messages addition operation to PST. Can you please verify this at your end and share your feedback with us?
3

Thanks.

I created the message using independentsoft’s jmsg, and the msg file’s size is 2.34 MB.
Also, it’s performance is way better than Aspose.email.
It took only 1.3 seconds to create 1000 message files.
And it took minutes for Aspose to create MapiMessage

So the performance could be improved.

4

Hi Curtis,


Thank you for sharing your concern.

Can you please share if you are directly creating MSG file or converting MIME message to MSG using the IndependentSoft JMsg API? This will help us identify the exact issue and log it for investigation by our Product team. If possible, please share the sample code with us for our investigation and performance comparison.
5

Here is the source code:



I only added 2000 To addresses because JMSG only allow these number of addresses. The msg size is: 2.44MB
Total time used to create 1000 messages: 2452ms

import java.io.IOException;
import java.util.Date;
import java.util.List;

import com.independentsoft.msg.DisplayType;
import com.independentsoft.msg.Encoding;
import com.independentsoft.msg.Message;
import com.independentsoft.msg.ObjectType;
import com.independentsoft.msg.Recipient;
import com.independentsoft.msg.RecipientType;

public class JmsgCreateApp {
    
    public static void main(String[] args) throws Exception {
        long startTime = System.currentTimeMillis();
        for(int i=0; i<1000; i++) {
            createMessage(i);
        }
        System.out.println(System.currentTimeMillis() - startTime);
        System.out.println("DONE");
    }
    
    
    private static Message createMessage(int msgCount) throws IOException {
     
        int totalAddr = 2000;
        
        StringBuilder toLine = new StringBuilder();
        for(int i=0; i<totalAddr; i++) {
            toLine.append(msgCount + "_" + i + "@foo.com, ");
        }
        toLine.append("\n");
        
        Message msg = new Message();
        msg.setEncoding(Encoding.UNICODE);
        msg.setClientSubmitTime(new Date());
        msg.setMessageDeliveryTime(new Date());
        msg.setSubject("test");

        msg.setSenderEmailAddress("from@foo.com");
        msg.setSenderName("from@foo.com");
        
        List recipients = msg.getRecipients();
        for(int i=0; i<totalAddr; i++) {
            Recipient recipient = new Recipient();
            recipient.setAddressType("SMTP");
            recipient.setDisplayType(DisplayType.MAIL_USER);
            recipient.setObjectType(ObjectType.MAIL_USER);
            recipient.setEmailAddress(msgCount + "_" + i + "@foo.com");
            recipient.setDisplayName(msgCount + "_" + i + "@foo.com");
            recipient.setRecipientType(RecipientType.TO);
            recipients.add(recipient);
        }
        
        msg.setDisplayTo(toLine.toString());
        
        msg.setBody("body");
        
        //msg.save("/temp/jmsg_" + msgCount + ".msg");
        
        return msg;
    }

}<totaladdr; i++)="" {="" toline.append(msgcount="" +="" "_"="" i="" "@foo.com,="" ");="" }="" toline.append("\n");="" message="" msg="new" message();="" msg.setencoding(encoding.unicode);="" msg.setclientsubmittime(new="" date());="" msg.setmessagedeliverytime(new="" msg.setsubject("test");="" msg.setsenderemailaddress("from@foo.com");="" msg.setsendername("from@foo.com");="" list
6

Hi Curtis,


Thank you for sharing the sample code.

We have compared the performance of Aspose.Email for Java API with IndependentSoft.de API and have logged the problem as performance bug with id: EMAILJAVA-33545. With respect to the message size issue, the MSG file generated by Aspose.Email API is 3.3 MB as compared to the one generated by the JMSG API which is 2.4 MB which is not a big difference. Please let us know if you have any further concerns about this problem.

7

The issues you have found earlier (filed as EMAILJAVA-33545) have been fixed in this update.

This message was posted using Notification2Forum from Downloads module by Aspose Notifier.
8

Just tested aspose 5.9 using the test application PSTSizeCheckApp.zip I set to you a while ago.
It still took 17 minutes to add 1000 messages to PST which is not improved.
With aspose 5.8, it took around 17 minutes too.

9

Hi Curtis,

Thank you for sharing your feedback.

We logged this issue as “MapiMessage creation is slow” in light of our findings where we found that adding this much large recipients consume lot of time. This issue was investigated at our end and we have improved this performance issue. You can compare the old and new performance by the following comparison code and update your code to reflect this change for improvement. Here are the performance improvement results at our end:

Time taken for Creating 10 MapiMessages with Old method

Execution Started: Wed Nov 11 14:40:46 PKT 2015

Execution Finished: Wed Nov 11 14:45:34 PKT 2015

Time taken for Creating 10 MapiMessages with New method

Execution Started: Wed Nov 11 14:45:34 PKT 2015

Execution Finished: Wed Nov 11 14:45:37 PKT 2015

Old Code Sample:

System.out.println("Execution Started: " + new Date());

for (int iMsg = 0; iMsg < 10; iMsg++)

{

[//System.out.println](https://system.out.println/)(“Creating Msg # " + iMsg + " at " + new Date());

MapiMessage mapiMsg = new MapiMessage();

//add 2000 recipients

for (int i = 0; i < 2000; i++)

mapiMsg.getRecipients().add(“to”+i + "@domain.com”, “to” + i, MapiRecipientType.MAPI_TO);

}

System.out.println("Execution Finished: " + new Date());

**New Code Sample**

System.out.println("Execution Started: " + new Date());
for (int i=0;i<10;i++)
{
MapiRecipientCollection rc = new MapiRecipientCollection();
for (int a = 0; a < 2000; a++)
{
rc.add("to" + a + "@domain.com", "to " + a, MapiRecipientType.MAPI_TO);
}
MapiMessage mapiMsg1 = new MapiMessage();
mapiMsg1.setRecipients(rc);
}
System.out.println("Execution Finished: " + new Date());