Slow PST File Creation


I'm trying to generate a pst file that will contain about 13K messages. After about 5K messages, the performance slows to a crawl.

I've also noticed that it appears the pst file may be buffered in memory as I don't see any bytes written to disk during this time. (I'm passing a buffered FileOutputStream) to the PersonalStorage.create(). There is plenty (2G+) of available heap (all generations). Periodic stack traces show something like this:

java.lang.Thread.State: RUNNABLE
at com.aspose.email.ms.System.IO.f.c(Unknown Source)
at com.aspose.email.ms.System.IO.f.b(Unknown Source)
at com.aspose.email.ms.System.IO.b.a(Unknown Source)
at com.aspose.email.gM$b.a(Unknown Source)
at com.aspose.email.gM$b.b(Unknown Source)
at com.aspose.email.gM$b.a(Unknown Source)
at com.aspose.email.gM.a(Unknown Source)
at com.aspose.email.kA.a(Unknown Source)
at com.aspose.email.hu.(Unknown Source)
at com.aspose.email.eq.a(Unknown Source)
at com.aspose.email.eq.a(Unknown Source)
at com.aspose.email.hz.a(Unknown Source)
at com.aspose.email.FolderInfo.addMessage(Unknown Source)

I'm running the 2.8.0 release, on a linux/sun 1.6 jdk.

Are there performance limits, or suggestions on how to improve the performance, and limit the overhead (mem/cpu) when creating pst files?


Hi Aaron,


Thanks for writing to Aspose.Email support team.

We are analyzing the issue and request you to spare us little time so that we may establish the environment. We will write back here as soon as our investigation is completed and findings are ready to share.

Your patience and understanding is highly appreciated in this regard.

Hi Aaron,


I have tried to re-produce the issue here but am afraid to inform that I could not succeed. About 20K messages are used and added to a newly created PST but no drastic change is observed in the average message processing time during this process.

Could you please run the following sample program and send us back the output.txt file which contains statistics while messages are added to the PST. Similarly share your sample running code as console application. We would also request you to provide all the 13K messages which can be used to create PST and perform analysis.

static void CreatePSTAndAddMessageToPST()
{
PersonalStorage pst = PersonalStorage.create(“D:/LargeOutput.pst”, 0);
pst.getRootFolder().addSubFolder(“TestItems”);
FolderInfo folder = pst.getRootFolder().getSubFolder(“TestItems”);
AddMessagesToPST(folder);
}

static void AddMessagesToPST3(FolderInfo folder)
{
File folders = new File(“D:/TheEmls2/”);
File[] listOfFiles = folders.listFiles();
try
{
PrintWriter out = new PrintWriter(“D:/TheEmls2/Output.txt”);
int iCount = 0;
for (File file : listOfFiles)
{
if (file.isFile())
{
FileInputStream fin;
int iSize;
try
{
fin = new FileInputStream(“D:/TheEmls2/” + file.getName());
try
{
iCount++;
if(iCount % 500 == 0)
{
out.flush();
}
long startTime = System.currentTimeMillis();
iSize = fin.available();
MailMessage message = MailMessage.load(fin, FileCompatibilityMode.SkipValidityChecking);

//MailMessage message = MailMessage.load(fin,MessageFormat.getEml());
MapiMessage mapimsg = MapiMessage.fromMailMessage(message,OutlookMessageFormat.Unicode);
folder.addMessage(mapimsg);
long endTime = System.currentTimeMillis();
long totalTime = endTime - startTime;
System.out.println(iCount + "=>Size = " + iSize + "=>Time = " + totalTime);
out.println(iCount + "=>Size = " + iSize + "=>Time = " + totalTime);
fin.close();
}
catch (java.io.IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
catch (FileNotFoundException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
out.close();
}
catch (FileNotFoundException e1)
{
// TODO Auto-generated catch block
e1.printStackTrace();
}
}

Your cooperation and understanding is highly appreciated in this regard.

Thanks Kashif.


I’ll gather the statistics, and provide them with our code sample as well.

The only difference I can see is that we construct the MailMessage as follows:

MailMessage mailMessage = MailMessage.load(emlMessageInputStream, MessageFormat.getEml());

Hi Aaron,


Thank you for the feedback.

Sure. Please share your statistics with us once you are done with your testing. Meanwhile, I am performing some more tests on creating PST with such a large number of messages and shall share my findings with you soon.