We're sorry Aspose doesn't work properply without JavaScript enabled.

Free Support Forum - aspose.com

Serious memory problems

Aspose.Email Product Family
#1

I'm using Aspose.Email 1.1, and am seeing some serious memory issues that are causing my application to fail.

Aspose.Email.Outlook.Pst.PersonalStorage.ExtractMessage() and Aspose.Email.Outlook.MapiMessage.Save() are both throwing System.OutOfMemoryException. Examples are below.

These memory problems are occurring on a machine that has 4GB RAM, under conditions where it shouldn't be running out of memory. Can anything be done to address this?

Best regards,
Rob


System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at ’.‘.›‚(Byte[] , Byte[] )
at ’.˜.( , )
at ’.˜.œ( , ƒ , )
at ’.˜.›(UInt16 , UInt32 , )
at ’.˜.œ( )
at ’.˜..ctor(’ , Œ , )
at ’.œ.”( , ƒ )
at ’..‚( , MapiMessage )
at ’..‚(UInt32 , MapiMessage )
at Â’..ReadMessage(UInt32 )
at Aspose.Email.Outlook.Pst.PersonalStorage.ExtractMessage(Byte[] entryId)


System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at “.˜.e(Stream , Stream )
at “..ƒ(MemoryStream srcStream, Boolean isForceFat, Int32& sectorCount)
at “..ƒ(– , š )
at “..ƒ(– , š )
at “..Save(Stream )
at Aspose.Email.Outlook.MapiMessage.Save(Stream stream)
at Aspose.Email.Outlook.MapiMessage.Save(String fileName)


System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at ’.‘.›‚(Byte[] , Byte[] )
at ’.˜.( , )
at ’.˜.œ( , ƒ , )
at ’.˜.›(UInt16 , UInt32 , )
at ’.˜.œ( )
at ’.˜..ctor(’ , Œ , )
at ’.œ.”( , ƒ )
at ’..‚( , MapiMessage )
at ’..‚( , MapiMessage )
at ’..‚(UInt32 , MapiMessage )
at Â’..ReadMessage(UInt32 )
at Aspose.Email.Outlook.Pst.PersonalStorage.ExtractMessage(Byte[] entryId)

#2

Hi Rob,


Sorry for the delayed response.

I am trying to replicate your problem on our end. So far I have not succeeded to make my test application (VS2008 .NET Framework 2.0 project) throw your said exception. My largest PST file has more than 100 messages. I am able to extract these messages and save them to disk by using latest version of Aspose.Email for .NET v1.1.0.

Please find below my source code for reference.
C#

// load the Outlook PST file
PersonalStorage pst = PersonalStorage.FromFile(@“C:\temp\archive.pst”);

// Get the Root folder
FolderInfo root = pst.RootFolder;

//Get Contacts folder
FolderInfo inbox = root.GetSubFolder(“Inbox”);

// Loop through all the contacts in this folder
MessageInfoCollection messageInfoCollection = inbox.GetContents();

int i = 0;

foreach (MessageInfo messageInfo in messageInfoCollection)
{
i = i + 1;
MapiMessage message = (MapiMessage)pst.ExtractMessage(messageInfo).ToMapiMessageItem();
message.Save(@“C:\temp\messages” + i + “.msg”);
}

I suggest you share with us your sample project and if possible then your PST as well. So we can reproduce and isolate the problem for correction purposes. Also, please share the details of your development environment.

We apologise for any inconvenience caused.

Best Regards,

#3

I want to upload a PST containing the messages that cause the System.OutOfMemory exception, but the PST is 2.4 GB in size.

So I want to extract a few problem MSGs from this PST, but I find a problem: MessageInfo.EntryIdString looks different from the EntryId format I've seen before.

Specifically, I use a tool to extract messages from PST files based on the EntryId. The tool expects the EntryId in a format like "00000000FBC220167A57F744A74C90993B42D2D604242000", but MessageInfo.EntryIdString is a format like "AAAAAKvuHSSqOqZGsAz/I/O1HAMkViEA". What is the difference between the two? Can I get the longer format using Aspose.Email?

Thank you,

Rob

#4

Hi Rob,


Thank you for the details.

I also believe that the OutOfMemory exception is caused by some problematic messages in your PST file. You can identify these problematic messages as below and remove them from the PST file manually.

try { extractMessage () call }
catch { get the subject, recipient etc from MessageInfo and print on screen }

Once all such message have been removed from the file, please execute your program and see if it still throws the said exception. Also, provide the problematic messages in an archive for our review.

Regarding the difference in EntryId, can you please state what tool are you talking about. So we may compare it with the EntryId format produced by Aspose.Email for .NET.

Thank you for your cooperation.
#5

Babar,

Thank you for the information. You are right, the OutOfMemory exception is caused by some corrupt messages. My problem is that I need to handle these unexpected messages gracefully, and the OutOfMemory issue causes my application to bog down.

I use Outlook Spy to see the EntryID:

http://www.dimastr.com/outspy/home.htm

I realized the difference between the two EntryID formats --- Aspose is converting the byte[24] to base64, and Outlook Spy is converting it to hex.

Thank you,

Rob

#6

Hi Rob,


Thank you for your feedback on EntryId values.

To evaluate your actual issue, we definitely need some problematic messages from your PST file. Please try to extract such messages and attach them with your reply in an archive.

Thanks again for your patience.
#7

I have spent several hours working to get you the corrupt MSGs, but my attempts result in messages that are not corrupt anymore. Exporting from the original PST, and using a tool to build a PST subset with only the corrupt messages, both result in Outlook "fixing" the corrupt messages. They would be worthless to you for reproducing the out-of-memory error.

I will continue to try to get these messages to you in some corrupted form!

Best regards,

Rob

#8

Hi Rob,


Thank you for your efforts.

Just to bring in your knowledge that Aspose.Email for .NET v1.1.0 lets you create a PST file from scratch and add messages to any sub-folder of it. If you have the extracted messages, instead of using any other tool you can use Aspose.Email for .NET to create a PST subset.

Please check the below source code for your reference,

C#

// Create new PST
PersonalStorage pst = PersonalStorage.Create(“PersonalStorage.pst”, FileFormatVersion.Unicode);

// Add new folder “Inbox”
pst.RootFolder.AddSubFolder(“Inbox”);

// Select the “Inbox” folder
FolderInfo inboxFolder = pst.RootFolder.GetSubFolder(“Inbox”);

// Add some messages to “Inbox” folder
inboxFolder.AddMessage(MapiMessage.FromFile(“message-1.msg”));
inboxFolder.AddMessage(MapiMessage.FromFile(“message-2.msg”));

#9

Hi again,


Just to bring in your knowledge that with the release of Aspose.Email for .NET v1.2.0, you can now delete messages from any sub-folder of the PST file. Please check the sample code for your reference.

C#

// Select the “Inbox” folder
FolderInfo inboxFolder = pst.RootFolder.GetSubFolder(“Inbox”);
MessageInfoCollection messageInfoCollection = inboxFolder.GetContents();
foreach (MessageInfo messageInfo in messageInfoCollection)
{
Console.WriteLine(“Subject: " + messageInfo.Subject);
Console.WriteLine(“EnryID: " + messageInfo.EntryIdString);
Console.WriteLine(”------------------------------”);
if (Some condition)
{
inboxFolder.DeleteChildItem(messageInfo.EntryId);
Console.WriteLine(“Message Deleted”);
}
}


#10

Babar Raza,

Thank you for the info on creating new PSTs. That is an excellent feature that I've been waiting for.

However, I am unable to extract the corrupt MSGs from my PST because PersonalStorage.ExtractMessage() fails. See the first post at the top of this thread:

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
[...]
at Aspose.Email.Outlook.Pst.PersonalStorage.ExtractMessage(Byte[] entryId)

I have a corrupt PST I really want to send to you for analysis, but it belongs to my client. If you will sign a non-disclosure agreement doc, I will send you the file immediately for analysis. Please let me know if you will do this. Thank you!

Best regards,
Rob W

#11

Hi Rob,

Please check the End User License Agreement for clause 8. The Non-Disclosure clause is part of the agreement. When anyone downloads and uses Aspose component, this agreement automatically applies.

We insure you that we keep the user’s data only to replicate any issue and delete it as soon as the issue is resolved. Moreover, if you make your thread “Private”, no one else but Aspose Team members and the owner of thread can view or access it’s contents/attachments.

Still if you need a word from management then I can refer you to the concerned authorities.

Thanks and regards,

#12

Thank you for the NDA information. Where can I send you a 1.5 GB zip file privately, not on the forum?

#13

Hi Rob,


Normally we recommend our customers to create an account on Banckle and upload the file using Banckle File Sharing facility. But I am afraid that 1.5 GB is a very large data file even for Banckle.

I will check and get back to you on this issue.
#14

Hi Rob,


Will it be possible for you to arrange your own FTP or Web Server to host this file and provide us the anonymous account for download purposes?

Thanks and regards,
#15

Thank you very much for following up on this OutOfMemory exception. It is the MOST important and critical issue for my continued use of Aspose.Email.

As soon as I receive permission from my client to deliver broken PST files to you, I will immediately send you FTP information.

Thank you very much!

Best regards,

Rob

#16

An additional stack trace:

[Recovered Yogesh Bahl_ANSI 2-Dec-2011 15.55.34{d15920ea-d26e-4754-be7c-fcf52a3a6d99}.pst] - Error extracting MSG from PST (System.InvalidOperationException: Failed to compare two elements in the array. ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Globalization.TextInfo.nativeChangeCaseString(Int32 win32LangID, Void* pNativeTextInfo, String str, Boolean isToUpper)
at System.Globalization.TextInfo.ToUpper(String str)
at System.String.ToUpper(CultureInfo culture)
at ?.?..?.??(Object , Object )
at System.Array.BinarySearch(Array array, Int32 index, Int32 length, Object value, IComparer comparer)
--- End of inner exception stack trace ---
at System.Array.BinarySearch(Array array, Int32 index, Int32 length, Object value, IComparer comparer)
at System.Collections.SortedList.IndexOfKey(Object key)
at System.Collections.SortedList.get_Item(Object key)
at Aspose.Email.Outlook.MapiMessageReader.ReadMessage()
at Aspose.Email.Outlook.MapiMessage.FromStream(Stream stream)
at Aspose.Email.Outlook.MapiMessage.(? , Byte[] )
at Aspose.Email.Outlook.MapiMessage.Save(Stream stream)
at Aspose.Email.Outlook.MapiMessage.Save(String fileName)

Thank you,

Rob

#17

Hi Rob,


Thank you for your response.

It would be of great help that you also provide a working console application along with the sample PST to reproduce the exact exception at our end.

Looking forward to hear from you.

Regards,
#18

Babar Raza,

Please privately send me an email address to which I can send you a link to download broken PSTs and other info. Thank you very much!

Best regards,

Rob

[rob.wingate@discoverthewave.com](mailto:rob.wingate@discoverthewave.com)

#19

Hi Rob,


Sorry for the delayed response on this.

You can send me the details via a private message. Its is completely safe and reliable. As soon as I receive the details, I will forward it to the concerned personnel.

Thank you.
#20

Hi Rob,


Thank you for your efforts.

I have performed my initial testing on your provided PST files. Below are the results for your reference. Please note that I was unable to use your provided source code because your code reference many namespaces that were unavailable to me. Changing your code and writing my own from scratch seemed to be the same. So for now, I have used my own code to extract messages from the all PST files. I would recommend that you provide us a working console application that we can run on every PST file for confirmation of our results.

  1. PST file by name Exchange2010 Test2.pst threw exception of type NullReferenceException on source line if(folderinfo.HasSubFolders == true) when FolderInfo is “Test”. Extracted Contents : 165
  2. PST file by name Exchange2010 Test1.pst threw exception of type NullReferenceException on source line if(folderinfo.HasSubFolders == true) when FolderInfo is “QATesting” or “RecoverableItems”. Extracted Contents : 86
  3. RV.pst file is broken as stated by you. I was unable to extract a single message from this PST. I met an exception of type InvalidOperationException when extracting first message (pst.ExtractMessage())
  4. I haven’t found any exception while extracting contents from OF_GDG.pst. Extracted Contents : 13272

Source Code:

// loop through all the messages in this folder
MessageInfoCollection messageInfoCollection = folderinfo.GetContents();
foreach (MessageInfo messageInfo in messageInfoCollection)
{
MapiMessage message = pst.ExtractMessage(messageInfo);
message.Save( ++i + “.msg”);
}
// call this method recursively for each sub-folder
if (folderinfo.HasSubFolders == true)
{
foreach (FolderInfo subfolderInfo in folderinfo.GetSubFolders())
{
ExtractMsgFiles(subfolderInfo, pst);
}
}

I have performed these tests on a machine having 6GB of RAM. I also have monitored the memory utilization during this testing and there were no major spikes in memory usage. The maximum memory used for the test process was 470,000k when processing OF_GDG.pst.

I am still working on the PST files to find the cause of problem. Once confirmed, I am planning to log issues in our tracking system for above points 1,2 and 3.

Regards,