FetchMessage (~100Mb) from Exchange Server

Hi,
I’m fetching messages from an Exchange Server mailbox. This works fine as long as the size of the messages (their attachments) is not too big.
Recently I had to process a message of approx. 100Mb in size, which failed, resulting in a OutOfMemoryException (see attached file for stacktrace).

I’m using the following code:

NetworkCredential credential = new NetworkCredential(, , );
IEWSClient client = EWSClient.GetEWSClient(, credential);
ExchangeMessageInfoCollection msgs = client.ListMessages(client.MailboxInfo.InboxUri, true);
foreach (ExchangeMessageInfo msgInfo in msgs)
{
try
{
MailMessage msg = client.FetchMessage(msgInfo.UniqueUri);
}
catch (OutOfMemoryException e)
{
logger.error(e.ToString());
}
}


Question is how to process such large messages.

Many thanks in advance

Scheu




Hi Scheu,


Thanks for writing to Aspose.Email support team.

Normally mail servers do not allow such large attachments with the mails, however servers can be configured to set the attachment size. I tried to arrange sample message with 100MB attachment, however I could not upload it to any exchange server for further testing.

We need little assistance to proceed further and request you to please arrange some test account where the sample EML with large attachment size is already present in the INBOX folder. It will help us to access the message and re-produce the issue here. You may provide us the test account credentials temporarily so that we perform the testing.

Thanks in advance for your cooperation.

P.S. Make this thread private before providing the test account credentials.

Hi,

I know, normally this should not be the case, but it’s a requirement from our customers in this case.

Unfortunately, it is not possible/allowed for me to setup such an account, but here’s the way I managed to get a 100MB mail on to Exchange Server (since such large attachments are forbidden here as well).

First I created the mail manually with the following code:

MailMessage m = new MailMessage();
m.From = "me@domain.com";
m.To = "me@domain.com";
m.Subject = “A big file”;
m.TextBody = “this contains a big file”;
m.Attachments.Add(new Attachment(@"<path to 100mb file>"));
m.Save(@".msg", MailMessageSaveType.OutlookMessageFormatUnicode);

Then I dragged the MSG-File into my Outlook mailbox manually (took some time to upload).
After that I could run the code from the original post, which then led to the described problem.

Could you give that a try as well?

Thanks in advance

Scheu

Hi Scheu,


Thank you for porviding further details.

I would request you to please spare us a little time to accomodate such a large size message file onto our exchange server and test the issue at our end. I’ll write back here after investigating the issue further at my end. We appreciate your patience in this regard.

Hi Scheu,


I was finally able to upload a test message, size 120 MB, to my exchange server, but was unable to reproduce the mentioned issue at my end using the latest version of Aspose.Email for .NET 3.5.0. The message was successfully downloaded and saved to the disc without causing any exception. I am using Exchange Server 2010 at my end. To investigate this issue further, could you please confirm to us if you are using the latest version of Aspose.Email for .NET at your end?

Hi,

yes I updated to Version 3.5.0 of Aspose.Email .NET some days ago.
<!–[if gte mso 9]>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:HyphenationZone>21</w:HyphenationZone>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>DE</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:DontVertAlignCellWithSp/>
<w:DontBreakConstrainedForcedTables/>
<w:DontVertAlignInTxbx/>
<w:Word11KerningPairs/>
<w:CachedColBalance/>
</w:Compatibility>
<w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel>
<m:mathPr>
<m:mathFont m:val=“Cambria Math”/>
<m:brkBin m:val=“before”/>
<m:brkBinSub m:val="–"/>
<m:smallFrac m:val=“off”/>
<m:dispDef/>
<m:lMargin m:val=“0”/>
<m:rMargin m:val=“0”/>
<m:defJc m:val=“centerGroup”/>
<m:wrapIndent m:val=“1440”/>
<m:intLim m:val=“subSup”/>
<m:naryLim m:val=“undOvr”/>
</m:mathPr></w:WordDocument>
<![endif]–><span style=“font-size:12.0pt;font-family:“Times New Roman”,“serif”;
mso-fareast-font-family:“Times New Roman”;color:black;mso-ansi-language:DE;
mso-fareast-language:DE;mso-bidi-language:AR-SA”>We use Microsoft Exchange Server 2007
SP3 (8.3.83.0 or higher<!–[if gte mso 10]>

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Normale Tabelle"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}

<![endif]–>) but our customers use the 2010 version and have the same issue.

Perhaps this helps for further investigation. When I’m fetching the big message (which takes quite some time before OutOfMemory exception occurs), I can observe (in Task Manager) the memory usage of my test application rising to about 400 to 500MB (or higher). But this should be no problem, I got 16GB of memory available.

Hope this helps.

Thanks for your efforts so far.

Scheu


Hi Scheu,


I have retested this issue at my end using both Exchange Server 2010 as well as Exchange Server 2013 and the exception didn’t arise at my end. Test application memory did rise on average to 350 MB, but the file was downloaded and saved to disc properly without any issue. My test message file contains two attachments of size 51MB and 64MB (total message size is 120 MB). Could you please tell if this issue arise for any sample message with size greater than 100 MB?

Hi,

just tested with a sample message containing two 48MB files, which doesn’t work either.

Did you use the code I provided in the original post to fetch the email? Simply to make sure, there’s no difference.
Is it possible for you to test on Exchange Server 2007, just to be sure that it works for you like with 2010/2013?

Thanks.

Scheu


Hi Scheu,

Yes, I have used the same code as you shared in your first post. Please spare us a little time to prepare and test this issue with Exchange Server 2007 as well. I’ll return to you on this as soon as I am done with testing this issue at my end and appreciate your patience in this regard.

Hi,

is there any new insight in this regard? I have to ask because this is kind of pressing.

Thanks.

Scheu

Hi Scheu,


I am afraid to inform that we could not re-produce the issue here. For further investigation, we request you to provide us a test account with credentials having a message with such large attachment. It will help us to observe the issue by downloading the message and provide assistance as soon as possible.

Hi,

unfortunately, that is not possible, as I mentioned in an earlier post.

Since we both used the same code and the same version of Aspose, perhaps the system makes a difference. Could you tell me your system specs, operating system and so on?

Besides, I found out that now streaming a message is possible, which wasn’t available in prior versions of Aspose. Consider the following code:

NetworkCredential credential = new NetworkCredential(, , );
IEWSClient client = EWSClient.GetEWSClient(, credential);
ExchangeMessageInfoCollection msgs = client.ListMessages(client.MailboxInfo.InboxUri, true);
foreach (ExchangeMessageInfo msgInfo in msgs)
{
try
{
using (var stream = new System.IO.FileStream(@“myMessage.msg”, FileMode.Create))
{
client.SaveMessage(msgInfo.UniqueUri, stream);
stream.Close();
}
}
catch (OutOfMemoryException e)
{
logger.error(e.ToString());
}
}


I assumed, this would work, but instead of starting to write to disk immediately, this method does exactly the same like the original one, namely allocating the whole message in memory, which then leads to the same OutOfMemory exception.

Any ideas on this approach?

Thanks.

Scheu

Hi Scheu,


We regret the inconvenience you are facing till yet with this issue.

Since I am not able to reproduce this issue at my end with the available test environments, I have forwarded this information to our development team for further investigation and feedback. Once there is some feedback available in this regard, I’ll write back here to update you. We appreciate your patience until then.

Hi Scheu,


I would like to update you about the status of this issue. We have logged it as NETWORKNET-34182 in our issue tracking system for further investigation by our development team. As soon as there is an update or a fix version available in this regard, we’ll notify you here via this thread.

We are sorry for the inconvenience caused to you in this regard.

The issues you have found earlier (filed as NETWORKNET-34182) have been fixed in this update.


This message was posted using Notification2Forum from Downloads module by Aspose Notifier.

Hi,

I’m afraid to say that on my end nothing changed with the new version 3.7.0

I’m just wondering if you were able to reproduce my exact same problem, since you didn’t seem to be able to at the beginning.

Could you please tell me, what has been changed and how that was tested exactly.

So, for testing, did you use an email with an attached file of a 100mb in size and did you save it as a msg file using a stream or not, since in my client neither works.


best regards

Scheu



Hi Scheu,


We are sorry for any inconvenience caused to you in this regard.

Yes, the issue was tested at our end with Exchange Server 2007 and was fixed with Aspose.Email for .NET 3.7.0. A test message was created with 6+ attachments, leading to 100+ MB attachments size. It was then fetched from the server using the FetchMessage service and saved to stream without any exception using this latest version. Could you please assist us with some of our queries as following:

  1. Please share your system details with us i.e. Operating System version (32/64 bit), installed Memory size.
  2. Please provide us again the exact sample code that raises this issue.
  3. Please provide us the complete stack trace of the issue
  4. Also, please point out the exact line of code that raises this exception.

At my end, using 64-bit Windows 7 OS, I am unable to observe this issue now with the latest version of Aspose.Email for .NET 3.7.0. Your answers to the above queries will help us assist you further.

Hi,

1. 64-bit Windows 7 OS, 16Gb of RAM, Core i5

2. Two possibilities, but neither works:
a. Non-streaming:
NetworkCredential credential = new NetworkCredential(uname, pw, dom);
IEWSClient client = EWSClient.GetEWSClient(uri, credential);
ExchangeMessageInfoCollection msgs = client.ListMessages(client.MailboxInfo.InboxUri, true);
foreach (ExchangeMessageInfo msgInfo in msgs)
{
try
{
MailMessage msg = client.FetchMessage(msgInfo.UniqueUri);
}
catch (Exception e)
{

}
}
b. using streaming:
NetworkCredential credential = new NetworkCredential(uname, pw, dom);
IEWSClient client = EWSClient.GetEWSClient(uri, credential);
ExchangeMessageInfoCollection msgs = client.ListMessages(client.MailboxInfo.InboxUri, true);
foreach (ExchangeMessageInfo msgInfo in msgs)
{
try
{
using (var stream = new System.IO.FileStream(@“myMessage.msg”, FileMode.Create))
{
client.SaveMessage(msgInfo.UniqueUri, stream);
stream.Close();
}
}
catch (OutOfMemoryException e)
{

}
}

3. Stacktrace:
14.01.2014 09:45:42 - 7152: [ERR] System.OutOfMemoryException: Eine Ausnahme vom Typ “System.OutOfMemoryException” wurde ausgelöst.
bei System.Text.StringBuilder.set_Length(Int32 value)
bei System.Xml.XmlTextReaderImpl.ParseText()
bei System.Xml.XmlTextReaderImpl.ParseElementContent()
bei System.Xml.XmlTextReaderImpl.Read()
bei System.Xml.XmlTextReader.Read()
bei System.Xml.XmlReader.ReadStartElement()
bei Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderExchangeServiceBinding.Read303_MimeContentType(Boolean isNullable, Boolean checkType)
bei Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderExchangeServiceBinding.Read418_MessageType(Boolean isNullable, Boolean checkType)
bei Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderExchangeServiceBinding.Read487_ArrayOfRealItemsType(Boolean isNullable, Boolean checkType)
bei Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderExchangeServiceBinding.Read490_ItemInfoResponseMessageType(Boolean isNullable, Boolean checkType)
bei Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderExchangeServiceBinding.Read678_ArrayOfResponseMessagesType(Boolean isNullable, Boolean checkType)
bei Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderExchangeServiceBinding.Read693_GetItemResponseType(Boolean isNullable, Boolean checkType)
bei Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderExchangeServiceBinding.Read795_GetItemResponse()
bei Microsoft.Xml.Serialization.GeneratedAssembly.ArrayOfObjectSerializer85.Deserialize(XmlSerializationReader reader)
bei System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
bei System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle)
bei System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
bei System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
bei Aspose.Email.Exchange.Schema.Exchange_2013.ExchangeServiceBinding.GetItem(GetItemType GetItem1)
bei .??.FetchMessage(String )
bei .??.SaveMessage(String , Stream )

4. Either:
a. MailMessage msg = client.FetchMessage(msgInfo.UniqueUri);
b. client.SaveMessage(msgInfo.UniqueUri, stream);


However, there seems to be some misunderstanding which might make a big difference I suppose.
I always spoke of an email with a single file attached, that has the size of about 100MB - not multiple files. That is the case I need to get working.
(but then again, even an email with 6+ attachments like you used, did not work for me)


best regards

Scheu

Hi Scheu,


Thank you for providing the details.

As per the difference outlined by you in the testing scenario, I now uploaded a test message with a single zip attachment of size 104 MB to the server and fetched it on my machine that is running Windows 7 OS 64-bit. However, this didn’t raise the issue. For further testing, I carried out the same test with a 32-bit OS client machine and I was able to face the OutOfMemoryException. I have reopened the associated ticket with id: NETWORKNET-34182 so that development team can consider it for further investigated and feedback. Once there is any update available, I’ll update you here via this thread.

We are sorry for the inconvenience you have faced in this regard.