Concurrent issue with MapiMessage.fromMailMessage

Aspose.Email Product Family
1

When we are using MapiMessage.fromMailMessage in multiple threads, we noticed some weird exception. The exception will only throw when we use multiple threads. And worked fine with ONE thread.
It’s rare and we only notice 2-5 times with a 100K message set testing.
Anyway, it show up many times when we repeated the test. So the issue should be there.

Thanks,

{code}
The mailMsg is created by us. We add headers, bodies add attachments to it.

MapiMessage.fromMailMessage(mailMsg, MapiConversionOptions.getUnicodeFormat());

java.lang.NullPointerException
at com.aspose.email.MapiMessage.a(Unknown Source)
at com.aspose.email.MapiMessage.b(Unknown Source)
at com.aspose.email.MapiMessage.a(Unknown Source)
at com.aspose.email.MapiMessage.a(Unknown Source)
at com.aspose.email.MapiMessage.fromMailMessage(Unknown Source)
{code}

2

Hi,


Thank you for writing to Aspose support team.

I have investigated this issue at my end with latest version of Aspose.Email for Java 5.0.0 and a few hundred email messages (that’s all I have), but was unable to observe the issue. Could you please cross-check at your end if the messages are valid ones (i.e. open properly in Outlook) that give rise to exception using the API? It is just to make sure that the exception is raised for valid message files. I would further request you to share the sample message files only if it is possible for you? This will help us develop our pool of messages as well as test the issue with the sample data that raises problem at your end.

3

We tried with 650K messages(number not size), not just a few hundred message.
The issue only show up when we use multiple threads.
Using one thread, all 650K messages could be processed.
When we are using multiple threads, all exceptions coming from the call:
MapiMessage.fromMailMessage(message, MapiConversionOptions.getUnicodeFormat());

We could not give our test samples to you because of company policy. What I can tell is that we have samples from various sources. Some are Envelope Journey messages. Some are malformed, some are having special characters.

The concurrent issue only show up a few time with 650K message test. But it’s indeed there because every time I run the test, I saw random exception here and there. With one thread, no issue. Also, if I synchronized the MapiMessage.fromMailMessage call, NO ISSUE. So it’s must be a bug in your code.

We are at final stage to decide whether select your product or not. And those bugs I raised already delayed our decision several times. Some issues, I found a work around. Some of them like the thread issue wasted lots of time. I had to wrote special code to bypass those bugs. MapiMessage.fromMailMessage is too slow. we want to use multiple threads to improve performance. On our server, it can only handle avg 33 messages per second. If you can find the bug and fix it, then it can save us time. And you don’t offer source code, otherwise, I can fix the bug myself. :frowning:

Thanks,


4

Hi Billow,


We are very sorry for the inconvenience you are facing. We have currently tested this issue with a 1024K messages that were duplicated by copy-paste. However, we could not reproduce the issue. It seems that more randomness of messages is required to reproduce the issue. We are currently preparing this much large set of messages to test the issue in detail and would request you to spare us some time for this purpose.

I would also like to share that issues are reported with the API from time to time and fixed accordingly in the next release. We value our customer’s feedback and issue reporting as it serves towards the improvement of our API. Thus, you can trust the Aspose.Email team in that if the issue is reproduced at our end, it will be reported immediately and fixed accordingly. We appreciate your understanding in this regard.
5

Based on my testing, the concurrent thread issue usually show up when I create MailMessage with attachments, then convert the MailMessage to MapiMessage. Create MailMessage was fine, no exception. The exception usually show up at:
MapiMessage.fromMailMessage(mailMsg, MapiConversionOptions.getUnicodeFormat());
When it tried to read the attachment, some value of the attachment was NULL, so throw null exception.

It seems that one of your static function shared a value. And in concurrent thread, the value was set to NULL, and the other thread was trying to use it at the same time.

My sample dataset have lots of email with attachments, calendar, nested messages.
If I try it on email without attachment, no such issue.

I have to use STATIC lock to get rid of those random exceptions.

Hopefully it will help you to find the bug.

Thanks,

Ying

6

Hi Ying,


Thank you for sharing additional feedback.

We further worked out this issue with a total 650K (count) messages that were all random and included messages, calendar items, and messages with attachments as well. But we could not reproduce the issue at our end.

Could you please share which JDK version you are using? Also, if possible please share your sample code with us that we could use for possible reproduction of this issue at our end. We shall look into it and assist you further.
7

I will write one and submit tomorrow.
BTW, I had the code submitted a while ago in another thread.

Thanks,

Ying

8

Hi there,

Please check the attached file.
It has a java source code with four output files using different arguments.

If you check the file, there were two of them worked fine without any issue.
nolock-1-thread.txt: no lock, one thread
syncmapi-8-threads.txt: with lock, 8 threads

And two of them throw random exceptions and had failures.
nolock-8-threads.txt: no lock, 8 threads
synclock-8-threads.txt: with lock on MailMessage.load(bis, mailLoadOption), no lock on MapiMessage.fromMailMessage(mailMsg, MapiConversionOptions.getUnicodeFormat());

9

Hi Ying,

Thank you for sharing the sample code and details.

We have run the test with your sample code using the latest version of Asopse.Email for Java 5.1.0 and the following settings:

  • Total Threads: 8
  • syncLoad = false
  • syncMapi = false
The test ran over the complete set of messages in our repository and no exception was raised. Could you please share if there needs to be some different values of these parameters as per your testing criterion? If the settings are the same as mentioned above, then we’ll be in need of your sample messages to reproduce this issue at our end.

10

You need to create some malformed emails.
Such as malformed content-type, missing boundary, wrong encoding, child/parent boundary with the same boundary name, nested messages, envelop journey messages.
Many of samples we used, have malformed data in them.
I had 395 samples.

It did work fine with well-formatted email. Tried several thousands of well-formated email. worked fine.

But when we are testing, we focus more on malformed emails. They can help you to find edge cases.

11

Hi Ying,


The issue then seems to be with the messages and not with multi-threaded usage of the API. As you have mentioned that you have 300+ such messages, I would request you to please share those with us so that we can use these for our investigations and assisting you further. We appreciate your understanding in this regard.
12

Hey I am facing same issue . I am using Aspose.Email for .net I am convert .eml file using MapiMessage.fromMailMessage in multiple threads getting error .NullPointerException

13

Hello @ashish007 ,

Thank you for sharing the issue. Could you share more details, such as your code snippet and the exact error message?

14
  • Aspose.Email’s PST writer is not thread-safe. Without the lock:

    • :x: Corrupted PST files.

    • :x: InvalidOperationException (“Object is busy”). For making threads safe
      // Process message
      lock (asposeLock)
      {
      using var mailMsg = MailMessage.Load(tempFilePath);
      using var mapiMessage = MapiMessage.FromMailMessage(mailMsg);
      inbox.AddMessage(mapiMessage);

      Interlocked.Increment(ref totalMessageInPst);
      Interlocked.Increment(ref totalProcessedMessages);

      if (totalProcessedMessages % 100 == 0)
      {
      Log.Information(“Total messages processed: {Count}”, totalProcessedMessages);
      }
      } i am doing so not getting error

15

also i am getting new error [07:30:27 ERR] Error processing message -2147008364
System.ArgumentOutOfRangeException: Index and length must refer to a location within the string. (Parameter ‘length’)
at System.String.ThrowSubstringArgumentOutOfRange(Int32 startIndex, Int32 length)
at System.String.Substring(Int32 startIndex, Int32 length)
at mm.c(String d)
at Aspose.Email.Calendar.Recurrences.CalendarRecurrence.d(String d)
at Aspose.Email.Calendar.Recurrences.CalendarRecurrence…ctor(String pattern)
at Aspose.Email.Mapi.MapiCalendarRecurrencePatternFactory.d(jK d, List`1 v, MapiCalendarTimeZone c, MapiMessageItemBase t)
at Aspose.Email.Mapi.MapiMessage.d(LK d, OutlookMessageFormat v, String c, Int32 t)
at Aspose.Email.Mapi.MapiMessage.n(MailMessage d, MapiConversionOptions v)
at Aspose.Email.Mapi.MapiMessage.m(MailMessage d, MapiConversionOptions v)
at Aspose.Email.Mapi.MapiMessage.d(Attachment d, MapiConversionOptions v)
at Aspose.Email.Mapi.MapiMessage.v(Attachment d, MapiConversionOptions v)
at Aspose.Email.Mapi.MapiMessage.d(MailMessage d, MapiMessage v, MapiConversionOptions c)
at Aspose.Email.Mapi.MapiMessage.v(MailMessage d, MapiConversionOptions v)
at Aspose.Email.Mapi.MapiMessage.FromMailMessage(MailMessage message, MapiConversionOptions options)
at Aspose.Email.Mapi.MapiMessage.FromMailMessage(MailMessage message)
at Program.ProcessSingleMessage(SimpleMessage msg, String databaseId, FolderInfo inbox, SearchSoapClient searchSoapClient) in

16
  • Is there a hard limit of 100 emails conversion .eml to .pst file at a time in the trial version?
    and why visual studio output window give this error Exception thrown: ‘System.FormatException’ in Aspose.Email.dll
17

@ashish007,

Aspose.Email guarantees that one thread can safely work with one PST file, but the PersonalStorage itself is not thread-safe.
What does this mean? You can use different PST files in different threads without any issues. You cannot write to the same PST file from multiple threads simultaneously without using a lock. Otherwise, you may encounter errors like corrupted PST files/Object is busy.

The email may contain a recurring calendar event with an incorrectly formatted recurrence rule.
Or the recurrence rule string might be shorter than expected, leading to an invalid substring operation. Can you please provide an EML file causing this issue?

Evaluation version of Aspose.Email provides the following limitation in PST: Only 50 emails can be extracted from a folder.

We can’t determine the cause of the error based on such limited information. Please provide more details, such as:

  • The specific method/code example being performed when the error occurs.
  • Whether the error happens with all emails or only specific ones.
  • A sample file that reproduces the issue.
  • etc.