@dgreenwood,
Regarding WORDSNET-20774, we disabled CPU turbo boost feature to get more correct results and also updated a bit the sample project and have got these results (see test-results.zip (973 Bytes)).
Next, we analyzed the threads concurrency. You can see the concurrency traces in the attached files.zip (172.8 KB). We did not find any thread synchronization issues in Aspose.Words’ code. The reason of this performance degradation issue is memory management time. Memory management takes 12% of process time for 2 threads, 29% for 4 threads and 41% for 8 threads. Threads are blocked by an event that is associated with a memory management operation such as Paging or Allocation. During this time, a thread has been blocked in an API or kernel state. It looks like a .NET Garbage Collection feature rather than a Aspose.Words’ bug.
To reproduce this issue without Aspose.Words, we replaced the sample action with fake action:
Action<string> fakeAction = (string fileName) =>
{
var sw = Stopwatch.StartNew();
for (var i = 0; i < 10000000; i++)
{
var m = new MemoryStream();
var b = new byte[128];
m.Write(b, 0, 128);
}
sw.Stop();
Console.WriteLine($"Fake thread finished {fileName} in {sw.ElapsedMilliseconds}ms.");
};
And got the similar results:
Processing 8 documents using 1 threads
Fake thread finished amosched1.rtf in 1921ms.
...
Total: 15189
Processing 8 documents using 2 threads
Fake thread finished amosched2.rtf in 2402ms.
...
Total: 9461, faster in 1.61 times
Processing 8 documents using 3 threads
Fake thread finished amosched3.rtf in 2880ms.
...
Total: 8076, faster in 1.88 times
Processing 8 documents using 4 threads
Fake thread finished amosched2.rtf in 3479ms.
...
Total: 6995, faster in 2.17 times
Processing 8 documents using 6 threads
Fake thread finished amosched1.rtf in 5475ms.
...
Total: 8007, faster in 1.9 times
Processing 8 documents using 8 threads
Fake thread finished amosched2.rtf in 7463ms.
...
Total: 7483, faster in 2.03 times
We have started working on memory management time to improve the performance in multi-threading mode.
Regarding WORDSNET-20769, we will most likely close this issue with “Not a bug” status. Please check the following points:
- Aspose.Words.Document is thread-safe
- lock() works as expected, it decreases load and save time in context of the locked thread, but increases total execution time because it locks parallel processing.
- Performance degradation in multi-thread mode is reproduced on our end and as described above (WORDSNET-20774), we have some ideas how to improve it.
We will keep you posted here on any further updates.
Thank you and the team for a very detailed and thorough analysis. I’m impressed. I agree that there is a lower-level block in kernel state (such as memory allocation/access), and that a more efficient approach to memory management would be a possible solution. Your last point is music to my ears… “3. Performance degradation in multi-thread mode is reproduced on our end and as described above (WORDSNET-20774), we have some ideas how to improve it.” I’ve read several good articles about LOH memory management, especially with regards to memory streams. Also, take a look for the thousands upon thousands of string manipulations, or concatenations, as string objects are immutable and get reallocated, causing more GC events. There are known solutions for avoiding. Anyways, sounds like you are into the details and have a plan. Thank you and the team again for all the hard work. Once solved, this will certainly improve the product and make you a stand-out in this market. Cheers!
One favor to ask, if you would be so kind. To help us all learn, can you please share your entire “updated a bit the sample project” and the entire fakeAction program? You see, when I run either the sample project or your fakeAction test and view the results in Concurrency Analyzer, I see something very different. Rather than memory overhead, I see a lot of time in Synchronization. Memory Management is 0% for me. As well, my results show “Worker Thread” rather than “CLR Worker Thread”. I would like to know what you are doing differently to obtain 1% Synchronization, and CLR Worker Threads. Please advise. Thank you!
@dgreenwood,
I have logged your request/comment in our issue tracking system and will keep you posted here on any further updates.
Hoping for a response soon. I’m eager to try your test programs and verify the same results. Thanks.
@dgreenwood,
Strangely, we can no longer reproduce our previous results with high memory management. Now, we get the same execution percentage with high synchronization and zero memory management on our test project. So, now we are getting results similar to yours. We will keep you posted here on any further updates.
I am checking back after several months, eager to hear if you had any additional light to shed on the performance issues noted above (high synchronization, etc.). Perhaps you have made or are making other enhancements to improve threaded performance?
Thanks
@dgreenwood,
I can only update you about the current statuses of these issues i.e. the implementation of the fix of WORDSNET-20769 has been “Postponed” and WORDSNET-20774 is currently “In Development” phase. There are no estimates available at the moment. I will keep you posted here on any further updates. Sorry for any inconvenience.
The issues you have found earlier (filed as WORDSNET-20774) have been fixed in this Aspose.Words for .NET 21.5 update and this Aspose.Words for Java 21.5 update.
The issues you have found earlier (filed as WORDSNET-20769) have been fixed in this Aspose.Words for .NET 21.6 update and this Aspose.Words for Java 21.6 update.