Aspose.Words.net Upgrade Performance Degradation

Aspose.Words Product Family
1

Good morning, we have an older Aspose license (18.1) and are considering re-purchasing new licenses that would take us to 24.5. While running some preliminary testing on an unlicensed package, I’m noticing about a 50% performance degradation, however.

My test is pretty simple, read a document template from a local folder, merge it against a pre-built data table of data, and save the resultant file as a PDF to another local folder. In order to get some real-world numbers, I’m doing this 1000 times. Then I’m doing 5 runs (of 1000) and taking the average number of seconds per run.

Using Aspose.Words 18.1, this takes 126 seconds on average. Using Aspose.Words 24.5, this takes 191 seconds on average. I’ve also tried several older versions. The times are listed below.

18.1 - 126 seconds
19.12 - 161 seconds
20.12 - 172 seconds
21.12 - 143 seconds
22.12 - 153 seconds
23.12 - 184 seconds
24.5 - 191 seconds

While we’d like to upgrade to take advantage of new features and fixes, I’m hesitant to recommend doing so if the performance is going to take this hard of a hit. Are these standard performance expectations, or is there something I should be doing to make the later versions run faster?

I have seen other threads on this forum about similar performance issues, but from what I can see they are mostly around the Java version of Aspose, and they seem to all resolve with a new fix being issued. Apparently that’s not the case for the .net version though.

Any help would be appreciated as we assess moving to a new version. Thank you.

2

@mattmac If possible could you please attach your template, sample data and code you are using for testing? We will check the scenario on our side and provide you more information.
It is seen that the 24.5 version is about 50% slower in your scenario, but this performance degradation looks acceptable for 6 years or development. Most likely some more complicated and a bit slower algorithms are required to render your document to PDF. But it would be better if you provide a sample application your are using for testing.

3

Thank you,

I can’t provide the original template, as it contains confidential information, but I put together a very simple template to test with. However, when I ran this template this morning, the problem is even more apparent. With 18.1 the same test as before took 10 seconds on average. With 24.5, it took 70 seconds on average. With a template this simple, I wouldn’t think the PDF rendering would be so different. I also tried the same test while commenting out the line to save to PDF, and there is still a pretty large performance gap between the versions.

Here is the test code:

using System.Data;

DirectoryInfo di = new DirectoryInfo(@"D:\Temp\MergeTest");

var dt = new DataTable();
dt.Columns.Add("MailedAddressLine1");
dt.Columns.Add("MailedAddressLine2");
dt.Columns.Add("MailedAddressLine3");
dt.Columns.Add("MailedAddressLine4");
dt.Columns.Add("MailedAddressLine5");
dt.Columns.Add("MailedAddressLine6");
dt.Columns.Add("MailedAddressLine7");

DataRow dataRow = dt.NewRow();
dataRow["MailedAddressLine1"] = "Line1";
dataRow["MailedAddressLine2"] = "Line2";
dataRow["MailedAddressLine3"] = "Line3";
dataRow["MailedAddressLine4"] = "Line4";
dataRow["MailedAddressLine5"] = "Line5";
dataRow["MailedAddressLine6"] = "Line6";
dataRow["MailedAddressLine7"] = "Line7";

dt.Rows.Add(dataRow);

double seconds = 0;
int times = 5;
for (int j = 0; j < times; j++)
{
    foreach (FileInfo file in di.GetFiles()) // clean up the files from the previous run
    {
        file.Delete();
    }

    int max = 1000;
    Console.WriteLine(DateTime.Now.ToString());
    DateTime now = DateTime.Now;
    for (int i = 0; i < max; i++)
    {
        Aspose.Words.Document doc = new Aspose.Words.Document(@"D:\Temp\MergeTest.docx");
        doc.MailMerge.Execute(dt);

        doc.Save($@"D:\Temp\MergeTest\{i}.pdf", Aspose.Words.SaveFormat.Pdf); // remove this line to test the merge alone.
    }
    Console.WriteLine(DateTime.Now.ToString());
    Console.WriteLine($"{j} {(DateTime.Now - now).TotalSeconds} seconds.");
    seconds += (DateTime.Now - now).TotalSeconds;
}
dt.Dispose();
Console.WriteLine($"Average Seconds: {seconds / times}");

MergeTest.docx (13.2 KB)

4

@mattmac
We have opened the following new ticket(s) in our internal issue tracking system and will deliver their fixes according to the terms mentioned in Free Support Policies.

Issue ID(s): WORDSNET-26946

You can obtain Paid Support Services if you need support on a priority basis, along with the direct access to our Paid Support management team.

5

@alexey.noskov, thank you for submitting this issue. Has there been any progress on it? I know this is submitted as a free support request, but I would think other customers would appreciate any performance improvements that are possible. We would also consider buying a new license if the performance wasn’t as degraded as what I’m finding.

As a followup, I re-opened my test project and tried again, now that Aspose 24.8 is out. I’m now getting an even bigger gap in performance. Aspose 18.1 only takes 9 seconds on average per run (of 1000 documents merged) while 24.5 and 24.8 take 74 seconds. Perhaps it has something to do with the extra packages that the 24 versions require that 18 doesn’t? I’m not sure, but thought these metrics might be helpful. Thank you.

6

@mattmac We have completed analyzing the issue. It has to be noted, that it may be trikky to mesuare performance with such a small template/scenario. There are a lot of side effects that can affect benchmark results. And even with accurate results, we cannot extrapolate them to real cases.

Also, you underestimate the PDF rendering impact:

With a template this simple, I wouldn’t think the PDF rendering would be so different.

Even with such a simple template, PDF rendering is the most time-consuming part (over 80%). With a real document it could be even more. There is a noticeable perfomance drop in PDF rendering between 18.01 and 25.07, but it worths nothing to investigate it without real template/scenario.

We have not found any performance bottleneck in import/mail merge scenario. Moreover, release builds without obfuscation show pretty much the same results in 18.01 and 25.07:

Method Version (source code) Mean Error StdDev Ratio
Import 18.01 302.74 ms 1.201 ms 1.760 ms 0.74
ImportMailMerge 18.01 410.63 ms 2.369 ms 3.472 ms 1.00
Import 21.09 289.20 ms 6.109 ms 8.363 ms 0.89
ImportMailMerge 21.09 326.99 ms 8.126 ms 11.654 ms 1.00
Import 22.09 302.86 ms 13.853 ms 19.868 ms 0.89
ImportMailMerge 22.09 342.24 ms 12.421 ms 18.206 ms 1.00
Import 22.10 272.65 ms 1.446 ms 2.074 ms 0.89
ImportMailMerge 22.10 307.15 ms 1.623 ms 2.379 ms 1.00
Import 22.12 272.21 ms 2.451 ms 3.354 ms 0.84
ImportMailMerge 22.12 327.76 ms 19.385 ms 26.534 ms 1.00
Import 23.10 280.11 ms 2.852 ms 3.998 ms 0.78
ImportMailMerge 23.10 364.11 ms 20.161 ms 29.552 ms 1.00
Import 23.11 303.10 ms 6.766 ms 10.127 ms 0.89
ImportMailMerge 23.11 341.64 ms 14.550 ms 20.867 ms 1.00
Import 24.05 289.73 ms 4.472 ms 6.555 ms 0.82
ImportMailMerge 24.05 355.95 ms 10.806 ms 16.173 ms 1.00
Import 24.07 315.47 ms 14.631 ms 21.446 ms 0.90
ImportMailMerge 24.07 349.47 ms 7.587 ms 11.356 ms 1.00

Performance results with nuget versions tend to be vary. It could be caused by using virtualization feature

Method NuGetReferences Mean Error StdDev Median Ratio
Import Aspose.Words 18.1.0 239.06 ms 3.273 ms 4.589 ms 237.59 ms 0.88
ImportMailMerge Aspose.Words 18.1.0 271.03 ms 1.695 ms 2.431 ms 270.88 ms 1.00
Import Aspose.Words 18.12.0 532.51 ms 4.863 ms 7.128 ms 529.97 ms 0.94
ImportMailMerge Aspose.Words 18.12.0 567.55 ms 3.992 ms 5.596 ms 569.01 ms 1.00
Import Aspose.Words 19.12.0 253.13 ms 2.701 ms 4.043 ms 253.31 ms 0.88
ImportMailMerge Aspose.Words 19.12.0 287.46 ms 2.217 ms 3.318 ms 287.75 ms 1.00
Import Aspose.Words 20.12.0 234.40 ms 2.032 ms 3.041 ms 234.25 ms 0.88
ImportMailMerge Aspose.Words 20.12.0 267.77 ms 2.013 ms 2.950 ms 268.13 ms 1.00
Import Aspose.Words 21.12.0 302.70 ms 1.636 ms 2.240 ms 303.35 ms 0.91
ImportMailMerge Aspose.Words 21.12.0 334.11 ms 3.128 ms 4.385 ms 334.97 ms 1.00
Import Aspose.Words 22.1.0 301.98 ms 2.284 ms 3.419 ms 302.01 ms 0.90
ImportMailMerge Aspose.Words 22.1.0 336.32 ms 2.158 ms 3.095 ms 337.35 ms 1.00
Import Aspose.Words 22.2.0 301.53 ms 3.325 ms 4.873 ms 301.21 ms 0.90
ImportMailMerge Aspose.Words 22.2.0 335.65 ms 3.178 ms 4.558 ms 334.64 ms 1.00
Import Aspose.Words 22.3.0 303.91 ms 3.384 ms 4.960 ms 302.97 ms 0.90
ImportMailMerge Aspose.Words 22.3.0 337.25 ms 5.368 ms 7.699 ms 334.55 ms 1.00
Import Aspose.Words 22.4.0 303.01 ms 2.497 ms 3.660 ms 303.56 ms 0.91
ImportMailMerge Aspose.Words 22.4.0 332.19 ms 1.807 ms 2.592 ms 332.08 ms 1.00
Import Aspose.Words 22.5.0 313.07 ms 1.668 ms 2.497 ms 313.55 ms 0.91
ImportMailMerge Aspose.Words 22.5.0 344.39 ms 2.323 ms 3.331 ms 344.52 ms 1.00
Import Aspose.Words 22.6.0 307.87 ms 1.445 ms 2.072 ms 307.37 ms 0.89
ImportMailMerge Aspose.Words 22.6.0 345.85 ms 1.167 ms 1.674 ms 346.44 ms 1.00
Import Aspose.Words 22.7.0 310.90 ms 1.984 ms 2.845 ms 311.99 ms 0.91
ImportMailMerge Aspose.Words 22.7.0 342.95 ms 2.026 ms 2.774 ms 342.17 ms 1.00
Import Aspose.Words 22.8.0 312.07 ms 2.228 ms 3.266 ms 312.11 ms 0.91
ImportMailMerge Aspose.Words 22.8.0 344.19 ms 2.742 ms 3.932 ms 344.81 ms 1.00
Import Aspose.Words 22.9.0 313.22 ms 2.693 ms 3.948 ms 312.84 ms 0.91
ImportMailMerge Aspose.Words 22.9.0 342.49 ms 2.087 ms 2.925 ms 342.18 ms 1.00
Import Aspose.Words 22.10.0 788.37 ms 15.717 ms 23.038 ms 779.78 ms 0.95
ImportMailMerge Aspose.Words 22.10.0 828.31 ms 7.756 ms 11.123 ms 824.13 ms 1.00
Import Aspose.Words 22.11.0 772.63 ms 2.993 ms 4.097 ms 774.33 ms 0.96
ImportMailMerge Aspose.Words 22.11.0 808.03 ms 4.178 ms 5.857 ms 810.39 ms 1.00
Import Aspose.Words 22.12.0 787.98 ms 5.754 ms 8.612 ms 785.61 ms 0.96
ImportMailMerge Aspose.Words 22.12.0 824.08 ms 5.551 ms 7.961 ms 820.66 ms 1.00
Import Aspose.Words 23.1.0 771.31 ms 3.468 ms 5.083 ms 770.77 ms 0.95
ImportMailMerge Aspose.Words 23.1.0 812.33 ms 6.387 ms 9.362 ms 808.63 ms 1.00
Import Aspose.Words 23.2.0 771.03 ms 4.650 ms 6.669 ms 768.50 ms 0.94
ImportMailMerge Aspose.Words 23.2.0 816.73 ms 4.938 ms 7.237 ms 817.32 ms 1.00
Import Aspose.Words 23.3.0 779.11 ms 2.702 ms 4.044 ms 779.32 ms 0.96
ImportMailMerge Aspose.Words 23.3.0 814.70 ms 4.147 ms 6.079 ms 814.43 ms 1.00
Import Aspose.Words 23.4.0 783.07 ms 2.943 ms 4.404 ms 781.76 ms 0.95
ImportMailMerge Aspose.Words 23.4.0 821.03 ms 6.295 ms 9.227 ms 817.81 ms 1.00
Import Aspose.Words 23.5.0 797.13 ms 10.743 ms 16.080 ms 794.18 ms 0.95
ImportMailMerge Aspose.Words 23.5.0 835.63 ms 11.596 ms 16.256 ms 829.65 ms 1.00
Import Aspose.Words 23.6.0 795.58 ms 8.454 ms 12.654 ms 793.96 ms 0.93
ImportMailMerge Aspose.Words 23.6.0 852.58 ms 16.032 ms 23.997 ms 845.80 ms 1.00
Import Aspose.Words 23.7.0 1,047.65 ms 10.998 ms 16.120 ms 1,041.16 ms 0.99
ImportMailMerge Aspose.Words 23.7.0 1,060.51 ms 8.040 ms 11.785 ms 1,058.32 ms 1.00
Import Aspose.Words 23.8.0 928.67 ms 7.844 ms 11.741 ms 926.40 ms 0.85
ImportMailMerge Aspose.Words 23.8.0 1,087.05 ms 25.642 ms 37.586 ms 1,076.31 ms 1.00
Import Aspose.Words 23.9.0 926.10 ms 6.199 ms 9.086 ms 921.75 ms 0.87
ImportMailMerge Aspose.Words 23.9.0 1,059.77 ms 1.361 ms 1.908 ms 1,060.06 ms 1.00
Import Aspose.Words 23.10.0 842.82 ms 4.474 ms 6.124 ms 840.81 ms 0.98
ImportMailMerge Aspose.Words 23.10.0 861.01 ms 4.668 ms 6.543 ms 858.14 ms 1.00
Import Aspose.Words 23.11.0 845.03 ms 3.681 ms 4.915 ms 843.80 ms 0.79
ImportMailMerge Aspose.Words 23.11.0 1,071.18 ms 7.365 ms 10.325 ms 1,065.92 ms 1.00
Import Aspose.Words 23.12.0 828.19 ms 7.459 ms 10.698 ms 822.92 ms 0.95
ImportMailMerge Aspose.Words 23.12.0 867.72 ms 12.990 ms 18.630 ms 863.09 ms 1.00
Import Aspose.Words 24.1.0 866.37 ms 7.166 ms 10.505 ms 868.00 ms 0.80
ImportMailMerge Aspose.Words 24.1.0 1,078.59 ms 8.193 ms 12.263 ms 1,078.64 ms 1.00
Import Aspose.Words 24.2.0 846.75 ms 6.287 ms 9.215 ms 846.98 ms 0.97
ImportMailMerge Aspose.Words 24.2.0 868.98 ms 7.324 ms 10.736 ms 864.82 ms 1.00
Import Aspose.Words 24.3.0 853.02 ms 9.595 ms 13.450 ms 844.62 ms 0.99
ImportMailMerge Aspose.Words 24.3.0 864.75 ms 4.907 ms 7.193 ms 860.50 ms 1.00
Import Aspose.Words 24.4.0 859.87 ms 14.440 ms 21.613 ms 856.89 ms 0.86
ImportMailMerge Aspose.Words 24.4.0 996.87 ms 7.051 ms 10.112 ms 992.65 ms 1.00
Import Aspose.Words 24.5.0 1,024.52 ms 13.979 ms 20.923 ms 1,021.84 ms 0.92
ImportMailMerge Aspose.Words 24.5.0 1,110.79 ms 13.545 ms 19.425 ms 1,111.09 ms 1.00
Import Aspose.Words 24.6.0 954.26 ms 4.585 ms 6.276 ms 0.95
ImportMailMerge Aspose.Words 24.6.0 1,008.77 ms 10.681 ms 15.987 ms 1.00
Import Aspose.Words 24.7.0 1,005.45 ms 16.039 ms 23.509 ms 0.97
ImportMailMerge Aspose.Words 24.7.0 1,034.09 ms 8.520 ms 12.219 ms 1.00
7

@alexey.noskov, Thank you for the follow-up. I completely agree that the PDF rendering is a large portion of the delay, and your metrics are certainly much more detailed than mine, so I appreciate the time spent testing this.

I also understand wanting to test against a real-world template, but unfortunately that isn’t feasible right now. However, for the sake of discussion, let’s assume that the template I provided is the real template, albeit a simple one. I certainly won’t pretend to know the underlying code or functionality of your product, but I would think that a more complicated “real” template would have worse performance than this simple one, and I understand that will be the case for any version.

That being said, if I’m understanding your metrics properly, it looks like the Nuget performance of the ImportMailMerge for 24.7 in your testing is about 3.8 times slower than in 18.1 (1,034.09 ms vs 271.03 ms). I’m not sure why the Nuget package would have such worse performance than the source code, but obviously I’m not going to be able to test or use the source code, so we’ll have to go with Nuget performance.

Also, I changed my test code to forgo the PDF rendering and just do the import and merge. Version 24.8 took 6 seconds on average (for 1000 imports and merges), while 18.1 took 2.6 seconds on average, roughly in line with your results.

Additionally, if I change my code to save the documents as .docx, rather than PDF, the performance is even worse for both versions. 24.8 takes 43 seconds on average, while 18.1 takes 24 seconds (again per 1000 iterations). So based on those numbers it appears that it’s actually more performant to render to docx than PDF, although the disparity between versions isn’t as large as PDF.

I appreciate your time looking into this. Obviously, I wouldn’t be putting in this much effort on my end if we weren’t seriously considering upgrading.

Thank you

8

@mattmac Thank you for from feedback. I forwarded it to our development team. It looks like the main reason of the performance degradation is the library obfuscation and we have limited abilities to act here, since we have to balance between the library protection and it’s performance.