Cells Through OutOfMemoryError: GC overhead in a multi-thread application

eboraks2017 · April 27, 2022, 12:31pm

I am evaluating Cells Java, and one of my requirements is to be able to run on a multi-threaded application. I have no problems with converting Cells to PDF with 6 threads.

But I do get “java.lang.OutOfMemoryError: GC overhead limit exceeded” error when converting to HTML with only 3 threads. I am giving the program 8GB of memory, but only about 4GB is being used. It seems to me that the application is just opening too many temp files or objects and doesn’t do a good job closing and cleaning them. The following post about GC overhead is interesting - https://stackoverflow.com/questions/1393486/error-java-lang-outofmemoryerror-gc-overhead-limit-exceeded.

Here is an example Excel file I am using - https://storage.googleapis.com/csi-se/eboraks/big_excel_to_immitate_sharepoint-3.xlsx.zip

Error Stack:
Exception in thread “pool-1-thread-2” 2017-07-05 22:00:05 INFO ExcelConverter:32 - Start converting big_excel_to_immitate_sharepoint-4.xlsx
java.lang.OutOfMemoryError: GC overhead limit exceeded
at com.aspose.cells.zbrg.a(Unknown Source)
at com.aspose.cells.zbrm.a(Unknown Source)
at com.aspose.cells.zbad.a(Unknown Source)
at com.aspose.cells.zfl.a(Unknown Source)
at com.aspose.cells.zaao.l(Unknown Source)
at com.aspose.cells.zaao.d(Unknown Source)
at com.aspose.cells.zahz.a(Unknown Source)
at com.aspose.cells.zku.b(Unknown Source)
at com.aspose.cells.ConditionalFormattingResult.getColorScaleResult(Unknown Source)
at com.aspose.cells.ConditionalFormattingResult.a(Unknown Source)
at com.aspose.cells.Cell.a(Unknown Source)
at com.aspose.cells.zafu.d(Unknown Source)
at com.aspose.cells.zafu.e(Unknown Source)
at com.aspose.cells.zafu.a(Unknown Source)
at com.aspose.cells.zafu.b(Unknown Source)
at com.aspose.cells.zafu.a(Unknown Source)
at com.aspose.cells.zafk.a(Unknown Source)
at com.aspose.cells.zafk.b(Unknown Source)
at com.aspose.cells.zafk.a(Unknown Source)
at com.aspose.cells.zafm.b(Unknown Source)
at com.aspose.cells.zafm.a(Unknown Source)
at com.aspose.cells.zafn.a(Unknown Source)
at com.aspose.cells.Workbook.save(Unknown Source)
at com.aspose.cells.Workbook.save(Unknown Source)
at com.cambridgesemantics.se.aspose.ExcelConverter.run(ExcelConverter.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

shakeel.faiz · July 6, 2017, 12:41pm

@eboraks2017

We have looked into this issue and found both Microsoft Excel and Aspose.Cells is taking much time to convert it to HTML, but both of them convert it to HTML successfully without any error. Please spare us some more time to look into this issue, we will update you asap.

eboraks2017 · July 6, 2017, 1:05pm

Shakeel,

It’s important to note that the problem only occurs when saving three and more presentations in parallel using threads. I was able to do two presentations in parallel without a problem. So, I know the Cells work. The problem is with scaling it up.

shakeel.faiz · July 6, 2017, 1:10pm

@eboraks2017

Sure, I understand your point that the problem occurs in multiple threads not in a single thread and problem is related to scaling up. We will update you tomorrow hopefully after looking into this issue further.

shakeel.faiz · July 7, 2017, 10:42am

@eboraks2017

Thanks for using Aspose APIs.

We were able to replicate this exception with 3 threads running simultaneously. Please see the sample code given below as well as the exception.

We have logged this issue in our database for product team investigation and for a fix. Once, the issue is fixed or we have some other news for you, we will let you know asap.

This issue has been logged as

CELLSJAVA-42340 - Exception in thread “Thread-2” java.lang.OutOfMemoryError: GC overhead limit exceeded

Here is the sample code.

Java

import java.util.*;

import com.aspose.cells.*;


public class ClsTestThread {
	
	public class SimpleThread implements Runnable
	{
		public String srcFile="";
		public String dstFile="";
		public String myThreadId="";

		@Override
		public void run() {
			
			try
			{
				
				System.out.println(myThreadId);
				System.out.println(srcFile);
				System.out.println(dstFile);
				System.out.println("----");
				
				Workbook wb = new Workbook(srcFile);
				wb.save(dstFile);
			}
			catch(Exception ex)
			{
				System.out.println("Simple Thread Exception Message:");
				System.out.print(ex.getMessage());
				System.out.print(ex.getStackTrace());
			}
		}
		
	}
	
	public void TestThreads() throws Exception
	{
		String sFile = dirPath + "sample.xlsx";
		
		SimpleThread st1=new SimpleThread();
		SimpleThread st2=new SimpleThread();
		SimpleThread st3=new SimpleThread();
		
		st1.srcFile=sFile;
		st2.srcFile=sFile;
		st3.srcFile=sFile;
		
		st1.dstFile = dirPath + "out1.htm";
		st2.dstFile = dirPath + "out2.htm";
		st3.dstFile = dirPath + "out3.htm";

		st1.myThreadId="Peter";
		st2.myThreadId="John";
		st3.myThreadId="Maria";
		
		Thread t1 = new Thread(st1);
		Thread t2 = new Thread(st2);
		Thread t3 = new Thread(st3);
		
		t1.start();
		t2.start();
		t3.start();
		
		t1.join();
		t2.join();
		t3.join();
	}
	
	static String dirPath = "F:/Download/";

	public static void main(String[] args) throws Exception {
		
		SetLicense();
		
		ClsTestThread p = new ClsTestThread();
		p.TestThreads();
				
		
		System.out.println("Done");
	}
	
	public static void SetLicense() {
		
		String licPath = "F:/Download/Misc/Aspose/Licenses/Aspose.Total.Java.lic";
		
		com.aspose.cells.License lic = new com.aspose.cells.License();
		lic.setLicense(licPath);
				
		System.out.println("Aspose.Cells for Java v"+ CellsHelper.getVersion());
	}
	
}

Exception

Exception in thread "Thread-2" java.lang.OutOfMemoryError: GC overhead limit exceeded
	at com.aspose.cells.zbrk.a(Unknown Source)
	at com.aspose.cells.zbrq.a(Unknown Source)
	at com.aspose.cells.zbag.a(Unknown Source)
	at com.aspose.cells.zfl.a(Unknown Source)
	at com.aspose.cells.zaap.l(Unknown Source)
	at com.aspose.cells.zaap.d(Unknown Source)
	at com.aspose.cells.zaia.a(Unknown Source)
	at com.aspose.cells.zku.b(Unknown Source)
	at com.aspose.cells.ConditionalFormattingResult.getColorScaleResult(Unknown Source)
	at com.aspose.cells.ConditionalFormattingResult.a(Unknown Source)
	at com.aspose.cells.Cell.a(Unknown Source)
	at com.aspose.cells.zafv.d(Unknown Source)
	at com.aspose.cells.zafv.e(Unknown Source)
	at com.aspose.cells.zafv.a(Unknown Source)
	at com.aspose.cells.zafv.b(Unknown Source)
	at com.aspose.cells.zafv.a(Unknown Source)
	at com.aspose.cells.zafl.a(Unknown Source)
	at com.aspose.cells.zafl.b(Unknown Source)
	at com.aspose.cells.zafl.a(Unknown Source)
	at com.aspose.cells.zafn.b(Unknown Source)
	at com.aspose.cells.zafn.a(Unknown Source)
	at com.aspose.cells.zafo.a(Unknown Source)
	at com.aspose.cells.Workbook.save(Unknown Source)
	at com.aspose.cells.Workbook.save(Unknown Source)
	at ClsTestThread$SimpleThread.run(ClsTestThread.java:26)
	at java.lang.Thread.run(Unknown Source)

Amjad_Sahi · July 12, 2017, 9:40am

@eboraks2017,

Please try our latest version/fix: Aspose.Cells for Java v17.6.7:

Your issue should be fixed in it.

Let us know your feedback.

Thank you.

eboraks2017 · July 14, 2017, 9:48am

Is there any chance you can put this jar in your Maven repository?

Amjad_Sahi · July 14, 2017, 10:03am

@eboraks2017,

Aspose.Cells for Java v17.6.7 is a hotfix and we post hotfixes into the forums for the users who cannot wait for our official releases (in Downloads section and maven repos.). The hot fixes are like official releases that contain functionality, fixes and enhancements (full fledged) and can be used on production server without any issue. We publish official releases once in a month (which contain all the previous fixes) in the Downloads module. Moreover, we publish official releases to maven repos all well on the same day. Our next official release of the product, i.e, Aspose.Cells for Java v17.7 is due in the third or fourth week of this month, the date is not finalized yet as releases are published when ready. So, if you need to get the version (which includes the fix for your issue) from maven repos., you may wait a week or so.

You will also be notified here in the thread once we publish the next release.

Let us know if we can be of any further help.

Thank you.

eboraks2017 · July 14, 2017, 10:23am

No problem. I added manually to Maven.

eboraks2017 · July 14, 2017, 10:24am

I tested the solution, it’s better and much faster but I still get the error when running it with 10 threads.

Amjad_Sahi · July 14, 2017, 1:01pm

@eboraks2017,

Well, I think the OutOfMemory error would be inevitable in your case as you are loading/manipulating your big Excel file in 10 threads. Also, it would be a big process which itself demands huge memory cost. I think you may try to make sure you have allotted sufficient amount of memory to JVM for the bigger process.

Thank you.

Amjad_Sahi · March 25, 2020, 6:39am

The issues you have found earlier (filed as CELLSJAVA-42340) have been fixed in this update.

This message was posted using Notification2Forum from Downloads module by Aspose Notifier.

Amjad_Sahi · March 25, 2020, 6:41am

The issues you have found earlier (filed as CELLSJAVA-42340) have been fixed in this update.

This message was posted using Notification2Forum from Downloads module by Aspose Notifier.