Error in reading saved file on AIX

sqa · March 23, 2007, 7:41am

I saved this file on UNIX and then when I try to read it, I get the following error
- System Error: End of file reached.

java.io.IOException: End of file reached.
at com.aspose.words.b.c(Unknown Source)
at com.aspose.words.b.c(Unknown Source)
at com.aspose.words.b.e(Unknown Source)
at com.aspose.words.ar.(Unknown Source)
at com.aspose.words.jr.b(Unknown Source)
at com.aspose.words.jr.b(Unknown Source)
at com.aspose.words.jr.(Unknown Source)
at com.aspose.words.kx.a(Unknown Source)
at com.aspose.words.kx.(Unknown Source)
at com.aspose.words.eb.(Unknown Source)
at com.aspose.words.Document.a(Unknown Source)
at com.aspose.words.Document.a(Unknown Source)
at com.aspose.words.Document.(Unknown Source)
at com.aspose.words.Document.(Unknown Source)
at com.fidelity.efras.server.models.ExportModel.removeBookMark(ExportModel.java(Compiled Code))

The file is attached.
Thank you,
Sharad

sqa · March 23, 2007, 8:03am

and this file(3141_fs_L_09302006.doc), I can open with word but not with Aspose.
I get the following error

- System Error: Index: 4, Size: 4
java.lang.IndexOutOfBoundsException: Index: 4, Size: 4
at java.util.ArrayList.RangeCheck(ArrayList.java(Compiled Code))
at java.util.ArrayList.get(ArrayList.java(Inlined Compiled Code))
at com.aspose.words.cw.a(Unknown Source)
at com.aspose.words.jr.a(Unknown Source)
at com.aspose.words.jr.a(Unknown Source)
at com.aspose.words.jr.b(Unknown Source)
at com.aspose.words.jr.(Unknown Source)
at com.aspose.words.kx.a(Unknown Source)
at com.aspose.words.kx.(Unknown Source)
at com.aspose.words.eb.(Unknown Source)
at com.aspose.words.Document.a(Unknown Source)
at com.aspose.words.Document.a(Unknown Source)
at com.aspose.words.Document.(Unknown Source)
at com.aspose.words.Document.(Unknown Source)

Thank you,
Sharad

DmitryV · March 23, 2007, 8:46am

Hi Sharad,
Thank you for reporting the issue to us. Could you please specify what application you used to save this files? Note also that MS Word fails to open the first attached file as well.

sqa · March 23, 2007, 9:02am

Both the files were saved using Aspose.
Yes, Word also cannot open the first file.
The same code works fine on Windows. ie I can save the file using Aspose and open it both with Word and Aspose.
But on AIX, something is not write with the way the files are saved, sometimes the save works fine and I can open the file but sometimes the file is corrupted. I am wondering if it has anything to do with the way you flush/close the stream.

DmitryV · March 23, 2007, 10:57am

I will ask our Java developer to take a look at this issue. Meanwhile I have logged the issue as #1538 in our defect database. Thank you for reporting it to us.

sqa · March 28, 2007, 4:26am

Thanks Dmitry,
How will now ,when its fixed?
Sharad

Konstantin · March 28, 2007, 4:55am

Hi, Sharad,
Sorry for inconvenience, can you also attach the code snippets that generate these files? And i did not understand fully: AIX not always generates bad files – some time good, another time bad?
Best Regards,

sqa · March 29, 2007, 10:44am

Konstantin,
Yes, sometimes it works, I am going to say if I execute the same code it probably works about 70-80% of the time. i.e. I can open the file both with Aspose and word and everything is fine.
all the code to generate the doc is spread in number of files.
Here is the last bit of code that actually does the save.
Originally we were using the commented line to do the save, but since it didn’t work I have been playing around with streams to save.

// this.document.save(this.outFilePath+this.outFileName+FILE_EXTENSION);

File file = new File(this.outFilePath+this.outFileName+FILE_EXTENSION);
FileOutputStream fo = new FileOutputStream(file);
fo.getChannel().lock();
this.document.save(fo, SaveFormat.DOC);
fo.getChannel().force(true);
fo.close();

Konstantin · March 29, 2007, 9:56pm

Hi, Sharad,
It is corrupted structured storage inside your files – i.e. some streams inside the SS have wrong length that causes different exceptions. The one thing already changed - in the next aspose.words for java release all these exceptions will be wrapped into one FileCorruptedException, but I don’t think that this solves your problem:)
The first thing I thought was that this is some kind of endianness issue (Windows is little-endian, your AIX most likely is big-endian). But you talk that the same template doc with the same code works under AIX in 80/20 probability (?). So this is transient error and endianness is unrelated to ours problem.
Probably some streams inside the doc incorrectly saved. It can be some pictures, ole objects, etc. We have to figure out these unsafe objects. Can you attach 1) the template document and 2) the code that produces (not saves) the destination document. They will help me to locate the error.
Best Regards,

sqa · March 30, 2007, 5:45am

Konstantin, The zip file contains, two java files and two templates.
There are no OLE objects or antyhting fancy.
The code starts with template and deletes the book marks that are not needed and replaces the book mark values.
The code works fine on Windows 100% of the time, but is sporadic on AIX.
I was wondering if you could send me the code for the Docuemnt.save().
Thank you,

Konstantin · April 3, 2007, 4:31am

Hi, Sharad,
I skipped through your 80K+ code snippets and template docs and noticed that you using nothing like shape, ole, macro, etc. It is only text and bookmarks (may be I miss something?).
I checked even ours internal streams flashing and closing logic – all okay here (else you didn’t be the first with this type of questions:)
For the first time I can suggest you three things:

Try the new release of Aspose.Words for Java (will be published within 2 days) – it contains several changes improving bookmark resilience.
Try to increase a memory available for your AIX JVM.
I see you play with a file channel locks. File locks are generally needed when integrating with non-Java applications, to mediate access to shared data files but generally not appropriate for intra-JVM coordination between threads. Is your Aspose application launched on several threads or on several JVMs under AIX? Is it possible some output file sharing conflict in your AIX config? We encountered similar file sharing conflict with saving pictures as separate files for html and pdf documents. We solved this by making unique file names for these pictures – by including new UUID in each file name.

I have noticed also that all failed files have the same size. Did the good files have other sizes?
Best Regards,

sqa · April 10, 2007, 10:12am

Konstantin,
I don’t see the new release yet. This is what I see.

2.0.0.0
Version: 2.0.2.0
Date Published: 12/13/2006

If the newer version is available, Can you please provide the link.
Thank you,

DmitryV · April 10, 2007, 10:55am

Hi,
The new version will be available in 1-2 days.

sqa · April 11, 2007, 9:17am

what about the file that word can open and aspose cannot?
It was saved using Aspose.
Thanks,

Konstantin · April 11, 2007, 10:00am

Hi, Sharad,
It is corrupted structured storage inside of both your files, and both files have the same size – 58K. Word’s structured storage is like a FAT inside a file (are you remember the FAT file system?). If using FAT analogy: your files probably have a good file table, but wrong length of some files (or unexpected end of media). MS Word is more tolerant to some errors and can open some fault files but this is just a lottery.
What about my questions about JVM memory and file sharing in your AIX environment?
I can’t reproduce your bug using 80K+ code snippet, - can you reproduce the bug in AIX within some kind of sandbox: compact code snippet that not depends on external libraries, servlet containers and so one?
Best Regards,

sqa · April 13, 2007, 12:58pm

We are not sharing any files, JVM has more memory on AIX boxthen on Windows where its working fine.
I will try to write some code and see if I can break it.
In the mean time can you look at the file and tell me what is really corrupted, is it like missing end tag for a book mark, or arbitrarily terminated file or soemthing.
Thanks

Konstantin · April 14, 2007, 9:59pm

Hi, Sharad,

The files are corrupted on the structured storage level. It is the very low level in the MS Word. Only after the structured storage successfully loaded, the document model (including bookmark tags) will be build.
It’s seems that something wrong with in-memory streams when the structured storage constructed on AIX or with file IO when the document saved (since both files have equal size).
Regards,

sqa · April 16, 2007, 7:56am

File size being same is just a coincidence, I have files which are different sizes and wouldn’t open with Aspose but open with Word.
Here is something else I noticed, When I start my server, It works fine for first 5-6 times and then it starts creating bad files. Could there be any static values that might be getting corrupted?

Konstantin · April 16, 2007, 11:55pm

Hi, Sharad,
All static values are final in the Aspose Words for Java… so I don’t think that they can be corrupted. Any way the error is platform- or application- specific because we didn’t get such an error in other java platforms.
It’s interesting info about first 5-6 good docs. It confirms ours versions that it is something wrong with in-memory streams (or GC) or with file IO. Can you check output files – may be some of them accidentally modified or overwritten? Or another question, what is different in Windows and AIX configurations?
Anyway I can do something real with the error only when you provide to me compact code snippet that reproduces the error.
By the way, did you try the new Aspose Words for Java release?
Best Regards

sqa · April 17, 2007, 6:11am

Konstantin,
I fully agree with you that its hard for you to fix my problem unless you see some code that produces it. The only difference that I can think of between windows and AIX is that on Windows its running on Tomcat in Eclipse where as on AIX its running under websphere. Main reason I started this thread was in case you knew of any known issues between AIX and windows.
I will try the new jar and let you know if I can reproduce the error in a snippet.