Layout detection issues on technical and multi-column scientific PDFs using Python via .NET

Aspose.OCR Product Family
1

Hello,

My use case is extracting semantic content from PDFs for downstream chunking and RAG processing. I am currently evaluating Aspose.OCR for Python via .NET (aspose-ocr-python-net==26.5.0) and ran into layout detection issues on both a single-column technical guideline and a multi-column scientific paper PDF.

I tried the following approaches:

  • PDF → DOCX conversion using multiple strategies. This produced many severe issues for my use case, and seems more focused on visual fidelity than semantic extraction.
  • api.recognize(input_data, settings) with DetectAreasMode.MULTICOLUMN, using the raw recognition outputs.
    • For text, this was mostly usable after I added my own reading-order correction, especially for multi-column pages.
    • However, tables, formulas, figures/images, and other semantic elements were not extracted separately enough for my use case.
    • Rerunning each page with MULTICOLUMN, TABLE, and FORMULA modes and then combining the results seems expensive and likely to introduce duplicate/conflicting output.
  • I then tried detect_document_layout() as a routing step: detect semantic regions first, then run the appropriate OCR method on each detected area and merge the results. This seemed like the most promising architecture, but I encountered repeated issues with the quality of the detected layout areas.

I am wondering whether there is a better recommended pipeline, additional configuration, preprocessing, model setup, or mitigation for this use case.

I attached a representative sample of problematic pages, along with bbox renderings and detection reports for each. Page numbers refer to the original document page numbers. I also attached the script used to generate the visualizations and reports.

DNV_5_pages.pdf:

  • page 17 — list bbox is too narrow; part of the list content is outside the detected region
  • page 28 — equation region is misclassified as paragraph/header; another equation/content area appears to be missing from layout detection
  • page 38 — list bbox is too narrow; equation region is missed or misclassified as header
  • page 40 — equation regions are missed or misclassified as header/image
  • page 78 — figure is misclassified as table; equation regions are missed or misclassified as header/paragraph

Resnet_4_pages.pdf:

  • page 1 — figure/image and its caption are detected as a single paragraph
  • page 7 — table is detected as image; another content area appears to be missing
  • page 8 — table is not detected as any semantic area; figure/image and its caption are detected as one image region
  • page 11 — tables are detected as paragraphs

Minor paragraph/header/caption differences are not the main concern. The blocking issues are tables, equations, figures/images, missing regions, and bboxes that do not cover the full content, because I need to use detected regions as the routing layer for downstream extraction.

I would appreciate guidance on the following:

  1. Is detect_document_layout() intended to distinguish paragraphs, lists, tables, equations, images/figures, captions, and headers reliably enough for downstream routing?
  2. Is there a better-supported Aspose.OCR Python via .NET pipeline for extracting PDF content into separate semantic elements while preserving reading order as much as possible?
  3. Are there known limitations around reading order, semantic structure, element-level extraction, or layout detection for this type of document?
  4. If detect_document_layout() is not intended for this use case, is there another Aspose API/product that would be more appropriate?

For preprocessing I used auto_skew and contrast_correction_filter. I also tried other settings during evaluation, and detect_defects did not report any defects.

Thanks,
Piotr
ocr_layout_topic.zip (4.8 MB)

2

We have opened the following new ticket(s) in our internal issue tracking system and will deliver their fixes according to the terms mentioned in free support policies.

Issue ID: https://issue.saltov.dynabic.com/issues/OCRPY-113

You can obtain Paid Support Services if you need support on a priority basis, along with the direct access to our Paid Support management team.

1 Like
3

Unfortunately, at the moment detect_document_layout is the only available method for detecting document regions, and your implementation is correct.

However, we are aware that this method does not always provide the desired level of accuracy. We are currently planning to integrate a more modern region detection approach that should significantly improve detection quality.

Our current estimate is to have this improvement available within the next 1–2 months.

Thank you for your feedback and patience. Your use case is very helpful in guiding these improvements.

1 Like
4

This is a preliminary result from the approach we are currently evaluating. We are sharing these intermediate results so you can evaluate our progress and provide feedback. We expect significantly better accuracy after integrating the new region detection model.
ocr_layout_topic.zip (4.8 MB)