An interesting problem, particularly with handwritten documents. And not limited to what they worked on, from their conclusion: "...the number of clustering levels is not limited to two, since document layout tasks are extremely diverse in nature. Paragraphs can further be clustered into text columns or sections, which may belong to even higher level blocks. Figure 16 shows a physical paragraph, or part of a semantic paragraph which spans across multiple text columns..." Not to mention text wrapping around figures.
The general field is called "document structure analysis" or "document layout analysis." There's been lots of work--at a cursory glance at this article, I'm not sure they've discussed that literature.
I worked on a similar problem a decade or so, although our work was done mostly by hand. We were trying to not only read in (bilingual) dictionaries using OCR, but turn them into dictionary entries, and then parse each entry into its parts (headword, part of speech, definitions or glosses, example sentences, subentries...). I won't go into details, but to our surprise one of the most difficult parts for the machine to get right was recognizing bold or italicized text.
Interesting. Two engineers at Apple worked on something similar that would slurp character bounding boxes from a PDF page and reconstruct paragraphs, columns, tables, etc.
It was surfaced in iOS a decade ago as "tap to zoom" feature for PDFs. It's funny — as with a lot of things there was a lot of sophisticated engineering under the hood and then marketing simply wants it to detect a tap in a paragraph and zoom to its bounds.
I can't think of the last time I read a PDF on my phone or I would test it to see if it still works as I remember.
Wow, I had no idea. That’s pretty cool. I don’t read a ton of pdfs on my phone, but I always find it super frustrating when that functionality is missing somewhere else.
An interesting problem, particularly with handwritten documents. And not limited to what they worked on, from their conclusion: "...the number of clustering levels is not limited to two, since document layout tasks are extremely diverse in nature. Paragraphs can further be clustered into text columns or sections, which may belong to even higher level blocks. Figure 16 shows a physical paragraph, or part of a semantic paragraph which spans across multiple text columns..." Not to mention text wrapping around figures.
The general field is called "document structure analysis" or "document layout analysis." There's been lots of work--at a cursory glance at this article, I'm not sure they've discussed that literature.
I worked on a similar problem a decade or so, although our work was done mostly by hand. We were trying to not only read in (bilingual) dictionaries using OCR, but turn them into dictionary entries, and then parse each entry into its parts (headword, part of speech, definitions or glosses, example sentences, subentries...). I won't go into details, but to our surprise one of the most difficult parts for the machine to get right was recognizing bold or italicized text.
Interesting. Two engineers at Apple worked on something similar that would slurp character bounding boxes from a PDF page and reconstruct paragraphs, columns, tables, etc.
It was surfaced in iOS a decade ago as "tap to zoom" feature for PDFs. It's funny — as with a lot of things there was a lot of sophisticated engineering under the hood and then marketing simply wants it to detect a tap in a paragraph and zoom to its bounds.
I can't think of the last time I read a PDF on my phone or I would test it to see if it still works as I remember.
Wow, I had no idea. That’s pretty cool. I don’t read a ton of pdfs on my phone, but I always find it super frustrating when that functionality is missing somewhere else.
2022, and we need this in browser reader modes.