Why does OCR fail on P&ID Extraction?

What is P&ID and why extract data from it?

Piping and Instrumentation Diagrams (P&IDs) are schematic blueprints of piping, equipment, sensors, and control logic in facilities such as oil and gas fields and refineries. They are the map for the process system.

Engineers extract equipment tags, design parameters, and topology—the relationship of what connects to what—and store them in a structured format for digital twins, process simulations, asset management, and equipment monitoring.

Other technical drawings engineers rely on include:

• Single Line Diagram (SLD): an electrical drawing showing how power flows through a facility.
• Equipment Layout: a drawing showing the spatial arrangement of equipment.

What is OCR?

Optical Character Recognition converts images of text into machine-encoded text. It enables useful applications such as real-time menu translation.

When extracting data from technical drawings, tools such as Tesseract, Azure AI Vision OCR, and Amazon Textract start to fall short.

Why does OCR fail on P&ID data extraction?

1. Text is rotated

Most OCR engines are optimised for documents where text runs left to right, lines are parallel, and orientation is consistent. Engineering drawings contain text rotated at 45° or 90°.

2. Drawings are spatially complex

P&ID drawings are spatially complex, with multiple tags often appearing on a single line that OCR cannot distinguish as separate entities.

3. Lines overlap

P&ID drawings contain dense piping networks, crossing and branching lines, arrowheads, tee joints, and control-loop bubbles.

• Dense piping networks
• Crossing and branching lines
• Arrowheads, tee joints, and control-loop bubbles

4. Tags use mixed formats and fonts

Engineering tags are structured codes, and OCR often breaks their structure:

• P-101A → P-101,A
• XV-2001 → XV200I
• 8" CS150-P → 8 CS ISO-P
• 2" LG-1001-SS → 2 LG-1001-5S

So what would actually work?

The answer is not abandoning OCR. P&ID extraction needs a multi-layered approach:

1. Preprocessing handles image resolution and improves the drawing before recognition.
2. Object detection recognises and locates symbols and asset tags.
3. Targeted OCR extracts the clear text labels and tags it does best.
4. Vision Language Models tie everything together by understanding spatial relationships and context.

Each technology handles what it is good at, solving what any single approach cannot.

We have spent the last year building exactly this and call it RosaryVision. Early results show 95% accuracy in tag extraction, reducing manual verification from hours to under 5 minutes.