Problem statement

The client wants to build a system to extract information from a wide variety of medical documents in a structured format. The format includes sections with key-value pairs, columns, free text, and titles.

Example: https://drive.google.com/drive/u/0/folders/1xzPt4zdDMUHnFPb0ESdo87qmUcCJVLn-

Untitled

{
  "document_title": "INTRAOPERATIVE NEUROPHYSIOLOGY",
  "print_time_stamp": "08-17-2019 08:59 AM EST",
  "surgery_details": {
    "surgery_date": "08-12-19",
    "patient": {
      "dob": null,
      "age": null,
      "sex": null
    },
    "surgeon": "KHODAM-RAD PAYMAN, M.D.",
    "anesthesiologist": "Joel",
    "ionm_technologist": "Octavio Calderon, CNIM",
    "location": "AVANGUARD SURGERY CENTER",
    "diagnosis": "lumbar stenosis",
    "procedure": "TLIF L5 - S1"
  },
  "timing": {
    "post_baseline_time": "08:00",
    "final_trace_time": "12:00",
    "total_professional_time": "04:00"
  },
  "conditions_of_the_recording": {
    "description": "All studies were performed on the aforementioned patient under real-time physician direct supervision via internet communication allowing continuous or immediate contact between the interpreting neurologist and surgeon. Please see tech notes for details of stimulation and recording."
  },
  "description_of_the_recording": {
   "description": "Somatosensory Evoked Potentials (SSEPs) were recorded to monitor the integrity of the somatosensory pathway. Sterile 12 mm subdermal needle electrodes were inserted post induction. The International EEG 10-20 electrode placement system was used. Paired surface disc electrodes were used for stimulation of all peripheral nerves and dermatomes. A differential amplifier system with a digital recording signal averager was used. Lower extremity Free running Electromyography (EMG) was performed to monitor the integrity of the motor system and for nerve/root irritability. Recording electrodes were placed in muscles appropriate to the site of the procedure. Electrical Stimulation of the pedicle screws were evaluated by using triggered EMG performed by stimulating each screw and recording a compound muscle action potential response in the appropriate muscles. A response to stimulus intensities of 10.0 mA or less in lumbar levels or 6.0 mA or less in thoracic levels raises concern for improper screw placement and potential breach in the pedicle wall. Corresponding thresholds were noted and communicated to the surgeon."
  },
  "summary": {
    "SSEPs showed signals with clear morphology and amplitude in bilateral upper and lower extremities from baseline traces.",
    "EMG showed no activity at baseline. During the course of the procedure there were no significant changes noted in SSEPs. No significant sustained discharges were noted to suggest nerve root irritation or mechanical disruption. Pedicle screw thresholds indicate low probability of pedicle crack or breach. The surgeon was kept informed of signals during the course of the procedure."
  },
  "impression": "see summary",
  "page": "1 of 2",
  "surgery_date_footer": "08/12/2019"
}

NOTE: This is the cleaned up JSON. The most raw output contains bounding box information with confidence scores.

Challenge

Untitled

Untitled

Untitled

Untitled

Untitled

Solution Outline

We focus on extracting information from readable PDFs and scanned PDFs

Untitled

File content Reader: