Doctopus: A System for Budget-aware Structural Data Extraction from Unstructured Documents

Extracting structured data from unstructured documents is essential for applications like analytical SQL queries and decision-making. Strategies such as pre-trained language models (PLMs) can be employed, but they often fall short in quality. Large language models (LLMs) have shown effectiveness in attribute extraction but are costly, making them impractical for large-scale document sets. To best trade off quality and cost, we present Doctopus, a system designed for accurate and cost-effective attribute extraction. Overall, Doctopus combines LLMs with non-LLM strategies to achieve an optimal quality-cost balance. First, the system employs an index-based approach to efficiently identify and process only relevant chunks. Afterwards, it further estimates the quality of multiple strategies for each attribute. Finally, based on the cost and estimated quality, Doctopus dynamically selects the optimal strategies through budget-aware optimization. With a real-life scenario, we demonstrate that Doctopus allows users to extract attributes accurately and affordably. The corresponding video is available at http://youtu.be/Cxl-PfvZY10?si=NYoHt2SyD9KHqd6V.