Data is for databases. It’s clear that information in all its forms generally resides in some form of data store, data repository or data coalition, collection and coalescence tool so that we can go and get it when we need it. This core truth means that data is for databases and databases are for database administrators.

But data is also for developers.

Software application development engineers take a primary role in wrangling with data management in many scenarios, not least of which is the act of extracting reliable and consistent data from business documents. Because the world of enterprise business has moved (and is still moving) to a digital-first base of operations (you may have heard about so-called digital transformation, just maybe), organizations need to ingest and encode valuable documents (some digital, but still a lot of paper) so that the content they contain can form a functional part of business workflows.

But business workflows can get lumpy and suffer from disruptions. What we want are intelligent automation workflows that happen inside business operations with digital documentation from the start. This is the pain point that intelligent automation company Abbyy seeks to address directly with the launch of its ABBYY Document AI service. Stylizing its brand in capitals as it does, Abbyy’s new intelligent document processing tool is accessible through what here is defined as a “self-service” (meaning developers don’t need the operations team to enable it) application programming interface.

Making (Structured) Sense Of Unstructuredness

The company says that its Document AI API was built with the developer’s experience in mind. It allows software engineers to transform unstructured business documents into structured, accurate data with just a few lines of code. This functionality makes it easier to integrate and work with optical character recognition and intelligent document processing solutions.

“As a vanguard of OCR, Abbyy has long had a vibrant community of cutting-edge developers creating transformational solutions with our advanced document AI,” said Nick Hyatt, vice president, engineering R&D at Abbyy. “We are providing them with a new API with minimal setup as well as access to ample community resources, pre-trained models for building proof-of-concepts and a predictable pay-as-you-go pricing model. Abbyy Document AI API is a major step forward for developing automated document workflows.”

According to analyst house IDC, the intelligent document processing market is projected to grow from $2.4 billion in 2023 to $10.5 billion in 2028, This 34.9% CAGR is thought to be driven by a number of factors, with key drivers including increasing cloud adoption and cloud-native development, the maturation of AI services as we move out of the intelligence hypecycle into practical use cases and expanded document AI use cases in general.

Hallucinations & Data Inconsistencies

“In the age of AI, optical character recognition is experiencing a true renaissance,” said Amy Machado, senior research manager for enterprise content and knowledge management strategies at IDC. “Developers struggle with extracting reliable data from documents and will often begin with general large language models for this process. However, they quickly face challenges with hallucinations, data inconsistencies and errors in document processing. [They also]

often lack support for multiple [human] languages, handwriting recognition and complex document structures. There is a need for purpose-built solutions specifically designed for document processing that prioritizes easy integration, flexibility, scalability, accuracy and consistency.”

Abbyy says that the Abbyy Document AI API enables software developers to enhance workflows with “pre-trained models to extract data” from documents, which in turn naturally empowers teams to be able to accelerate automation for complex business processes like KYC (a set of guidelines and principles used mainly by financial institutions to verify and validate new clients, standing for know your customer as it does), business account openings of all forms, customs clearance, invoice processing, expense management and order processing.

Precision-Engineering For OCR

Abbyy Document AI API enables quick, accurate and effortless data extraction to quickly convert business documents of any type, format or language. According to Abbyy, this new software offering provides “precision OCR”, capable of flawlessly preserving a document’s logical structure to provide AI-ready data that is essential to unlocking insights in generative AI and retrieval augmented generation. It can also help with core tasks associated with forming the robust foundation needed to train language models.

This news comes on the heels of the company establishing new AI labs across the United States, Hungary and India to accelerate the development of purpose-built AI for intelligent document processing and process automation.

“Our proprietary datasets, AI platform and model research and development combined with our deep domain knowledge create foundational intellectual property that will significantly enhance our core solutions and enable expansion into adjacent enterprise applications. We’re building upon decades of leadership in OCR, machine learning, computer vision, and natural language processing while extending innovations in AI with our industry expertise. This integrated approach powers next-generation multimodal models to deliver more robust, consistent outcomes that transform business processes,” said Sanjay Nichani, vice president for, AI & computer vision at Abbyy.

Developer Experience Matters, After All

We hear a lot from enterprise technology vendors who tell us about their focus on customer experiences and now, more recently, the need to make great customer experiences happen by first enabling good software developer experiences. Underling this truth, Abbyy VP Hyatt has said that, “The developer experience is a crucial aspect of our product strategy. Our teams look forward to making next-generation ABBYY AI easier to consume with modern APIs and developer tools.”

This sector of enterprise technology has been on a journey. We can reasonably track intelligent document processing and today’s approach to AI-driven process intelligence back to the data mining techniques popularized throughout the nineties, if not before. CMO at Abbyy Bruce Orcutt says that no firm “touches” more documents than Abbyy, with so many deployed instances of its platform now spanning markets across banking and financial services, healthcare, software & IT services, insurance and retail.

“Only around 25% of work executed in systems of record is core transactional work related directly to business processes,” said Orcutt. “When we move forward to AI-driven process intelligence with intelligent document processing, we can start to reduce that 75% of repetitive administration grunt work in poorly documented, poorly integrated and poorly managed processes. This enables us to find out who the most valuable people are in any given team and then (and this is positively not about AI taking over human jobs) be able to reallocate employees into new value-add roles, where they can do their jobs better with AI tools underpinning them. While we have focused on product and process owners inside any given organization, we are now expanding to the software application developers actually building the automations we can now make workplaces better with.”

With so much at stake in the data management arena, we clearly need to think about data for databases and database administrators, data for developers as showcased here… and the resultant new data services that consumers will be able to use when AI and automation enters its Industry 3.0 phase which it must next logically do.