An AI journey: The revolutionary role of Transformers in finance AI

Co-founder Kunal Verma explains the role of Transformers in developing generative AI and how they’ve contributed to AppZen’s success.

Kunal co-founded AppZen in 2012 and developed its core artificial intelligence technology. He's passionate about developing AI-based solutions to solve real-world business problems and is responsible for AppZen’s product vision. As CTO, Kunal oversees the company’s R&D and data science teams. He earned his Ph.D. in Computer Science from the University of Georgia with a focus on semantic technologies, is a published author with over 50 refereed papers, and holds several granted patents.

Changing the face of finance

The last few years have seen a revolution in artificial intelligence (AI) technology. One of the most important developments has been that of the Transformer. To give you some context, the “GPT” in ChatGPT stands for “generative pre-trained Transformers.” These Transformers have paved the way for the rapid development and deployment of generative AI, which creates detailed responses to questions and requests. It’s a faster AI training technology that has profoundly redefined how we approach almost every aspect of business, including finance.

In 2017, a team of Google researchers published a groundbreaking paper, “Attention Is All You Need,” shifting the focus away from traditional machine learning to developing the Transformer model. The Transformer is a deep learning AI architecture that can generalize many different kinds of tasks. It’s primarily based on an attention mechanism that allows it to capture deep relationships between all tokens, or sub-words, within a domain (or specific body of knowledge). When Transformers are given an input sentence, they can generate the next few sentences based on the relationships between tokens they have already captured. Models that use Transformers in this way outperform other technologies in quality, speed, and efficiency.

AppZen’s Transformer-enabled AI journey

Every organization’s finance team understands how critical capturing accurate financial data is for business continuity. Unfortunately, even the best automation tools often leave teams with a lot of manual work. AppZen initially set out to alleviate the burden of automation that still left auditors feeling overworked.

In our first steps down this road, AppZen Expense Audit used conventional AI and machine learning to set benchmarks for back-office finance automation. But our AI engineers quickly recognized the potential of Transformers. They saw how Transformers could enable the building of AI models that could handle such complex issues as finding duplicates across reports, reducing false positives with a deeper understanding of transaction lines, and supporting global business operations through multi-lingual audits.

Layers of improvement

AppZen’s custom-trained Transformers are much smaller than their large language model (LLM) counterparts, yet they maintain those models’ robust capabilities. They’re optimized for efficiency and maintain a much better balance between recall (how much information they can draw from to get the right answer) and precision (how often they’re correct). We use them at every level of transaction processing. They’re incorporated into the computer vision and extraction layers to pull raw data from receipts and documents, such as invoices, and classify that data for use within the workflow.

The result is a 40% reduction in processing costs and a 50% increase in the resolution of transactions. Using Transformers within audit models in the data analysis layer decreases auditor workload by an additional 10-15%. They also contribute to the system's self-improvement by incorporating user feedback and final processing outcomes. And custom models can be tailored to client-specific auditing needs for even more improvement.

Deeper finance domain understanding

Our Transformer-enabled models were first trained with general finance domain expertise for enterprise spend, then on millions of receipts per day. This created powerful solutions with a deep understanding of finance transactions and auditing processes. The reason ChatGPT works so well is its ability to extrapolate from known language elements to understand what is unknown. In the same way, our AI uses analogies to make sense of information it wasn’t directly trained to recognize. When a model encounters an unfamiliar expense item that it knows is in the same category of unauthorized spend as other items it’s seen, it can flag that out-of-policy purchase.

Traditional automation, such as optical character recognition (OCR), can recognize words on a receipt and enter the information into a template, then flag anything that it’s been told to watch for, such as “spa” on an expense receipt. In this example, our AI not only recognizes text but also understands contextual nuances. It can read that same receipt, written in Dutch, and understand that “spa” is a bottled water brand. Transformers help it understand not only the language but also the relationships between each element within a transaction.

In the next example, our AI was not trained to recognize “ferrous sulphate” as a personal care product on a receipt. Despite that, the AI knew that this was a type of medicine. It also knew this item was purchased at a store that commonly sells medicines and was therefore highly likely to be in the class of products that should be flagged as personal expenses. (The AI also recognized a potential duplicate line item.)

Future-proofing your business with generative AI and self-learning

We’ve also created a transformative solution for Accounts Payable. Our latest tool for AP email inbox management uses cutting-edge generative AI powered by the same Transformer technology we’ve worked with for years. With unprecedented speed and accuracy, AppZen Inbox automates the organization and processing of and response to AP emails. It categorizes and prioritizes emails with an understanding of content and context. It manages touchless invoice processing from the moment an invoice reaches you. It even handles supplier communications and is compatible with Gmail and Microsoft Outlook email clients. This integration of generative AI right at the start of your AP workflow promises to elevate stakeholder engagement, streamline communication, and free up time for strategic decision-making.

Our AI solutions also continuously improve through user feedback, adapting autonomously to new data without the need for additional input or manual tweaks. This seamless integration allows the system to self-learn and evolve, enhancing accuracy with minimal user intervention. In finance, where transparency is key, our AI not only becomes more efficient but also remains explainable, ensuring that any data processing can be clearly justified to our customers. This approach negates the need for future model deployments or updates, embodying a touchless, self-sustaining system tailored to the dynamic needs of your finance organization.

Keeping our attention focused on the future

With a deep commitment to relentless innovation, we've designed a suite of AI-powered solutions: AppZen Expense Audit for T&E, AppZen Autonomous AP for accounts payable processing, AppZen Card Audit for corporate card transaction auditing, and AppZen Inbox for supplier query management. These streamline back-office finance by removing common workflow bottlenecks. And there’s much more on the horizon.

The transformative AI journey that began with a visionary team at Google helped to define our own approach to AI training early in our products’ development. As we continue to advance AppZen’s product capabilities, features, and offerings, we constantly keep a finger on the pulse of AI development and push the boundaries of what’s possible. By closely examining our customers’ pain points and searching within new technologies for novel solutions, we remain focused on that original spark of “attention” that is igniting a revolution for finance teams everywhere.