Governed lending AI agents on Azure now verify new loan applications by voice at every branch, so specialists only see authentic, qualified applicants.
A Texas consumer lender wanted to find out whether agentic AI could take repetitive calls off its call center and lending teams without putting customer data or compliance at risk. Discovery workshops came first, then a test of the market for packaged tools, then custom lending AI agents built on Azure. The first one, a voice agent that verifies new loan applications, is in production across all branches.
The client is a consumer lending company that has offered personal, credit starter and signature loans since 1968, including to people from underserved communities. Beyond its home state of Texas, it runs service centers in Oklahoma, Virginia and Missouri.
It already had a robust proprietary loan management system for core processing and servicing. What it lacked was a safe way to use large language models and agentic AI to push efficiency and profitability further. The company had worked with our lending engineering team for over five years, so it asked the people who already knew its systems to validate the idea, plan the agents and deploy them inside live workflows.
Consultants ran focused workshops with the lender's stakeholders to map operational needs, expectations for the agents and the constraints on AI implementation.
For each high-impact use case, the team delivered an agent concept, a detailed software requirements specification and a tailored set of performance, quality and user acceptance metrics. Those documents gave the lender a clear basis for choosing what to build first, and gave the engineers a fixed yardstick for testing later.
The agents were specified to switch instantly between English and Spanish during calls, so multilingual customers get consistent communication. Verification is live; the other two follow in later projects.
Picks up applications that passed initial credit bureau checks and calls the applicant on a consented voice call. It verifies identity with a Social Security number and date of birth and confirms borrowing intent. Only authentic, qualified applications reach human specialists.
Planned to call debtors, walk them through repayment options and propose alternative terms on the fly within FDCPA rules. It would collect approvals for saved payment methods, take new card and bank details, and capture promises to pay and renewal confirmations, while the backend triggers real-time gateway payments and ACH direct debits.
Intended to triage inbound calls, work out the intent of each request and route it to the right loan servicing specialist. Automated call dispositioning would cut wait times and free servicing capacity.
The lender first planned to adapt ready-made agentic AI tools. A shortlist of the most suitable vendor offerings came first, then a feasibility study before anyone wrote code.
| Option | Feasibility finding | Impact |
|---|---|---|
| Customize packaged agentic AI tools | Most commercial agentic AI products were not ready for enterprise-grade rollout. | Extensive rework needed to handle the lender's real lending processes. |
| Build custom agents on cloud AI services and commercial LLMsRecommended | Cheaper than customizing packaged tools, with comparable release timelines. | More control over agentic workflows and solution scaling. |
The lender chose the custom path. The project manager proposed building the agents one at a time, each as a separate project with its own work breakdown, team plan, estimates and risk steps. That limits scope creep, keeps the same team on the whole initiative and lets later agents reuse tested components with minimal QA overhead.
The architects built the verification agent from modular, event-driven components and used managed cloud services wherever possible to keep costly custom code down. LiveKit Cloud configures the calls, the LiveKit Agent SDK holds the agent logic, and Azure Container Apps runs the workflows on Azure, where the lender's systems already lived.
Workflows scale horizontally on demand, based on resource use and active sessions. A call dispatch service with queue-based load leveling and configurable session limits prevents overprovisioning. The team compared LLM hosting options such as Azure OpenAI and OpenAI Realtime, then picked configurations that balance tokens, concurrency limits and pricing tiers.
Voice calls need near-real-time responses. Real-time audio streaming through LiveKit was chosen for low-latency concurrent audio between agent and customer, and its streaming APIs for agent and LLM interaction cut integration work and help keep conversations flowing under heavy load.
The agent talks to the lender's application platform and outside services through standardized APIs. New application sources, customer channels or AI models plug in with minimal changes to the core.
The agent acts only through structured function calls and predefined business logic, so every action is deterministic, auditable and aligned with business rules. Control modules handle instructions, action tracking and approval, and full interaction tracing lets the team reconstruct any decision.
In lending, a voice agent handles identity data on every call. These controls were part of the design, not added at the end.
Engineers connected the custom modules and third-party services and integrated the agent with the lender's website, the main source of applications. The admin panel gained agent control dashboards where specialists track verification status and resolve open cases by hand.
QA confirmed the agent reads user input correctly, calls back-end services and follows its interaction logic, then validated the links between the agent, telephony services, LLMs and the website.
Full verification calls were simulated to check conversation flow, response accuracy, latency and customer experience.
Testers fed the agent ambiguous or inconsistent answers, interrupted calls, unexpected inputs, API timeouts and slow model responses. It had to recover without breaking the conversation or exposing sensitive data.
Load tests on varying data volumes measured scalability, latency and concurrent call capacity. Dedicated voice tests checked speech quality, responsiveness and coherence on live calls.
In user acceptance testing the agent handled verification accurately and moved data cleanly across systems. A pilot at one branch followed, and as of April 2026 the agent runs in production at all branches.
Throughout the past five years, the team has always brought structure, technical insight, and a strong understanding of lending operations to our IT initiatives. Seeing that they approached AI engineering the same way gave us confidence from the start. We knew our agents would be secure, well-governed, and aligned with the way we work without overspending. The first live agent fit naturally into our application review process and brought results right away.
Early feedback from the pilot branch was clear: the agent reduced manual workload and made application verification more consistent. Automated checks at an early stage also tighten risk control, because weak or fake applications stop before a specialist spends time on them. The design work for the debt collection agent is now under way, using the same architecture principles.
If you are planning lending AI agents of your own, three lessons stand out. Test packaged tools against your real processes before you commit. Ship one narrow agent at a time with the same team. And make every action auditable from day one, with tracing on open standards such as OpenTelemetry.
LiveKit's agent framework offers SDKs in Python and Node.js, so if you need more capacity for agent logic and integrations, you can hire Python developers or Node.js developers. You can also browse more case studies, or get started with a scoping workshop.
The feasibility study found most commercial agentic AI products were not ready for enterprise rollout and needed extensive rework for real lending processes. Custom task-specific agents on cloud AI services and commercial LLMs were cheaper, had comparable timelines and gave more control over workflows and scaling.
Sensitive data is encrypted again before it enters the communication layer and decrypted only inside isolated agent runtimes. The agent runs in a dedicated Azure Virtual Network with private endpoints, so the data stays within controlled components.
GLBA for personal and financial data handling and residency, TCPA for customer outreach such as consent and call timing, and FDCPA for the planned debt collection agent.
After functional, integration, end-to-end, edge case, load and voice testing, the lender ran user acceptance testing, then a pilot at one branch. Full production at all branches followed by April 2026.
That is part of the design. The agents were specified to switch instantly between English and Spanish during calls. The verification agent is the first in production; the debt collection and call handling agents are still to be built.
Azure OpenAIOpenAI Realtime API
Azure Container AppsAzure Service BusAzure Key VaultAzure Virtual NetworkAzure Private Endpoints
PostgreSQL
RedisAzure Application Insights
A secure enterprise platform that gives organizations controlled access to GPT-4, GPT-4O, and Perplexity through a single interface. A local screening model checks every prompt before it reaches external LLMs, with role-based access, custom system prompts per department, and a complete audit trail.
A US medical technology startup had a sensor-equipped slant board prototype and a tight deadline. We built the firmware, iOS, Android and web apps and AWS back end for a connected rehabilitation platform, plus the software documents for its FDA 510(k) submission.
A US real estate firm ran its whole operation in Podio but had no single view of performance. A live analytics layer over 14 Podio apps now gives eight teams dashboards that load in under a second and update within a minute, cutting reporting time 40-50%.
Tell us where the work is stuck today. In one call you get a senior engineer's view of the approach, the risks worth planning for, and a realistic first release.
