Big 4 client · fully on-premise
Enterprise RAG Chatbot for a Big 4 Accounting Firm
Multimodal retrieval over a Big 4 firm’s full compliance library, tuned for citation instead of summary — every answer arrives with its source paragraph attached.
The problem
Junior auditors at a Big 4 firm were reading thousands of pages of compliance whitepapers and audit guidelines to answer a single question. Every senior became a search engine, and the same question got asked and re-answered across teams that never saw each other’s work.
What we built
A multimodal retrieval system over the full compliance library, running entirely on the firm’s own infrastructure. OCR and table parsing mean the semi-structured exhibits — the rate tables, the disclosure matrices, the scanned annexes — are searchable rather than silently skipped. Retrieval is tuned for citation rather than summary, so the system returns the passage that supports the answer alongside the answer itself.
Before
- Juniors reading thousands of pages per question
- Seniors acting as search engines
- Tables and scanned annexes unsearchable
How it works
Parse by layout. Each page is split into text, table, figure and caption regions by an ensemble of layout models. Tables and charts are cropped with their captions and read by a vision model, so exhibits are searchable rather than skipped.
Index with provenance. Text is chunked by meaning; every chunk carries its file, page and, for a figure or table, the image it came from — so each answer can point to the exact page.
Fine-tune on the firm’s own GPUs. Phi-3 and Llama language models and a Jina embedding model were fine-tuned on the firm’s own H100 GPU cluster, so neither the documents nor the training data left its infrastructure.
Retrieve two ways. Keyword and semantic search are blended (about 30/70), with diversity reranking, because rule numbers and names are exactly what embeddings alone miss.
Break the question down. Multi-part requests (“summarise the 2024 risk updates as a table with these columns”) are split into sub-questions, and each is checked for relevance before it runs.
Pick the tool per part. An agent routes each sub-question to retrieval or to a table formatter; a final check confirms the combined answer covers everything that was asked.
Keep the better answer. When a table is requested, the table and the plain answer are both scored for relevance; the better one is returned, or the table is refined from the retrieved text.
After
Big 4 client · fully on-premise
Stack
RAGOCROn-PremiseLLM Fine-TuningH100Why it holds up
In audit work, an answer you cannot cite is not an answer — it is a liability. That single constraint shaped the whole build: the ranking, the chunking strategy, the refusal behaviour when the library genuinely does not cover the question. A confident guess is worse than no answer at all when the output ends up in a working paper.
Related service: Hong Kong RAG solution
More case studies
200k calls/month · 67% booking rate vs 77% human
24/7 AI Agent Call Center: 200k Calls a Month Across 60 US HVAC Brands
Voice agents orchestrating 200,000 calls a month across 60 US HVAC brands for a 5,000-person enterprise, booking 67% of calls against 77% for human agents — within 10 points of a trained team, around the clock. Built as a product, an AI front desk that answers, books and routes, not a pilot.
- Night and weekend calls unanswered
- Staffing every shift hard to justify
US$1M collected in the first 3 months
AI Payment Collection and Reminders
The companion product to the AI front desk: it chases unpaid invoices with polite, timed reminders and a payment link, and stops the moment the customer pays or replies. It collected over US$1 million for the client in its first three months.
- Chasing payment is awkward and late
- Billable time lost to collections
Ask the EDI in plain English
Natural-Language EDI for a US EDI Solutions Vendor
Business users ask questions of raw X12 — “what are the net terms on invoice 4471?” — and get answers in plain English, from a Llama 3 model fine-tuned on EDI and grounded in the parsed transactions.
- Business and EDI teams speak different languages
- Rejected transactions decoded by hand
Get started
Talk to the engineer who will build it.
You will speak with me directly. Bring the use case, the data and the constraints, and I will give you a straight view on feasibility, effort and the first thing to ship.
