GLM 5.2 is nearly as accurate as a human book keeper
An AI-powered bookkeeping system benchmarked on UK VAT returns is reported to be nearly as accurate as a careful human bookkeeper, at a fraction of the cost. Commenters explore where this could be genuinely useful—automating routine data extraction and categorizations for small businesses—while raising concerns about liability, fraud, non‑deterministic errors, and the fact that tax authorities ultimately hold business owners, not software vendors, responsible. Many see a near‑term role for AI as an assistive tool with strong controls and human review rather than a full replacement for professional accountants.
Benchmark and “Human-Level” Metric
- Benchmark compares GLM 5.2 to human bookkeepers on UK VAT preparation.
- Some argue it’s a relevant metric for an AI bookkeeping service; others say “as accurate as a human bookkeeper” is weak because humans are error-prone.
- The human in the test was unusually careful, knowing it was a benchmark, so “typical” humans might perform worse.
- The model’s VAT result was off by only pennies, but made several classification/handling errors highlighted in the write-up.
Accuracy vs Liability and Risk
- Major theme: even if AI matches or beats human accuracy, the legal liability still falls on the business owner.
- Some stress that accountants are regulated and can bear professional responsibility or insurance-backed liability; an AI vendor’s ToS explicitly disclaims such responsibility.
- Others counter that, in practice, tax authorities usually pursue the filer regardless of whether a human accountant was used.
Real-World Tax Enforcement and Materiality
- Multiple commenters say tax agencies (IRS, EU VAT authorities) tolerate small errors and often correct returns rather than punish, especially for small businesses.
- Concept of “materiality” is raised: tiny discrepancies are often ignored.
- Others insist accuracy still matters because serious errors or fraud can be very costly or, in extreme cases, criminal.
Scope and Limits of Automation
- Many see bookkeeping as highly automatable: constrained domain, structured data, clear outputs.
- Common pattern suggested: AI prepares books; a human reviews, especially for edge cases and judgment calls.
- Thread emphasizes that humans also miss things and that multiple layers of review (human + AI) may be better than one.
Data Access, Retrieval, and Hallucinations
- Practical bottleneck: getting high-quality, structured inputs from banks, credit cards, email receipts.
- Some describe successful custom systems (email parsing, bank integrations) with LLMs plus scripts.
- Others are wary of letting LLMs “fetch invoices” or generate line items because hallucinated transactions have been observed in finance apps.
Non-Determinism, Judgment, and Accounting Complexity
- Accounting often involves estimates, gray areas, and interpretation; not purely deterministic.
- Commenters note that large firms already show internal disagreement among senior accountants and auditors.
- Concern: LLMs may sometimes “cheat” to satisfy goals, potentially fabricating numbers or masking mismatches.
Fraud, Controls, and Social Engineering
- Strong concern about AI-driven accounts payable being exploited via “social engineering the LLM” (fraudulent invoices, spoofed instructions).
- Proposed mitigations: traditional controls (purchase orders, approvals, segregation of duties), hard guardrails, and limiting AI’s authority to initiate payments.
- Some argue AI could actually enforce controls more consistently than humans, if designed with strict rules.
Use Cases and Attitudes
- Solo practitioners and small businesses report good experiences using LLMs to assist with bookkeeping and tax prep, with manual review.
- For larger or more complex organizations, many call full AI outsourcing “insane” but see value in automating rote tasks.
- Overall sentiment: promising and improving, but human oversight and clear accountability remain essential.