Interesting write-up. Having been a bookkeeper a long time ago, I'm not too surprised at this being susceptible to automation by an LLM backed system.
It seems also that the classes of error they encountered could be handled by improved skills/knowledge base access on the fine points of relevant tax legislation.
The important part for their software ofc is, will they take responsibility for the output if HMRC come calling? Without that users are adopting the risk which they may not be keen to do (dealing with HMRC is not fun), with that it could be a very nice saving for a lot of small companies (and bad for the employees of a lot of accountancy firms)
I'd be scared shitless to even try something like this. There is just a pretty website, a video, and a blog post. No info on the founders, I can't find anything on LinkedIn and a company Vineyard Finance LTD that was incorporated last year.
We're all unhinged about the data we're giving LLMs but here I'd draw the line. I'd rather keep paying for the small amount I pay to have my accounts done.
The company I work for, Digits, has been regularly updating our AI-vs-human bookkeeper benchmark. Look at page 8 -- many models are nearly as accurate as a human bookkeeper
Parses emails or other sources, extracts numbers, correlates different transactions, web search, asks questions, stores notes (regex based, very simple).
The hard part is getting good data, I'm sure that lexus nexus or whoever can get API access to my bank account and all my credit cards, but I can't. Email turned out to be the best way for most of my providers. Managed to avoid 2factor auth so far, but it will suck when I need it.
I was one of the human book-keepers for this benchmark (the preparer; my co-founder verified the VAT submission once ready), and given that at the time of doing this I knew I was eventually going to use this data for evaluating the models, I was super careful. So I guess this is a "good book-keeper". In the previous company our book-keepers made lots of mistakes; some serious enough that we had to restate our company's accounts.
Then why only have one human bookkeeper? Surely two would be better, since you can compare their results. But then, perhaps you should hire three, so you can figure out which one is right.
Oh, I'm actively doing this at the moment. FreeAgent grabs my transactions from Wise already, and then I give it [Claude Code, in fact] a folder of PDFs to attach to my invoices, including figuring out VAT, and it's uploading what it found using the FreeAgent API. My accountant hasn't complained yet, and it seems considerably more accurate than when my wife was doing it.
It's not hard to imagine that will be able to do as good a job as a human accountant in the not too distant future.
It's also not hard to imagine tax authorities using AI to audit everyone's tax returns every year.
We sure live in interesting times.
Interesting write-up. Having been a bookkeeper a long time ago, I'm not too surprised at this being susceptible to automation by an LLM backed system.
It seems also that the classes of error they encountered could be handled by improved skills/knowledge base access on the fine points of relevant tax legislation.
The important part for their software ofc is, will they take responsibility for the output if HMRC come calling? Without that users are adopting the risk which they may not be keen to do (dealing with HMRC is not fun), with that it could be a very nice saving for a lot of small companies (and bad for the employees of a lot of accountancy firms)
I'd be scared shitless to even try something like this. There is just a pretty website, a video, and a blog post. No info on the founders, I can't find anything on LinkedIn and a company Vineyard Finance LTD that was incorporated last year.
We're all unhinged about the data we're giving LLMs but here I'd draw the line. I'd rather keep paying for the small amount I pay to have my accounts done.
Info on the founders coming soon -- we're just going public with this.
For slightly out of date founder bios (both Adam and Iva) were also co-founders here:
https://www.biomage.net/our-team
The company I work for, Digits, has been regularly updating our AI-vs-human bookkeeper benchmark. Look at page 8 -- many models are nearly as accurate as a human bookkeeper
https://digits.com/downloads/beyond-the-hype-evaluating-llms...
This doesn't surprise me at all. You can really constrain this problem, give very narrow context, and get pretty reliable and reproducible results.
I've gotten very good results with some vibe-coded deepseek book keeping. https://github.com/traverseda/beansync
Parses emails or other sources, extracts numbers, correlates different transactions, web search, asks questions, stores notes (regex based, very simple).
The hard part is getting good data, I'm sure that lexus nexus or whoever can get API access to my bank account and all my credit cards, but I can't. Email turned out to be the best way for most of my providers. Managed to avoid 2factor auth so far, but it will suck when I need it.
> nearly as accurate as a human book keeper
Anything to avoid using the metric system.
Though seriously, what is this metric? Why would I care if an LLM is accurate as a human bookkeeper? Humans aren't exactly known for perfect recall.
Hey, author of the blog post here.
I was one of the human book-keepers for this benchmark (the preparer; my co-founder verified the VAT submission once ready), and given that at the time of doing this I knew I was eventually going to use this data for evaluating the models, I was super careful. So I guess this is a "good book-keeper". In the previous company our book-keepers made lots of mistakes; some serious enough that we had to restate our company's accounts.
It’s a service that provides an AI bookkeeper so it’s a pretty relevant metric.
You'd care if you had a human bookkeeper and you were considering replacing them with this company's AI bookkeeper.
Why would I want both a less accurate book keeper and to incur all of the liability of doing the books myself?!
It's not less accurate. As commented above, the control knew they were being tested against the machine, so made sure to be super careful.
In everyday life the human is less careful, and the machine costs 1% of the human.
Presumably your current book keeper is not your slave, and you have to pay them...
...again: liability is the key issue. Cost savings a not exactly an isolated issue here.
Then why only have one human bookkeeper? Surely two would be better, since you can compare their results. But then, perhaps you should hire three, so you can figure out which one is right.
Oh, I'm actively doing this at the moment. FreeAgent grabs my transactions from Wise already, and then I give it [Claude Code, in fact] a folder of PDFs to attach to my invoices, including figuring out VAT, and it's uploading what it found using the FreeAgent API. My accountant hasn't complained yet, and it seems considerably more accurate than when my wife was doing it.
Quiet plug for https://github.com/pjlsergeant/byre which I use for all my little projects like this.
Very cool!
We've used the following CLI to do the freeagent upload:
https://github.com/anjor/freeagent-cli
How are you dealing with finding the receipts? Would you like to try a receipt finder that grabs them from your mailbox/ google drive?
> They've done studies, you know. Sixty percent of the time, it works every time.