Ledger

The problem

Shared household costs lived in a spreadsheet. Every month somebody opened four or five bills in four or five different formats, typed the numbers across, worked out who owed what, and hoped the totals still agreed. It worked, in the way that manual reconciliation always works right up until the month somebody is busy.

The bills do not arrive in one shape. Some are PDF statements, some are email bodies, some are order confirmations, some are card exports. None of them agree on where the amount sits, what the billing period is called, or whether the month on the page is the month the bill is for.

And the tolerance for error is low in a specific way. This is money between family members, so the tolerance for a wrong number is very low. Double-counting a bill because an import ran twice is the worst available outcome.

The engineering decision

Where should the messiness of the inputs be allowed to live?

Constraints

• Re-running an import must never change the result.
• A displayed total must always equal the rows above it.
• A source that changes its layout must fail visibly, not silently produce a wrong number.

Options considered

• Keep entering the numbers by hand Reliable while somebody is paying attention, which is exactly the assumption that fails. It is also the thing worth automating: the task is monthly, identical, and entirely mechanical.
• One general parser with heuristics for any bill Attractive until the first ambiguous document. A parser that guesses is a parser that will quietly assign an amount to the wrong month, and nothing downstream would know.
• One small parser per source, all returning the same record Chosen. Each knows exactly one format and nothing else, and every one of them emits the same normalised shape.

Decision

The messiness is confined to the parser layer and never escapes it. Each parser takes raw text and returns one record shape: period, category, amount and note. So everything downstream deals in one shape regardless of where the number came from.

Implementation

Around 1,400 lines across five parser modules, each carrying its source’s period convention explicitly. Ingestion polls a dedicated mailbox and de-duplicates on message id, so running it twice is a no-op. Ownership is resolved per line item and independently of category, with defaults that apply automatically to recurring bills that always belong to one person.

Result

Bills file themselves. Every total on a page is the sum of the rows shown above it, so a rounded line can never disagree with the total it belongs to.

Architecture

1. PDF statementsForwarded emailsOrder confirmationsCard exports
2. Per-source parsers
3. Normalised records · period, category, amount, note
4. Split + budget rules
5. Expenses · Money · Cards

A fan-in. Four kinds of input, four parsers that know one format each, and a single normalised record from there on, so the split and budget rules never have to know where a number came from.

The month a bill is for is not the month it arrives

The subtlest bug in this whole system is a calendar problem. A statement dated the 22nd of October, due the 1st of November, is the November bill. A phone bill names its own month in the body text. An internet bill uses its bill date. Three sources, three different answers to “which month is this?”, and no way to infer it from the amount.

Get it wrong and nothing crashes. The number is right, the category is right, and it lands one month out. That shows up later as a month that looks unusually cheap next to one that looks unusually expensive, long after anybody would connect it to an import. So each parser carries its source’s period convention as an explicit, commented rule rather than inferring one.

The same instinct drives de-duplication. Ingestion keys on message id rather than on amount and date, because two identical amounts in one month is a real thing that happens and must not be silently collapsed into one.

Evidence

• 1,400 lines of parsers five modules, one format each
• 0 double-counted imports de-duplicated on message id
• 2 processes web and ingest worker, separately restartable

Running the ingest worker as its own container is the cheapest reliability win here: a slow mailbox, a malformed PDF or a parser stuck on an unexpected layout cannot block the page somebody is trying to read.

What I would improve

• The parsers are regular expressions over extracted text. That is the right amount of machinery for four known formats and the wrong amount for forty, and a vendor redesigning a statement will break one.
• Worse, that break is quiet. A parser that stops matching produces nothing rather than an error, so the failure looks like a bill that never arrived. It needs an expected-bills check that notices absence, which is a different thing from validating presence.
• Amounts are stored as numbers rather than as integer minor units. Nothing has gone wrong yet and it is the sort of thing that goes wrong exactly once.
• Everything is one SQLite file on one machine, backed up but not replicated.