What your iPhone actually does when it scans a receipt

You snap a photo of a crumpled receipt. A second later, the app usually has the total, the merchant name, the date, and the currency. It feels like magic — but it’s not. It’s your phone doing real work, right on the device, with no upload in sight.

Here’s what actually happens between the tap and the result.

Cloud OCR vs on-device OCR

Many receipt scanner apps upload your photo to a server. A cloud service runs text recognition, extracts the data, and sends it back. That server sees every receipt — what you bought, where you were, when, and how much you spent. Your spending history lives on someone else’s infrastructure.

On-device OCR does the same processing but never sends the image anywhere. The photo stays on your phone. The text extraction runs on your phone. The parsed result stays on your phone. The server never enters the picture because there is no server.

This isn’t just a privacy difference. It also means the scan works without an internet connection — in a market overseas, on a flight, in a subway station. The receipt doesn’t need to wait for a round trip to a data center.

How Apple’s Vision framework works

Apple ships a text recognition engine called Vision as part of iOS. It runs on the iPhone’s Neural Engine — dedicated hardware designed for machine learning tasks. Here’s what happens when you point it at a receipt:

Text detection. The framework scans the image for regions that contain text. It identifies blocks — lines, paragraphs, clusters of characters — and maps their positions in the photo. It knows where the text is before it knows what the text says.

Character recognition. Each detected region gets processed through a neural network trained on printed and handwritten text. The model converts pixel patterns into actual characters. It handles different fonts, sizes, and orientations — including the slightly crooked angles you get from a quick phone snap.

Language correction. After the raw characters are extracted, the system applies a linguistic correction pass to fix recognition errors. If the neural network is 90% sure a character is an “O” but the surrounding word makes more sense with a “0”, it corrects. This step is what pushes accuracy from decent to usable.

Structured output. The result comes back as positioned text — each line tagged with its location in the image. An app can then walk through these lines and look for the patterns that matter: amounts, dates, currency symbols, store names.

All four steps run on the Neural Engine. No network call. No API key. No waiting for a response from somewhere else.

What gets extracted

Raw OCR gives you text. Turning that text into expense data is a second step — pattern matching and heuristics that identify specific fields.

Total amount. The scanner looks for number patterns near keywords like “total” or “amount due.” Receipts almost always put the total near the bottom, formatted with two decimal places. The system finds the last match — since subtotals and line items also look like totals, the final one is usually the actual total.

Merchant name. The store name is typically the first line of text on the receipt. It’s often in a larger font or all caps. The scanner grabs the top line as a starting point. Some apps also check against your past merchants — if the OCR text contains a name you’ve logged before, that match takes priority over guessing.

Currency. Currency codes like USD, EUR, PHP, or SGD appear on many receipts. The scanner checks for these three-letter ISO codes in the extracted text. Currency symbols ($, EUR sign, yen sign) are less reliable since several currencies share the same symbol.

Date. Receipts use wildly inconsistent date formats. DD/MM/YYYY, MM/DD/YYYY, “Apr 05, 2026”, or just “05-04-26.” The scanner runs multiple date patterns against the text and picks the most plausible match. When the format is ambiguous — is 04/05 April 5th or May 4th? — it can guess wrong. That’s one reason manual review still matters.

What it can’t reliably get

On-device OCR is good, but it has real limitations. Being upfront about them matters more than overselling.

Itemized line items. The OCR can read the text, but mapping which price belongs to which item — especially when columns don’t align perfectly — is fragile. Sometimes it works. Sometimes it scrambles the associations.

Tip amounts. Tips are often handwritten on printed receipts. Handwriting recognition is significantly less accurate than printed text. A scrawled “5.00” might read as “S.00” or not get detected at all.

Tax breakdowns. Receipts format tax differently — single line, multiple rates, or embedded in the total with no separate line. Reliably separating tax from the total across all formats is an unsolved problem for on-device processing.

The honest answer: on-device scanning reliably gets you the total, the merchant, the currency, and the date. Everything beyond that is best-effort. A well-designed app lets you review and correct before saving.

Why accuracy varies

Not all receipts are created equal. Several factors affect how well on-device OCR performs:

Print quality. Thermal receipts fade. A receipt that’s been in your wallet for two weeks may be too light for the neural network to read confidently. Fresh receipts scan better.

Receipt format. A clean POS receipt with consistent spacing scans well. A handwritten bill from a street vendor is a different challenge entirely.

Language and script. Vision supports many languages, but accuracy varies. Latin-script receipts tend to perform best. Other scripts — Chinese, Japanese, Thai — work but may need more correction.

Photo quality. Blur, shadows, and extreme angles hurt accuracy. A straight-on shot in decent lighting gives the Neural Engine the best input. You don’t need a perfect photo, but a readable one helps.

Layout complexity. A receipt with a single total at the bottom is easy. One with discounts, loyalty points, and service charges is harder — not because the text can’t be read, but because each number’s meaning depends on context.

What this means in practice

When you scan a receipt in an expense tracker that uses on-device OCR, your photo never leaves your phone. The processing happens on dedicated hardware built into your iPhone. The extracted data — amount, merchant, currency, date — gets saved locally.

Gastos, a private, local-first expense tracker for iPhone, uses Apple’s Vision framework for receipt scanning. The image stays on-device. The extracted fields go into a review screen where you can correct anything the scanner got wrong before saving. No cloud. No account. No upload.

The technology isn’t perfect — no OCR is. But it’s good enough for the core job: turning a photo into a logged expense in a few seconds, without your spending data touching a server.

---

Gastos is a local-first expense tracker for iPhone. Log expenses by text, receipt photo, or voice. On-device AI, Travel Mode, and everything stays on your phone.

Download on the App Store