July 7, 2025/707 words/4 min read

Digitizing Old Notebooks

Building a pipeline to rescue old notebooks from slow decomposition.

Note There are no images in this post. The scans are personal, so this is just about the process.

Every time I visit my parents I end up in the attic, like a squirrel after a long winter. Opening dusty boxes and hoping I find something interesting is a habit at this point. This time I found nine old notebooks from around 2001 to 2003, buried under a pile of toys I do not remember ever playing with. Back then I spent most of my free time writing down game concepts, sketching fantasy maps, and trying out whatever idea happened to be in my head. I didn't remember most of what was in them, but I recognised it all instantly as mine.

Inside were really stupid RPG systems with more rules than any actual game could survive, fantasy maps covered in symbols I invented on the spot, and little diagrams for machines that ignored logic. It's the kind of stuff a kid makes when they have more imagination than skill. It's a little embarrassing.

The handwriting is also part of the story. I'm very left-handed, but the Eastern European school system I grew up in insisted I write with my right. That pretty much never took. The muscle memory never formed, so my handwriting is cramped and uneven and every letter looks slightly off. I stopped writing by hand as soon as I could and never went back, so flipping through page after page of it felt like stumbling into a part of my childhood I usually skip.

The notebooks themselves are in rough shape, with yellowed pages, cracked edges, weird stains, and the general wear of over twenty years in an attic. Seven-year-old me cared deeply about drawing worlds, not so much about preserving them. If I left the notebooks alone, they wouldn't survive much longer, so I decided to digitise everything and build a small pipeline to handle it.

I started with my phone, but that lasted three pages. There were shadows everywhere, warped angles, and terrible lighting. Doing nine notebooks like that would have driven me mad. I ended up borrowing a document camera from my library, mounted it above my desk, set up two cheap LED panels at 45 degrees, and put a dark desk mat underneath for contrast. Much better!

For each page I saved an archival JPEG at full resolution and a PNG copy for processing. The capture script handles the boring parts.

#!/bin/bash
set -e
 
NOTEBOOK_ID="$1"
PAGE="$2"
RAW_DIR="./raw/${NOTEBOOK_ID}"
mkdir -p "$RAW_DIR"
 
# Capture from the document camera (outputs JPEG)
gphoto2 --quiet --capture-image-and-download \
  --filename="${RAW_DIR}/page_${PAGE}_archival.jpg"
 
# Convert a copy to PNG for processing
convert "${RAW_DIR}/page_${PAGE}_archival.jpg" \
  "${RAW_DIR}/page_${PAGE}_process.png"
 
# Clean up the processing version
convert "${RAW_DIR}/page_${PAGE}_process.png" \
  -fuzz 12% -trim +repage \
  -deskew 85% \
  "${RAW_DIR}/page_${PAGE}_clean.png"

After capturing each page, I tag it in a small Electron viewer I threw together because that was faster than searching for a better tool. It lets me assign categories, estimated dates, keywords, and a quick quality note. The metadata looks like this.

{
  "notebook_id": "nb_03",
  "page": 47,
  "year_written": "2003",
  "categories": ["game_design", "rpg", "combat"],
  "keywords": ["turns", "mana", "classes"],
  "condition": "water_damage",
  "notes": "Illegible handwriting with several unclear symbols"
}

OCR failed immediately. Tesseract handled the printed labels just fine, but the handwriting might as well have been ancient runes. So I split the task. Google Vision does the printed text (it's excellent at that), and Claude handles the handwriting, which it's weirdly good at. The transcription script feeds in the page and metadata and asks for both text and diagram descriptions.

import anthropic
import base64
from pathlib import Path
 
def encode_image(path):
    data = Path(path).read_bytes()
    return base64.b64encode(data).decode("utf-8")
 
def transcribe(image_path, meta):
    client = anthropic.Anthropic()
    encoded = encode_image(image_path)
 
    prompt = (
        "This is a scanned notebook page from around "
        f"{meta.get('year_written', 'unknown')}. "
        "The handwriting is extremely difficult to read. "
        "Transcribe everything visible and mark unclear text with [?]. "
        "Also describe drawings and diagrams in detail."
    )
 
    msg = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=4096,
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "image",
                     "source": {"type": "base64", "media_type": "image/png", "data": encoded}},
                    {"type": "text", "text": prompt}
                ]
            }
        ]
    )
 
    return msg.content[0].text

The results are hilarious. Claude recognises my messy scribbles way better than I can!

I store everything in SQLite with full-text search.

CREATE TABLE pages (
    id INTEGER PRIMARY KEY,
    notebook_id TEXT,
    page INTEGER,
    date_written TEXT,
    transcription TEXT,
    diagram_description TEXT,
    image_path TEXT,
    metadata JSON
);
 
CREATE VIRTUAL TABLE pages_fts USING fts5(
    transcription,
    diagram_description,
    content='pages',
    content_rowid='id'
);

Now I can search for things like all pages about elemental magic or all maps with mountain ranges. It's instant and strangely satisfying to use.

These notebooks aren't valuable to anyone else. They're messy and half-formed, but they're a clear record of who I was before I knew what I was doing. I was a kid trying to build things I didn't have the skills for yet, who thought putting a dungeon inside a giant crab made perfect sense. Now I can read and search them without handling pages that are already falling apart.

Six notebooks done, three left at the time of writing. The pipeline works well for something I threw together over a few evenings, and I'm curious to see what else shows up in the remaining pages. So far it has cost me a few dollars in API calls and some late nights, which feels like a good deal.

Update, July 15. All nine notebooks finished. About ten EUR and maybe twenty hours of work total. Absolutely worth it!