Code

GPT OCR followup#148

Code

Merged

danvk merged 16 commits intomasterdanvk/oldnyc:masterfrom gpt-ocr-followupdanvk/oldnyc:gpt-ocr-followupCopy head branch name to clipboardOct 27, 2024

+32,898-63,970

Commits

Commits on Oct 27, 2024

• clean GPT OCR in ingest.py

  danvk

  committed
  8aaa661

  Copy full SHA for 8aaa661

• split interior whitespace

  danvk

  committed
  ff3d3d5

  Copy full SHA for ff3d3d5

• normalize more interior whitespace

  danvk

  committed
  7eeac9a

  Copy full SHA for 7eeac9a

• pipeline-style

  danvk

  committed
  a3df2de

  Copy full SHA for a3df2de

• try being much more lax about is_negative

  danvk

  committed
  6eba892

  Copy full SHA for 6eba892

• 2/426 false positives

  danvk

  committed
  dddda1a

  Copy full SHA for dddda1a

• this is pretty good

  danvk

  committed
  77ea822

  Copy full SHA for 77ea822

• images.ndjson diff script

  danvk

  committed
  596de3f

  Copy full SHA for 596de3f

• split up (1), (2), (3) etc on single one; 180 matches, 2 false positives

  danvk

  committed
  9857cd8

  Copy full SHA for 9857cd8

• remove F. S. LINCOLN stamps

  danvk

  committed
  2919b9c

  Copy full SHA for 2919b9c

• strip NEW YORK PUBLIC LIBRARY stamp (108)

  danvk

  committed
  4bb8815

  Copy full SHA for 4bb8815

• Drop "President Borough of Manhattan" (862!)

  danvk

  committed
  16ed45c

  Copy full SHA for 16ed45c

• update keep-ids with new cleaner

  danvk

  committed
  2d48b06

  Copy full SHA for 2d48b06

• add another manual list

  danvk

  committed
  53feee9

  Copy full SHA for 53feee9

• run cleaner on old site OCR, too

  danvk

  committed
  318a8bf

  Copy full SHA for 318a8bf

• squash excessive newlines

  danvk

  committed
  bc211db

  Copy full SHA for bc211db