A loose thought: when will human experts stop being the gold standard for ground truthing niche classification problems? That's already shifted in somewhat in domains like radiology but even there there's issues with classifying in the wild of lots of hospital with messy sensors. Is this something that will generalize in the coming years with AI?

from Pioneering Spirit (Blyg) ↗ · snapshot of v1

This has happened quite a few times. LLM's capabilities are quite jagged so they'll be superhuman in some areas and not in plenty others. Jury is still out where though.

• 1610s — Astronomy: Telescopes replace naked-eye observation for distinguishing celestial objects.

• 1670s — Microbiology: Microscopes reveal and enable classification of previously invisible organisms.

• 1860s — Chemistry: Spectroscopy identifies elements by their characteristic light signatures.

• 1880s — Infectious disease: Microscopy and bacterial cultures replace symptom-based identification of pathogens.

• 1900s — Blood typing: Laboratory tests establish blood compatibility rather than clinical inference.

• 1910s — Mineralogy: X-ray diffraction identifies crystal structures more reliably than visual inspection.

• 1950s — Hematology: Automated cell counters replace manual microscopic blood-cell counting.

• 1970s — Pregnancy testing: Hormone-based tests replace clinical assessment of pregnancy symptoms.

• 1980s — Forensics: DNA profiling supplants expert interpretation of physical resemblance and less discriminating biological evidence for identification.

• 2010s — Clinical microbiology: Mass spectrometry (MALDI-TOF) automates bacterial species identification, replacing much expert biochemical classification.