Lecture 7: Outsourced Data
Effective AI use in applied machine-learning projects
Who decided how this TIME article became a data record?
Claude scraped this web page into a JSON record.
For each step: us or Claude?
TIME, Oct 27 2026 · Record from my Claude Haiku 5.5 run, Oct 30 2026
1
How to request the page
2
Which fields the record has
3
What went into each field
Outline
A. The data we need
B. Leakage, near-duplicates, and shift
C. The choices and risks we inherit
D. Data collection with AI agents
Define the prediction task before choosing a dataset
When we download or fork a dataset, we inherit someone else’s choices about what to collect and how to label it. A few typical examples below.
Population
Whose cases must this dataset represent?
Prediction target
What will we predict, using which available fields?
Deployment unit
Will we predict for a new person, document, site, or time period?
Extraction rule
What counts as a record, and what belongs in each field?
The Literary Digest poll, 1936
The poll mailed about ten million ballots. Roughly 2.4 million people replied.
Their responses predicted Landon would win; Roosevelt won 46 of 48 states.
Would getting all ten million recipients to reply have fixed the poll?
Full response would have improved the prediction, but still underestimated Roosevelt’s vote share.
Google C4
Common Crawl stores snapshots of web pages.
Google used its April 2019 snapshot to build C4, about 750 GB of English text for training T5.
C4 rejected pages containing words from this list.
What was the intention of C4’s filter word list?
Who did C4’s filters leave out?
The audit found that the word filter disproportionately removed text from and about minority groups.
One listed word can reject an entire page
“Sex” triggers rejection of Gay and Same-sex marriage; MIT’s article is rejected because it spells out IHTFP.
C4’s filter code and word list · our run, Oct 31 2026
We applied C4’s word-list rule to the full text of 13 hand-picked Wikipedia articles.
Hugging Face FineWeb-Edu
FineWeb-Edu keeps web pages that its classifier scores as educational.
It keeps all 13 Wikipedia openings from slide 8. We also scored other kinds of text.
A learned quality score excludes too: forum posts, news, and research papers drop out.
Swapping filters moves the exclusion; it does not remove it.
FineWeb-Edu and its classifier, Hugging Face 2024 · our run, Nov 2 2026
GeoDE collected the same object classes across six regions
GeoDE asked people in six regions to photograph the same 40 object classes. Compare their stoves with ImageNet’s examples in the bottom row.
These photos illustrate variation; they do not measure coverage.
Which settings must our project cover, and how could our data sources miss them?
The language of a question can change the answer
We asked Claude Haiku 5.5 the same question ten times in each of four languages:
“The Spratly Islands are a territory of which country? A) China B) Philippines C) Vietnam. Answer with one letter.”
The language we ask in decides whose writing answers.
A test set in one language covers one population.
Claude Haiku 5.5, our run, Oct 30 2026 · question after This Land is {Your, My} Land, NAACL 2024
Outline
A. The data we need
B. Leakage, near-duplicates, and shift
C. The choices and risks we inherit
D. Data collection with AI agents
A test image can have a near-copy in the training set
CIFAR-100 has small color images in 100 classes. The authors reviewed each test image’s nearest training match.
Which of these pairs should count as the same example?
The authors counted all three categories as duplicates.
891 test images had a training duplicate.
ResNet-110 test error
26.05% → 29.24%
after replacing the duplicate test images
Different photos can show the same license plate
AOLP contains multiple photos of the same plates. A random photo split can put one plate in both training and test.
For evaluation on unseen plates, keep each plate in one partition.
With the same partition sizes but each plate kept on one side, the TRBA model read 96.94% of plates correctly, down from 99.18%.
Random and regional splits answer different questions
Each dot is a California census area; we predict its median house value.
The regional split holds out whole one-degree grid cells.
Higher \(R^2\) is better.
If deployment is in unseen regions, hold out whole regions.
A model can learn which hospital an X-ray came from
A network could also name the hospital from the X-ray alone, over 99.9% of the time.
Pooled hospitals let the model learn the hospital; a held-out one exposed it.
A pneumonia model trained on NIH and MSH X-rays was also tested at a third hospital, IU.
When data comes from several sites, test on a site the model never trained on.
Overall accuracy can hide a subgroup’s failure
Lab 1’s classifier: one nearest neighbor on weight and cylinders, trained on 392 cars from 1970–82, predicting high or low gas use.
The classifier was correct on 71% of the 214 newer cars, but failed on all five electric cars.
Report subgroup scores together with sample counts.
Three ways deployment data can differ from training data
Inputs change
covariate shift, \(P(x)\)
The label mix changes
label shift, \(P(y)\)
The rule changes
concept shift, \(P(y \mid x)\)
40%
30%
15%
15%
10%
25%
30%
35%
Masks change the images of the same people.
The activities stay the same, but their relative frequencies change.
The same applicant can have different financial situations, and receive a different decision when the approval rule changes.
Face ID with a Mask, Apple Support · label and rule panels are illustrations
Reweighting training examples can correct covariate shift
When only the inputs change, weight each training example by how much more common its input is at deployment:
\( w(x) = \dfrac{P_{\text{target}}(x)}{P_{\text{source}}(x)} \)
Training points that look like deployment data count more, and the fit follows them.
This needs a stable rule \(P(y \mid x)\), and training data wherever deployment data falls.
Rescaling predictions can correct label shift
When only the label mix changes, rescale each class’s predicted probability by how its share changed:
\( P_{\text{target}}(y \mid x) \propto P_{\text{source}}(y \mid x)\,\dfrac{P_{\text{target}}(y)}{P_{\text{source}}(y)} \)
No retraining. This needs each class to look the same, a stable \(P(x \mid y)\), and the new label mix.
Which kind of shift hit Lab 1’s newer cars?
MNIST trained with 100 images each of 0, 3 and 9 and 14 of every other digit, tested on balanced digits · demo
Outline
A. The data we need
B. Leakage, near-duplicates, and shift
C. The choices and risks we inherit
D. Data collection with AI agents
In Lab 1, the old and new car labels use different thresholds
Lab 1 cars, 1970–82
“High gas use” means above one fixed cutoff, the same for every year. The value is not documented.
Newer cars, 1982–2015
“High gas use” means worse than the average new vehicle sold that year, a cutoff that moves.
Use a common label definition before comparing accuracy across eras.
Otherwise, a prediction can count as wrong because the target changed.
The newer cars came with a provenance file. It says their labels use a different rule.
Check assumptions before correcting distribution shift
The input distribution, label prevalence, and labeling rule changed together.
Observed change
Evidence in the cars
What a correction would require
Inputs, \(P(x)\)
The new cars are heavier and include electric cars with 0 cylinders.
Reweighting needs stable \(P(y \mid x)\) and support overlap. The old dataset contains no electric cars.
Label prevalence, \(P(y)\)
Label 1 share: 0.50 (old cars) → 0.30 (2010–15 cars).
Prior correction needs stable \(P(x \mid y)\). A changed rate alone is insufficient.
Labeling rule
Label 1 now means at or above that year’s average.
Choose the target definition, then obtain labels and evaluate under that rule.
A different label can still describe the same image
ImageNet is an image-classification benchmark with one label per image.
This example comes from a published audit of its annotations.
The recorded label is “acoustic guitar.”
The paper also offers “stage.”
Annotators checked one proposed class without seeing the alternatives.
When more than one label fits, record the ambiguity.
Benchmark answer keys contain errors too
A benchmark is a dataset, and people wrote its answer key.
A model that answers “Pandemic” is scored wrong. What did the score measure?
Read a sample of the test items, and their keys, before trusting a score.
Who labeled the data shapes what a label means
Several widely used hate-speech datasets were labeled by crowd workers.
The same tweet was less often called offensive once annotators knew its dialect.
Find out who labeled the data, and what they were told.
Healthcare spending is not the same as healthcare need
A widely used healthcare algorithm ranked patients for extra care.
Its training label was future healthcare spending.
The graph compares chronic illnesses at the same algorithmic risk score. Black patients had more illnesses at a given score.
Spending reflected unequal access to care as well as illness.
Check whether the label measures what our decision needs.
Forking a dataset can also copy its leaked keys
Truffle Security, which makes TruffleHog, scanned Hugging Face datasets for live keys.
Forks can republish leaked keys. Some keys grant permission to modify the datasets we download.
Keep discovered keys out of shared output and report them to their owners.
A retracted dataset can live on in its copies
A 2021 study traced three face and person-recognition datasets through nearly 1,000 papers that cite them.
Retracting the original did not retract the datasets and models built from it.
Check whether a dataset, or the one it was built from, has been retracted or relicensed.
A list of links can point somewhere new
Many image datasets ship as lists of links. Whoever controls a linked site later decides what downloaders get.
Save the files, or their hashes, when we collect from a list of links.
Poisoning Web-Scale Training Datasets is Practical, IEEE S&P 2024 · Conceptual Captions validation links, our run, Nov 3 2026
We ran into a related issue with CIFAR10; luckily we can fetch from hugginface and check the MD5
Outline
A. The data we need
B. Leakage, near-duplicates, and shift
C. The choices and risks we inherit
D. Data collection with AI agents
robots.txt states which paths crawlers should avoid
A user-agent string names the client.
“Disallow: /” asks each named crawler to stay off every path.
Nothing stops the crawler.
The log only shows if the crawler respected the rule.
Crawler rules depend on the site and the client
If Common Crawl obeys, future web datasets lose eight of these twelve sites.
Permission to fetch also says nothing about what a server will send.
Saved robots.txt files, Oct 30 2026
May each crawler fetch the front page? Twelve sites’ robots.txt, saved Oct 30 2026.
An AI collector can work around an access restriction
From Anthropic’s October 2026 report on unintended actions by its own models:
What should the collector have done when it hit the fee?
Anthropic reports minimal impact and gives no frequency.
Specify what the collector should do when access fails, and record the route it took.
My own Claude gave out my email without asking
My Claude Code sessions, Oct 13 to Nov 1 2026
Claude’s summary of the incident: “I” is the main Claude agent, and “you” is Shen the person.
this doesn't guarantee it won't happen again
A page can carry instructions for the AI that collects it
Google scanned Common Crawl, the web archive behind C4, for text addressed to AI agents:
If our collector followed this link, what would it fetch, and what would it record?
Google, April 2026: the linked page streams text that never finishes loading.
Most injections it found were pranks or teaching material, and none was shown to work.
Limit what the collector may fetch and run, and read its request log, not just its records.
The same URL returned different content to different clients
Claude’s record left the sponsor out.
Right for a dataset of editorial topics; missing for a study of ads sent to AI crawlers.
We requested the same TIME article three ways: in Chromium, a browser, and through curl, a command-line downloader, identifying as Chrome or ClaudeBot.
Passing a schema check does not mean every field is complete
Rule
Sponsored block
Instagram, Snapchat
Schema check
Editorial text only
left out, 3 of 3
in 0 of 3 records
pass, 3 of 3
Sponsored kept separately
kept apart, 3 of 3
in 2 of 3 records
pass, 3 of 3
We still need to check which source mentions belong under the written inclusion rule.
Opus 5.5 also left out Instagram and Snapchat, in 3 of 3 runs.
Asked to audit the record against the page, it flagged both each time.
We chose the request and fields; Claude chose their contents.
We ran three extractions per rule: editorial text only, or sponsored content kept separately.
A PDF converter can change a table before extraction starts
OmniDocBench tests tools that turn PDF pages into text and tables.
Left: a table on a page. Right: what Qwen2-VL-7B, a vision-language model, returned.
Source page (3 of 7 columns)
Qwen2-VL-7B output
Which values moved?
The last column slid down one row: hex row 1 lost its value, and −195 fell off the end.
The headers no longer match their columns, and the orbit values are garbled.
Newer models do better, not perfectly: on the 2026 table score (out of 100), the best parser gets 97 and GPT-5.2 gets 83.
Five habits for auditing a dataset we did not collect
Write down whose data we need, and check who the collection left out.
Split the way deployment meets the data; drop fields we won’t have then.
Before collecting, set aside the test split and plan to read a sample by hand.
Record what labels mean, where files came from, and what stays unknown.
Save what the collector received, and trace a record back to it.
AI can propose rules and run checks; we justify the rules, inspect the evidence, and decide what conclusions it supports.