How do we know the AI output is accurate? What observable evidence is there? From the article:

> Inside this huge machine, which is called a synchrotron, electrons are accelerated to almost the speed of light to produce a powerful X-ray beam that can probe the scroll without damaging it. ...

> The scan is used to create a 3D reconstruction, then the layers inside the scroll - it contains about 10m of papyrus - have to be identified. ...

> After that artificial intelligence is used to detect the ink. It's easier said than done - both the papyrus and ink are made from carbon and they're almost indistinguishable from each other.

> So the AI hunts for the tiniest signals that ink might be there, then this ink is painted on digitally, bringing the letters to light.

The ML ink-detection models aren't spitting out Greek text. They're just predicting ink locations, which can be calibrated and cross-checked by manual inspection with the original. (E.g., an earlier article showed how ink particles showed up as a shift in texture.) They operate on a lower level than letters and words, so if the ink does correspond to Greek letters that come out to recognizable Greek words forming sensible passages, it's good evidence that the output is correct.

Presumably, it's possible for errors to slip through, but human labelers can similarly make errors.

I think this is where most of the ML/AI practitioners need to focus on.

Instead of obsessing over other accuracy technique for fully autonomous detection they should focus on human assisted detection to improve the quality/value of the solution. In this case recall/sensitivity is the utmost important accuracy metric and try to get it closer to 100% if possible. Hence even if the ML/AI got it wrong (due to false positive detection), at least the human experts can have a look at it and hopefully eliminate any false positive. It also considerably reduce the burden of the upstream manual based inspection/automation since it is machine automated. This can also mitigate and hopefully prevent false negative detection because false negative detection will not has the opportunity for further inspection/verification by the human experts or trained inspectors. Essentially the AI/ML is functioning as the filter shifting through the massive data very quickly (with minimum or zero false negative), and then the results are then verified by the much slower human experts.

Very clever

> Presumably, it's possible for errors to slip through, but human labelers can similarly make errors.

That's often an argument for AI systems: Is it better than humans (than avoiding car accidents, reading text, reading x-rays, etc.). But in science we need observable evidence.

> But in science we need observable evidence.

The observable evidence here is the raw x-ray data. Based on the article, what is being done here was using ML to process that X-ray data, to create another set of output that the humans could analyze.

I understand jumping on the anti-AI bandwagon, but this ain't ChatGPT. Using ML to process complex instrument output has been used since at least the 70s. This is much closer to statistics based approaches than it is to some random startup trying to sell your manager on some snake oil AI-assisted whatever.

I don't know your background, so I'm sorry if I'm assuming anything. Any chance you articulate what specifically you don't like about the method they used to process the data, other than "It's ML?"

We have plenty of observable evidence that the output of these detection models correponds to real ink residue, at least on one of the scrolls where the distinct texture is most obvious [0]. The models definitely aren't making things up out of pure noise.

And if they did, then it almost certainly wouldn't come out as identifiable Greek letters, since these models are strictly looking at local patches of texture to identify the presence or absence of ink. Again, these aren't just black boxes spitting out long passages of Greek text, everything past the initial ink detection is still done by hand.

[0] https://scrollprize.org/firstletters#caseys-crackle-pattern

Things like OCR or detection ML models don’t generally have the same drawbacks / hallucinations as LLMs for example. They’re not 100% accurate but they also won’t return made up results. Detecting a pattern is very different from any kind of “reasoning”

Two things:

1. OCR absolutely does "hallucinate", and the word better suits that domain of OCR than the domain of LLMs as OCR will sometimes "see" text that doesn't actually exist.

2. This isn't OCR, it's more like contrast enhancement on a CT scan. Looking at a the project website, there's other work to figure out the shape of the paper surface because that's not obvious or simple either.

The output of this model is what might be given to an OCR, but given how noisy the picture in the article is, I suspect not.

It's doing a search operation, which AI is fantastic at. The operator can then verify the output.

> How do we know the AI output is accurate?

From the blog post where they announced 2023 grand prize: they did a couple of things to verify the results, including scanning the same area multiple times and making sure that multiple models produce similar results.

https://scrollprize.org/grandprize#how-accurate-are-these-pi...

They also created their own scroll, cooked it, and scanned it to create a ground truth dataset.

Recent and related:

News from Scroll 5 - https://news.ycombinator.com/item?id=42955356 - Feb 2025 (3 comments)

Additional backstory:

First word discovered in unopened Herculaneum scroll by CS student - https://news.ycombinator.com/item?id=37857417 - Oct 2023 (210 comments)

Vesuvius Challenge 2023 Grand Prize awarded: we can read the first scroll - https://news.ycombinator.com/item?id=39261861 - Feb 2024 (216 comments)

That CS student was Luke Farritor -- now part of the infamous DOGE team.

I get the ideologues here being mad… but I don’t get mad to the point where people need to pretend this guy isn’t highly intelligent. Doesn’t it make it better that there are really sharp people on the “DOGE” team.

Shouldn’t people be more mad at dummies doing it?

It's good that smart people are involved if we take it at face value that they're trying increase "government efficiency," as opposed to, say, dismantling the welfare and regulatory state; if the latter are their aim, and if we don't want them to accomplish such a goal, we should not be cheered to learn that they're such clever lads.

> dismantling the welfare and regulatory state

Pretty sure that's exactly what the US voted for in the last election, considering that was pretty explicitly promised. I think most of the US is cheering them on as they do it.

That's what 49.8% of the voters voted for. So not really "most" for any reasonable meaning of the term "most".

People don’t elect presidents, states do, and states elected Trump with a 312 to 226 electoral vote margin — so the states overwhelmingly voted for Trump. The purpose of the United States federal government is very strictly spelled out — and the 10th amendment makes it very clear that powers not expressly mentioned in the Constitution are reserved for the states. The U.S. is a republic, not a democracy.

Trump won by a very small margin, 1.5%, and didn't even get 50% of the popular vote. It isn't a "landslide" or "mandate" no matter how many times they claim it. Yes, Trump won, but that isn't grounds for ignoring the constitution. There is a separation of powers and Trump can't delegate powers which aren't his to Musk to delegate to technoboys.

Unfortunately the only leverage Congress has to stop Executive overreach is impeachment.

If they refuse to impeach, then they are tacitly handing the Executive ever more power, essentially moving us from a republic to an elected dictatorship.

And given that the GOP has a majority, no matter how slim, it seems very unlikely they will impeach.

The executive branch systematically dismantling itself doesn't sound like "ever more power".

312 to 226 isn’t a “small margin.”

You don’t measure a baseball game by number of hits or strike outs, but by the number of runs scored. The popular vote is literally irrelevant.

Who is saying he isn’t intelligent? And can’t smart people do bad things?

What’s bad about finding waste? Many appropriations are expired yet still keep getting paid out. That’s crazy.

The liberal media echo chamber is preventing people from using common sense. When media and congressional democrats literally call us “Nazis” — it’s hard to take any of it seriously.

“Trump is a fascist”

If that’s true, he’s the first fascist in history to attempt to make the government smaller.

Apparently being interested in free speech and smaller government and reducing waste, fraud, and abuse is fascism.

The people screaming loudest about USAID for example huge hypocrites, or have a very short memory: https://foreignaffairs.house.gov/press-release/icymi-washing...

https://x.com/tracking_doge/status/1888032469699498379?s=46

Are things you disagree with bad by definition or assumption? Or because you were told… because unless you have a front row seat, everything you know is because someone else already parsed that info for you.

> unless you have a front row seat, everything you know is because someone else already parsed that info for you.

Taking this to its logical conclusion, it is pointless for us to discuss anything we're not actively doing.

In the future, perhaps we may see "first glimpse inside burnt floppy disk after 2k years"?

"Electron microscope image of partially-melted optical disc"

They said that the work was a Greek Epicurean work, but described it as finding fulfillment in the pleasures of life. The Greek Epicureans were of the opinion that avoiding pain and suffering was the object of ethical philosophy, which is not the same thing, at all.

> Epicureanism ... declares pleasure to be its sole intrinsic goal.

https://en.wikipedia.org/wiki/Epicureanism

What the OP actually says is 'fulfilment can be found through the pleasure of everyday things' which is very much in line with Epicurean thinking.

This is largely all new to me, but this section defining pleasure within your link does seem to align with what GP is saying:

https://en.wikipedia.org/wiki/Epicureanism#Pleasure

> Epicureans had a very specific understanding of what the greatest pleasure was, and the focus of their ethics was on the avoidance of pain rather than seeking out pleasure.

So whether the description of the work (as GP critiques) is correct, really comes down to whether the definition of 'pleasure' used is as in Epicureanism. Certainly someone unfamiliar would misunderstand it.

Seriously. Working overtime…

What are you talking about?

The BBC doesn’t get any USAID funding. The BBC News website is supported by UK television license payers, and for non-UK folks it has banner ads which help pay for hosting and other costs.

There's a separate non-profit, BBC Media Action (formerly the BBC World Service Trust) with a completely different remit to the BBC that receives some USAID funding, but is mainly funded by the UK government.

You may, or may not be asking in jest. In any case, I think you are alluding to the fact that Luke Farritor, winner of the Vesuvius challenge in 2023 and former SpaceX intern, is now employed by Elon Musk in the Department of Government Efficiency, ostensibly to find waste, fraud, and abuse in government spending.

https://www.youtube.com/watch?v=zpflT8XuSyg&t=11s

well, that's cursed knowledge

It's likely that the talents needed to solve a technical challenge such as finding text in CT scans of heavily burned pages using machine learning doesn't mean he will have the same level of success when dealing with insights about socioeconomic issues in both the local and global scale and all the politics that such endeavor needs, specially when there is a heavy bias to an existing decision, the decision to reduce funds (meaning, even if he concludes that there is one particular endeavor where the government should spend more, it would be frowned upon by his boss and peers)

Just the cutting of everything USAID shows how little he understands of politics, the soft power that such organization earns for the country cannot be overstated, but in the business context Musk knows best there is really not equivalent, so he thinks such spending counts as "inefficiency", turns out countries are not companies and Musk and friends are in for a rude awakening and the country is the one who will suffer because of it as it increasingly alienates it's allies.

Without comment on the guy himself — I assume he's enthusiastically trying to do the best for the world, like most people — there's a huge gap between "let's find and stop very obvious fraud" and "let's cancel all soft power spending everywhere immediately because I don't know what value it has and don't want to spend money for the time it takes to find out".

The former is police work.

The latter is a practical demonstration of Chesterton's Fence, which is the "conservative" part of the word "conservative".

You say that like 6 year olds regularly use papyrus.

Despite that it was once common, very few people today have any more reason to know about it than they have to know about silphium or electrum.

https://en.wikipedia.org/wiki/Silphium

https://en.wikipedia.org/wiki/Electrum

College kids can be scientists. Anyone doing science is a scientist.

There are even middle-school kids doing science. Or random adults with non-STEM jobs collecting data as a hobby for Citizen Science.

We should be careful not to gatekeep the words science or scientist. The more mad scientists we have throwing spaghetti on the wall to see what sticks, the more things we can discover.

I absolutely agree, and the (possibly badly said) intention was to highlight that credit is not being given where credit is due. "scientists" being a way to obscure the actual people who pulled this off.

Yup as long as it’s done in a scientific way and published under peer review doesn’t matter who did it - that’s the great thing about blind publication review in a lot of fields

Exactly. Science is a process not an ideology.

The difference between screwing around and science is writing things down. - Abraham Lincoln (probably)

You’re gonna be shocked to learn how many research papers the “primary author” is the faculty advisor, that the second author is a grad student who wrote the entire paper, and the professor only observed/inspired the project and proofread the paper.

I'm pretty sure Luke has a lot more things to worry about right now than being interviewed by the BBC, lol.