How Accurate Is AI at Identifying Rocks?

In July 2026 we sent 76 specimens with known correct answers through anyrockidentifier.com itself — the live tool, not a separate test copy — and scored what came back. Thirty-five were our original blind-test specimens. The other 41 were picked on purpose to be hard: quartz against calcite against selenite, jasper against agate, moonstone against opal against labradorite, granite against diorite against gabbro.

Sorting the results by the confidence score the tool showed:

The pattern is the point. The confidence number is honest — high scores really were reliable, low scores really were shaky. Every wrong answer that still looked confident sat between 78 and 82: a hematite read as magnetite, a green fluorite read as emerald, a variscite read as malachite.

That mattered because our "Confident" badge used to start at 75, so it was being given to a band that was only about 81% right. We raised the badge to 85 and above. Fewer results get called Confident now, and the ones that do have earned it. We also replaced four photos in our own field guide, including quartz and granite, after the test showed they were unrepresentative enough to fool the tool.

One honest limit: 76 specimens is a real test, not a lab study, and 22 out of 22 from 22 tries is a clean result rather than a promise. The model also samples, so runs vary — we ran the same 35 specimens twice an hour apart and got slightly different mistakes each time. Read the bands as the signal and any single result as an anecdote.

Our first write-up, in June 2026, came from a blind test of 35 specimens. It reported 82.9% strictly correct on the exact mineral or rock name, rising to 88.6% when we allowed the right mineral family or an accepted synonym — calling massive purple quartz "amethyst" rather than "quartz," for example. On common everyday specimens it scored about 91.7%, which we summarised as roughly nine in ten. We also reported that it never made a confidently wrong call.

Both of those claims are now withdrawn, for two separate reasons.

First, that June run went through a separate test harness rather than the live site, so it never measured what a visitor actually gets. Its numbers are not comparable to the July ones and we no longer quote them as current.

Second, the harder July set broke both claims outright. Confidently wrong calls did happen once look-alikes were in the mix. And on the common, everyday specimens — the ones "nine in ten" was about — the re-run scored 9 out of 12, which does not support the claim. So we stopped making it.

We would rather publish this than quietly leave the old number up. The numbers in the section above are the ones we stand behind.

Accuracy claims are cheap, so we built the test to be hard to game. One step assembled 35 photos of real specimens from authoritative sources, saved each under a scrambled, meaningless filename, and sealed the list of correct answers away. A separate step identified every photo cold — no peeking at the answer key — and wrote those predictions to disk. Only then was the key opened and the predictions scored against it. The model never saw what it was supposed to say while it was saying it.

We did not stack the deck with easy wins, either. The set ran from iconic, beginner-friendly specimens down to deliberately nasty cases: minerals that look almost identical to one another, and a handful of dim, cluttered, real-world photos. We also recorded a confidence number for each call, so we could check not just whether the answer was right, but whether the model knew how sure it should have been.

A single accuracy percentage hides the useful part. Three things do most of the work:

Here is the uncomfortable truth about identification tools: being right most of the time is not enough. What ruins trust is being wrong while sounding certain. A tool that says "this is malachite, 98%" about a dyed howlite has done real damage — someone overpays, or mislabels a piece, or stops looking. A tool that says "possibly malachite, but I am not certain — also consider chrysocolla, and check the hardness" has done its job even when it does not nail the name.

That property has a name: calibration. A well-calibrated model's confidence tracks reality — high confidence is almost always right, and low confidence is the model honestly flagging a coin-flip. In our test the calibration was strong. Every error landed in the low-confidence band; the high-confidence calls were right across the board. The model knew when it did not know, which is the single hardest and most valuable thing a vision model can do here.

So we built the rock identifier around that instead of hiding it. You get a real confidence score, not a fake one. When the model is unsure, it shows you the runner-up candidates rather than forcing a single guess. And every result points you to a simple at-home test to confirm it yourself. We would rather tell you "I think it is X, here is how to be sure" than impress you with a number we cannot stand behind.

It is worth understanding why this is genuinely hard, because it explains both the misses and the fix. Geologists do not identify minerals from looks alone. They scratch them to gauge hardness, drag them across a plate to see the streak (the powder color), weigh them for density, drip acid on them, test them with a magnet. A photo throws all of that away. You are left with color, shape, and luster — and plenty of different minerals share those. Some look-alikes are close to impossible to separate from a picture, no matter how good the model is. This is just as true for a crystal identifier as for any rock — two clear, glassy crystals can be different minerals entirely.

Two situations cause most of the trouble. The first is genuine twins: labradorite, for instance, is unmistakable when its blue-green flash catches the light, but photographed flat with no flash showing it is just a gray feldspar that any system will hedge on. See labradorite for what that flash should look like. The second is bad inputs — dim light, a busy background, no sense of scale, a thumb in the frame. Garbage in, hedged answer out, which is the honest behavior but not the satisfying one.

The good news is you can do a lot to swing the odds. Better photos alone close most of the gap, and a single physical test usually settles whatever the photo could not.

When you do this, the AI stops being a final verdict and becomes what it is good at: a fast, well-read first opinion that narrows a mystery rock down to a short, testable shortlist. Pair it with a five-minute test and your real-world accuracy climbs well past any single photo on its own.

One deliberate limit, because it is part of being honest about accuracy. We identify rocks, crystals, minerals, gemstones, and fossils — and nothing else. We do not identify mushrooms, plants, berries, or wildlife. A misidentified rock costs you a label. A misidentified mushroom or snake can cost someone their life, and no confidence score is good enough to take that risk. Staying in the safe lane is a feature, not a gap.