On the 10th of September, Google and Janelia published the wiring diagram of an entire male fruit fly nervous system: 166,000 neurons and 125 million synapses, traced from electron microscope slices and checked by hand. We spend a lot of our week teaching software to recognise chart patterns. So we did the obvious silly thing and asked the fly to help.

Spoiler: it did not help. But the way it failed says a lot about how pattern detection works, and about how we do it at Scanr.ai. Here is the whole experiment, scoreboard included.

What you can and cannot do with a brain map

You cannot train it. A connectome is a map of which neuron touches which, with a synapse count on each edge. There are no weights in the machine learning sense, no learned function and nothing that takes an input and gives an output. Asking it to learn your chart patterns is like asking a road atlas to drive you home.

You can, however, borrow one circuit from it. In 2017 Dasgupta, Stevens and Navlakha noticed that the fly's smell circuit is already an algorithm. Around 150 projection neurons carry the smell in. They fan out onto around 2,000 Kenyon cells, each Kenyon cell sampling about six inputs at random. Then a single inhibitory neuron shuts down all but the top five percent. The sparse set of cells left firing is a fingerprint of the smell, and two similar smells leave overlapping fingerprints. Computer scientists call this a locality-sensitive hash. The fly uses it to tell whether a smell is one it has met before.

Our labelling problem has the same shape. We have 871,000 candidate chart windows, each flagged by our current detector as a possible pattern, and a human has to confirm or reject each one before it can train anything. Ordering that queue well means asking, for every unlabelled chart, how much it looks like the ones a person already accepted. That is a similarity search.

The experiment

We downloaded the 1.1 GB synapse table, pulled out every antennal lobe projection neuron and every Kenyon cell in the right hemisphere, and kept the edges between them. Out came a matrix of 155 inputs by 1,912 cells with 11,062 connections and a median of six inputs per cell. The textbook geometry, straight from the release.

Each chart window is 128 daily bars. We took the closing price, standardised it, stretched it across the 155 projection neurons, pushed it through the fly's wiring and kept the top five percent of Kenyon cells. That sparse code is the fly's opinion of the chart. On top of it we fitted the same one-line readout our labelling queue already uses: the average code of the charts a human accepted, minus half the average of the ones they rejected.

Three arms went on the scoreboard beside it.

The floor. Plain correlation between standardised closes. Zero parameters. If an arm cannot beat this, it has subtracted information.
The control. The fly's wiring with the partners shuffled. Every Kenyon cell keeps its six inputs and loses only the biology's choice of which six. Any gap between this and the real wiring belongs to the fly.
Our encoders. Twenty-four neural networks we trained ourselves, which we will come back to.

The judge was 314 human labels, 160 accepted and 154 rejected, scored leave- one-ticker-out so no arm could win by memorising a price series. The number is the probability that a random accepted chart outranks a random rejected one. A coin flip scores 0.5.

The scoreboard

Arm	Score
Our best encoder	0.715
Our worst encoder	0.620
Shuffled fly wiring, ten seeds	0.55 to 0.59
The real fly wiring	0.534
The floor	0.478

The fly beat the floor. Then it lost to a shuffled copy of itself, and it lost to the worst network we have ever trained by a wider margin than that network loses to our best.

We tried to rescue it. We swept the winner-take-all fraction from one percent to twenty. We swapped the on-or-off wiring for synapse counts. At every setting the real fly sat at the bottom of the shuffled range or below it. Dasgupta's paper reports the same thing about smells: the trick is sparse random expansion, and the fly's particular choice of partners adds nothing on top.

Why a fly cannot read a chart

Three reasons, and the first is the one that matters.

A smell has no order. A chart is nothing but order. The fly's 155 inputs are glomeruli, each tuned to a family of molecules, and the wiring evolved around which molecules co-occur. When we stretched a price series across those inputs, bar 40 landed on some glomerulus for no reason at all. The connections that make the circuit good at smells are the ones that scramble a chart. Shuffling them removes the scramble, which is why the shuffled fly scored higher.

The expansion never learns. The release fixes every number in the fly's matrix before it sees a chart. The only thing that learns is a one-vector readout on top, and 314 labels is all it gets. Six of our fifteen pattern classes have fewer than ten accepted examples. A readout fitted on four accepted Bear Flags and eighteen rejected ones is guessing.

It hashes the wrong similarity. Winner-take-all over a linear projection approximates correlation of the input, and correlation is the floor we just beat. Correlation thinks a double bottom on a six-week base and the same double bottom on a nine-month base are different shapes. A trader does not.

Which patterns the fly could see

Four classes carry enough labels to read. On all four the fly lands within 0.05 of plain correlation, the zero-parameter floor it was supposed to improve on, and the encoder sits well to the right of both. The per-pattern view says what the headline said, one row at a time: a sparse hash of a correlation ranks charts the way the correlation does.

The pair at the top is the row worth staring at. Double Top is the easiest of the four for every arm, Double Bottom one of the hardest, and the gap holds across ten shuffles of the fly's wiring. Both come from the same detector's proposals, so the difference lives in what we reject. Our reading is that a rejected double top usually has the wrong shape, while a rejected double bottom has the right shape and fails on something correlation cannot see, such as where the confirming bar falls. That is a note about our labels rather than about the fly, and it has already sent us back to the contact sheet of rejected bottoms.

Would a bigger dataset fix it?

Partly, and less than you would hope. More labels would steady the readout, and the classes with four accepted examples would stop producing noise. Every arm on the scoreboard would benefit from that, so the fly would climb along with the rest.

What more data cannot do is teach the expansion, because the expansion is frozen. The shuffled control will still match or beat the real wiring, since the wiring's structure is about molecules and no quantity of charts changes that. And our encoders eat data in a way the fly cannot: they train on hundreds of thousands of unlabelled windows, so a larger corpus makes them better before anyone labels anything. Give both sides ten times the data and the gap widens.

How we actually find patterns

The detector running in the scanner today is the one that flagged those 871,000 candidates. It finds pivots, fits trendlines between them and checks the geometry against 17 pattern definitions, then reports the keypoints so you can see the pattern drawn on the chart. It is fast, it explains itself and it runs across the whole market every session. Its weakness is that it is literal. A pattern that a trader would accept and the rules would not has no way in.

The next version starts from the idea the fly got half right: a chart is a point in a space, and similar charts should sit near each other. Instead of a fixed random expansion we train an encoder to build that space itself. It never sees a label. It sees two views of the same window, cropped, rescaled and bent in time, and it learns to place them together and away from every other window in the batch. After hundreds of thousands of windows the space knows that a base is a base whether it took six weeks or nine months, because we told it so with the augmentations.

On top of that space, a pattern class is one vector. We fit it from human labels, the same readout the fly got, except now the space underneath was built for the job. Scoring a chart is a dot product, which is what lets us run it over every ticker at scan time rather than over a shortlist. It is also what lets us fit a vector from a handful of examples you mark yourself, so a setup only you trade can become a scan without anyone writing a rule for it. That part is still in the workshop, and we plan to release our v2 chart patterns encoder later this year.

The 314 labels that judged the fly are the same labels that will train those vectors. Every chart a person accepts or rejects moves the queue, and the queue chooses which chart that person sees next. That loop is what we are betting on.

What the fly kept

A badge. The proposals it made are in our labelling tool under a green FLY tag, with its score on each one, and anything a person accepts from that pile stays signed by the fly forever. Nobody wants a training set where the insect's picks and the human's are indistinguishable.

It also earned a permanent lane on the scoreboard. The next time somebody suggests that a brain from nature will find patterns a trained model cannot, we have a number: 0.534, against 0.715, and a shuffled copy in between.

Questions about the method, or an experiment you would like us to run next: [email protected].

— Eoin & The Scanr.ai team