Tokyo’s Sakana AI has just introduced the Continuous Thought Machine (CTM), a fresh take on artificial intelligence that mimics the brain’s own way of handling time. Unlike traditional models that process inputs as static slices, CTM syncs synthetic neurons gradually, keeping a memory of past activations. This brain-inspired method – built on neuron-level models – transforms how AI handles tasks by imagining a real internal tick of time.
Since its launch in 2023, Sakana AI has looked to nature for new ideas. Co-founder Llion Jones, known for his work on the Transformer architecture, now steers a project that stresses thoughtful, multi-step processing. Instead of solving a problem with a one-shot pass, CTM works through several phases, letting it take on more complex challenges.
The technology uses a so-called “synapse model” to manage neuron states alongside external inputs. This process generates pre-activations synchronized over time, which in turn help the system attend to important details and make good predictions. Tests on ImageNet 1K produced a respectable 72.47% top-1 accuracy and 89.89% top-5 accuracy, underscoring CTM’s potential even if raw performance wasn’t the main goal.
One of the most interesting findings came from maze navigation experiments. Here, CTM guided itself through complex paths step-by-step – a capability that wasn’t pre-programmed, but rather emerged from its unique design. In head-to-head tests, CTM outshone Long Short-Term Memory networks and simple feedforward models in sequence sorting and parity tasks. Even with more varied neuron activity, researchers are still exploring the practical benefits of this complexity.
Although the model draws its inspiration from neuroscience, it opts for a functional twist rather than strict biological accuracy. The design’s recursive nature means training can be slower due to limited parallelisation and heavier computational demands. Still, Sakana AI has open-sourced both the code and the model checkpoints to encourage further exploration into biologically inspired AI.
