heywood on 22/8/2026 at 16:02
Several MIMD (multiple instructions, multiple data) parallel computing architectures emerged in the 1980s. The INMOS Transputer was the first I can recall. When I was an undergraduate student, our school's main multi-user system was a 20 CPU MIMD machine called the Encore Multimax. In grad school, I took a class on parallel computing architecture and one of our programming assignments was on another MIMD machine called the Sequent Symmetry. Encore and Sequent were small companies who pioneered the technology, but by the mid 1990s they couldn't compete with multiprocessor systems from big dogs like IBM, Sun, Intel, SGI and clusters of cheaper systems. The idea never died. Today's multi-core CPUs are essentially transputers on a single chip.
DuatDweller on 22/8/2026 at 21:40
And so DIGITAL Equipment Corporation couldn't compete with the PC compatibles that started flooding in the market.
They wanted to build a PC but it had no chance for compatibility.
taffernicus on 23/8/2026 at 09:24
Quote Posted by heywood
Several MIMD (multiple instructions, multiple data) parallel computing architectures emerged in the 1980s. The INMOS Transputer was the first I can recall. When I was an undergraduate student, our school's main multi-user system was a 20 CPU MIMD machine called the Encore Multimax. In grad school, I took a class on parallel computing architecture and one of our programming assignments was on another MIMD machine called the Sequent Symmetry. Encore and Sequent were small companies who pioneered the technology, but by the mid 1990s they couldn't compete with multiprocessor systems from big dogs like IBM, Sun, Intel, SGI and clusters of cheaper systems. The idea never died. Today's multi-core CPUs are essentially transputers on a single chip.
needless to say the inventor behind the transputer was ahead of his time, and its legacy lives on today in the form of CPUs with numerous cores.
What's actually interesting is its concurrent message passing for parallel computing systems like this but it seems to have lost out to shared-memory systems in terms of performance (and it reminds me of Golang programming language quote "Do not communicate by sharing memory; instead, share memory by communicating")
The x86 architecture also went through phases, starting with single-CPU systems, then moving on to multicore CPUs, and finally to multicore CPUs with NUMA topology.
If I'm not mistaken, the figure behind the transputer also founded XMOS, and the transputer's influence is still evident there.
While browsing yesterday, I happened to come across a discussion on computational RAM and PIM (Processing-In-Memory). As for computational RAM itself, the Transputer can be said to have been influential in that area. Hopefully, computational RAM and PIM will resurface within the next decade. From my research, it turns out that PIM actually already exists in HBM memory, but it hasn't garnered much attention because it likely hasn't yet targeted the non-enterprise market.
Link to ArXiV paper regarding PIM : (
https://arxiv.org/abs/2012.03112)
taffernicus on 24/8/2026 at 12:08
Quote Posted by DuatDweller
In the 1970s, most microprocessors could add and subtract—but they couldn't multiply.
This video tells the story of one of the most overlooked limitations in early computing: the missing multiply instruction. From the Intel 8080 and MOS 6502 to the rise of VisiCalc, flight simulators, and early CAD software, programmers were forced to perform multiplication through slow software routines that consumed valuable processing time.
Discover why chip designers left multiplication out of early CPUs, how William Kahan and Intel's 8087 coprocessor changed the industry, and how a feature we take for granted today helped make modern spreadsheets, 3D graphics, engineering software, and personal computing possible.
A fascinating story about engineering trade-offs, transistor budgets, and the innovation that quietly transformed computing forever.
[video=youtube;6bT5dyEBK0s]https://www.youtube.com/watch?v=6bT5dyEBK0s[/video]
Aside from home computing, if only this could get the recognition it deserves...
[video=youtube_share;BVtHh9JoS3s]https://youtu.be/BVtHh9JoS3s?si=6uz5TPaIPHduvt6_[/video]
Let's make room for dataflow architecture in the future
DuatDweller on 25/8/2026 at 21:50
Prepare for Ternary logic.
or H, L and - L (1, 0 and -1).
[video=youtube;AlDosLcKhio]https://www.youtube.com/watch?v=AlDosLcKhio[/video]
heywood on 2/9/2026 at 12:33
Quote Posted by DuatDweller
Prepare for Ternary logic.
or H, L and - L (1, 0 and -1).
That video is AI slop. Ternary computing is as old as binary computing, and various ternary computers have been built before. About 10 years ago, Samsung funded development of ternary semiconductor logic for chip manufacturing. But ternary computing has never caught on because it's less efficient than binary computing. The same calculations require more gates in ternary logic.
heywood on 2/9/2026 at 16:59
Quote Posted by taffernicus
Aside from home computing, if only this could get the recognition it deserves...
[video=youtube_share;BVtHh9JoS3s]https://youtu.be/BVtHh9JoS3s?si=6uz5TPaIPHduvt6_[/video]
Those black cubes with red blinking lights looked great on magazine covers, but weren't usable for much. The CM-1 was primarily designed for implementing semantic networks for AI research, particularly language processing. Its 1-bit processors only supported integer math and local memory was only 512 bytes per processor. University AI labs couldn't afford it, and applications outside of AI were extremely limited. The CM-2 added floating point co-processors and more memory to support scientific computing applications. The CM-2 failed in the scientific computing market because it was far less efficient than vector supercomputers. The failure of the CM-1 and CM-2 led Thinking Machines to give up on their massively parallel SIMD architecture. The CM-5 as seen in Jurassic Park had 32 Sun SPARC processors with vector co-processors connected by a fat-tree network. It supported SIMD for backwards compatibility, but it was a MIMD machine.
I studied the CM-1&2 in the course I mentioned above, as well as some other massively parallel architectures. Then I had to do a programming assignment on a MasPar, which was a competitor to the Connection Machine. It could execute certain algorithm primitives very fast, like prefix summing, but we couldn't think of any real world problems to solve where calculating massive prefix sums take most of the execution time. It could do parallel graph traversal, but loading the graph into local memory became a bottleneck that got worse as you increased the number of processors in use faster than the gains from parallelizing the traversal. It was a similar story with matrix multiplication. The other problem with SIMD is conditional logic. If any of the processing units has to branch based on its data, all the other processing units have to follow or wait until the branch is complete.
I guess I should introduce Amdahl's law at this point. Gene Amdahl defined the "speedup" of parallel processing as the ratio of program execution time on a single processor divided by program execution time on multiple parallel processors. Speedup is typically plotted on a graph as a function of the number of processors. The upper limit is linear speedup, e.g. if you double the number of processors it executes in half the time, which is impossible because some parts of every program require some of the processors to wait, e.g. when executing sequential code that can't be parallelized, branching, waiting on memory, I/O. If 90% of program execution time is fully utilizing all processors in parallel, the remaining 10% limits speedup to a maximum of 10 no matter how many processors you have. Here is an illustration from Wikipedia:
Inline Image:
https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/AmdahlsLaw.svg/960px-AmdahlsLaw.svg.png?utm_source=en.wikipedia.org&utm_campaign=index&utm_content=thumbnailIn the real world, it's worse than Amdahl theorized. Due to memory contention and communications overhead, the waiting time often increases as your increase the number of processors, so it looks more like this picture (from Cornell):
Inline Image:
https://cvw.cac.cornell.edu/parallel/efficiency/EfficiencyRealisticSpeedup.gifEven trivially parallel algorithms involve memory waits, so in practice you could never effectively use all 64k nodes in a full CM-1. They designed the CM-2 and the MasPar to be partitioned into smaller virtual machines to support running multiple programs simultaneously, but because they share the same memory system and other resources, you still couldn't effectively utilize all the nodes.
My grad research was in computational electromagnetics. I tried implementing the algorithms used in our solvers and mesh generators on all of the parallel computers we had access to in that course, and all the speedup curves looked like the lower curve. The MasPar was by far the hardest machine to program for and achieve speedup on. Programming difficulty was another reason why massively parallel SIMD machines died out.
A final reason was the RISC wars. RISC workstation CPUs kept leapfrogging each other in the 1990s. The 32 MHz SPARC CPUs in the CM-5 were state of the art in 1993 when it launched, but just a few years later you could buy a 500 MHz Dec Alpha workstation for $8-10k. A few of those on a LAN in an office environment could handle the same computing loads as a 32-node CM-5 tower that cost ~$1.5M and needed to be put in a server room with a chiller.
Quote:
Let's make room for dataflow architecture in the future
Research interest in dataflow architecture has been making a comeback due to the growth of AI and big data workloads. But GPUs have become massively parallel and pipelined, and support data parallel stream processing using conventional programming methods. A dataflow architecture (going by definition) doesn't have a control program instructing nodes what to do each cycle, each node in the network executes the calculations it's been configured to perform whenever it's input arguments are available. That makes sense for specific applications where the calculations to be performed don't depend on the data, like DSP. But conditional processing is a problem.
demagogue on 2/9/2026 at 18:55
Reading things like that makes puts me in awe of the brain's architecture to process as much as it does as fast as it does. The number they bandy about is 10^15 - 10^18 FLOPS over 100 trillion synapses on a 20W energy budget.
I feel like there should be a lesson about computational architecture in there somewhere, but I have no doubt very smart people have looked very deeply into it and there are very good reasons why it's difficult to model.
heywood on 3/9/2026 at 14:44
Quote Posted by demagogue
Reading things like that makes puts me in awe of the brain's architecture to process as much as it does as fast as it does. The number they bandy about is 10^15 - 10^18 FLOPS over 100 trillion synapses on a 20W energy budget.
I feel like there should be a lesson about computational architecture in there somewhere, but I have no doubt very smart people have looked very deeply into it and there are very good reasons why it's difficult to model.
There's been a lot of studies of perception that show how much our brains "cheat" to keep up with all of our sensory inputs, which is the basis for a lot of illusions and tricks. I'm not sure if there are analogous studies investigating how we reason efficiently and what cheats or shortcuts are involved that can be exploited similarly.
Interestingly, state of the art AI models have 1-2 trillion parameters, which is only one order of magnitude greater than the estimated 80 billion neurons in our brains. Given the breadth of their training and knowledge, I would have expected the ratio to be higher. It suggests to me that artificial neural networks may use efficiencies and cheats somewhat similar to our own. Studying how these models reason may lead to hypotheses about how we do it that can be tested.
I'm more amazed by the contrast between the complexity of our hardware layers. I know life has had a long time to evolve, but it still blows my mind to see what can be built with organic chemistry. And with such small blueprints too, only 6 billion base pairs in our genome. OpenAI reportedly used 10,000 NVidia A100 GPUs to train the original ChatGPT 3.5, and each of those GPUs has 54 billion transistors.
