Microprocessors and Microsystems, Guest Editorial for 1355 Special Issue, V21, Nos 7, 8, March 1998, by Paul Walker, 4Links, paul@4Links.co.uk

In his seminal book about paradigms and paradigm shifts [1], Thomas S Kuhn suggests that a paradigm can get into crisis when there are anomalies, either too many or too great, which the paradigm can not cope with. The current paradigm for connecting microprocessors and microsystems is, and has been, the system bus. The bus suffers from the anomaly that as more devices are added to the bus, the bus needs to go faster, but the extra load on the bus means it actually goes slower. With processor speeds increasing faster than Moore's Law, the bus speed is more and more the system bottleneck, system interconnection is in crisis, and the bus is becoming more and more of an anomaly.

These are exactly the circumstances where Kuhn suggests that a new paradigm is due, and the technology described in this special issue offers such a different paradigm. The IEEE 1355 interface is based on point-to point links, which are connected by multi-port switches between the various devices which need to communicate. As new devices are added to the network, so more paths exist through the network, and so the available bandwidth grows instead of reducing to a bottleneck.

In any new paradigm, a number of similar developments build the consensus needed for the new paradigm to gain acceptance. In the case of interfaces and networks, we can see the change from bussed Ethernet towards switched Ethernet, the development of FibreChannel, of ATM, ServerNet, and MyriNet, all as evidence of a shift towards switched networks.

While these other technologies show the trend, there is one aspect of IEEE 1355 that sets it apart from them and which is particularly appropriate for Microprocessors and Microsystems. The other technologies were all designed for a particular application: Ethernet as a LAN, FibreChannel for disks and servers, ATM for a global telecomms network, and ServerNet and MyriNet for Networks of Workstations (NoWs). IEEE 1355, on the other hand was originally designed as a heterogeneous interface between microprocessor chips. As any electronic appliance now contains at least one microprocessor chip, an interface between these chips becomes appropriate to any interface between appliances, whatever the application.

Of course it is hard enough to promote a new paradigm. It is even harder to persuade people that this heterogeneous interface that was not actually designed for their application is in fact more suitable than the interface they have specifically developed for their own application. It is perhaps not surprising if some of the companies involved early in the technology - even the company that originated the technology - reckon it's more profitable, in the short term at least, to concentrate on the latest bus standards and on the application specific switched networks.

In the promotion of a new paradigm, Kuhn recognises that there are inevitably huge difficulties. Of its proponents, however, he writes:

"Nevertheless, if they are competent, they will improve it, explore its possibilities, and show what it would be like to belong to a community guided by it. And as that goes on, if the paradigm is one destined to win its fight, the number and strength of the persuasive arguments in its favor will increase. More scientists will then be converted, and the exploration of the new paradigm will go on. Gradually, the number of experiments, instruments, articles, and books based upon the paradigm will multiply. Still more men, convinced of the new view's fruitfulness, will adopt the new mode of practising normal science, until at last only a few elderly hold-outs remain."

This special issue of Microprocessors and Microsystems is evidence of just these improvements, explorations of possibilities, persuasive arguments, etc. that Kuhn expects. The papers show a substantial community of interest in this new paradigm of interfacing microprocessors and microsystems, and that they give a snapshot of some excellent work on developing the paradigm and applying it. Whether it will prevail over the bus paradigm is yet to be seen, but an analogy with another recent paradigm shift --- in modern manufacturing techniques --- offers a useful pointer. You don't have to be a manufacturing expert to know that over the last 50 years, manufacturing has been turned upside down. Particularly Japanese companies [3], and now companies the world over, found that by using these modern techniques they could make good profits while selling product for less that their competitors' cost price, and the companies unprepared to change no longer exist. Key features of 1355 switched networks are shown in Table 1, compared with the characteristics of modern production and of the old production line. The similarities between 1355 and modern manufacturing are so strong that if 1355 itself does not prevail, then something remarkably similar will.

 

Old production line

Modern manufacturing

IEEE 1355

Bottlenecks

Spare or flexible capacity to eliminate bottlenecks

Multiple paths and adaptive routing to eliminate bottlenecks, provide fault tolerance

Rigid production lines

Flexible manufacturing

Flexible encapsulation of other protocols,
flexible network topologies

Quality doesn't matter/repair if faulty

Quality is paramount, Zero defects

Reliable links,
buffers never overflow

Don't change it

Continuous improvement

Adding nodes and links continuously improves the network

Lead time is irrelevant

Just-In-Time because delay reduces quality, reduces customer service, and reduces throughput

Worm-hole routing and flow-control minimise delay

Long conveyor belt

Many small, independent, cells

Many independent links

Large batch sizes

Small batches, in trays, preferably one unit per tray

Small packets, appropriate for the application

"Push" components onto the line

When a tray is used, the tray goes back as an order to "pull" more components

Feedback flow-control "pulls" data when there is space for it

Long change-over time

Very fast tool-change

Fast, widely distributed, independent, arbitrations;
fast changeover between different packets and protocols

Environment does not matter

Waste of any kind is damaging to the environment (as well as profit)

Eliminate wasted data from buffer overflow, eliminate wasted power driving unnecessary wires, eliminate unnecessary RF pollution

Inventory is asset

Inventory is liability

Minimise data buffers

Make 50% profit margin once per year

Make 20% profit margin 10 times per year

1/5th the link speed in a switched network can offer 20 times the overall throughput of the bus

Table 1: Analogies between IEEE 1355 and the recently shifted paradigms in manufacturing

 

Papers could be written about each of the topics listed in Table 1, and far more could be written than are included in this Special Issue. We start with a demonstration that the essential 1355 circuit uses as little logic as an RS232 interface, and probably considerably less logic than a full UART [Cook]. We finish with the results of experiments with 256 and 1024 nodes for a huge data acquisition system at CERN (like a large scale embedded supercomputer) and showing communication results which are outstanding for such a machines [Haas2]. In consumer volumes, the UART is a tiny corner of a microcontroller chip costing a few cents. The supercomputer may cost tens or hundreds of millions. Yet the few cent microcontroller and the supercomputer can both use this same IEEE 1355 technology.

Part of the reason for the supercomputer-like performance is actually the low cost of the circuit. In writing about MyriNet, Chuck Seitz [2] says that the most important thing about a network is the degree (number of ports) of the routing switch. The cheaper and smaller the interface circuit is, the more ports can be put onto each chip, and the 32 ports of the STC104 has (at the time of writing) yet to be matched by any other commercial chip.

Of course the communication needs to be reliable, and the second paper [Haas1], from CERN, reports on the data rate and cable-length performance and reliability of buffered copper (DS-DE) and optical fibre (TS-FO) media of the 100 Mbits/s versions of the standard.

Where problems may be perceived with a new paradigm, the proponents must offer improvements to overcome these problems, and three papers offer such improvements. The AutoLabeller tool [Firth] provides a labelling for STC104 networks which is guaranteed not to deadlock, and which is optimum for a number of networks and close to optimum for all networks. While the overall network throughput and latency of 1355 networks is better than many other networks, the STC104 is susceptible to "Head of Line Blocking", which increases latency and reduces bandwidth, and a possible way to reduce this susceptibility is to incorporate central buffering in the switch [Waaderland]. It is very easy with a bus to send the same data to multiple places --- less easy with a switched network --- and a Message Duplicator [Brown] provides a flexible way to send the same data to more than one destination

Several different applications are covered by the papers, including data acquisition in the CERN papers and Image Processing in Brown's paper. A characteristic needed by more and more applications is fault-tolerance, and [Skeie] describes how fault-tolerance can be applied to a grid --- a network not intuitively easy to make fault-tolerant. A paper about an ATM switch [Serpanos] describes the use of the HS-Links variant of IEEE 1355, which runs at 1 Gbaud The ATM switch includes central buffering similar to that described by Waaderland, and also includes an additional flow-control mechanism.

Users need to predict the performance of the networks, and the three final papers describe three different approaches to performance: Analysis, Simulation, and Measurement. In the long term, analysis offers the lowest cost method of estimating performance, and [Lysne] shows what promises to be an interesting and useful analytical model. PACT have done a huge number of simulations of a wide variety of topologies, using OpNet, which they have gathered together into their "Network Designers' Handbook" [4], and which they have summarised in [Jones]. Finally CERN have built a large network and characterised both its performance and reliability [Haas2] --- and shown remarkable agreement with the PACT simulations. Both show average latencies of a few tens of microseconds, for average link utilisations up to 7.5 Mbytes when the theoretical maximum link data rate is around 8 Mbytes/s, and this is the average for 64 links up to 256 links and in the CERN case up to 1024 links. In spite of the conservative speed on individual links, the system bandwidth, when the avearge link bandwidth is multiplied by the number of links, goes well beyond 1 Gbyte/s. The papers also show saturation of the links, and close to saturation they show a small probability of a much longer latency. This is worthy of further investigation and possible improvement, but the results show that the efficient network topologies possible with 1355 are immensely better in this respect than the one-dimensional bus and ring topolologies.

Even with ten papers on IEEE 1355 in this special issue, there is plenty omitted. These papers do not show how easy it is to map other protocols onto 1355 --- although it is much easier to map ATM or IP or MPEG or disk accesses than it is to map some of the bus protocols. The papers do not show anything for the consumer or domestic market, except demonstrating the low cost of the circuits. Nor do they show the correspondence between 1355-based communication and the Communicating Sequential Processes (CSP) model of concurrent software, with embedded operating systems such as Virtuoso, or with Java or the new Inferno OS for some embedded applications. For these, and for further information about the IEEE 1355 standard and its applications, the 1355 Association's web site [5] offers information and contacts.

I hope the papers do go some way to show the performance, cost, reliability and flexibility of IEEE 1355.
I'm grateful to authors, referees and publishers for their work, their encouragement, and their patience.

[1] Thomas S Kuhn "The Structure of Scientific Revolutions", Univ. of Chicago Press, 1962.

[2] Nanette J. Boden, Charles L. Seitz, et al. "Myrinet -- A Gigabit-per-Second Local-Area Network" IEEE Micro, February 1995

[3] James C Abegglen, George Stalk Jr., "Kaisha, The Japanese Corporation", Basic Books, New York, 1985.

[4] A M Jones, M A Firth, C J Wright, N J Davies. "The Network Designer's Handbook", To be published IOS press 1997.

[5] http://www.1355-association.org

Brief bio:

Paul (C P H) Walker set up 4Links as a consultancy in interfaces such as IEEE 1355. 4Links now offers boards, chips, IPR, as well as consultancy, for links. Before setting up 4Links, he was at INMOS where he managed the subsystems group and defined the TRAM standard for transputer modules, a standard which has been copied by standards for other processors with links. The success of the TRAM standard suggested the value of a standard for links, which led to developing the groundwork which resulted, with the work of many others, in the IEEE 1355 standard. With the formation of the 1355 Association, he was elected editor to the association.

He has an MA from reading Mechanical Sciences at Cambridge, is a Member of the IEE, and an Associate Member of the IEEE.