Editor's Note: This being one of the biggest vacation weeks (4th of July
week) for most Americans, I thought I'd publish on ESNUG an article that
had a bit more of a meta-view of engineering software than our usual
down & dirty EDA/ASIC/FPGA details you're used to reading about in ESNUG.
Although this was written by Charles Small, an editor at Computer Design
magazine, it's never been published anywhere before. After reading it,
you might find it interesting to ask yourself why synthesis has been so
successfully used for chip design (due to synthesis being inherently a
text-to-gates process.)
- John Cooley
the ESNUG guys
P.S. For those of you who have sent in your DAC Trip Reports to me,
thanks! (It's been extremely interesting to discover stuff I hadn't
personally seen at DAC that unrelated engineers at different companies
each independentally got simutaneously excited about from DAC. You'll see
more next Wednesday when the ESNUG DAC Trip Report goes out.) And for
those who haven't e-mailed me your views and/or your own DAC trip report,
please do -- there's no such thing as too much data when it comes to
reviewing a conference that had over 10,000 engineers attending!
P.P.S. Have a great 4th of July weekend, ya'll! :^)
( ESNUG 294 Item 1 ) ---------------------------------------------- [6/98]
From: Charles H Small <charles.small@worldnet.att.net>
Subject: Bad Software Explained
Bad Software Explained
----------------------
One of the most annoying puzzles of our age is, "Am I stupid or does this
program just really stink?" Luckily for your self esteem, but unluckily for
your blood pressure, the answer is, "No, you`re not stupid. Software is bad.
And it`s not your fault."
The puzzle is all the more puzzling when you consider that software
developers are making money faster than they can deal with the steady stream
of armored cars dropping off moneybags on their loading docks. Software
developers lack nothing for resources. And marvelously powerful computers
are widely available for amazingly small amounts of money. Hapless users,
such as you and me, are desperate for good software for our spiffy, new
personal computers. Yet, despite this confluence of need, money, and
[computing] power, things just don`t seem to be coming together.
The Answer
----------
I have the answer. Simply put, software is bad because...
SOFTWARE PROGRAMMERS ARE THE WRONG KIND OF PEOPLE DOING THE WRONG JOB WITH
THE WRONG TOOLS ... WRONG.
Can 100,000 Frenchmen Be Wrong?
Certainly numbered among those currently devising software are dedicated,
highly trained, hard-working programmers who are honestly trying their best
to do a good job. But then, the same could have been said for Torquemada`s
inquisitors. Sincerity is a highly overrated virtue.
Strangely, although researchers who study how people actually think
("cognitive" psychologists) all know the answer to the mystery of bad
software, the enlightening results of their research into human
intelligences [plural] have not become common knowledge. This article seeks
to rectify this deficiency.
The cognitive psychologists` determinations that lead, inexorably, to the
conclusion that programmers are the wrong kind of people doing the wrong
job using the wrong tools go like this:
- The human mind has separable intelligences.
- Everyone has a different mix of higher or lower intellect in these
separable intelligences-a different "profile."
- Different profiles suit certain tasks better than others.
So far so good. But, counter-intuitively, and most importantly for
understanding why software is so bad, the cognitive psychologists` key
finding is:
- High visual intelligence correlates strongly with mild-to-severe
dyslexia.
To corroborate the preceding list, we must begin with the best known
proponent of the Theory of Multiple Intelligences (Reference 1), Professor
Howard Gardner of Harvard University (Cambridge, Massachusetts, USA).
Gardner strongly objects to the notion that a only single human intelligence
exists. And that a single metric (i.e., "IQ") is sufficient to characterize
this supposedly seamless, monolithic human intelligence. Gardner`s mission
in life is nothing less than eradicating the pernicious influence that the
notion of a single human intelligence has had on education.
Intelligences, Not Intelligence
-------------------------------
Gardner notes that what Western culture thinks of as "intelligence" is just
one particular combination of what he terms "verbal" and "logical"
intelligences. If a person has just this right proportion of these two
intelligences, he will sail through the educational system and be uniquely
qualified to be a law-school professor-and little else. Let me put it this
way, can you imagine a law-school professor piloting an airliner? Or
designing a large suspension bridge? Better yet, would you let a law-school
profession pilot your plane? Would you drive over a bridge that a
law-school professor had designed? Probably not. Instead you probably have
some sense that, despite our culturally determined conceits, different
kinds of jobs require different kinds of intelligences.
Garnder cleverly draws upon common knowledge to give credence to the notion
of multiple intelligences. Gardner posits that if both child prodigies and
idiot savants exhibit extraordinary ability in a certain area, then that
ability could be a separable human intelligence. Examples are child
prodigies who display an eerie ability to do mathematics or play music at an
age so young that they could not have possibly been taught these skills.
And we are all familiar with the rare idiot savants who also display
extraordinary math ("Rainman") or musical ability despite being otherwise
severely retarded.
So mathematical ability and music ability are probably separable human
intelligences. Among the other intelligences that Gardner identifies, his
"visual" intelligence -and its link with mild-to-severe dyslexia- is key
to understanding the origins and continuance of bad software.
Visual Intelligence Overlooked
------------------------------
In fact, Gardner is particularly incensed about the complete lack of
recognition of visual intelligence in our culture despite its obvious
importance to many endeavors. Notwithstanding the importance of subjects
such as reading and writing or nuclear physics, such abilities were probably
not essential to human evolution until relatively recently. Visual
intelligence, on the other hand, has been, and continues to be, manifestly
important to humankind. Given primates` comparatively poor senses of smell
and hearing, visual intelligence was essential for early man`s successful
hunting and food gathering. Today visual intelligence is just a vital for
engineering, physics, chemistry, industrial design, etc.
Conversely, textual intelligence had been, for most of human history, a
rarely needed skill. Until the advent of widespread literacy, professional
"rememberers" committed important information to memory. When writing first
became commonplace, only a tiny number of professional scribes were needed.
So, because textual intelligence can hardly have been essential for human
survival, extraordinary textual intelligence is therefore rarely encountered
in the human gene pool.
Poor Textual Intelligence Is Probably Normal
--------------------------------------------
That textual intelligence is a rare, separable intelligence may seem an odd
notion, give the present importance of literacy. The high proportion of
humans who suffer from what educators term "learning disabilities" means
that these abnormalities are, in fact, quite normal. Such handicaps almost
always translate to difficulty with reading and writing. Such so-called
learning disabilities are very real, serious problems for today`s educators
and students. But, given the course of human evolution, we should no more
expect most ordinary humans to have high textual intelligence than we would
expect them to excel at hopping on two legs like kangaroos.
Because of the undue emphasis placed on textual and verbal intelligence by
teachers, schoolchildren get little or no opportunity to develop or display
their visual intelligence. I, personally, despite getting consistently good
grades in school, never felt at ease until I encountered the wonderful world
of plane geometry. I could solve geometry problems faster than I could draw
them out. I was mystified that my schoolmates could not see the
obvious-to-me-solutions to the problems. This wonderful, breezy feeling
continued for me in college when I took "Statics and Dynamics," a series of
engineering courses based on applying Newtonian Physics to mechanical
engineering problems.
Gender Linked
-------------
At the risk of being Politically Incorrect, it is worth noting that over a
half a standard deviation appears to exist between the average visual
intelligence of males versus females. Small differences at the mean amount
to large differences at the margins. So out at the high end of the
visual-intelligence scale, 99 out of 100 persons gifted with high visual
intelligence (and cursed with mild-to-severe dyslexia) are male.
On the other hand, most early education teachers are women. Do we see a
problem here? Early education teachers are, after all, quite intolerant of
inherently male behavior. If a little boy acts too much like a little boy
(squirming, calling out in class, fighting, running around, etc.) the
teachers in the United States will drug him into submission until he begins
to act "properly" (i.e., like a good little girl). Typically, they
understand so little of visual intelligence, that if they are aware it
exists at all, they erroneously call it "spatial ability" as if visual
intelligence was good for no more than remembering where you left your socks
last night.
Famous Dyslectics of History
----------------------------
Examples of famous engineers and scientists from history prove illuminating.
Michael Faraday, Thomas Edison, and Albert Einstein were all thought to be
stupid when they were young because they had sub-par textual intelligences.
I, for one, cannot imagine someone like Thomas Edison or Michael Faraday
actually being able to get through engineering school. Indeed, most
practicing engineers regard engineering school as an torturous initiation
rite to be endured, rather than proper preparation for a profession. Yet we
all know that these three were anything but stupid. They all had the most
extraordinary visual intelligences.
Faraday has an almost mystical insight into the nature of electromagnetic
radiation. Yet he could scarcely read or write. Edison could see the most
marvelous inventions in his mind`s eye. Typically for a person having such
an intelligence, he communicated not in words but in sketches. Despite his
brilliance, Edison did poorly in school. Einstein could envision the most
abstruse details of the subatomic world. For example, he invented
magnetic-resonance as a thought experiment just by discovering the
phenomenon in his extraordinary visual intelligence. Yet the best job he
could get after obtaining his PhD in physics was as a fourth-grade patent
clerk.
Engineers Can`t Write
---------------------
The folklore of engineers fully supports the notion of mild-to-severe
dyslexia linked to extraordinary visual intelligence. Engineers, for
example, are notorious for writing poorly. Engineers cannot talk about
anything without simultaneously drawing diagrams all over any handy, flat
surface. Engineers often pause as they talk. Brain scans reveal that they
use these pauses occur because they are attempting -usually with considerable
difficulty- to translate pictures into words.
On a personal note, I have given Reference 2 to several gifted engineers.
They, as I did, all found the book extremely disturbing to read. They found
themselves reliving the scorn and prejudice they endured as children in
school because they did not have the "right" kind of intelligence. So, by
all means, read the book. It`s fascinating. But, if you think you have
high visual intelligence, be warned that you could be repeatedly overcome
by the return of long-suppressed, hurtful memories.
Written Records Nonexistent
---------------------------
Going beyond anecdotes about a few famous men, Henry Petroski, a civil
engineering professor, in his surprisingly entertaining and readable book,
"The Pencil," establishes that throughout history, technologists have been
primarily visual thinkers. The book professes to be a history of the humble
lead pencil. But, because the lead pencil was perhaps the first
mass-produced object made from a deliberately engineered material, the book
is actually a history of modern research-and-development (R&D) based
engineering design.
Petroski`s pet peeve is that technologists leave precious little for
historians of technology to chew upon. Technologists make drawings, models,
and-of course-complete technical artifacts, but little or no written records.
Such is certainly the case for engineering accomplishments ranging from the
Great Pyramid to the invention of the microprocessor.
Petroski`s most important observation seems, at first glance, like a
commonplace. He states that engineering is a combination of science and
art. There`s more to this assertion than first meets the eye. Firstly, by
"art," Petroski simply means that which is learned by doing, by practice.
Non-engineers cannot appreciate how important "standard practice" is to
engineers. The phrase "standard practice" means no more or less than the
art of engineering codified, shared, and universally adhered to by all
practicing engineers voluntarily.
Petroski`s observation also correctly counters the popular notion that
engineering is somehow a poor relation to science, dependent on science for
all its really good ideas. This notion is wrong. Large-scale engineering
predates science by at least 6000 years. Engineers got along for millennia
on just standard practice. But the notion that science precedes engineering
is so entrenched in the popular mind, that scientists often get credit for
engineers` accomplishments. For example, most people think scientists put
men on the moon when engineers were the ones who actually did the job.
Standard Practice Is Codified Art
---------------------------------
This simple-seeming notion of standard practice is, in fact, a crucial
difference between technological endeavors performed by real technologists
and the work of programmers. Consider, that for all new technological
fields, except programming, rapid progress at exponential rates followed an
initial period of confusion. Examples are automobiles, aviation, and
electronics. The rapid progress is a direct result of engineers` ability
to codify and assimilate their collective art into standard practice.
But in all the time that programmers have been hacking away, poking little
groups of characters into text files, they have not been able to accumulate
the powerful mixture of art (practice) and science that practitioners in
other technological fields have. For example, reusable software components,
such as operating-system utilities and mathematical library program that
have been in use for more than 30 years, are still found to be full of bugs.
The overwhelming importance of standard practice -codified art- to most
technologists stands in sharp contrast to programmers` view of programming.
Programmers, too, are quite fond of calling programming an "art." But, as
is so often the case, programmers have their own Tweedledum-Tweedledee
definition of common-sounding terms. By "art," programmers mean the term
as hijacked by fine artists: A fit of inspired, emotional, visceral,
non-rational creation by a half-crazed genius. Van Gough, in other words.
Creative "Geniuses"
-------------------
Now it`s perfectly true that most programmers are neither as creative nor as
crazed as Van Gough. My point is that they wish they were and they think
that producing technological artifacts as Van Gough would produce paintings
is perfectly acceptable -- a bone-chilling idea.
Most people know better than to peek into the kitchen of their favorite
Chinese restaurant. One glance and you will probably never be able to eat
there again. Similarly, if most software users could peek into software
developers` cubicles and take in the goings-on, the hapless users would be
properly horrified. Certainly after observing the antics of programmers,
most of the users` questions about why their software is so bad would be
answered.
In fact, the tools and practices of software developers are appallingly
crude and have not changed significantly since the dawn of programming. As
most people have no idea exactly what it is that programmers do when they
"write" program, or-for that matter-just what it is that engineers do when
the eventually get back to their drawing boards, a brief description is in
order.
Engineers Work Visually
-----------------------
Engineer`s tools are overwhelmingly visual. Engineer's tools suppress
unwanted detail and emphasize the relationships between exactly those
elements they want to see. Subsumed beneath engineers` charts and diagrams
are powerful mathematical engines and vast collections of distilled,
real-world data. For example, an engineer can measure the performance of
a system, plotting certain key data on a so-called "pole-zero" diagram.
Just by glancing at the disposition of the "poles" and "zeros," the engineer
can tell if the system will be stable.
If the system`s performance is not satisfactory, the engineer can literally
move the poles and zeros around to his liking. Then a simple operation can
extract specifications for re-engineering the system so that it will perform
according to the revised pole-zero plot.
The only engineering-style visual tool that most laymen have encountered is
the diagram of the London Underground. This wondrous diagram is not the
work of a mapmaker or a graphics designer. An electronics draftsman devised
the diagram. The proof of its utility and power to communicate exactly what
people want to know about a subway system while suppressing extraneous
detail is that every subway system in the world has plagiarized it.
Engineer Produces Greatest Graphic Ever
---------------------------------------
Reference 4 contains another glimpse of the power of extraordinary visual
intelligence. Tufte`s book hammers home the power of presenting complex
information visually. The author presents a sobering "river-of-time" diagram
of Napoleon`s ill-starred invasion of Russia that he characterizes as the
greatest graphic ever. The diagram displays seven different variables
clearly and comprehensibly. At a glance, anyone can see The Grand Army
dwindle from a multitude to a small band of stragglers. The diagram is the
work, of course, of a military engineer.
Although we all benefit from their results, no one except practicing
engineers appreciate the beauty, elegance, and power of these visually
oriented engineering tools. That`s a pity because these tools, like The
Calculus they are largely based on, are among the most marvelous of
mankind`s creation.
So What Do Programmers Do?
--------------------------
That programs are "written" is purely the result of happenstance: When
computers were invented, the Teletype was there, lurking, waiting, to be
pressed into service as an input/output device.
In its intended application, the completely non-electronic, Teletype was a
marvelous example of pinball-machine electromechanical design. You could
hook a pair of such machines to either end of a phone line and send actual
typewritten messages back and forth at a sustained rate of about 30
characters per second-far faster than most of us can type. The industrious
"chug-chug-chug-DING!" sound of the Teletype was a staple sound effect of
new programs for years.
It turns out that setting up even the most rudimentary computer to simulate
one end of a Teletype transaction is easy. So, almost from the first,
programmers used the Teletype not only as a way to get readable messages
from a computer but also as a means to send commands to the computer.
Because the Teletype can send only letter, programmers encoded their
commands of letters.
Amazing Lack of Progress
------------------------
But computers have been with us for 40 or 50 years so programming must have
progressed mightily in all that time. Right? Teletypes disappeared long
ago. That computers are increasing in power and decreasing in cost at
exponential rates is a commonplace. But, believe it or not, you can still
connect a WW2-vintage Teletype to the most modern of computers, invoke a
dreadful piece of software known as a "line editor," and program away in any
of the most modern computer "languages," eking out the millions of lines
code needed to do anything significant with today`s powerful computers.
I contend that the crude, unsophisticated methods programmers use doom their
efforts. Text-based programming systems have a crucial weakness that
renders them unfit for technological purposes: While a engineer`s drawing
can mimic, or symbolize, the actual structure of a device having parallel
paths or feedback loops, a linear string of text is inherently a poor way to
represent such structures.
Now, I would understand perfectly if the average person could not believe
that programming has not only not progressed at the same rate that computers
have, but hardly progressed at all. The situation is exactly as if you
dropped by your local newspaper`s printing plant expecting to see tons of
smoothly running machinery only to find a scriptorium where the copies
of the newspaper were being copied out by hand.
But what I am saying is all too true.
Real Technologists Cooperate
----------------------------
As if programmers feeble tools were not bad enough, trying to make people
who are genetically not suited to work as technologists pull together on a
project is as feckless an enterprise as trying to teach a pig to sing. As
Mark Twain observed, when you try to teach a pig to sing, you fail on two
counts: You are wasting your time and will succeed only in annoying the
pig.
Believe it or not, analyses of studies of programmer productivity have
consistently yielded very similar mathematical models of overall
productivity versus the number of programmers assigned to the job.
Alarmingly, these functions all go though zero when then number of
programmers rises above 50 or so. Indeed, very few documented cases exist
of large groups of programmers producing anything, let alone anything good.
Those not familiar with software-development practices would be astounded,
I am sure, at how small the teams of developers were who wrote many
important programs.
This inability to bring large amounts of manpower to bear on complex
problems is one of the roots of bad software. The folklore of the software
industry is that programmers are unorganizable and undisciplinable by their
very natures. I believe this folklore to be true because I believe the
current crop of programmers to be much better suited to being poets than to
being technologists. The idea of getting 100,000 poets to cooperate
effectively on a project is laughable.
Proponents of text-based systems can point to relative improvements in
their craft with pride. However, in absolute terms, text-based programmers
produce the most fault-ridden technological products available. Error rates
in the percentages (of lines written) are the norm for large programs.
Hardware projects of a similar level of complexity have error rates in the
parts per million.
Advocates of text-based tools argue that new programs have high defect rates
because all the components of the programs are crafted from scratch, whereas
a complex new piece of hardware-an airplane, for example-is created out of
existing components using established design methods.
Circular Arguments
------------------
Even if you grant the textually minded their view, just why does software
development have to differ from other forms of technological development?
According to proponents of text-based tools, the answer is that software is
inherently much more flexible than hardware; hardware is very constrained,
presumably giving hardware designers fewer wrong turns to take compared with
the virtually infinite ways that software can go wrong. This observation
merely begs the question: How did hardware come to be constrained in the
first place? This constraint did not happen naturally to the hardware
components as they ripened, hardening like nuts on a tree, nor will it
happen naturally to software components. Indeed, the only evidence for
software`s peculiar properties is existing software itself.
Real technologists, on the other hand, are, as Thomas Carlisle snottily, but
accurately, observed, "Little men in black coats who do something important
whilst trying to do something else." Carefully and thoroughly working out
all problems, adding tried and proven methods to their standard practice,
engineers can work together in huge groups and accomplish marvelous things.
For example, the Boeing Civilian Aircraft Company of Seattle, Washington,
USA recently debuted the 777 airliner, a technological artifact which is
arguably as complex as any program. Hundreds of thousands of technologists,
many of them working for over 20,000 subcontractors [!], all cooperated to
design this airplane. And, Lo!, it flies!
The success of the 777 brings to mind the worst software "crash" in history.
During a demonstration flight of a new Airbus airliner full of dignitaries
and potential customers, the airplane simply flew into the ground, despite
the frantic efforts of the pilots, because the computer-directed control
system decided that the pilots should not be allowed to pull up because the
aircraft was too close to the ground [!?].
The Solution
------------
The solution to the pressing problem is as drastic as it is obvious once you
have the facts about human intelligences: Technologists having high visual
intelligence and possessing visually oriented tools must replace the current
crop of textually oriented programmers and their antique text-based tools.
Sidebar
-------
Totally unrecognized by the computer-science crowd, a few software
development-tool vendors have refuted the false imperative that "writing"
programs necessarily involves stuffing endless streams of plain ASCII
characters into bottomless text files. Most of these software-development
tools were devised for test engineers. Such engineers have to automate the
testing of manufactured goods as they come down an assembly line. The hard
part of such a job is knowing how to make precise, repeatable measurements
at high speed. Typically, test engineers assemble a network of
special-purpose, precision instruments needed for a particular measurement,
controlling the instruments and collecting their data with a computer.
Such engineers need visually oriented programming tools because, firstly,
they are engineers. Secondly because their software must control complex
systems having feedback and parallelism. And lastly, the engineers have
lots of important things to do beside fooling with text files.
The difference between the visual and textual methods is dramatic. A
diagrammatic specification for a complex control program takes only a few
pages. An engineer blessed with extraordinary visual intelligence can easily
comprehend the structure of such a system. And equivalent textual version
of the same program would span many pages grouped in to many separate
files -- literally incomprehensible by even those having extraordinary
textual ability.
The first, and argueable the best, of these systems is LabVIEW from National
Instruments, Austin, Texas, USA. With LabVIEW a test engineer can quickly
program elaborate test sequences diagramatically. LabVIEW will command the
instruments to make measurements, collect the data that the instruments
take, and display those data in human-readable formats. Unlike VisualBasic
and other software that allow you to configure just graphical user
interfaces, LabVIEW is a complete comiler that produces optimized programs.
References
----------
1. Gardner, Howard, "Frames of Mind, the Theory of Multiple
Intelligences," Basic Books, New York, NY, USA. ISBN 0-465-02508-0.
2. West, Thomas G, "The Mind`s Eye," Prometheus Books, Buffalo, NY, USA.
3. Petroski, Henry, "The Pencil," Alfred A. Knopf, New York, NY, USA.
4. Tufte, Edward R, "The Visual Display of Quantitative Information,"
Graphics Press, Cheshire, CT, USA.
|
|
