( ESNUG 459 Item 7 ) -------------------------------------------- [12/14/06]
Subject: customer benchmarks confirm Synopsys TetraMAX 12X speedup claims
> I first saw the story in EEtimes.com. It was almost a word-by-word rehash
> of the official Synopsys press release. From the release:
>
> "This is the 2nd major performance enhancement Synopsys has engineered
> for TetraMAX in the span of a year," said Graham Etchells of Synopsys
> test marketing. "Taken together, the 2 most recent TetraMAX versions
> have achieved on average more than a 12x speed-up in ATPG results."
>
> So what am I missing here? Is this a Synopsys Marketing contrived "study"
> rigged to show the world that (obviously) 4 out of 5 choosy mothers choose
> Jif? Or are the everyday users actually seeing real life benchmarks
> where TetraMAX is now a kickass 12X faster? Inquiring minds want to know!
>
> - from http://www.deepchip.com/wiretap/061102.html
From: Zahi Abuhamdeh <zahi=user domain=txc spot calm>
Hi John,
First, thanks for your efforts to keeping the EDA vendors to their word.
I read your post on the claimed speeds of TetraMAX, and thought to shed some
light on the subject. Yes, we did experience a 4x speed improvement between
2004.12 vs 2005.09. It came when we were in tape out and our Tmax 2004.12
runs were too slow. We moved to the new version 2005.09, and there it was:
~4x faster. The same was true for 2005.09 to 2006.06 with 3x improvement.
So, I do believe Synopsys's claim of 12x.
To compare apples to apples though, we did re-run a test case through all 3
versions, and attached are the results.
TetraMAX Vector Run Speed Up Speed Up
Version Size Time (hrs) vs. 2004.12 vs. 2005.09-SP2
----------- ---------- ---------- ----------- ---------------
2004.12 17,017 15.33 1x --
2005-09-SP2 17,137 3.52 4.36x 1x
2006.06-SP2 15,796 1.15 13.33x 3.06x
In all 3 cases, for all 3 runs, test coverage was 99.15% to 99.18% and the
fault coverage was 98.97%.
The design had 300K FFs. All tests were run on the same machine:
CPU: 2 x 3400 MHz
Intel Xeon
2048 KB Cache
Memory: 16 GB
RAM, 33 GB Swap
Since we saw 13.33x, I do believe Synopsys' 12x claim to be true.
We were so surprised at the speed up between 2004.12 and 2005.09, that we
chucked the TetraMAX 10x multi-CPU license that we had bought before.
- Zahi Abuhamdeh
TranSwitch Corp. Bedford, MA
---- ---- ---- ---- ---- ---- ----
From: [ The Horse With No Name ]
Hi, John,
Have me completely anonymous, so Synopsys doesn't get pissed.
I never heard anyone at ITC talking about this. If TetraMAX was so much
faster, then lots of people would be talking about it. Often people say
TetraMAX is somewhat faster than FastScan, but not at the "I'm switching
products" level.
No one at the Synopsys booth at ITC pushed this either. They pushed their
approach for ATPG for small delay defects (feed PrimeTime data into TetraMAX
so its transition fault ATPG will propagate on the longer path). This is
better than plain old transition fault testing. STARC gave a talk on what
I gather will be the future TetraMAX delay test improvement, but I and other
colleagues were underwelmed. There is an improvement, but it is by no means
state-of-the-art.
I interpreted the Synopsys speedup press release to be the usual incremental
improvement in tool performance over time, not something revolutionary.
Keep in mind the bulk of the effort within test tools today is in compacting
or compression patterns, not basic ATPG.
- [ The Horse With No Name ]
---- ---- ---- ---- ---- ---- ----
From: Roberto Mattiuzzo <roberto.mattiuzzo=user domain=st spot calm>
Hi, John,
Although for many of the designs we manage, ATPG run time is not the major
aspect, we do care about it. Here's the performance improvement we've seen
between TetraMAX v2004.12 and TetraMAX v2006.06.
v2004.12 v2006.06
total faults 1749114 1749114
test coverage 96.16% 96.05%
# of patterns 4149 4086
Total CPU time 4212.63 450.60
This is for single stuck-at runs, executed on a Linux AMD 64 platform.
As you can see, run time improvement is 9.34X. Coverages are alike, and the
vector count has decreased slightly. That 9.34X increases further in the
case of Transition Faults.
- Roberto Mattiuzzo
STmicroelectronics Milan, Italy
---- ---- ---- ---- ---- ---- ----
From: John Ford <jford=user domain=dftdigest spot calm>
Hi John,
Saw your article today about the TetraMAX PR. I wouldn't call it BS, but
to me, it's kind of a non-story. The truth is that all the ATPG vendors
are refining their algorithms almost constantly. Whether or not they
decide to release it in the press is another thing.
To be fair to Synopsys, they did at least specify, "the same high-quality
test patterns", for "both stuck-at and transition delay fault testing",
"on all design-types". I know that's a little nebulous, but this *is* a
press release.
I've used both TetraMAX and FastScan over the years. In my experience
they're comparable in speed, but that's not really my biggest concern. An
ATPG tool could be blazing fast but generate a crapload of patterns, and
I'd be screwed anyway, because I can't fit them all on the tester.
On top of that, they could be less than completely effective, because up
until recently, the ATPG vendors were still struggling on how to generate
*quality* tests for the more subtle defects seen in 90nm and below.
For me it's more about the *quality* of patterns than how fast they're spit
out. Maybe I could paint your house 10x faster than another guy, but how's
it going to look?
- John Ford
DFT Digest Laguna Beach, CA
---- ---- ---- ---- ---- ---- ----
From: Jon Colburn <jcolburn=user domain=nvidia spot calm>
Hi John,
Regarding Wiretap No. 061102, you asked your readers if the Synopsys
marketing claim of a 12X speedup in TetraMAX was real. We are heavy users
of TetraMAX and have definitely noticed that it has gotten a lot faster
recently. To test this claim we took a couple designs, both graphics
processors, and ran them through the various TetraMAX versions on the same
machine configuration to get a real apples-to-apples comparison.
2004.12 2006.06-SP2
Design A (560K flops)
full stuck-at ATPG 49 hrs 3 hrs
patterns 19K 8K
1k transition 16 hrs 1 hr
4k transition 78 hrs 6.5 hrs
Design B (1.3M flops)
full stuck-at ATPG 115 hrs 7 hrs
patterns 16K 10K
In full stuck-at ATPG runs Design A went from about 49 hours to less than
3 hours (16.3X) and Design B went from 114 hours to 7 hours (also 16.3X).
In both cases the pattern counts actually went down significantly for the
same coverage as well (60% and 35% reductions respectively).
I only did a partial run on the smaller of these designs for transition
faults, but for the first 1K patterns the old version took 16 hours and the
latest took ~1 hour; this is on a design with 12+ test clocks. At higher
pattern counts the performance delta for transition tests gets even larger,
2k patterns at 57 hrs vs. 2 hrs.
So we've seen 16-18X speedup in real designs. Hard to believe, but we know
that Synopsys had developers working on this for over a year. These are all
single CPU runs, but we have also seen considerable speedup of distributed
mode ATPG because it benefits from these optimizations.
- Jon Colburn
Nvidia Corp. Santa Clara, CA
---- ---- ---- ---- ---- ---- ----
From: Francesco Mainieri <francesco.mainieri=user domain=st spot calm>
Hi John,
I work with TetraMAX since... I cannot count... several years... In this
last period a lot of improvements has been developed in this tool. For
instance, it's much faster then before, I really don't know if it is 12x.
If not, it should be closed to 12x. Before we used to use the distributed
processor -- today only one processor is enough.
I can say that Tmax is leading edge!!
- Francesco Mainieri
STmicroelectronics Castelletto, Italy
---- ---- ---- ---- ---- ---- ----
From: [ A Foo Fighter ]
Hi, John,
I'm a DFT engineer in Canada. Please do not disclose my name. My designs
are larger than average with multi-clocks domains (15+). The coverage we
had with TetraMAX is in the high 90s. I can tell you from my first hand
experience that between the 2005.09 and 2006.06, the TetraMAX runtime has
significantly improved. Even though I can not confirm the 12X Synopsys
has claimed, I can tell you from my experience that it's close.
- [ A Foo Fighter ]
---- ---- ---- ---- ---- ---- ----
From: [ The Masked Avenger ]
John, keep me anon...
Reading the Mentor vs. TetraMAX marketing debate is always amusing. To be
honest, users of EITHER of these tools should go and thank their managers!
I work in a shop where our DFT tools are 99% homegrown and it is a painful
experience! Tylenol-and-Jack-Daniels-for-lunch kind of headaches.
We did have a very positive experience with TetraMAX a couple of years ago.
Our inhouse homegrown ATPG tool could not generate patterns on 3 of our
large designs; they would run for 96 hours and core dump with no results.
TetraMAX was able to handle all of the designs.
- [ The Masked Avenger ]
---- ---- ---- ---- ---- ---- ----
From: Anna Tesone <anna.tesone=user domain=st spot calm>
Hi, John,
Here's a few lines about my experience on a real design. I was in trouble
because of complexity (multiple clocks, 130 k FFs, high speed patterns).
Our main concern was the very long ATPG runtime. Initially I used TetraMAX
2004.12 then I moved to 2006.06-sp3. These are the results comparing the
two versions (only for stuck-at fault models):
2004.12 2006.06-sp3
test coverage 97.06% 97.02%
patterns 5,618 5,161
CPU time 13,063 1,478
I got these data using two trials running on the same machine. The complete
patterns generation requires now almost 1 week. Obtaining the same patterns
with the old 2004 version of TetraMAX would have been a real nightmare.
- Anna Tesone
STmicroelectronics Arzano, Italy
Index
Next->Item
|
|
