( DAC 09 Item 5 ) ----------------------------------------------- [12/11/09]
Subject: Nusym Denibulator, Breker Trek, Jasper, RealIntent
GROWING PAINS: As I've said many times before to EDA marketing people who
were fretting when they saw engineers publically complaining about their SW:
"Engineers who actually use a tool will have some praise and lots of
gripes. It's human nature. No software is bug free. When you're
seeing lots of public bug talk about a tool, you truely know that it's
being widely used. When you see only public happiness and sunshine
about a tool; be aware that you're the one being used."
- Industry Gadfly, Feb 11, 2000
In this case, technical complaints from prospective Nusym users mean that
they're actually thinking about buying it. On top of that, the shear volume
of user comments about Nusym said something, too. That is, Nusym has moved
from the "tire kicking" stage and into the "serious customers" stage. Nice.
Jasper, Avery and RealIntent were mostly hanging out this DAC while we got
to welcome two new bug hunters in "stealth" mode: NextOp and Vennsa.
And congrats to Breker Trek for moving from the "unknown" category and into
the "initial honeymoon" phase. They're actually getting noticed by some!
What were the 1 or 2 or 3 or 4 INTERESTING specific tools that
you saw at DAC this year? WHY where they interesting to you?
---- ---- ---- ---- ---- ---- ----
How to do coverage-driven verification:
1. create a coverage goal: a list of all the functions and corner-cases
that need to be covered during verification. The list is implemented
in terms of code coverage metrics and coverage points. (Coverage
points are states or sequences of states that need to occur at some
time during test regression; they're in PSL or System Verilog.)
2. create a test suite to achieve the coverage goal: a set of directed
tests that allows each function to be easily debugged. (However,
directed tests are expensive to create, so directed random testing
is used to increase coverage at the expense of simulation time.)
3. the tests and test-bench must be constructed to ensure that any bugs
exposed by the tests are detected by the test bench.
Nusym addresses the second of these three tasks. Achieving coverage
through random testing can be very expensive. Some coverage points can be
extremely difficult to reach in simulation; they can either require
excessively long simulation times before they are hit or simply a lot of
luck. Nusym optimizes directed-random tests using the existing testbench
to achieve coverage goals sooner and IDs any unreachable coverage goals.
Lots of companies still use coverage only as a sanity check for their
verification. It's not the driving force, so for them achieving coverage
is not so important -- though it should be and will be.
- Adrian Lewis of Corner Case Research
---- ---- ---- ---- ---- ---- ----
Nusym's Denibulator. Random coverage closure is taking way too long.
This is the most promising tool in the intelligent verification space
since it can work within our existing environments and can possibly even
eliminate coding the directed part of directed random and all those truly
directed tests we need to close coverage.
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
I didn't have much time to spend with Nusym at DAC. Conceptually, their
technology sounds intriguing enough to warrant a deeper look. The first
test will be how seamlessly it weaves into our existing flow.
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
The techniques Nusym showed in their slides/demo are not truly path
breaking but simple ones that aid in not wasting time with unreachable
coverage holes. It is that simplicity that made me very interested in
their stuff. But it is unclear if Nusym can go beyond the "static
analysis" part and offer more sophisticated means to analyze/exclude
coverage holes.
The other not-so-much-spoken feature is their "replay" technique; perhaps
it is still maturing, but sure enough it is one of those very useful
techniques in regression runs.
- Shalini Pandey of CVC Pvt Ltd
---- ---- ---- ---- ---- ---- ----
Nusym sounds great for small directed test case environments where closing
on code coverage is a problem -- but that does not fit our model.
We have very complex stimulus generators where a large set of inputs can
be randomized which affect both the DUT configuration as well as a huge
driver code base. It's very challenging to setup a testcase correctly
which I think Nusym would struggle with. We already make use of in-house
randomization techniques which once running in full regression mode are
effective so far as closing on code coverage goals. That has not been a
problem for us.
- Robert McAlister of Redback Networks
---- ---- ---- ---- ---- ---- ----
When I asked the lead in our verification group for his opinion of Nusym,
he thought that so far the coverage generated by randomness in our current
testbench environment is reasonably converged so he's not in a hurry to
evaluate a new tool right now. I have also heard of Nusym has problems
with IPs (and we use many Denali IPs) but I think Nusym can work with
customers to resolve these issues.
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
Nusym was uninteresting to us because it's tied to Vera (which we don't
use and don't plan to use) and it's yet-another-closure-through-coverage
tool. Nusym also works with System Verilog testbenches which we also
don't use. The Nusym salesmen seemed disinterested and had no product
literature to offer.
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
There are two problems I see for the Nusym product offering:
1. How scalable is their tool? My current impression leads me to
believe that Nusym has not yet proved the scalability issue.
Fine, it works at a block level verification, what about
bigger subsystems or a full chip?
2. Support for Specman 'e'? System Verilog support is great, but
still most big companies use 'e' to verify complex chips.
I do like that Nusym works on the question of when do I know that my
verification intent is complete?
- Mohsin Riaz of PMC-Sierra
---- ---- ---- ---- ---- ---- ----
I was kind of skeptical at first, but seeing Nusym demo the way the tool
converges on the last remaining coverage, it might be worth looking into.
Nusym doesn't substitute the need to do a quality OVM/VMM environment, but
it at least buys down the simulation time on the farm."
- Alvin Cheung of Northrop Grumman
---- ---- ---- ---- ---- ---- ----
I think Nusym's intelligent verification product is going to make Nusym
the next acquisition target by one of the larger EDA companies. Their
tool will automatically shorten the time to complete functional coverage.
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
I had an interesting time talking technology with Nusym.
- Jong-gu Jeon of Samsung
---- ---- ---- ---- ---- ---- ----
We find lots of bugs, but have a difficult time hitting 100% of our
defined coverage. Determining whether coverage is missed because of
poor constraints, incorrect coverage, or just not enough random cycles
is an extremely time consuming manual process. We believe Nusym could
work within our current methodology by generating the tests to hit
coverage missed by constrained random testing, and identifying coverage
cases that cannot be hit.
We have decided to do a closer eval of Nysym to check their claims.
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
A couple companies at DAC this year were interesting to us.
Avery introduced a formal product called Insight. In an eval with us,
they were able to verify our reset sequence for a large ASIC without
any trouble. Their tool can be used in many other ways that we are
looking forward to exploring.
NextOp also had an interesting product called "Hima" that generates
assertions. Hima is unique. It could change the assertion part of
the verification game.
- Dylan Dobbyn of Teradyne, Inc.
---- ---- ---- ---- ---- ---- ----
I am a big fan of Certitude from Springsoft. They have added some new
optimizations that lets it be deployed earlier in the process.
Jasper Active Design is attacking some of the re-use issues we have
been concerned about.
NextOp (one of the many "stealth" operations) has some promising
technology in the automatic assertion space, but needs a some features
and a UI to make the tool more interactive with the user.
Vennsa (another "stealth" operation) has technology in the automatic
debugging space. Their tool, OnPoint, focuses on root cause of errors
once your testbench fails.
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
Jasper Design Automation, Inc. Jasper has some interesting technology for
formal verification and a growing list of customers. I saw a demo of one
tool that claims to do a formal proof of your System Verilog Assertions
(and coverage) and generates vectors (and corresponding waveforms) to show
an example of where each assertion fails or passes.
The results on the Jasper demo were quick and impressive. This tool
avoids the time spent writing and debugging test benches and test vectors,
at the block or subsystem level. This could really minimize integration
time of design blocks. The formal proof could in some cases, find bugs
constrained random testing may never reach. Of course a simple demo
didn't show the limitations of the tools regarding large complex design
and overly complicated assertions, but this is definitely a company to
check out.
- Edward Paluch of Paluch & Associates, Inc.
---- ---- ---- ---- ---- ---- ----
Since we already know all the Cadence, Synopsys, Mentor, Magma roadmaps
and tools, what got my attention this year at DAC was Trek from Breker.
The area of interest is test development productivity. Their graph
based approach looks to be a promising variant on the top of existing
verification technologies already used internally.
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
(1) Breker Systems, (2) BEECube and (3) Cadence's TLM
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
Breker Systems Logic Verification
They have a cool modeling methodology and then a coverage directed test
vector generation which uses analysis of the model to generate stimulus
to cover targets in the output space. A very nice visualization. The
coverage model is good on state space, but does not offer much for cross
coverage (states and value combinations).
- Steve Meier, former Synopsys VP of R&D
---- ---- ---- ---- ---- ---- ----
My team is looking at Breker's Trek tool closely during an eval. What
we initially like about this tool is that it an add-on to any existing
environment. (We use Verilog, SV, VMM and OVM for now).
Their BNF syntax looks interesting and for the uninitiated it may take
a while, but certainly no big deal.
Trek' coverage results annotation and reachability analysis is promising
as it presents the test-coverage at a higher level of abstraction than
traditional SV. In SV world one needs to code the complex covergroups,
code/generate tests, correlate them and then view lower level coverage
data (GUI/HTML/TEXT) to extract same kind of information.
They claim to be eliminating the need for complex checkers; this is
something we are still wary about and would like to delve deep into
during our eval. In our view the checker part is hard and will be hard
even with Trek. Our view of this part of Trek is it is an ability to
correlate the testcase and coverage to the checking mechanism; hopefully
at a higher level of abstraction. If this works, we'd be happy.
- Shalini Pandey of CVC Pvt Ltd
---- ---- ---- ---- ---- ---- ----
RealIntent: They are still alive and thriving -- surprising! It seems
their tool has matured over a period of time.
One-Spin: It seems they have put new spin to verification. Demo looks
promising, but not sure how many companies using it. They
have tough competition, too.
Nusym: It seems they are going nowhere. Loss of Jayant may be an
issue for them. Anyway, they got a good concept. Until
Synopsys catches them.
- [ An Anon Engineer ]
---- ---- ---- ---- ---- ---- ----
I liked the RealIntent Meridian product above Mentor 0-in. They seem
to have thought out all the domain crossing issues that Mentor 0-in
conveniently masks, like coming in and out of reset across clock domain
synchronization circuits.
- Dan Bezzant of Prequel, Inc.
Subscribe to Newsletter
Send feedback to John... (He likes it!)
Index
Next->Item
|
|
