( DAC 03 Item 15 ) ----------------------------------------------- [ 01/20/04 ]
Subject: Atrenta SpyGlass
DON'T CALL ME THAT! Ask any Atrenta saledroid about their line of code
coverage tools and they'll give you an earful about how they're NOT a linter
nor a code coverage company. Walks like a duck. Squalks like a duck.
Eats like a duck. Smells like a duck. But, no, it's not a duck. (Atrenta
just doesn't want to go against the free code coverage & linter tools already
built into VCS, NC, and ModelSim. It's a marketing thing.)
I am a CAD manager so my perspective is from tool support how proactive
are they on working on problems with respect to the tool. The usual
scenario is that when I have a problem reported by user Atrenta
responses within one to two hours with a solution to the problem
(sometimes a SpyGlass switch or feature that will fix the problem) or
identifies the bug and assigns a VI #. Within 1 to 2 weeks Atrenta
identifies the release that the bug will be fixed in. I have not
investigated their rivals. The tool was already in use when I took over
support. Currently, I really don't have any negative things to say
about Atrenta.
I did take an internal user survey and the responses were very
positive; below is one of the most informative responses:
Here are some points that I have about the tool:
1. It is easy to learn and quick to ramp up on
2. It chews on our design very quickly (~100 Kgates).
3. Reports are legible.
4. It has a clean and simply GUI that can be used to sort on warnings
and errors, (although I prefer the batch method of text reports
along with UNIX greps and sorts to post-process.)
5. It looks like it has a lot of advanced capability that we are
unfamiliar with, but I sure would like to tap into this side of the
tool (DFT checks, etc.)
6. Documentation is well laid out, easy to read, and easy to find stuff
in. Sure there have been plenty of bugs as have been reported via
SPR, but it really looks like these guys are very responsive to
these and get fixes into the tool fairly quickly. In brief, these
guys are a breath of fresh air compared to most of the EDA companies
out there.
As far as the usage of the tool is concerned, I have been pleased with
it.
- Rick Stanton of Cypress Semiconductor
Atrenta SpyGlass strengths:
- The only tool which simply works in mixed language
environment (Verilog & VHDL)
- Extremely user friendly and a lot of meaningful option to turn on
and off and produce a meaningful reports
- Quick responses to reported bugs and issues from AE and R&D and
willingness to come on site and help when needed and enhance the
tool
- Well organized set of policies which contain rules that can be
filtered out if user choose to do so
- Does a good job of detecting clocks and clock domain crossings
- SpyGlass's incremental schematic window does a good job of
graphically displaying the crossings
Atrenta SpyGlass weaknesses:
- Just like any other EDA tool, it has bugs, especially in mixed
language environment
- SpyGlass uses busses instead of record elements in VHDL, feature
enhancement has been requested, Atrenta is looking into it currently
- SpyGlass does not behave like ModelTech, NC-SIM, Debussy in Verilog
environment. You can not compile libraries and put them away and
refer to them from other libraries, feature enhancement has been
requested, Atrenta is looking into it currently
I have looked at a total of eight HDL Checkers: Nova-VHDLlint from
Avanti; Nova-ExploreRTL-VHDL from Avanti; nLint from Novas; vncheck
from TransEDA; Hal from Cadence; Leda from Synopsys; SureLint from
Verisity (Verilog only); SpyGlass from Atrenta.
Interestingly enough 0-in, Real Intent, @HDL are not among them.
- Mehdi Shahbazi of Broadcom
Atrenta's Spyglass is impressive. It very quickly compiles and
"synthesizes" your RTL. Among other things, it identifies the design top
level (useful for looking at other designer's code), traces back clock
nets to potential sources and domain crossings, displays hierarchy, and
has cross checking to the source RTL for identified potential problem
areas. The number of default rules can get a bit overwhelming but they
provide the ability to tailor these for both the associated warning text
and the severity so you can craft a set for your own needs. Others from
our group reviewed the constraints checker and were impressed enough that
we are going to bring it in house for review. All in all, I think this
was the best overall RTL analysis tool that I saw for our particular
needs.
- Jeff Waite of Chip Express
The Atrenta ERC demo looked impressive. We have some "classic"
benchmarks, which we would like to evaluate the tool with. Hopefully
they will release it soon. They did say they will give us a beta release
for evaluation, that has not happened yet!!!
Atrenta is an awesome tool, but expensive. I am waiting for them to get
the constraint checker released.
- Nicco Bhabu of Chip Express
I have used SpyGlass for three different projects in the last year. I
would not consider myself a power user at this point.
1. Evaluation of in-house developed Verilog design.
This was for a large design project involving four design centers.
All code was written in Verilog. For this project I had created a
"design guideline". This specified things ranging from aesthetics to
issues dealing with DFT and clocking. Atrenta helped define which
rules/policies should be checked in the tool in order to meet the
guidelines. We did not start using SpyGlass until around 2/3 of the
way through our design cycle. As we would do a release, I would run
the checks on the code. Sometimes it was inconsistent about specifying
directories for include files. i.e. sometimes you had to tell it
additional directories and sometimes you didn't.
SpyGlass flagged several issues with the design regarding rule
violations. In particular, using positional association rather than
named association for instantiated components outputs not being
registered sensitivity list problems ports not connected on
instantiated all bits of a bus not being connected
Another thing it flagged which is useful is that an instantiated
component port order should match the order of the module port
definitions. We had an issue where synthesis was complaining about all
ports not being connected but did not specify which port. The designer
could not find this because the order of the instantiation was so
jumbled from the module definition. SpyGlass helped us identify which
port was missing.
We worked with the AE to develop some custom rules. The rules are all
customizable via Perl although I think you have to be a real Perl
wizard.
2. Evaluation of an IP module received from an outside supplier.
As part of negotiations of the completion of a contract I was asked to
run SpyGlass on IP provided by a third party supplier. In less than a
day I was able to read the design into the tool and run an RTL
analysis. Much to my surprise this came back extremely clean with no
violations or warnings.
3. Evaluation of inter-clock domain synchronization of a legacy VHDL
design block
I used SpyGlass to check the synchronization of a peripheral interface
containing several clock domains. SpyGlass has a predefined set of
methods for synchronization. Due to a custom synchronization method
with an extra reset, it did not recognize these paths as being
synchronized.
After I got all of the clocks defined, it flagged 91 asynchronous
crossings that were not synchronized and 2 that were synchronized.
After painstaking evaluation of the 91 paths, it was determined that
all but two were actually synchronized. The two had a legitimate issue
that was fixed prior to tapeout.
One drawback of the tool is that it does not allow you to create
relationships of inputs and clock domains or creation of a virtual
clock for truly asynchronous IO. This has been identified to Atrenta
as something that should be incorporated in future revisions.
Summary:
There are so many rules/policies in the tool that you must be very
careful not to turn them all on at once. You will be overwhelmed. It
does separate the issues nicely into categories when looking at the
violation via the GUI so that helps. After spending some time with the
AE, I learned how to search the rules/policies in order to just turn on
specific items. That is much more useful.
I often have to look at a design after it is "done" and it is a very
quick way to get an assessment. A good way to keep the designers
"honest".
- Dan Talley of Skyworks
At Cypress, I've been engaged with Atrenta for over a year now, helping
determine the subset of all SpyGlass rules Cypress we require RTL code
to pass through. I was responsible for incorporating basic SpyGlass
training into Cypress' internal training and also ran the early tool on
legacy designs to benchmark the effort required bring legacy RTL code
in line with the rules we'd selected.
SpyGlass has evolved substantially and improved over the past year.
Throughout that period, they have been without exception very
responsive to all my questions and requests for information. I've been
engaged with individuals from sales/marketing to support to the CTO of
the company, and found every one of them to be very knowledgeable,
articulate, and proficient. They have grown over the past year, but
they still deal with Cypress as though we were their only customer.
SpyGlass has a Perl interface which includes the ability to overload
specific properties of the rule checking paradigm; at Cypress, we have
created our own customized sets of warning levels and warning messages,
so SpyGlass fit well into our design flows and nomenclatures.
Initially (~1 yr ago) the modifications to the tool required some Perl
expertise, but the tool has since been enhanced to make this
overloading process straightforward and easy to maintain and propagate
from release to release.
We did some initial tests of one other tool before we selected SpyGlass
(Synopsys' Leda), creating a matrix articulating our requirements with
relative weightings for each tool. We made the decision to use
SpyGlass based on that overall score and have never had occasion to
look back. At the time of the evaluation, in our view Leda's only real
edge over SpyGlass was its mixed language capability; now mixed
language support is fully implemented in SpyGlass.
- Dave Harris of Cypress
Our experiences with SpyGlass have been positive. They do a good job
in analyzing our RTL designs and giving us early feedback into design
problems, in addition to doing the basic lint-type checks. I've had
RTL design engineers tell me SpyGlass found things that would have been
either painful to find after synthesis or generated gates that would
have been difficult to meet timing on.
One of the complaints that we've had about it are that the rule set is
overwhelming if you blindly use it as you would a lint checker. We try
to address this by training (we've got it down to a one hour overview
when people are getting started) and are working to develop a standard
template that would make it easy for people to run only the rules that
other engineers here have found to be the lint-type checks to do. The
only other big complaint is from the advanced users we have when they
try to do one of the more complex checks and the results either aren't
as we would expect. The source of these problems is usually either a
mismatch between what the designer thinks the rule should be doing and
what it is, or what I would consider a typical bug in a relatively new
cad tool. In either of these cases, Atrenta has been very responsive
to working with us either to fix the code or help us understand what
its doing.
The other work that we've done with Atrenta is to take our RTL coding
guidelines and develop custom rules when the standard checks don't do
what we want them to. We were able to implement about 80% of our
guidelines with either no or minimal modifications to the supplied
checks, and we expect to be able to get to the 95% level with an
acceptable amount of work.
Overall, we're happy with the tool. We have confidence Atrenta will be
there to work with us when we run into problems and we're hopeful their
next-generation technology will continue to add value.
- [ An Anon Engineer ]
I'm responsible for a DFT team which develops the Motorola common DFT
flow (central CAD organization). Thus, I'm NOT a designer which uses
SpyGlass DFT on a daily basis.
At Motorola, we started working with what is today called Atrenta in
1999. At that point in time we selected SpyGlass as it provided the
best tool for performing Code Checking plus writing custom rules.
Since then SpyGlass is used in Motorola to perform simple RTL rules
checking. (As you might know Motorola has its Semiconductor Reuse
Standards, which among others includes Coding and DFT rules).
When SpyGlass was extended to also cover DFT rules checking we were
very interested in it and worked with Atrenta to again write custom
Motorola DFT rules. This did not work out properly due to immense
effort and knowledge which was required to develop and maintain custom
rule sets for DFT. Since then we changed our strategy: we now rely on
DFT rules which Atrenta provides. We have our custom file in which we
define which Atrenta rules are applied to our designs to adopt our DFT
methodology.
Design teams are very interested in this approach. The main benefits we
are seeing are:
- Early DFT rules checking combined with RTL coding checking: A single
tool which performs both checks (RTL coding and DFT!) enable the
identification of DFT problems earlier
- Easy to run once rules are defined
- Using Synopsys DRC checker always requires the expensive Synopsys
licenses and is significantly slower than SpyGlass . Syntest is not
in our tool suite.
- Recently SpyGlass added RTL test coverage analysis to get an
estimate of test coverage and test controlability issues already at
RTL level. Correlation between SpyGlass and TetraMAX ATPG is OK now
(since several patches have been provided.)
Our Toulouse DFT team spent significant effort on evaluating SpyGlass
again for Motorola. They drove a lot of enhancements in SpyGlass. Now
SpyGlass DFT rules checking is supposed to be an integral part of all
big Motorola Wireless designs. Often design teams for those designs
are distributed to 30+ different. Defining RTL DFT rules checking as a
handoff from RTL to the integration team makes the task for the
DFT/integration teams much easier. However, I have to admit that
adoption still need to ramp up. But it is ramping up.
- Helmut Lang of Motorola
I only had a cursory look at 0-in and Real Intent a while ago and have
not formally evaluated them on our end. We brought in SpyGlass to
phase out Verilint and were able to do so.
With regards to comparison with other tools, all tools of this type of
RTL checking and exploration technology have an issue in that they
generate a volume of unmanageable violations and leave the rule
prioritization to the user. This leaves every site to spend nearly a
year to prioritize the rules, suppress violations that have no impact
on the function and the quality of result, improve accuracy of
violations by eliminating false negatives, develop and refine custom
rules and use-models. I feel that though most of the tools in this
space have close to comprehensive rules, what differentiates one
product from the rest is it's maturity in usage by end-users and
agility of vendor R&D responding to refinements to bring the needed
accuracy to violations. In my experience, I feel that SpyGlass
exhibits this characteristic of mature tool in this space with some
scope for further refinements.
Here is our usage experience and impressions on Atrenta's SpyGlass
product(s) so far. We have been using SpyGlass for over 2 years now
and it is now part of our methodology across all designs at our site
and is currently under consideration at other sites.
We use about 220 Verilog-pertinent rules out of a total of 650+ rules
from 12 rule-decks, partitioned into three categories: Mandatory,
Optional and Informational. The tool is used mostly in the batch mode
with the exception of debug in GUI mode during design exploration-
pertinent rule violations. Local rule-waiving is discouraged and
designs are audited with global override on rule-waivers.
SpyGlass has come a long-way over the past year in bringing accuracy to
a number of rules that were otherwise reporting violations that are
considered false-negatives from user's stand-point. Usability
improvements and accuracy refinements will be an ongoing process for
some more time with this technology and we are working with the vendor
in this direction.
We recently completed an evaluation of their DFT rule-deck for test-
coverage closure at RTL. We find the tool providing very good value in
this area. The value comes from it's ability to continually render
feedback on the test-coverage in response to RTL transformations to
improve on the fault coverage. It is effective to use the tool in the
GUI mode for DFT analysis, as the process demands effective interactive
and iterative debug.
At DAC'03, I attended presentation on rules-driven constraints checking
and formal technology for rules validation. The former will bring the
much needed formal checking to the design constraints while the latter
will bring the needed accuracy to prediction technology.
Overall, I find Atrenta's tools and technology representing excellent
value by exposing issues upfront at RTL which otherwise will amount to
expensive inter-design-task iterations. I find that it's a matter of
how well one exploits a technology such as this across all designs and
all design centers."
- Chandra Moturu of Hewlett Packard
SpyGlass is conceptually a good tool, but I have found that one of it's
biggest selling points is its biggest weakness. Atrenta marketing
loves to talk about how SpyGlass has over 25,000 rules that it can
check. This certainly makes it a very flexible tool, but it actually
gets in the way of getting something done in a reasonable amount of
time.
First, because it gives you so many things it can check, you and your
group need to go through the rules to figure out which ones are really
important for you to check. Even after paring down the list by
excluding specific sets of rules like Xilinx, VCS, latch-based design,
etc, there are still thousands of rules to consider. It's already
difficult for one person to go through these rules, understand what
they are checking for, and decide on their relevance to their design,
but usually you don't want to entrust these decisions to only one
person. So, you end up taking up the time of several senior engineers
to come up with a set of rules. Then you have to somehow tally
everyone's opinions. I had to write a Perl script to create short
summaries of the rules and another Perl script to tally the votes. If
a rule came out unanimous, then it was a no brainer. For mixed results
rules, we ended up just going with the majority, because it was so
time-consuming and painful and ended up with a somewhat non-
analytically selected set of rules.
Then we threw the rules against our design. We got hundreds of
thousands of messages. A tool is only as good as its output being
reviewed. With so many messages, no one was going to look at them.
So, I just went through our rules again and pared them down by 2/3.
This new smaller set of rules produced tens of thousands of violations.
This was somewhat in the range where we thought we could manage them.
In the end, because of time pressures, we had to prioritize the
violations and only fix the top categories of violations.
Another problem with SpyGlass is that it has several categories of
rules: lint, openmore, morelint, STARC. Lint is basically rules
applied by Verilint. Morelint are more lint rules. STARC and openmore
are coding standards generated by somebody. If you want STARC- or
openmore-compliant code, then you apply these rules. But without going
through all the rules of each policy, you don't know what rules are
checked by what policies. So you end up throwing all the policies
against your design. Some of the rules have aliases in other policies.
Some of them are functionally identical, but are not aliases, so you
end up getting duplicate messages that are worded differently. But the
worst problem is that each policy has a different scale to specify
severity. You have a mixed bag of warnings, errors, informations,
mandatory, recommended, etc. You don't know if a "mandatory" in STARC
is more serious than "warning" in openmore, or "error" in morelint.
So, if you mixed policies, you really had no way, besides manually
going through the rules to prioritize the messages. SpyGlass is very
flexible (it's rules are written in Perl), so you can overload rules
with information like severity that's normalized across all policies,
but it's something that the user has to do.
Atrenta has started addressing this problem with "templates" which are
prepackaged sets of rules which you can apply. But when I was running
my design through, they didn't have a "clean-synthesis" template (see
below).
I did have a lot of difficulties also with run times. My initial runs
took over 10 hours to run the entire design. SpyGlass compiles the RTL
into a netlist and it particularly had a hard time with large case
statements (4096 cases). (But in their defense, other tools like 0-in
Checklist had problems with large case statements as well). I
eventually got it to reduce the run time to about 2 hours buy using new
releases, and eliminating slow running rules. Also it was very
difficult to determine which rules were causing the run-time slowdown.
This whole issue of synthesis run-time really irritated me because
SpyGlass didn't have to take into consideration issues like timing and
design rules and just get to a functional netlist.
In the end, we did get the tool to produce some results which helped us
to clean up the design. But because it was so difficult to manage the
rules and its outputs, a lot of the results were ignored by the
designers and never fixed and it was only at the synthesis stage that
some problems were caught. I'd say that SpyGlass is potentially a
powerful tool, but it's very awkward and difficult to manage. A
colleague at another site told me that it took them a full year to get
SpyGlass working the way they want it.
As I used this tool, I had to question why we were going through this
experience. Although it would have been nice to revise our code so
that it was well written and easy to understand and conformed to some
coding standard, the bottom line of our efforts was to make sure that
our code didn't contain things that would shoot us in the foot later.
Basically, I wanted a "clean synthesis" policy which would make sure
that we wouldn't run into synthesis problem or the synthesis results
wouldn't have mismatches to simulation. So rather than "linting"
(which in my definition implies "dumb" syntax and text analysis which
produces a lot of false positives), I want "design checking" (where the
tool understands the design and will not produce false positives).
IMHO, linting as a concept is obsolete considering such issues as
imported IP and legacy code. I really don't care that one signal in
the design is over 32 characters long or that there's a tab in the file
-- considering our schedule pressures, I wanted to know about clock
domain crossing without synchronization or simulation/synthesis
mismatches. I don't care about cuts and scratches, but I do care about
stepping on a land mine and losing a leg.
Bottom line, if you are willing to put in the time and effort with
SpyGlass, it can be a very useful tool that can be fine tuned to meet
your exact needs. But if your goal is just get the design through
without investing a lot of time and effort, SpyGlass, at least in my
opinion, took too much time and effort.
- [ An Anon Engineer ]
As an ASIC Design Center our usage may be different than the norm of
customers. We are looking for a means to screen the incoming RTL or
netlist for problems prior to spend a large effort to get the
RTL/netlist into other tools such as Synopsys. We have
both a Magma layout and a Cadence layout flow but so far we have been
using mostly the netlist entry to Magma since this is what the customer
is delivering.
As the design environment transitions to a RTL handoff (we are seeing
more of these, particularly from FPGA), the necessity to qualify the
release becomes much harder. SpyGlass is helping us solve this
problem. We have been able to find real issues with
customer releases by using the tool which has sped up the debug cycle
that precedes the actual physical layout work. We have only seen a
demonstration at DAC of the new timing constraint policy, but our EDA
group in San Jose is actively looking at this function of the tool.
Constraints are almost harder to get clean than the netlist. The
syntax checking inside does not exactly know the requirements to
satisfy Synopsys so we have had to run both SpyGlass and Synopsys DC to
clean most RTL, but since Synopsys crashes at first occurrence on some
of this code it has been helpful to be able to find all occurrences.
Clock boundary crossing checks of asynchronous clocks has been an area
where our customers have asked for reports both from RTL and netlist.
We can not comment on the other tools, but this tool has definitely
made our work easier and faster.
- Jonathan Levi of Toshiba
SpyGlass pros:
1. Sorts VHDL files in correct order for compiling without errors.
Save a lot of time and effort getting the correct order without a
customer provided compile script. This feature is one I like the
best.
2. Syntax error check of RTL code.
3. Schematic viewer gives a quick overview of the design.
4. Check of multiple top level designs. The enables the user to
determines if there are any unused blocks if there is only 1 top
level.
5. Clock tracing and clock tree ability. This is another feature I
really like, being able to determine the clocks in the design.
SpyGlass cons:
1. Even though there are some checks for Synopsys coding styles, there
is not a template dedicated to checking Synopsys coding style.
2. Using the GUI, initial setting and changing the methodology and
Templates windows takes a long time for large RTL design. It even
takes longer for a gate level netlist.
3. The reports generated can get huge. It would be nice there is a
filter when generating reports.
- Roger Mar of Toshiba comments
I like Atrenta Spyglass myself.
- Gangadhar of DigiPro Design
Atrenta's Spyglass is quite intriguing. It offers a lot of potential.
We're evaluating it at the moment. Like most tools it suffers from a
poor UI. (One day EDA vendors will hire human factors folk and we'll all
be happier.)
- Kevin Jones of Rambus
I have been using SpyGlass for early checks on gate-level netlists, not
RTL. This is not the "sweet spot" of SpyGlass' intended usage, but we
still find it useful for getting quick results and identifying problems
quickly.
Useful info we get from SpyGlass includes:
- unclocked registers
- both edges of clock used
- unconnected inputs
- unregistered outputs
- cross-clock synchronization points
- unsynchronized cross-clock paths
These last two items identifying cross-clock points are particularly
useful. It identifies these cross-clock points MUCH faster than
primetime and it even categorizes whether-or-not it is synchronized by
the standard multi-flop method. It could be much more helpful if it
had the option of reporting the path between the clocks and not just
the start and end points.
It appears that SpyGlass could be put to good use for testing DFT
issues, but we have not pursued that yet.
- Dean Ranger of Toshiba
Regarding SpyGlass usage, I think that the tool has some margin to
become more mature: both for reading the code and the libraries and for
some complex rules. But Atrenta is very pro-active in fixing issues.
- Olivia Riewer of STmicroelectronics
|
|
