( ESNUG 578 Item 3 ) ---------------------------------------------- [12/01/17]
Subject: Hands-on PSS user explains CDNS Perspec vs. Breker TrekSoC (pt 3)
In part 3, I will discuss PSS Coverage and Accelera PSS.
- [ Ant-Man, the Movie ]
PSS user explains CDNS Perspec vs. Breker TrekSoC (pt 2)
From: [ Ant-Man, the Movie ]
Hi John,
And here is my "part 3" on PSS Coverage and Accelera PSS..
- [ Ant-Man, the Movie ]
---- ---- ---- ---- ---- ---- ----
G. PSS COVERAGE
A critical aspect of verification is always understanding your coverage.
There are two main types of coverage:
- Pre-Runtime Coverage
This is calculating your coverage before you exercise your tests. The
idea is that you will know the full spectrum of your coverage before you
run a single test case.
By definition, PSS pre-runtime coverage only exists for automatically
generated tests.
Example: Let's say you have a PCI transfer action. The PCI transfer
action has multiple parameters which are part of actions. Let's take
these 3 parameters:
- Which processor is doing the transfer?
- Which memory the transfer is going to?
- What's your memory size?
For every parameter you have multiple options: 5 different processor
options, 10 different memory options, and 20 different options for
memory size.
Pre-runtime coverage shows your coverage with respect to all those
parameters for each action. You can define your parameters and the
PSS tool will automatically define your coverage in advance:
- Did every processor get exercised or not?
- Did detection go to every memory?
- Did every processor go to every memory size?
- Runtime Coverage
This is collecting your coverage after you've run all your tests.
There are two kinds of runtime coverage, basic and advanced:
1. Basic runtime coverage is collecting the same coverage that was
defined for pre-runtime and making sure that it is hit. You
monitor whether the actions got executed or not. And if they
did get executed, what the parameters were, etc.
If your verification environment and everything else were perfect,
your pre-runtime and runtime coverage would match or correlate
very closely to each other. But what if something happens during
simulation and an action got skipped and was never executed? This
is why runtime coverage is an important check for us.
2. Advanced runtime coverage is created using information from
your test, design, and testbench live and while it's happening
during simulations.
All the information needed for runtime coverage calculations is dumped
in your simulation log files. So runtime coverage could be calculated
manually by processing the log files -- you could write a parser, and
do some scripting to figure out whether the processor executed
everything or not. But it would take a lot effort.
Cadence Perspec
Perspec has broader coverage functionality than Breker. It calculates
your pre-runtime coverage for a scenario when the tests are first
generated (Cadence calls this "Gen-Time" coverage), and then it
calculates your coverage after your tests are run (runtime coverage).
The Cadence SLN language lets you define PSS coverage that is almost
SystemVerilog-like, i.e. like item coverage, cross coverage, etc.
Since Perspec is integrated into Cadence VManager, this PSS coverage
gets reported along with your other coverage -- you see it as one single
coverage report in VManager regardless of where it was created.
- Pre-Runtime Coverage ("Gen-Time" coverage)
Perspec's Gen-Time coverage is more elaborate than Breker's, plus
it is more integrated into the standard CDNS coverage flow.
- Runtime Coverage
Perspec has both basic runtime coverage (as described above) and
advanced runtime coverage.
Perspec's *advanced* runtime coverage uses live monitoring of the
activities within the tests, as well as the design and testbench.
Perspec will show you:
- when certain transactions have started
- when certain transactions have completed
- if other transaction you want to make happen in between
have happened
You can define not only the coverage of an action, but also the
coverage across various actions, e.g. you can use Perspec to
determine whether the action happened, and if it did, while it was
executing, did these other actions happen together?
Example: You want to make sure your FIFO level got executed.
And you also want to know when your FIFO was executed, did
other transactions get tied up?
With Perspec you can code for any of these arbitrary, more complex,
white-box kinds of coverage. The events could come directly from
your C code, or from monitoring your actual design -- and you can
extrapolate those events and create coverage from it. You can
create cross-coverage or simple event coverage.
Let's say you want to monitor at very low-level detail:
- whether something specific happens within the action
- and when that specific activity happens, whether
something else happened or not
Perspec's advanced runtime coverage post-processes your log file,
and calculates your coverage out of it. (With Gen-Time, you don't
need any logs because you haven't run your simulations, etc. yet.)
Perspec basically monitors the events from your log data (either
from C or Verilog) and puts some messaging out in a certain format
('prints it') into your log files.
Example: You want to check whether a data transfer of over 20 MB
size happens when your FIFO buffer is between 70% and 100% full.
Here's how Perspec does this.
1. You define a coverage group in Perspec, where one of the
items is your FIFO buffer size (e.g. 10%, 20%, 30%...).
2. Then, whenever your FIFO reaches any of those buffer
levels (10%, 20% ...), have Perspec 'prints' that
information into your log file.
3. In parallel, you create an event called "XYZ", which says
if transactions sized over 20 MB have happened. Have
Perspec do a 'prints' to the log file each time it happens.
4. The coverage you're looking for is when your FIFO activity
was between 70-100% and over 20 MB. Based on your log data,
you will be able to figure out if when your FIFO hit 70%,
did this happen correctly or not.
You may have 50 different things that you want to monitor like this.
Doing this advanced kind of complex system coverage was not feasible
before Portable Stimulus was invented.
Perspec's advanced runtime coverage is the main difference between
us verifying an entire complex SoC vs. us verifying just cores.
Technically, yes, you could theoretically write a post script to
monitor events and assemble all the data. You could then look at
various pieces together and figure out where your coverage holes
are. And theoretically, you could write a functional coverage group.
But it will only work for so long. And it all becomes very complex
and unsustainable.
Breker TrekSoC
Breker also has pre-runtime coverage, but it was a little bit more
simplistic in nature.
When I last looked at Breker a few years ago, they also had basic
runtime coverage, too, but nothing like Perspec's advanced runtime
coverage.
This might have changed. I'm not the guy to tell you about it.
You'll need a Breker user to update you on their coverage, John.
---- ---- ---- ---- ---- ---- ----
H. ACCELERA PSS LANGUAGE SUPPORT PLANS
Accellera is working on its PSS standard, which is expected to be finalized
in 2018. Both Cadence and Breker have committed to supporting *both* PSS
syntaxes: DSL and C++.
Once PSS is approved, it will have some built-in constructs for vendors to
add new functionality. It will then be up to each vendor to decide what
they want to support directly... or to put in hooks for the users to do
it for themselves. (An example of this would be for memory management,
which isn't PSS, but that both Cadence and Breker already support.)
The PSS being approved will also (theoretically) let us use our generated
tests from one tool vendor in another vendor's environment.
---- ---- ---- ---- ---- ---- ----
For us, Cadence Perspec is the best portable stimulus tool for our complex
SoCs. Compared to Breker, in Perspec it's easier / more flexible to model,
easier to specify test intent, it has better (and *integrated*) debug and
coverage -- plus more advanced coverage. Also the Perspec GUI works.
We've been happy with Perspec. It's saved us man-months per project. Plus,
with Perspec we've had a flood of extra test cases that we wouldn't have had
otherwise, due to our limited staffing and time constraints.
- [ Ant-Man, the Movie ]
---- ---- ---- ---- ---- ---- ----
Related Articles
Hands-on PSS user explains CDNS Perspec vs. Breker TrekSoC (pt 1)
Hands-on PSS user explains CDNS Perspec vs. Breker TrekSoC (pt 2)
Hands-on PSS user explains CDNS Perspec vs. Breker TrekSoC (pt 3)
Cadence Perspec wins #1 in #1 "Best of 2017" with Portable Stimulus
Breker TrekSoC takes #2 in #1 "Best of 2017" with Portable Stimulus
Mentor InFact was pretty much an early No Show in the PSS tool wars
Join
Index
Next->Item
|
|
