( DAC 03 Item 3 ) ----------------------------------------------- [ 01/20/04 ]
Subject: RunTime Flowtracer
THE FLOW KEEPER: RunTime sells a tool, Flowtracer, that documents and
automate your flow in a Tcl-like language. They claim it is better than
using "make" because it tracks file access as the job is being done. They
also have an LSF style job distribution feature that people like.
Runtime Design Automation sells network computing software like LSF but
says they have better failure analysis.
- John Weiland of Intrinsix
While we have been using RunTime for about a year, we have never had the
system fully loaded to a continual stress point nor have we poured into
every feature. How comfortable you are with TCL will define how much
you can dig and get out of Flowtracer. Usually I find that everything
is already there, just hidden from casual perusing.
Flowtracer-NC -- this is not the whole Flowtracer tool, but the batch
queue portion.
Flowtracer Highs
- Allows environments to be built in straight forward manner, that
provides tool isolation.
- We have been able to import foreign environments into our space
without having to resort to extensive shell scripting/programming.
- Batch system is straight forward.
- Painless to migrate from Platform.
- RunTime has been responsive to our requests for features to make
the migration easier (interactive mode).
- They are responsive to bugs/questions.
- Pricing and Functionality (you get a lot for the money).
- Not as confusing as Platform when it comes to batch daemons.
- Decent documentation
- Easy system reservation/lock out system.
Flowtracer Lows
- Initial installation can be confusing.
- Directory structure still needs work. (Their 7.X releases are
better and make more sense).
- Susceptible to vendor flexlm daemon changes.
- Command line UI can be a little confusing at times.
We have to create and maintain tool environments, and have migrated over
to Flowtracer being our tool environment setup tool.
- Winston Worrell of Microsoft
We built a computing farm with Flowtracer NC & a complicated enviromnent
including many EDA tools and heterogeneous machines. Flowtracer NC has
fairshare, resource control and policy configuration to optimize our job
priority and efficiency.
Flowtracer NC allows me to remotely control and summit my jobs in Taiwan
through their monitor GUI and the web access from Sunnyvale. The load
balance increases our throughput 3X compared with our orignal flow.
Flowtracer NC has vovflexlmd to communicate flexlm. However, they should
bundle Flexlm into one daemon. Otherwise, there will be a miss if a lot
of jobs are submitted inside and outside Flowtracer NC at the same time.
- Willis Shih of Faraday USA
RunTime Design Automation (RTDA) -- Flowtracer/EDA (FT/EDA)
Description:
Flowtracer/EDA is a resource management tool that tracks flows
consisting of jobs, files and the dependencies between them. It can
direct the parallel execution of jobs to bring files up-to-date
(a la Make) using any of the popular batch queuing systems. It has
command-line, X-Windows GUI and web-based interfaces to review and
control flows. The usage model has two main steps: first plan out a
flow by executing a TCL configuration file, second execute that flow
by triggering an update process. As the flow executes, it is possible
to monitor the progress as jobs pass, fail or wait to be executed.
Features:
1. Multiple approaches to capturing job/file dependencies. Our in-house
tool has a very simple dependency mechanism that is a little flaky;
Make's approach is not very flexible. My favorite technique is when
FT/EDA automatically captures file dependencies at runtime. We have
Make set up to do this for our C/C++ builds, but Flowtracer can do it
for most any kind of job. This eliminates a lot of the pain of manually
defining dependencies and of erroneous out-of-sync files due to missing
dependencies. Note that FT/EDA has to capture all file read/write system
calls to do this correctly.
I experienced some job crashes using runtime interception on Linux.
However, RTDA was quick about finding and fixing these bugs. As a
fallback, I used TCL templates to extract the obvious dependencies from
command-line arguments. At times, I also explicitly added I/O
dependencies to jobs to set up the basic skeleton of the flow, just
as you do with Make.
2. Automatically captures job details (stdout/stderr, status, cputime)
and makes them easy to review. Make doesn't do this; I wrote an in-house
tool in Perl to do the same kind of thing. When trying to track down
failures, complete trace information is invaluable. I also found
navigating both the GUI and the web interface after running a flow pretty
painless. (The command-line interface seems more suited to planning and
executing flows than monitoring and debugging them.)
3. Having a real programming language to describe flows is nice. I'm
sick of the quirky limitations of Make. FT/EDA has a predefined TCL
Flow API, but you can build on top of it or define your own API using
low-level procedures. FT/EDA is also more rigorous than Make. I
translated some Makefiles into FT/EDA flows, copying jobs and
dependencies one-to-one. I expected the flow to just run, but missing
dependencies were found by FT/EDA that caused the flow to halt. At
first I found this frustrating, but after awhile I got used to it. It
made me realize that I'm just not that good at identifying dependencies.
4. Integrates with your existing batch queuing system. FT/EDA uses a
TCL-based wrapper layer that can be customized to fit your installation.
RTDA had to do some custom work for us because they don't have LSF
in-house. RTDA has another tool called Flowtracer/NC that competes
with LSF. I haven't had a chance to do a complete comparison of the
two, but on some simple tests, FT/NC blew LSF away when dispatching
lots of short jobs.
5. Open (but still proprietary) implementation. While some of the key
components are binary, much of the implementation is written in TCL.
This makes customization and problem debugging within the system much
easier. It also provided a lot of source code examples for me as I was
learning TCL. I'm used to rolling my own tools, but with FT/EDA it
doesn't feel like I've given away my ability to be creative.
6. Complex tool. I would say using FT/EDA well is as difficult as
learning a new programming language. It is powerful and flexible and
open, but you have to give it a lot of information to get started. RTDA
could probably do a better job getting users to the first "Hello,
World!"-type flow. However, I think anything beyond that is always
going to take real mental investment. My experience with Make and Perl
was certainly that way.
7. You get what you pay for. We use a lot of open-source as well as
licensed tools in our environment. My bias is to purchase good,
well-supported infrastructure tools that can be integrated into our
existing environment. I don't like kitchen-sink tools that try to do
everything for me (often in ways I don't like), but also find the added
value of a commercial tool in its features, stability and support
generally justifies the cost. For me, FT/EDA fits right into my
preferred niche.
- [ An Anon Engineer ]
|
|
