No one is invincible!

Most organizations use project health as a way to manage trade-off between development time, cost, and quality. Various measurements are used to access the performance or effectiveness of a project team. Without measurements they would be subjective guess. However traditional methods like measure test coverage or measuring design stability are falling short because of fundamental changes that are occurring in electronic design. Here are some of the top trends in design and the impact they have on tracking project health.

1. It’s an intelligent mesh, not just a flow.

There is no longer a sequential design flow in electronic design: there is architectural, layout, and verification exploration. Layout and verification often start before architecture and implementation details are finished. IP blocks further parallelize, loop and interconnect the project flow. With a mesh development, the next extremely difficult question is resource allocation. How do you apply people, machine, licenses, and testing runs? What stage needs more resources? What will be the resource impact on development time, cost and quality?

2. Global teams

Expect more “follow the sun” design work. However for distributed engineering groups, it can be difficult to manage and track progress. Global teams must share data, status, and issues. Project team can no longer rely on self report to get a picture of where things are. Teams must find ways to automate sharing status and escalating and brainstorming on issues.

3. Multiple dimensions of analysis

Engineering is managing constraints and trade-offs. A workable plan must balance cost, performance, schedule and quality to develop a useful design. Whole teams are dedicated to power, performance, fault, routing and timing. Each group needs to communicate and escalate optimization and trade-off decisions.

4. Runtime jobs growing faster than transistor count

Sure, transistors counts are growing but the jobs running analysis are exploding. Teams routinely run many hundreds of builds and automated tests. Expect to see this trend continue. How are you going to manage it? Are you getting best use of software licenses and CPU resources? Are you spending more on licenses than hardware?

5. Summarizing and Analysis of More Testing:

As system size grows, manual testing typically cannot keep up. So everyone is turning to test-data generation. But test generation–regardless of the framework used–requires manual sorting and analysis of the test results. Developers must wade though an ocean of data looking for warnings and errors. The result is that–if you don’t automate the data collection and analysis–you will spend all your time grepping log files. If you add more CPU resources and run more jobs, can you analyze the output by hand? You must automate the analysis of the results as well after a certain point or further test generation is pointless.

Why Achilles Test Systems?

Achilles Test Systems’ DV Notebook is a flexible repository for collecting, analyzing and summarizing a large amount of test data from multiple sources. We excel in multi-vendor environments. The DV Notebook’s visualization enables users to sift through test results, categorize failures, and highlight areas of concern. It draws attention to the key points while retaining the relevant context and the lower level details.

The DV Notebook can parse log files and extract key pieces of information such as test name, test description, time of execution, duration of test, key parameters, random seed values, error messages, and pass/fail status. Built-in templates track verification testing and manage regression suites out of the box. They offer a starting point that can be customized to address your exact situation, extracting the precise data you need and working with the tools in your flow. Collecting key information and storing it in an organized and easily accessible dashboard saves time, avoids errors, and facilitates re-use.

About Our Organization
Achilles Test Systems products and services enable development teams to correlate results across multiple project data files, cutting debug time in a collaborative development process. Project status is always available and up to date with automatically generated tables and charts to highlight trends from historical data. Every member of a global team just needs a web browser to contribute insights and to access an integrated visualization of seed tracking, test data, and source-code revision history.

This entry was posted on Friday, May 29th, 2009 at 10:48 am and is filed under Engineering Practices. You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.