In recent blogs - specman-callback-coverage-api and improving-tests-efficiency-using-coverage-callback - we shared some ideas about how to employ the new Coverage Callback API for increasing the tests efficiency. This blog shows two more ideas for improving tests, using runtime coverage information.

Stop the run after goal is reached

The basic concept of Coverage Driven Verification is running many random tests, and every once in a while analyze the coverage report, and if there are holes – write tests directed at this hole. These new tests have a very specific goal, hitting the coverage hole, a specific cover item bucket is expected to be sampled. You write constraints and sequences that are expected to steer the test to the uncovered area and after the test ends you look at the coverage report and see if you succeeded.

Wouldn’t it be nice to be notified during the run when the test reaches its goal? You can decide to stop the test, you can also decide to add extra checks or messages at that point.

If the goal is “get the out_of_mem event to be emitted”, it’s easy to add code waiting for this event, no need for new features. But if the goal is defined as a combination of multiple attributes, it gets harder to implement a code waiting for this combination. In addition – it would require writing this code for each new test aiming at a different combination. But there is no need to write new code, you already have it – it is defined in the coverage model, using crosses and transitions.

Assume this coverage definition –

cover cover_me is  {
    item legal : bool   = cur_trans.legal;
    item kind  : kind_t = cur_trans.kind;
    item address : uint (bits : 16) =
             cur_trans.address using ranges = {
        range([0..10], "low_address");
        range([11..0xfff0], "mid_address");
        range([0xfff1..0xffff], "high_address");
        };
    cross legal, kind, address;
    };
};

And assume that the coverage analysis indicates that we never had a legal transaction of type D with its address in the high_address range. In terms of coverage, it means that we want the test hits the bucket TRUE_D_high_address of the cross item. Using the Coverage Callback API we can query the coverage sampling and notify when this bucket is hit.

struct cb_notify_cover_hit like cover_sampling_callback {
  event bucket_hit; 
  do_callback() is only {       
    var cr          : rf_cover_group = get_current_cover_group();
    var item_value  : string;

    for each (item) in cr.get_all_items() {    
      if item is a rf_cross_cover_item (cross_item) {
        if cross_item.get_name() == "cross__legal__kind__address" {
          item_value =
                  str_join(get_simple_cross_sampled_bucket_name(
                           cross_item), "__");
          if item_value == "TRUE__D__high_address" {
            message(LOW, "Sampled desired value - item ",
                    item.get_name(), " == ", item_value );
          };
          emit bucket_hit;
        };
      };
    };
  };
};

// For example, one can stop the run after
// getting the bucket_hit event
extend env {
    on my_cb.bucket_hit {
        start stop_after_delay();
    };
    stop_after_delay() @clock is {
        wait [drain_time] * cycle;
        stop_run();
    };
};

Note that if the coverage model definition is changed, for example – the ranges are defined in a different way, and “high_address” is defined with different values – no need to re-edit the callback code.

This is a very tailored piece of code, as it looks for a very specific bucket name. The open source on github, cover_cb_report_hit, implements a reusable utility that you can download and use. In this utility, one can register the name/s of the bucket/s in interest – and the do_callback() checks whether the sampled item is in the list of items of interest. If it is – it checks the bucket name.