Category

Why?

Writing and maintaining tests for VHDL designs can be a very time consuming and tedious activity. The sole act of running simulations one by one can take up loads of time. Multiple test benches are usually needed to cover various test cases and scenarios. Frequent design changes will therefore keep you busy constantly updating numerous VHDL files. With today’s agile development style, keeping your tests maintainable is an important issue to keep in mind.

Time factor aside, writing or even understanding tests for VHDL designs takes VHDL skills. A test engineer with little to no VHDL experience will have a hard time writing up even simple test sequences. And let’s be honest, VHDL is a language for describing hardware structures and behavior, and is ill-suited for pretty much everything else.

How?

In order to solve or at least alleviate some of these problems you can use Tcl files and batch scripts to build a sort of test framework. That’s exactly what we did for one of our clients projects.

VHDL Test Framework Overview

We wrote test benches for each design under test (DUT) that don’t contain actual test cases, but instead implement a library of functions, such as wait statements, logger functions, getters (validate response) and setters (apply stimuli). The actual test cases are written as text files (see code block below), using clear and easy to understand syntax, which is parsed and applied to the corresponding functions within the test bench. Decoupling the actual test case from the VHDL test bench makes it much easier to implement new test cases, as well as adjust existing ones. Furthermore, it keeps the effort needed to update the test bench to reflect design changes to a minimum.

Input File (describing test case)

FREQ Clock 1600

SET Reset 1
SET Input 1
SET OutEnable 1

WAIT FOR ms 5

VALIDATE ComparatorOut 1

WAIT FOR ms 5

SET Input 0

WAIT FOR ms 5

VALIDATE ComparatorOut 0
VALIDATE FrequencyOut 100

END

Log File

===============================================================
 Date:           12.04.2019 - 13:22:25
 Stimulus file:  $VHDL_TEST_DIR\tb_DUT\io_files\input\sim1.txt
===============================================================
     Sim time | Message
---------------------------------------------------------------
         0 ns   'Clk' frequency set to 1600Hz
         0 ns   'Reset' set to '1'
         0 ns   'Input' set to '1'
         0 ns   'OutEnable' set to '1'
   5000000 ns   VALIDATION 'ComparatorOut' (expected: '1'; actual: '1')                         Check passed
  10000000 ns   'Input' set to '0'
  15000000 ns   VALIDATION 'ComparatorOut' (expected: '0'; actual: '0')                         Check passed
  15000000 ns   FREQUENCY VALIDATION 'FrequencyOut' (expected: '100Hz'; actual: '100.000Hz')    Check passed
  15000000 ns   ++++++++++++++++end of simulation+++++++++++++++++
-------------   Total Checks = 2;  Failed = 0

Tcl File (ModelSim Do File)

#############################################################
# Add libraries and compile necessary files
#############################################################

# For all Testbenches
vlib sim_lib
vcom -2008 -explicit -work sim_lib ../../../../shared_lib/report_pkg.vhd
vcom -2008 -explicit -work sim_lib ../../../../shared_lib/parser_pkg.vhd
vcom -2008 -explicit -work sim_lib ../../../../shared_lib/basic_functions_pkg.vhd
vcom -2008 -explicit -work sim_lib ../../../../shared_lib/clock_gen.vhd

# DUT specific
vlib CS_lib
vcom -93 -explicit -work CS_lib $srcPath/SDC/Controllogic_svn4892/CS.vhd

# Testbench specific
vlib $libName
vcom -2008 -explicit -work $libName ../hdl/tb_CS.vhd

#############################################################
# Prepare output files
#############################################################

for {quietly set simCase $startSimCase} {$simCase <= $stopSimCase} {incr simCase} {
    # build path and file name
    quietly set logFile $outputPath/log$simCase.txt
    # create file
    echo "Empty output file created: $logFile"
    catch {quietly set outputLog [open $logFile "w+"]
           close outputLog
    }
}

#############################################################
# Run simulation for specified TCs
#############################################################

for {quietly set simCase $startSimCase} {$simCase <= $stopSimCase} {incr simCase} {

    echo "--------------------------------------------------------------"
    echo "Running simulation for TC #$simCase"

    # Gg_simCase sets generic parameter g_Simcase
    # -l writes simulation log to sim_log.txt
    # -t sets simulation time resolution
    vsim -l $simLog -t $timeRes -Gg_simCase=$simCase $simUnit

    # apply system time stimuli to simDateTime VHDL signals
    force simDateTime(1 TO [expr $timeStrLen]) [string range $timeStr 0 [expr $timeStrLen -1]]

    # get rid of warnings at 0 ns
    quietly set StdArithNoWarnings 1
    quietly set NumericStdNoWarnings 1
    run 0 ns;
    quietly set StdArithNoWarnings 0
    quietly set NumericStdNoWarnings 0

    if ($gui) {
        view objects
        view locals
        view source
        view wave -undock
        add wave *
        property wave -radix unsigned /tb_cs/AdressCSxSI
        property wave -radix unsigned /tb_cs/AdressGeneralxSI
        run -all
        wave zoom full;
    } else {
        run -all
        quit -sim
    }
}

echo "--------------------------------------------------------------"

Running Multiple Simulations

In order to make running multiple simulations easy, a Tcl script is written for every DUT that applies simulation settings, compiles the necessary source files, runs test cases in ModelSim and creates a summary of passes/fails of the run test cases. If you need to run simulations for multiple DUTs, a simple batch script can call the specified Tcl files one after the other within ModelSim.

rem ###############################################################################
rem # Execute do file in ModelSim, forwarding the parameters passed to batch script
rem ###############################################################################

if %gui%==true (
    rem run w/ GUI
    vsim -l sim_setup_log.txt -do "do ../scripts/do_run_tb_CS.tcl %*"
) else (
    rem run in console
    vsim -c -l sim_setup_log.txt -do "do ../scripts/do_run_tb_CS.tcl %*"
)

Conclusion:

In the long run, building a test framework for your VHDL design can save you lots of time and effort. To sum up some of its major advantages:

• Creating, altering and understanding test cases is very easy and doesn’t require knowledge of VHDL
• Multiple simulations can be run in sequence using Tcl scripts
• Tests for multiple DUTs can be simulated in sequence using batch scripts
• The test bench becomes more maintainable