Test Framework for VHDL Designs
Why?
Writing and maintaining tests for VHDL designs can be a very time consuming and tedious activity. The sole act of running simulations one by one can take up loads of time. Multiple test benches are usually needed to cover various test cases and scenarios. Frequent design changes will therefore keep you busy constantly updating numerous VHDL files. With today’s agile development style, keeping your tests maintainable is an important issue to keep in mind.
Time factor aside, writing or even understanding tests for VHDL designs takes VHDL skills. A test engineer with little to no VHDL experience will have a hard time writing up even simple test sequences. And let’s be honest, VHDL is a language for describing hardware structures and behavior, and is ill-suited for pretty much everything else.
How?
In order to solve or at least alleviate some of these problems you can use Tcl files and batch scripts to build a sort of test framework. That’s exactly what we did for one of our clients projects.
VHDL Test Framework Overview
We wrote test benches for each design under test (DUT) that don’t contain actual test cases, but instead implement a library of functions, such as wait statements, logger functions, getters (validate response) and setters (apply stimuli). The actual test cases are written as text files (see code block below), using clear and easy to understand syntax, which is parsed and applied to the corresponding functions within the test bench. Decoupling the actual test case from the VHDL test bench makes it much easier to implement new test cases, as well as adjust existing ones. Furthermore, it keeps the effort needed to update the test bench to reflect design changes to a minimum.
Input File (describing test case)
FREQ Clock 1600 SET Reset 1 SET Input 1 SET OutEnable 1 WAIT FOR ms 5 VALIDATE ComparatorOut 1 WAIT FOR ms 5 SET Input 0 WAIT FOR ms 5 VALIDATE ComparatorOut 0 VALIDATE FrequencyOut 100 END
Log File
===============================================================
Date: 12.04.2019 - 13:22:25
Stimulus file: $VHDL_TEST_DIR\tb_DUT\io_files\input\sim1.txt
===============================================================
Sim time | Message
---------------------------------------------------------------
0 ns 'Clk' frequency set to 1600Hz
0 ns 'Reset' set to '1'
0 ns 'Input' set to '1'
0 ns 'OutEnable' set to '1'
5000000 ns VALIDATION 'ComparatorOut' (expected: '1'; actual: '1') Check passed
10000000 ns 'Input' set to '0'
15000000 ns VALIDATION 'ComparatorOut' (expected: '0'; actual: '0') Check passed
15000000 ns FREQUENCY VALIDATION 'FrequencyOut' (expected: '100Hz'; actual: '100.000Hz') Check passed
15000000 ns ++++++++++++++++end of simulation+++++++++++++++++
------------- Total Checks = 2; Failed = 0
Tcl File (ModelSim Do File)
#############################################################
# Add libraries and compile necessary files
#############################################################
# For all Testbenches
vlib sim_lib
vcom -2008 -explicit -work sim_lib ../../../../shared_lib/report_pkg.vhd
vcom -2008 -explicit -work sim_lib ../../../../shared_lib/parser_pkg.vhd
vcom -2008 -explicit -work sim_lib ../../../../shared_lib/basic_functions_pkg.vhd
vcom -2008 -explicit -work sim_lib ../../../../shared_lib/clock_gen.vhd
# DUT specific
vlib CS_lib
vcom -93 -explicit -work CS_lib $srcPath/SDC/Controllogic_svn4892/CS.vhd
# Testbench specific
vlib $libName
vcom -2008 -explicit -work $libName ../hdl/tb_CS.vhd
#############################################################
# Prepare output files
#############################################################
for {quietly set simCase $startSimCase} {$simCase <= $stopSimCase} {incr simCase} {
# build path and file name
quietly set logFile $outputPath/log$simCase.txt
# create file
echo "Empty output file created: $logFile"
catch {quietly set outputLog [open $logFile "w+"]
close outputLog
}
}
#############################################################
# Run simulation for specified TCs
#############################################################
for {quietly set simCase $startSimCase} {$simCase <= $stopSimCase} {incr simCase} {
echo "--------------------------------------------------------------"
echo "Running simulation for TC #$simCase"
# Gg_simCase sets generic parameter g_Simcase
# -l writes simulation log to sim_log.txt
# -t sets simulation time resolution
vsim -l $simLog -t $timeRes -Gg_simCase=$simCase $simUnit
# apply system time stimuli to simDateTime VHDL signals
force simDateTime(1 TO [expr $timeStrLen]) [string range $timeStr 0 [expr $timeStrLen -1]]
# get rid of warnings at 0 ns
quietly set StdArithNoWarnings 1
quietly set NumericStdNoWarnings 1
run 0 ns;
quietly set StdArithNoWarnings 0
quietly set NumericStdNoWarnings 0
if ($gui) {
view objects
view locals
view source
view wave -undock
add wave *
property wave -radix unsigned /tb_cs/AdressCSxSI
property wave -radix unsigned /tb_cs/AdressGeneralxSI
run -all
wave zoom full;
} else {
run -all
quit -sim
}
}
echo "--------------------------------------------------------------"
Running Multiple Simulations
In order to make running multiple simulations easy, a Tcl script is written for every DUT that applies simulation settings, compiles the necessary source files, runs test cases in ModelSim and creates a summary of passes/fails of the run test cases. If you need to run simulations for multiple DUTs, a simple batch script can call the specified Tcl files one after the other within ModelSim.
rem ###############################################################################
rem # Execute do file in ModelSim, forwarding the parameters passed to batch script
rem ###############################################################################
if %gui%==true (
rem run w/ GUI
vsim -l sim_setup_log.txt -do "do ../scripts/do_run_tb_CS.tcl %*"
) else (
rem run in console
vsim -c -l sim_setup_log.txt -do "do ../scripts/do_run_tb_CS.tcl %*"
)
Conclusion:
In the long run, building a test framework for your VHDL design can save you lots of time and effort. To sum up some of its major advantages:
- Creating, altering and understanding test cases is very easy and doesn’t require knowledge of VHDL
- Multiple simulations can be run in sequence using Tcl scripts
- Tests for multiple DUTs can be simulated in sequence using batch scripts
- The test bench becomes more maintainable
