The first Xilinx FPGA to support dynamic partial reconfiguration was the XC6200 series released in 1995. Its taken me nearly 30 years to try my first partially reconfigurable design, simply because the functions I've implemented never benefited from it before. The benefits come from having a sequence of logic functions, where each function is used for a reasonable amount of time before no longer being required. In other words the desire is to multiplex the use of the FPGA fabric over time with function switching taking 10-100 milliseconds instead of seconds for a full initial configuration. If you have a design that fits this profile there are SWaP-C gains to be made from partial FPGA reconfiguration.

• Introduction
• Glossary
• Experimental Design
• Code Structure
• Constraints
• Build Process
• RMs using Out of Context Synthesis
• RMs using In Context Synthesis
• Demonstration
• Conclusions
• References

Introduction

This is perhaps best done by Xilinx's own QuickTake video embeded below. My aim is to put their described process into action for the first time and create a repeatable design pattern that can be built via TCL scripts without manual intervention.

Glossary

Xilinx's user guide UG909 provides alternative ways to create the various designs and sub-designs required. I think their guide is a little lean on how to create the reconfigurable modules (partial sub-components), and heavy on the alternative linking methods. I detail two methods I used for the creation of reconfigurable modules, fully scripting my (perhaps naïvely) preferred method, for a really simple design. Note that Xilinx does provide an example design as part of their "Vivado Design Suite Tutorial: Dynamic Function eXchange", UG947, but the reference design is behind a Design License Agreement which you may not be authorised to agree, or not want to. The design I present will work on a Zybo (legacy) development board from Diligent. It was chosen as it was a simple step away from an existing design used for educational purposes, ScratchVHDL.

Experimental Design

The design needs to prove that when the dynamic or reconfigurable region is swapped, the static region is not overwritten. This means I need the static region to retain some state provided by the dynamic region between partial reconfigurations. I want a minimal design to prove the point at the expense of realism. So I won't be swapping floating point operators or counters in and out, just a few registers to hold a value for display.

For visual confirmation I have a dual seven segment display which is already proven to work. The aim is to have the RM currently in the RP drive the right digit on the display with a unique hexadecimal number. Then on a button press, save the same value to state in the static partition, which drives the left display digit. When the RM is swapped, the left digit retains the old value and the right digit gets a new value. A button press is used to save the value in the static partition for the next iteration.

Each RM is trivial, just a few registers that would have been optimised away by constant propagation if it were not for the reset changing their values. This design did however suffer from some optimisation that changed the interface between the RMs and the RP, and that needed to be resolved (covered later).

Code Structure

A neat way to swap RMs in and out of the RP might be to use different architectures for the same entity. The method of swapping the architecture employed would then be via a VHDL configuration. However I found Vivado was unhelpful here. While it can allow selection of a top level configuration, it managed to 'black box' the PLL IP and got into an unhelpful state. I assume because the tool now believed the PLL to be defined twice and got confused.

An alternative, and perhaps with an advantage, is to use a generic passed down the hierachy that can then conditionally instantiate the functionality of each RM. This may prove preferrable because the same generic can be used in a memory map to allow it to be read back to identify the current RM loaded in the RP, or used for other internal purposes. If the functions of the RMs are realised by existing code that do not match the interface of the RP, a wrapper is useful to provide that matching interface.

Constraints

Ensure the RMs' netlists or design checkpoints (DCP) you are attempting to stitch into the RP have the same entity specification. When linking and optimsation fail, open synth_rmx.dcp in a Zip file extractor, find the file reconfig_rm_stub.vhdl (names will vary for other designs). Note the specification of the entity (shown below). Constant propagation can "optimise" the signals on the interface so that interface will no longer match the hole it has to stitch into and after link_design, opt_design will fail.

-- Copyright 1986-2022 Xilinx, Inc. All Rights Reserved.
-- Copyright 2022-2023 Advanced Micro Devices, Inc. All Rights Reserved.
library IEEE;
use IEEE.STD_LOGIC_1164.ALL;

entity reconfig_rm is
  Port ( 
    clk : in STD_LOGIC;
    reset : in STD_LOGIC;
    incr : in STD_LOGIC;
    buttons : in STD_LOGIC_VECTOR ( 3 downto 0 );
    leds : out STD_LOGIC_VECTOR ( 3 downto 0 );
    display : out STD_LOGIC_VECTOR ( 3 downto 0 )
  );

end reconfig_rm;

architecture stub of reconfig_rm is
attribute syn_black_box : boolean;
attribute black_box_pad_pin : string;
attribute syn_black_box of stub : architecture is true;
begin
end;

set_property KEEP_HIERARCHY true [get_cells {reconfig_rp}]

The above property prevents the interface between RP and RM being optimised and hence mutated to be incompatible.

# Define the reconfigurable region of the device.
create_pblock pblock_rp
set crp [get_cells reconfig_rp]
set prp [get_pblocks pblock_rp]
add_cells_to_pblock $prp $crp
resize_pblock $prp -add {SLICE_X36Y50:SLICE_X43Y74}
set_property HD.RECONFIGURABLE true $crp
set_property SNAPPING_MODE {ON} $prp

WARNING: [Vivado 12-4775] Pblock ranges need to include all sites of a tile. The following ranges will be automatically aligned to a tile boundary: SLICE_X36Y50:SLICE_X43Y74 [<path>/DFX/constraints/impl.xdc:19]
WARNING: [Constraints 18-8777] Unable to split tiles. All required files are not available.

A pblock must be defined for the RP. Some care is required with the pblock range. I found it hard to satisfy and avoid the warning messages shown above. Even the tool's own derived regions did not please it either. The SNAPPING_MODE property is supposed to assist here.

SNAPPING_MODE

SNAPPING MODE property is applied on Pblocks to automatically adjust the size and shape. The property can be applied to Reconfigurable Partition Pblocks, to have them automatically adjusted to meet DFX floorplan requirements.

Vivado Design Suite Properties Reference Guide, UG912

For completeness the HD.RECONFIGURABLE is explained here too, as it modifies the properties on the pblock.

HD.RECONFIGURABLE

This initiates the Dynamic Function eXchange features in the software that are required to successfully implement a DFX design. The HD.RECONFIGURABLE property implies a number of underlying constraints and tasks:

• Sets DONT_TOUCH on the specified cell and its interface nets. This prevents optimization across the boundary of the module.
• Sets EXCLUDE_PLACEMENT on the cell's Pblock. This prevents static logic from being placed in the reconfigurable region.
• Sets CONTAIN_ROUTING on the cell's Pblock. This keeps all the routing for the RM within the bounding box. Enables special code for DRCs, clock routing, etc.

Vivado Design Suite User Guide: Dynamic Function eXchange, UG909

The necessity of these parameters on the pblock are hopefully obvious!

Build Process

In short, the plan is to create an initial design, that's a configuration with the static partition and the first reconfigurable module, RM1. Save the initial, static and RM1 partitions from an implemented design. The subsequent RMs must now match the 'hole' in the static partition left by removing RM1. That means not just the same interface (VHDL ports), but also the net locations in the FPGA fabric. Hence why the partitions are saved from an implemented design. The problem is that subsequent RMs are still HDL code. They each need to be synthesised to a netlist, linked into the implemented design, then place and route performed to marry up the nets. When saving the static partition, the RM needs to be 'black boxed' and the routing locked.

update_design -cell [get_cells {reconfig_rp}] -black_box
lock_design -level routing
write_checkpoint -force "$src_dir/static.dcp"
write_bitstream -force "$src_dir/static.bit"

RMs using Out of Context Synthesis

To see the following in context open the TCL build script.

With this method I was able to synthesis a second RM as a separate Vivado project. The constraints need to be relevant to the RM, and ideally input and output constraints specified for the interface. Synthesis then needs to use synth_design -mode out_of_context to create the RM's netlist.

I found this awkward because:

• Code is split into separate projects for each RM. That's a management overhead.
• Out of context constraints need to be derived for the RP, but at least can be re-used across all RM projects. You might consider using a method like Specifying Boundary Timing Constraints in Vivado to derive these constraints.
• These Out of context constraints that are externally and manually specified might be the cause of timing issues.
• The build is split across separate project instances, so more involved to script.

Essentially this method worked. And UG909 provides four methods to manage the linking.

RMs using In Context Synthesis

With this code organisation, an integer generic is incremented for each partition for simplicity. The model above replaces the RP with each subsequent RM, completes implementation and then writes out the netlist for just the RM. The implementation of the static portion remaining locked. The advantage of this method is that:

• Timing across the interface is managed by Vivado rather than an additional constraints file whose specification might not be quite right.
• Easily scripted in a loop for each RM.
• Runtime should be no worse than for out of context synthesis.

The implemented static partition is locked, so it is not the case that implementation is repeated from the start, only each RMs has to be implemented, hence the assertion about the run time.

set crp [get_cells {reconfig_rp}]
foreach rm {2 3 4} {
  set_property generic rm_num_g=$rm [current_fileset]
  reset_run synth_1
  launch_runs synth_1 -jobs 14
  wait_on_run synth_1
  open_run synth_1
  write_checkpoint -force -cell $crp "$src_dir/synth_rm${rm}.dcp"
  # Create a new Vivado instance briefly for RM
  create_project -part xc7z010clg400-1 -in_memory "Stitch RM${rm}"
  add_files "$src_dir/static.dcp"
  add_files "$src_dir/synth_rm${rm}.dcp"
  set_property SCOPED_TO_CELLS {reconfig_rp} [get_files "$src_dir/synth_rm${rm}.dcp"]
  link_design -top {zybo_z7_10} -part xc7z010clg400-1 -reconfig_partitions {reconfig_rm}
  opt_design
  place_design
  route_design
  write_bitstream -force -cell $crp "$src_dir/rm${rm}.bit"
  close_project
}

Demonstration

To demonstrate the reconfiguration, the following script is run in a TCL shell outside of Vivado. The main reason being that the TCL procedure pause does not work inside Vivado, and the TCL shell is lighter weight with a faster start up time.

set prods {<path>/DFX/products}

# Suppress "INFO: [Labtools 27-1434] Device xc7z010 (JTAG device index = 1) is programmed with a design that has no supported debug core(s) in it."
set_msg_config -suppress -id {Labtools 27-1434}
# Suppress: "INFO: [Labtools 27-3164] End of startup status: HIGH"
set_msg_config -suppress -id {Labtools 27-3164}

# https://stackoverflow.com/questions/18993122/tcl-pause-waiting-for-key-pressed-to-continue
proc pause {{message "Enter to continue"}} {
    puts -nonewline $message
    flush stdout
    gets stdin
}

proc program {f} {
  set hwd [get_hw_devices xc7z010_1]
  set_property PROBES.FILE {} $hwd
  set_property FULL_PROBES.FILE {} $hwd
  set_property PROGRAM.FILE $f $hwd
  program_hw_devices $hwd
  refresh_hw_device [lindex $hwd 0]
}

open_hw_manager
connect_hw_server
open_hw_target
current_hw_device [get_hw_devices xc7z010_1]
refresh_hw_device -update_hw_probes false [lindex [get_hw_devices xc7z010_1] 0]

# Full / Initial
program "$prods/initial.bit"
pause "Initial BIT file loaded, Enter to continue"

foreach rm {2 3 4} {
  # Partial Reconfigure
  program "$prods/rm${rm}.bit"
  pause "Partial BIT file RM${rm} loaded, Enter to continue"
}

foreach rm {1 2 3 4} {
  # Partial Reconfigure
  program "$prods/rm${rm}.bit"
  pause "Partial BIT file RM${rm} loaded, Enter to continue"
}

# Static Revert gives "F" hex, nets must float high
program "$prods/static.bit"
puts "Static BIT file with black box"
pause "THE END, Enter to exit"

close_hw_manager
exit

The script above produces a sequence like the following illustration (shortened for brevity).

The final display is for the static partition with RP back boxed. The left digit has been initialised to 0. The right digit suggests that the unconnected nets are pulled up so the binary vector "1111" is displayed as F.

Conclusions

This blog illustrates an extremely simple reconfigurable design featuring:

• VHDL code structure
• Scripted synthesis and implementation
• Scripted demonstration

References

1. Github Source Code
2. Vivado Design Suite User Guide: Dynamic Function eXchange, UG909
3. Vivado Design Suite Dynamic Function eXchange Design Hub, DH230
4. How to take advantage of partial reconfiguration in FPGA designs, EE Times June 2006
5. FPGA Dynamic and Partial Reconfiguration: A Survey of Architectures, Methods, and Applications, ACMComputing Surveys, Vol. 51, No. 4, Article 72. Publication date: July 2018.