Modules

Interface

This document contains all the information needed to configure modules for your f4pga project as well as some info about the API used to write modules.

Configuration interface:

Modules are configured through an internal API by f4pga. The basic requirement for a module script is to expose a class with Module interface.

f4pga reads its configuration from two different sources: platform’s flow definition, which is a file that usually comes bundled with f4pga and project’s flow configuration, which is a set of configuration options provided by the user through a JSON file or CLI interface.

Those sources contain snippets of module configurations.

A module configuration is a structure with the following fields:

  • takes - a dictionary that contains keys which are names of the dependencies used by the module. The values are paths to those dependencies. They can be either singular strings or lists of strings.

  • produces - a dictionary that contains keys which are names of the dependencies produced by the module. The values are requested filenames for the files generated by the module. They can be either singular strings or lists of strings.

  • values - a dictionary that contains other values used to configure the module. The keys are value’s names and the values can have any type.

Platform-level configuration

In case of platform’s flow definition, a values dictionary can be defined globally and the values defined there will be passed to every module’s config.

Those values can be overridden per-module through module_options dictionary.

Parameters used during module’s construction can also be defined in module_options as params (those are not a part of module configuration, instead they are used during the actual construction of a module instance, before it declares any of its input/outputs etc.. This is typically used to achieve some parametrization over module’s I/O).

Defining dictionaries for takes and produces is currently disallowed within platform’s flow definition.

For examples of platform’s flow definition described here, please have a look at f4pga/platforms/ directory. It contains platform flow definitions that come bundled with f4pga.

Project-level configuration

This section describes project’s flow configuration.

Similarly to platform’s flow definition, values dict can be provided. The values provided there will overwrite the values from platform’s flow definition in case of a collision.

Unlike platform’s flow definition, project’s flow configuration may contain dependencies dict. This dictionary would be used to map symbolic dependency names to actual paths. Most dependencies can have their paths resolved implicitly without the need to provide explicit paths, which is a mechanism that is described in a later section of this document. However some dependencies must be provided explicitly, eg. paths to project’s Verilog source files. It should be noted that depending on the flow definition and the dependency in question, the path does not necessarily have to point to an already existing file. If the dependency is a product of a module within the flow, the path assigned to it will be used by the module to build that dependency. This is also used to in case of on-demand dependencies, which won’t be produced unless the user explicitly provides a path for them.

project’s flow configuration cannot specify params for modules and does not use module_options dictionary. Neither it can instantiate any extra stages.

Any entry with a couple exceptions* is treated as a platform name. Enabling support for a given platform within a project’s flow configuration file requires having an entry for that platform. Each of those entries may contain dependencies, values fields which will overload the dependecies and values defined in a global scope of project’s flow configuration. Any other field under those platform entries is treated as a stage-specific-configuration. The key is a name of a stage within a flow for the specified platform and the values are dicts which may contain dependencies and values fields that overload dependencies and values respectively, locally for the stage. Additionally a default_target field can be provided to specify a default target to built when the user does not specify it through a CLI interface.

The aforementioned *exceptions are:

  • dependencies - dependencies shared by all platforms.

  • values - values shared by all platforms

  • default_platform - default platform to chose in case it doesn’t get specified by the user

Those apply only to flow configuration file.

Internal environmental variables

It’s very useful to be able to refer to some data within platform’s flow definition and project’s flow configuration to either avoid redundant definitions or to store and access results of certain operations. f4pga allows doing that by using a special syntax for accessing internal environmental variables.

The syntax is ${variable_name}. Any string value within platform’s flow definition and project’s flow configuration that contains such patterns will have them replaced with the values of the variables referenced if those values are strings. Eg.:

With the following values defined:

{
  "a_value": "1234",
  "another_value": "a_value: ${a_value}"
}

another_value will resolve to:

"a_value: 1234"

If the value is a list however, the result would be a list with all entries being the original string with the reference to a variable replaced by following items of the original list. Eg.:

With the following values defined

{
  "list_of_values": ["a", "b", "c"],
  "some_string": "item: ${list_of_values}"
}

some_string will resolve to

["item: a", "item: b", "item: c"]

Be careful when using this kind of resolution, as it’s computational and memory complexity grows exponentially in regards to the number of list variables being referenced, which is a rather obvious fact, but it’s still worth mentioning.

The variables that can be referenced within a definition/configuration fall into 3 categories:

  • value references - anything declared as a value can be accessed by it’s name

  • dependency references - any dependency path can be referenced using the name of the dependency prefaced with a ‘:’ prefix. Eg.: ${:eblif} will resolve to the path of eblif dependency. Make sure that the dependency can be actually resolved when you are using this kind of reference. For example you can’t use the a reference to eblif dependency in a module which does not rely on it. An exception is the producer module which can in fact reference it’s own outputs but these references cannot be used during the mapping stage (more on that later).

  • built-in references - there are a couple of built-in variables which are very handy:

    • shareDir - path to f4pga’s share directory.

    • binDir - path to f4pga’s bin directory.

    • prjxray_db - Project X-Ray database path.

    • python3 - path to Python 3 interpreter.

    • noisyWarnings - (this one should probably get removed)

Module class

Each module is represented as a class derived from Module class.

The class should implement the following methods:

  • execute(self, ctx: ModuleContext) - executes the module in exec mode

  • map_io(self, ctx: ModuleContext) -> 'dict[str, ]' - executes the module in mapping mode

  • __init__(self, params: 'dict[str, ]') - initializer. The params is a dict with optional parameter for the module.

Each module script should expose the class by defining it’s name/type alias as ModuleClass. f4pga tries to access a ModuleClass attribute within a package when instantiating a module.

Module’s execution modes

A module has essentially two execution modes:

  • mapping mode

  • exec mode

mapping mode

In mapping mode the module is provided with an incomplete configuration which includes:

  • takes namespace: this maps names of input dependencies to the paths of these dependencies

  • values namespace: this maps names of variables to the values of those variables.

The module has to provide a dictionary that will provide every output dependency that’s not on-demand a default path. This is basically a promise that when executed in exec mode, the module will produce files for this paths. Typically such paths would be derived from a path of one of it’s input dependencies. This mechanism allows the user to avoid specifying an explicit path for each intermediate target.

It should be noted that variables referring to the output dependencies can’t be accessed at this stage for the obvious reason as their values are yet to be evaluated.

exec mode

In exec mode the module does the actual work.

The configuration passed into this mode is full and it includes:

  • takes namespace: this maps names of input dependencies to the paths of these dependencies

  • values namespace: this maps names of variables to the values of those variables.

  • produces namespace: this maps names of output dependencies to explicit paths. This should not be used directly really, but it’s useful for ModuleContext.is_output_explicit method.

  • outputs namespace: this maps names of output dependencies to their paths.

When the module finishes executing in exec mode, all of the dependencies described in outputs should be present.

Module initialization/instantiation

In the the __init__ method of module’s class, the following fields should be set:

  • takes - a list of symbolic dependency names for dependencies used by the module

  • produces - a list of symbolic dependencies names for dependencies produced by the module.

  • values - a list of names given to the variables used withing the module

  • prod_meta - A dictionary which maps product names to descriptions of these products. Those entries are optional and can be skipped.

Qualifiers/decorators

By default the presence of all the dependencies and values is mandatory (In case of produces that means that the module always has to produce the listed dependencies). This can be changed by “decorating” a name in one of the following ways:

  • ?suffix

    • In takes - the dependency is not necessary for the module to execute

    • In produces - the dependency may be produced, but it is not guaranteed.

    • In values the value is not required for the module to execute. Referring to it through ModuleContext.values.value_name won’t raise an exception if the value is not present, instead None will be returned.

  • !suffix

    • In produces - the dependency is going to be produced only if the user provides an explicit path for it.

Currently it’s impossible to combine both ‘!’ and ‘?’ together. This limitation does not have any reason behind it other than the way the qualifier system is implemented at the moment. It might be removed in the future.

Common modules