DenseLayer¶
Dense neural network layer.
This is a fully-connected layer, where each input feature is connected to each output feature. Optionally includes a bias term. As with all built-in layers, you can override the initializer for the weights and/or the bias, which default to Lecun Normal and zero, respectively. More Info... Version 0.2.0
Ports/Properties¶
data¶
Data to process.
w_init¶
Initializer for the weights.
b_init¶
Initializer for the bias.
w_prior¶
Optional prior distribution for the weights.
b_prior¶
Optional prior distribution for the bias.
units¶
Number of units (i.e . output features).
with_bias¶
Whether to include a bias term.
w_initializer¶
Choice of weight initializer. This can either be one of the provided initializers, or the value "custom", in which case one of the Initializer nodes must be wired into the respective input port. For beginners it is recommended to stick to the defaults, since initialization of deep net layers is nuanced and can be tricky, otherwise be prepared to experiment with different choices. In general, the variance-scaling (lecun, glorot/xavier, he/kaiming) initializers are recommended, except for very simple/small layers where you may have a good default assumption as to the distribution of the weights (e.g., truncated_normal or uniform). Bias layers are typically zero-initialized. For initializers that take arguments, you can also type out the arguments positionally as in "truncated_normal(1.0,0.0)" (note reversed order of stddev, mean). The following initializers have arguments (here listed with their defaults): constant(value), those ending in normal(stddev=1, mean=0), those ending in uniform(min=0,max=1), orthogonal(scale=1,axis=-1), identity(gain=1), and variance_scaling(scale=1, "fan_in" (default)/"fan_avg"/"fan_out", "truncated_normal"(default)/"normal"/"uniform",optional-axis-indices=auto). Note that glorot and xavier are aliases for each other, and likewise he and kaiming are aliases for each other.
b_initializer¶
Choice of bias initializer. This can either be one of the provided initializers, or the value "custom", in which case one of the Initializer nodes must be wired into the respective input port. For beginners it is recommended to stick to the defaults, since initialization of deep net layers is nuanced and can be tricky, otherwise be prepared to experiment with different choices. In general, the variance-scaling (lecun, glorot/xavier, he/kaiming) initializers are recommended, except for very simple/small layers where you may have a good default assumption as to the distribution of the weights (e.g., truncated_normal or uniform). Bias layers are typically zero-initialized. For initializers that take arguments, you can also type out the arguments positionally as in "truncated_normal(1.0,0.0)" (note reversed order of stddev, mean). The following initializers have arguments (here listed with their defaults): constant(value), those ending in normal(stddev=1, mean=0), those ending in uniform(min=0,max=1), orthogonal(scale=1,axis=-1), identity(gain=1), and variance_scaling(scale=1, "fan_in" (default)/"fan_avg"/"fan_out", "truncated_normal"(default)/"normal"/"uniform",optional-axis-indices=auto). Note that glorot and xavier are aliases for each other, and likewise he and kaiming are aliases for each other.
op_precision¶
Operation precision. This is a compute performance optimization. See jax documentation for details on these options. Note that this only applies to the operation, while the storage precision may be separately configurable depending on the node in question.
layername¶
Name of the layer. Used for naming of weights.
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.