Skip to content

← deep_learning package

NetWeightArray

Define network weight array.

This node can only occur inside the network graph wired into a Define Net (NetDefine) node. The node is used to request a weight array whose value can be used as part of a neural network computation. The weight array is implicit in the containing subnet, and is optimized during training, as in the built-in NN nodes (e.g., convolution, etc). As with all implicit parameters, if you want to optimize them manually (i.e., using the Calculate Gradient node) rather than using one of the higher-level nodes such as "Deep Model" (DeepModel), you first need to transform your subnet (or rather a computational graph in which it is materialized) into functional style, which is done using the NetTransform node. Version 0.2.0

Ports/Properties

w_init

Initializer for the weights.

verbose name
W Init
default value
None
port type
DataPort
value type
BaseNode (can be None)
data direction
IN

w_prior

Optional prior distribution for the weights.

verbose name
W Prior
default value
None
port type
DataPort
value type
Distribution (can be None)
data direction
IN

value

Value of the weights.

verbose name
Value
default value
None
port type
DataPort
value type
object (can be None)
data direction
OUT

name

Name of the weight array. Must be unique within the subnet where it is used.

verbose name
Name
default value
weights
port type
StringPort
value type
str (can be None)

shape

Shape (dimensions) of the weight array.

verbose name
Shape
default value
[1]
port type
ListPort
value type
list (can be None)

dtype

Data type of the weight array. Float32 is the easiest to use, but the 16-bit data types can be more efficient -- however, float16 has limited range, and can cause numeric problems, which typically manifest as the network failing to train. Some hardware supports bfloat16, which is as efficient as float16, but has a wider range and is therefore an easier drop-in replacement for float32 when it is supported.

verbose name
Data Type
default value
float32
port type
EnumPort
value type
str (can be None)

initializer

Choice of initializer. This can either be one of the named initializers, or the value "custom", in which case the initializer must be wired into the the initializer port. For some initializers that take arguments, you can also specify these positionally as in "truncated_normal(1.0,0.0)" (note order of stddev, mean).

verbose name
Initializer
default value
lecun_normal
port type
ComboPort
value type
str (can be None)

set_breakpoint

Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

verbose name
Set Breakpoint (Debug Only)
default value
False
port type
BoolPort
value type
bool (can be None)

metadata

User-definable meta-data associated with the node. Usually reserved for technical purposes.

verbose name
Metadata
default value
{}
port type
DictPort
value type
dict (can be None)