LongShortTermMemoryLayer¶
A long short-term memory (LSTM) recurrent layer, based on Hochreiter and Schmidhuber (1997).
This is a recurrent layer that conceptually retains internal network activations across successive time steps when the node is used to process time series data (see below for more details). In contrast to earlier recurrent cores, LSTMs have an improved ability to learn long-term dependencies using a gating mechanism that can selectively retain or forget past information. This implementation follows Zaremba et al. 2015, which is a minor modification that reduces forgetting during initial training. Like all recurrent layers, the node does not usually store and carry over these activations, but depends on either a special loop node (the RecurrentLoop node, see docs for more details) to step aross the time axis of some given data array, or requires that the user manually passes in and retrieves that carry state via the carry in/out port. Also like all recurrent nodes, this node will move the instance axis of the input first, optionally retain any axes listed in the parallel_axes port, and and flatten any other axes into a single feature axis at the end. When managing the carry state manually, the state for this node can be obtained via get_initial() or be constructed as either an array or packet (depending on the data that it is used with), where the first axis is of size 2, the last axis is of size units, and the middle axis is sized to be the product of the instance axes and other parallel axes of the data. More Info... Version 0.2.0
Ports/Properties¶
data¶
Data to process at current time step.
carry¶
Carried-over activations from previous time step.
units¶
Number of hidden units.
layername¶
Name of the layer. Used for naming of weights.
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.
parallel_axes¶
Optionally an axis or comma-separated list of axes that shall be processed in parallel by the layer, using the same learned weights. This is useful if you have multiple parallel processes that you wish to process separately but you assume that they are all ultimately governed by the same rules, you can learn a single kernel for all of them. Note that, for packet data, the instance axis (if present) and for plain-array data the first axis is always treated as a parallel axis, so you don't need to list it here.