Working with Pipeline Designer

Using Pipeline Designer to modify or create pipelines is fairly straightforward. When you load a pipeline (which have a .pyp extension), the pipeline graph (made up nodes and edges) will appear in the Pipeline Designer canvas. Each node represents a function, and each edge represents the flow of data between functions.

Adding nodes to a pipeline

Drag a node from the toolbox (or double-click on a node in the toolbox) to the canvas, then click on the node's input or output anchor (semi-circle line to the left or right of the node) and drag to the next node to connect them. This will create an "edge" (link) between two nodes in the graph. An edge connects an output port in one node and an input port in the other node (nodes can define their ports as either IN or OUT or both). The most commonly used pair of ports will be selected by default (in many cases this will be the data port in both nodes), and otherwise a window will pop up where you can select the two ports to connect. You can mouse over the port names in the window to see their documentation.

Pipeline Designer will allow you to connect one output port to multiple input ports as this may be necessary in some cases. However, each input port can only receive a single connection.

A faster approach is to right-click anywhere in the canvas and start typing to bring up a list of nodes. Besides searching for node names, it will also search for keywords that have been associated with nodes, to help you find nodes that perform certain actions (for example the Mean nodes is associated with the keyword "average" and "median" and will show up if you type either of those).

  • mousewheel: scroll vertically.
  • Alt + mousewheel: scroll horizontally.
  • Ctrl + mousewheel zoom in/out.
  • Ctrl-F: search for a node in the graph (will search the full text as displayed in the canvas). Uses "starts with" search.
  • Ctrl-G: Find next occurrence of a search result with Ctrl-F (pressing Ctrl-G repeatedly will cycle through all occurrences).
  • Ctrl-Shift-F: search for a node in the graph (searches the node name only).
  • Shift-C: center the graph on currently selected nodes.
  • Shift-V: shift view to make currently selected items visible.
  • Shift-F: fit selected items in view and zoom to fit
  • Ctrl+1..9: assign current location to a bookmark, Alt+1..9 shift view to bookmarked location.

Keyboard shortcuts for working with nodes and edges

  • Ctrl-S: replace the selected node with another node in place (edges will automatically reconnect)
  • Edge drag-and-drop: "Pull" an edge off of a node and attach it to another node by clicking on the edge close to the original node, and dragging and dropping it on the target node input or output receptor.
  • Click + drag: select multiple nodes and move them within the canvas.
  • Ctrl-D while hovering over a node or edge: delete a node or edge.
  • Alt-Delete with a node selected: remove the node and create an edge between the adjacent input and output nodes.
  • Shift while dragging a node or edge: duplicate the node or edge.
  • Ctrl-B: select node(s) that have a control relationship with the currently selected node (i.e., Function, Placeholder).
  • Ctrl + left-click (on a node): select all nodes that have a control relationship with it (along dotted edge).
  • F9: toggle the breakpoint on the selected node(s) (for debugging; see Developer docs on running a pipeline in a python debugger).

Duplicating nodes

You can select an existing node in your pipeline and press Control-C, Control-V to duplicate it. You can then drag the duplicate to another part of the graph. You can copy/paste individual nodes or groups of nodes (select the nodes first with click-and-drag). If your pipeline is already saved to disk, then copying a node in the pipeline will preserve all of its settings (meaning the duplicate copy will have all the same settings). If you add a new node, modify its settings and then copy/paste it without first saving the pipeline to disk, the duplicate node will have the default settings (not those you typed).

You can also copy/paste nodes between pipelines by running Pipeline Designer twice (as mentioned above), opening a different pipeline in each one, and then copy/pasting between them.

You can also copy/paste text annotations (and arrows) in the same way as nodes.

Changing your display settings

Many of the font and icon sizes (i.e., size of nodes in the canvas or the toolbox, help box, etc.) are configurable by going to File, Settings (or pressing Control-Shift-I), and selecting the View tab. Your settings are saved in your home folder under .pd/settings.ini; you can delete this file to restore the default settings.

Pipeline status bar

The status bar is at the bottom left of the window. Blue indicates PD is connected to Neuropype, while green means a pipeline is running (blue if pipeline is paused). Grey means PD is not connected. The Neuropype version number you are connected to is displayed in the status bar when first connecting.

Certain nodes may display status messages in the status bar, such as when calibrating or training (a model). (The absence of a message does not indicate a problem.)

Output window

The output window shows the log messages generated by Neuropype as a pipeline runs. It's important to monitor this window so as not to miss any warnings or errors. This window can be floating or can be docked at the bottom of the window. (Note on Linux, you need to drag the grey bar underneath the output window title bar in order to dock it; once the output window reaches the bottom of the main window, slide your mouse cursor until you see a grey shape indicating that the output window will dock there, then release the mouse.)

Setting node parameters

Double-clicking on a node in the canvas will open up a window with the node's parameters. Changes take effect immediately in your pipeline, but are not saved to disk until you save the pipeline (Ctrl-S).

Some nodes have a "ComboBox" parameter type which allows you to either select from a list of options or type in a value. If you type in a value that is not in the list, you must press Enter to confirm the value (rather than pressing Tab or clicking elsewhere), for the value to take effect.

Pipeline Designer and Neuropype

Pipeline Designer (PD) does not itself contain or execute any pipeline code, but rather communicates with Neuropype over the latter's REST API to fetch information regarding nodes, their ports and parameters, execute/pause/reload a loaded pipeline and retrieve the logs (which are shown in the PD output window). (You can also run pipelines without Pipeline Designer by using Neuropype Control Panel (accessible from the NP icon in the system taskbar and selecting Run Pipeline ...), the pipe.cmd (Windows) or pipe.sh (Linux) scripts in the Neuropype installation directory, or by communicating directly with the Neuropype API using the language of your choice (see Developer docs for the API documentation).

Running multiple Pipeline Designer instances

Neuropype supports multiple clients connecting to it at once. This means you can have multiple instances of Pipeline Designer open, each with a different pipeline. (Each instance of Pipeline Designer can only have a single pipeline.) Note that logging is unified across all instances (since its Neuropype that does the logging, not Pipeline Designer), so you will see interleaved log messages if multiple pipelines are running simultaneously.