Advanced Configuration: Weblinks

Background

Weblinks are special connections where the FastX Session is not running on that same server as a FastX Server. This is typically achieved when an admin creates a custom launcher script and launches a session on a system with FastX installed, but it is not part of a FastX cluster.

Why do this?

A typical FastX cluster can handle 300-500 instances of FastX before the load gets too large and synchronization slows down the transporter.

Administrators who have load balancers (lsf, slurm etc) may find it more efficient to simply add a small proxy to the Compute Node and isolate the FastX Servers on the Head Nodes. This saves resources and speeds up the load balancing since FastX Servers only need to be clustered on the Head Nodes.

Sessions that use the weblink proxy get the nodeID of the FastX server it connects to, and the configuration is transparent to the user. This allows for greater flexibility and scaling of FastX clusters.

All connections are initiated from the weblink proxy meaning the Compute Node can be on a system behind a firewall and the FastX Server can be on the other side adding to security

Enabling Weblinks

FastX Server

To enable the FastX Server to listen for weblinks add the WEBLINK_PORT variable to fastx.env (or the environment) and restart FastX

Setup

# add to fastx.env or as an environment variable
# By convention, port 3320 is the weblink port, but it can be anything
WEBLINK_PORT=3320

# Adding the Weblink Url will pass it as a parameter to the start script
WEBLINK_URL=https://your_head_node:3320

# Optional Uri.  only edit if you are running fastx in a subdirectory
WEBLINK_URI=/weblink

# disable authentication (only for testing)
WEBLINK_TOKEN_NOAUTH=1

# path to the custom launcher script
LAUNCHER_SCRIPT=/path/to/my/launcher.py

Launcher Script

The launcher script is a custom script that admins write that is used to launch sessions. The script can be written in any language. The input from FastX is passed on STDIN as a JSON string

See Job Scheduling

Weblink Proxy

The weblink proxy sits on the same Compute Nodes as your FastX Sessions. The weblink proxy is included with the FastX installation.

Setup

  1. Install FastX on the Compute Node
  2. Disable the FastX service(s)
    • sudo systemctl disable fastx4
    • sudo systemctl disable rlm
    • sudo systemctl disable fastx-ssh

Note, you can disable the services during installation.
The weblink proxy will connect directly to the FastX Head Node.

Launching Sessions on a Proxy

The launcher script is used to send data to a the remote node in some method (Typically a load balancer or SSH). Once the data is on the remote node, it needs to run a command

start-weblink

The command to run is /usr/lib/fastx/4/scripts/start-weblink. This command will establish a connection to the Head Node and then run the /usr/lib/fastx/4/scripts/start command to start the session. It will also maintain reconnects. An inital connection failure is considered fatal error

Environment Variables

The following environment variables are used.

  • WEBLINK_URL=https://localhost:3320 # your WEBLINK_URL
  • WEBLINK_URI=/weblink # your WEBLINK_URI
  • WEBLINK_PRIMARY=primary_jwt # the weblink.primary parameter passed to the start script
Starting a session

Run the weblink command. This command runs in the foreground. sessionId is required.

WEBLINK_URL=https://my.weblink.com:3320 LOGLEVEL=debug /usr/lib/fastx/4/scripts/start-weblink --command=xterm --sessionId=12345567

In a script, you should pass the start data on stdin

echo "$START_DATA" | WEBLINK_URL=https://my.weblink.com:3320 LOGLEVEL=debug /usr/lib/fastx/4/scripts/start-weblink --json=0