FastX allows you to fully customize the process of launching a session. Each app can attach a job scheduler allowing custom session launching on a per app basis. In FastX 5, custom job schedulers are located in /etc/fastx/schedule/[schedule.dir]
The purpose of a job scheduler is to create a pending session.
A pending session is a session is stared via a custom start script.
Admins can integrate commercial job schedulers such as SLURM into FastX
Or they can use FastX job scheduler forms to make customizable node selections for which session to start
The following is a summary of the files in the job scheduler
- metadata.ini — add metadata to the job scheduler
- preprocess — script that handles all processing before a session is started. It will call optional scripts if they exists. These optional scripts are only called if they are explicitly in the custom scheduler directory
- form_script — dynamically generate a form if metadata.ini does not have a forms[] parameter
- form.html — dump a static html form (if form_script does not exist)
- transform — used to format the start data for final processing by the start script
- node-filter — filter out launcher nodes that do not meet specific criteria. return an array of filtered nodes
- node-select — choose a nodeID to launch the session on
- start — custom script used to start the session
- status — periodically called on pending session to make sure it is still running
- terminate — used to cancel a pending session
- log — call logs on a pending session
- The following are typically not used.
- connect
- disconnect
- exec
Pending sessions
When launching a FastX session, there may be a delay from the initial start to the actual launching of a session. An example of this would be a slurm job scheduler waiting for a compute node to be available to process the job. At this stage, the session is “pending”. It is in the database, but it does not contain the started parameter. Custom schedulers can implement actions on the session that can be used to manipulate the pending sessions.
Common actions are terminate (cancelling a job) and logs (writing custom logs to the job).
Any of the session actions can be created. The only caveat is the action must return the same object as a running session.
Running Sessions
Once a session has started and is running, it will connect to the FastX web server and update the database. At this point the scheduler is complete and the FastX session actions will directly send messages to the session itself. Schedule scripts will no longer be used once the session is running
metadata.ini
This file contains metadata that the schedule scripts will use
# display name
name=
# display description
description=
# use a different scheduleDir as the parent.
# the job scheduler will check all the parents for the first script it finds with the proper name
parent=
# select which node to start the session on using the built in nodes
node-select=session
# forms to show to the client
# if the form is a string, interpret it as an html file
forms[]=form1.html
# if the form is a json object, file is required. default type is html
forms[]={ "file": "form2.html" }
forms[]={ "file": "form3", "type": "html" }
# you can execute a custom script to generate html by setting type to script
forms[]={ "file": "form4", "type": "script" }preprocess
The preprocess script will return the forms and also do load balancing. Preprocess will allow a user to completely change the data that is sent to the start script. In most cases, this script does not need to be changed from the default
input
{
nodes: [] # array of nodes for load balancing
me: {} # object of user data who is making the call
start: {} # start data
nodeID: "string" # nodeID of the node running the preprocessor
}output
The output of the preprocess script should return a json result object with either a “form” stage or a “data” stage.
The form stage tells the client to display the HTML form to gather more information
result: {
stage: "form",
form: "Obsolete. No longer used"
startForm: { html: HTML form, id: formId },
originalFormData: { } // start object that was sent to the form
}The data stage tells the server to pass the result.start object as the object that will be passed to the start script.
The result.start object should contain a nodeID parameter which tells the web server which tells the web server which node to run the start command on
result: { stage: “data”, start: {} //data to pass to the start script. must contain a nodeID param }
start
The start script is the main script that is used to create custom job schedulers. In previous versions, FastX had a global variable named LAUNCHER_SCRIPT which was used as the launcher script.
FastX extends the start script so each individual app can run its own launcher. This way you can have a heterogeneous cluster (some sessions start immediately, some may use slurm, others start Virtual Machines …)
The typical method for start is to fork a process and then execute /usr/lib/fastx/4/scripts/start –json=0 passing the input[‘start’] data to the forked process
input
{
start {} // Start Object. Pass to the fastx/4/scripts/start --json 0
me: {} // current user object who is execing the program
nodeID: "string" // nodeID the script was launched from
userData: {} // custom user data object
defaultStartScript: "string" // path to fastx/4/scripts/start
weblink: {} // weblink object to set on weblinks
rootDir: "string" // base dir of fastx /usr/lib/fastx/4
varDir: $FX_VAR_DIR
configDir: $FX_CONFIG_DIR
localDir: $FX_LOCAL_DIR
tmpDir: $FX_TEMP_DIR
uid: 1000 // user's uid
gid 1000 // user's gid
homedir: "string" user's $HOME directory
shell: "/bin/bash" // user's $SHELL
login: "user" linux username
}output
The output of this data will tell the web server that the session has been scheduled. You can add a custom scheduling message as well as add a schedule object to pass to the session data
{
result: {
"output": [
{ "line": "This is stdout", "type": "stdout" },
{ "line": "This is stderr", "type": "stderr"}
],
"stdout": "This is stdout as one string",
"stderr": "This is stderr as one string",
"message": "Test Start Message that the user can display",
"schedule": { // this object will be passed to the session data to be used with other actions
"jobId": 123,
"param2": "some_string"
}
}log
You can use this action to display custom logs to the user. Typically this can be used for debugging
input
{
session: {} // session data object
me: {} // user data object of user who is executing
}output
{ result: {
"log1": "output string 1",
"log2": "these logs will be displayed to the user when he goes to logs",
"log3": "log string can be anything"
} }status
Pending jobs may fail before ever contacting the web server. In these instances, the pending sessions will stay in the session list until they are manually purged. The status script is a way to periodically check on the job to verify it is still active. Exit code 0 means that it is still active (and do nothing). Non zero exit codes will tell the web server to clean the pending session from the database.
Make sure to clean up any data in the status script
input
{
session: {} // session data object
me: {} // user data object of user who is executing
}output
{ result: { } }terminate
This action should be used to manually cancel a job. Enter in any job cancellation code in the terminate script.
{
session: {} // session data object
me: {} // user data object of user who is executing
}output
{ result: {
"status" : "message you can send to the user to see"
} }