FastX 5 adds new features to Job Scheduling fully modularizing the way you start a session.
Multiple methods of session launching can run concurrently under a unified interface.
Whether it be classic VDI session launching, clustering, SLURM workload management, Kubernetes, or launching AWS, GCP or Azure VMs, FastX is flexible to handle any configuration.
In order to make a fully modular drop in replacement for job schedulers several breaking changes had to be made to simplify configuration. The power and flexibility of FastX still exists, in a well defined form.
For a summary of the requirements needed to upgrade job scheduling, see Job Scheduling Migration Cheat Sheet
GUI Interface
FastX 5 adds a GUI to job scheduling. Job scheduling existed in FastX 4, but it required a user to comb through documentation of how to integration it. As such, many Job scheduling features were simply overlooked. FastX 5 adds a GUI interface so that job schedulers are front and center in the admin section. Default schedulers are well documented which speeds up integration
Breaking Change: Well Defined Job Scheduler Locations
Attaching a job scheduler in FastX 4 meant copying over the default scheduler to a new location, modifying it and then setting the path in the app.
FastX 5 creates 2 well defined locations for job schedulers System /usr/lib/fastx/5/integration/schedule and custom /etc/fastx/schedule. This allows job schedulers to be easily configurable and modifiable and defined by the directory name
Custom directories with the same name as a system directory /etc/fastx/schedule/default will override the system directory. This allows for simplified modification.
Deprecated: scheduleDir
Previously apps defined the path for a job scheduler using the scheduleDir parameter. This has been deprecated for the more flexible [schedule] object
Migration Tip
- Move your job custom scheduler directories into /etc/fastx/schedule
- Modify your apps
[schedule]
dir=relative_schedule_pathBreaking Change: Forms Moved Into the Job Scheduler Directory
In FastX 4 custom forms were global objects attached to each app. In practice forms are tightly integrated to the act of starting a session (ie job scheduling). Forms have now been moved into the job scheduler. This allows FastX to ship with sensible default forms simplifying installation
GUI Form Builder
The GUI interface now comes with a form builder allowing admins to quickly create custom forms to gather data from the end user. Admins still have access to the pure html text for custom forms.
Defining Forms
The job scheduler has several methods for defining forms in the job scheduler. The order is well defined and allows a great combination of flexibility and simplicity
By default the form processor checks for the existence of forms in the following order
- metadata.forms — option in the metadata.ini file
- form_script — script file used to generate a custom html form based on user input
- form.html — static html file
Once the job scheduler finds one of these options, it stops and returns the form
form.html
The simplest configuration is to create an html file and stick it in the job scheduler directory. This is where the GUI form builder stores the form.
Admins who need 1 static form should use this file
form_script
Admins who want to create a form based on specific criteria can create an executable script named form_script. The script takes the job scheduler input and outputs html to be displayed to the user
metadata.ini
Adding the forms field in schedule/<your_scheduler>/metadata.ini allows advanced customization of forms. This allows you to have multiple forms of different types
# if the form is a string, interpret it as an html file
forms[]=form1.html
# if the form is a json object, file is required. default type is html
forms[]={ "file": "form2.html" }
forms[]={ "file": "form3", "type": "html" }
# you can execute a custom script to generate html by setting type to script
forms[]={ "file": "form4", "type": "script" }
Form Data Now Stored in formData Parameter
In FastX 4 all form data was stored in the params field of the start object. This caused several issues
- The form data was not isolated from the other data passed
- There was no way to determine which form created the data
- There was the possibility that a later form could overwrite earlier form data
- The client had to make decisions on which form had already been processed
FastX 5 puts all form data into the formData object that is sent with the start object
start["formData"] = {
"form1.html": { form_1_answers },
"form2_script": { form_2_answers }
}
The id of the form that was processed becomes the parameter name with the data of the form as an object
Processing forms is greatly simplified
- All forms have been processed once all the form filenames are formData parameters
- Skipping forms becomes as simple as setting the formParam: {} on the server side
- Each form is isolated making it easier to dump the values for actual job scheduling
Breaking Change: Load Balancer Moved Into the Job Scheduler Directory
Load balancing is the act of deciding which FastX Launcher Node to exec the start script on. In a standard installation where Launcher nodes also act as Compute nodes, load balancing is done via FastX
However, when using commercial job schedulers such as SLURM, FastX’s load balancing is largely redundant. SLURM will be doing the load balancing. FastX’s job is simply to pick a login node with the SLURM client (sbatch) installed.
This tends to cause confusion. Moving load balancing into the job scheduler directory allows the proper node to be selected based on the job scheduler’s requirements rather than a global configuration
Upgraded preprocessor file
There is a special executable script in the default scheduler named preprocessor. The role of this script is to compile and verify all the start data before sending it off to the start script. The preprocessor code remains largely unchanged across job schedulers, so it is recommended that you do not edit this file.
However, there are many customizations that are needed before actually starting a session. The new preprocessor will execute special named files if they exist in the schedule directory. We recommend you edit these special files to fit your needs. The preprocessor executes in the following order
Job Scheduler Node Filtering
Before any data processing can occur, we need to make sure that there exists a launcher node that can handle the custom job scheduler you chose. Launcher Nodes now advertise the job schedulers that they support. FastX will filter out any node that does not have the required job scheduler. This filtered set of nodes will be sent to the preprocessor
Form Processing
The preprocessor is responsible for compiling all the forms and making sure each form has been processed. This has already been covered in the above section regarding forms.
Start Data Transform
Once all the data has been compiled, it is a good idea to do server side validation of the start data.
The preprocessor will execute a file named transform if it exists. The result is a JSON object that will be used as the start data to send to the start script.
Non-zero return codes will result in an error, cancelling the start process.
Migration Tip
In FastX 4 the scheduler’s start script would handle any data transform. You can still use the start script to validate data. However, the transform file offers a cleaner separation of duties
Node Filtering
The preprocessor will exec a script named node-filter if it exists in the scheduler directory. The output is a list of FastX Launcher Nodes that can be selected. If no nodes are returned, FastX throws an error
Node filtering is often used to separate nodes in a cluster into different groups.
A typical example is some nodes have GPUs and others don’t. Or separate Development nodes from Production nodes. The node selection should be done based on only the nodes that satisfy the requirements.
Migration Tip
FastX 4 apps had an object named nodeMetadataFilter which was a set of key/value pairs.
This has been replaced by schedule.data.
# FastX 4 metatadata filtering
[nodeMetadataFilter]
key1=val1
key2=val2
# FastX 5 metadata filtering
[schedule.data]
key1=val1
key2=val2The reason for this replacement is due to having job schedulers that handle node selection differently.
In a SLURM cluster, nodeMetadata is not useful. However, setting default scheduler data (like maximum RAM) is very userful.
If you are using a built in FastX job scheduler, the only thing you need to do is filter out nodes that are not useful during selection. So having both nodeMetadataFilter, schedule.data, and params are all mostly redundant
Node Selection
The next step of the preprocessor is to select the node that will be used for launching. In FastX 4 we referred to this as load balancing. Node selection is a little more accurate when dealing with commercial job schedulers such as SLURM.
The preprocessor will exec a script named node-select if it exists in the scheduler directory returning the nodeID of the FastX launcher nodeID.
Migration Tip
FastX 4 had a param called LOAD_BALANCER_SCRIPT. This was the custom file used for selecting a node. The name has been replaced with node-select. Copy your custom load balancer into the scheduler directory and rename it
FastX 4 allowed you to select a built in load balancer by setting the environment variable LOAD_BALANCER.
In FastX 5, in the scheduler’s metadata.ini file set node-select=sessions (or whichever) to use a built in node selector