[Contents] [Prev] [Next] [End]
This chapter shows you how to use some of the basic features of LSF JobScheduler. After trying the following examples, you should be able to use JobScheduler for everyday tasks.
You will most commonly use JobScheduler to process periodic or repetitive jobs. To accomplish this, you need to first select or create a calendar. Next you must associate jobs with the calendar. Once you have performed these two tasks, JobScheduler will automatically execute your jobs according to the calendar schedule and the resource specifications associated.
You can also use JobScheduler to process jobs based on the occurrence of some other event in the system, for example, the creation (arrival) of a file.
JobScheduler offers many utilities to check the status and progress of your jobs and calendars. You can set up automatic notification or re-submission to handle job failure. Other utilities allow you to discover the resources available in the clusters on the network.
There are three possible types of calendars you can use to submit your job: a preconfigured system calendar, a calendar you or another user created, or an 'anonymous' calendar. See 'What is a Calendar?' for more details.
The first step is to create a calendar with the bcaladd command. Create a calendar called 'daily'. This calendar becomes active once a day at 8 a.m.
% bcaladd -d "Daily Calendar" -t "*:*:*:08:0" daily Calendar <daily> is created.
A calendar has a name, one or more time expressions, and an optional description.
A time expression is a colon-separated list that defines time events. An optional duration field in the time expression specifies the time duration in which the time event is active. Time expressions are specified with the -t option. To prevent the shell from interpreting any special characters, time expressions following the -t option should be placed within quotes. See 'Time Expression' for further details.
The optional description is a string of text declared with the -d option. If the string contains blanks or special characters, it should be placed within quotes as well.
Once your calendar is created, it can be used to drive jobs. A calendar can be modified after it is created.
Figure 2 shows the calendar creation window of xbcal.
Each item in the Time Expression area has an 'extras' button which leads to a further pop-up window. These pop-ups allow you to select the details for each field. Figure 3 shows the resulting pop-up from clicking the button beside the Days item.
The bcal command displays information about the calendars on the system.
% bcal CALENDAR_NAME OWNER STATUS DURATION NEXT_EVENT_TIME hourly user1 active 7 Tue Dec 24 16:00:00 1996 complex user1 inactive - Wed Dec 25 17:00:00 1996
By default, bcal displays all of your calendars. You can specify a calendar by name.
% bcal hourly CALENDAR_NAME OWNER STATUS DURATION NEXT_EVENT_TIME hourly user1 active 7 Tue Dec 24 16:00:00 1996
See 'Displaying Calendars' for details of the individual fields of the bcal output.
The same information is available using the GUI application xbcal. As shown in Figure 4, the user has requested xbcal to display all the calendars on the system.
You can use bcal to display more detail about your calendars.
% bcal -l
See 'Displaying Calendars' for further details.
Once you have created your calendar, you can associate jobs with it. Submit a calendar-driven job using the bsub command. The -w option is used with the calendar keyword to specify a calendar dependent job.
% bsub -w "calendar(daily)" progA Job <3456> is submitted to default queue <priority>.
A calendar-driven job will be executed each time the calendar becomes active.
A job can be associated with a calendar and also have dependencies assigned to it. For example, a cheque-writing program should not run until the payroll jobs have successfully updated the employee records. Job dependencies are specified using the "&&" (AND), "||" (OR), and "!" (NOT) operators.
% bsub -w "calendar(everyday) && done(job3)" -J cal-dep progB
See 'Calendar Dependencies' for further details.
You can use the GUI application xbsub to associate jobs with your calendar. Figure 5 shows submitting a job to the calendar 'daily.'
Use the bdel command to remove a calendar-driven job.
% bdel 3456 Job <3456> is being deleted
This command removes a specific job associated with a calendar from the system. If the job is currently running, bdel kills the process before removing the job from the system.
See 'Removing a Job' for details.
Note
There is also the bkill command. This command will kill the current run of a job and requeue it for later execution.
You can make a job depend on one or more prior jobs. For example, your job may require that the previous job has started processing (but it does not matter if it has completed).
% bsub -w "started(first_job)" -J second_job time_card
In normal processing, your job probably requires that the prior job finished successfully.
% bsub -w "done(pre_process)" -J main_process cheque_run
See 'Inter-job Dependencies' for more details.
You may want a job to run after some file event has occurred. JobScheduler will check for the creation of a file and run your job.
% bsub -w "file(arrival(file_loc))" command
You can also run a job if the file exists.
% bsub -w "file(exist(file_loc))" command
See 'File Event Dependencies' for all available file event functions and more examples.
After you have submitted jobs to the JobScheduler, you can check the status of your jobs by running the bjobs command.
% bjobs JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME 1004 user6 RUN sim hostA hostA myjob0 Sep 1 09:23 1235 user6 PEND priority hostB sleep 30 Sep 1 13:55 1234 user6 SSUSP normal hostD hostB sleep 30 Sep 1 10:09 1250 user6 PEND sim hostA myjob2 Sep 1 13:59
Running jobs are listed first. Pending jobs are listed in the order in which they will be scheduled. Jobs in high priority queues are listed before those in lower priority queues.
JobScheduler not only enables you to submit and monitor your production jobs, but also allows you to view your cluster resource and load information from any machine in your cluster. JobScheduler gathers cluster resource information as well as system state information from all server hosts in the cluster. A variety of utilities are available to present such information in various formats.
JobScheduler provides tools for users to get information about the system. The lsid command shows you the version of LSF, the name of the cluster, and the current master host.
% lsid LSF 3.0, Dec 10, 1996 Copyright 1992-1997 Platform Computing Corporation My cluster name is JScluster My master name is hostA
The lsload command displays dynamic load information about JobScheduler server hosts.
% lsload HOST_NAME status r15s r1m r15m ut pg ls it tmp swp mem hostD ok 0.4 0.3 0.1 14% 10 5 3 56M 203M 125M hostB ok 0.8 0.8 1.6 42% 7 4 0 10M 71M 43M hostA unavail
Note that the hosts are ordered so that lightly loaded hosts are listed first. The status 'unavail' indicates that the Load Information Manager (LIM) on that host is not running.
The lshosts command shows the static resource information about JobScheduler server hosts.
% lshosts HOST_NAME type model cpuf ncpus maxmem maxswp server RESOURCES hostA hppa HP715 4 1 64M 128M Yes (hppa hpux) hostB sunsol sparc 3 1 96M 128M Yes (sun solaris) hostD SGI R10000 10 8 512M 1024M Yes (sgi irix6)
Static resource information is either configured by your system administrator or automatically identified by LIM.
The bhosts command displays information about JobScheduler server hosts information.
% bhosts HOST_NAME STATUS JL/U MAX NJOBS RUN SSUSP USUSP RSV hostA ok - 1 0 0 0 0 0 hostB closed - 1 1 1 0 0 0 hostD ok - 4 2 2 0 0 0
These fields display the status and job counters on the JobScheduler servers. The status 'closed' indicates that the server will not accept any new jobs at the current time.
Job queues represent different job scheduling and resource allocation policies. All jobs submitted to a queue share the same policy. Each queue can be configured to use a subset of the servers in the JobScheduler cluster; the default is to use all servers.
The bqueues command lists the available queues.
% bqueues QUEUE_NAME PRIO NICE STATUS MAX JL/U JL/P NJOBS PEND RUN SUSP nightly 43 10 Open:Inactive - - - 5 5 0 0 accounting 30 20 Open:Active - - - 0 0 0 0 evaluation 20 20 Open:Active - - - 0 0 0 0
A job queue can have many parameters that define the behaviour of the queue. See 'Listing User Information' for further details.
All the operations described above can be achieved using the JobScheduler GUI tools.
The on-line help built into these GUI tools allows you to learn as you work.
Copyright © 1994-1997 Platform Computing Corporation.
All rights reserved.