[Contents] [Prev] [Next] [End]
This chapter describes how to use lsmake to perform parallel software builds and similar tasks.
lsmake is based on GNU make, and supports all GNU make features. Additional command line options control parallel execution. GNU make is upwardly compatible with the make programs supplied by most UNIX vendors.
To use lsmake you do not need to change your makefile, although reorganizing the contents of the makefile might increase the parallelism and therefore reduce the running time.
The lsmake(1) manual page describes the command line options that control load sharing. The gmake(1) manual pages describes the other command line options.
Note
Interactive tasks, including lsmake, are not supported on Windows NT.
Many tasks consist of many subtasks, with dependencies between the subtasks. For example, compiling a software package requires compiling each file in the package and then linking all the compiled files together.
In many cases most of the subtasks do not depend on each other. For a software package, the individual files in the package can be compiled at the same time; only the linking step needs to wait for all the other tasks to complete.
In an LSF cluster you can use lsmake to select a group of hosts and run parts of your make in parallel.
lsmake supports all the GNU make command line options.
The lsmake -j num_processors option tells lsmake to ask the LIM for num_processors processors. If fewer processors are available, lsmake uses all the available processors. If no processors are available and the local host meets all resource requirements specified using the lsmake -R option, all make tasks are run on the local host.
By default lsmake selects the same host type as the submitting host. This is necessary for most compilation jobs; all components must be compiled on the same host type and operating system version to run correctly. If your make task requires other resources you can override the default resource requirements with the lsmake -R resreq option.
To build your software in parallel on 10 processors, enter:
% lsmake -j 10
If you want to take advantage of parallelism between the CPU and I/O on a powerful host, you can also specify the number of concurrent jobs for each processor using the lsmake -c option:
% lsmake -j 10 -c 2
This selects up to 10 processors and starts two tasks on each processor.
lsmake can significantly reduce the response time of your make; however, it may also overload your file server or network if the jobs you are running are I/O intensive.
You can specify a threshold load so that parallelism is automatically reduced when the file server load is above a threshold and expanded when the file server load is below the threshold.
% lsmake -j 10 -F "r15s < 5 && pg < 20"
This lsmake job uses up to 10 hosts, and reduces the parallelism if the file server CPU load r15s goes beyond 5, or if the file server paging rate goes beyond 20 pages per second. lsmake automatically determines the file server for the current working directory.
The smallest unit that lsmake runs in parallel is a single make rule. If your makefile has rules that include many steps, or rules that contain shell loops to build sub-parts of your project, lsmake runs the steps serially.
You can increase the parallelism in your makefile by breaking up complex rules into groups of simpler rules. Steps that must run in sequence can use make dependencies to enforce the order. lsmake can then find more subtasks to run in parallel.
If your make job is divided into subdirectories, lsmake -M can process the subdirectories in parallel. The total number of parallel tasks is shared over all the subdirectories. Without the -M option, lsmake processes subdirectories sequentially, although tasks within each subdirectory can be run in parallel.
To process subdirectories in parallel they must be built as separate targets in your makefile. You must specify the make command for each subdirectory with the built-in $(MAKE) macro so that lsmake can substitute the correct lsmake command for the subdirectory.
Some makefiles may work correctly when run on a single machine, but may not work correctly when run in parallel through lsmake.
Figure 15 shows a makefile rule that uses a shell loop to process subdirectories.
DIRS = lib misc main
prog:
for subdir in $(DIRS) ; do \
cd $subdir ; $(MAKE) ; cd .. ;\
done
|
When this makefile is run on a single machine, the directories are processed sequentially; that is lib is built before misc and main. However, when run using lsmake using the -M option, all directories can be built in parallel. Therefore, it is possible for the misc and main directories to be built before lib, which is not correct.
Figure 16 shows a set of makefile rules to perform the same tasks and allows the subdirectories to be built in parallel in the correct order. Note that an extra rule is added so that the lib and misc subdirectories are built before the main directory:
DIRS = lib misc main
prog: $(DIRS)
$(DIRS):
cd $@ ; $(MAKE)
main: lib misc
|
make jobs often require a lot of resources, but no user interaction. Such jobs can be submitted to the LSF Batch system so that they are processed when the needed resources are available. lsmake includes extensions to run as a parallel batch job under LSF Batch:
% bsub -n 10 lsmake
This command queues an lsmake job that needs 10 hosts. When all 10 hosts are available, LSF Batch starts lsmake on the first host, and passes the names of all hosts in an environment variable. lsmake gets the host names from the environment variable and uses the RES to run tasks.
You can also specify a minimum and maximum number of processors to dedicate to your make job (see 'Minimum and Maximum Number of Processors'):
% bsub -n 6,18 lsmake
Because lsmake passes the suspend signal (SIGTSTP) to all its remote processes, the entire parallel make job can be suspended and resumed by the user or the LSF Batch system.
lsmake is fully compatible with GNU make. There are some incompatibilities between GNU make and some other versions of make; these are beyond the scope of this document.
When lsmake is running processes on more than one host, it does not send standard input to the remote processes. Most makefiles do not require any user interaction through standard I/O. If you have makefile steps that require user interaction, you can put the commands that require interaction into your local task list. Commands in the local task list always run on the local host, where they can read from standard input and write to standard output.
Copyright © 1994-1997 Platform Computing Corporation.
All rights reserved.