Using lstcsh

This chapter describes lstcsh, an extended version of the tcsh command interpreter. The lstcsh interpreter provides transparent load sharing of user jobs.

This chapter is not a general description of the tcsh shell. Only load sharing features are described in detail.

Interactive tasks, including lstcsh, are not supported on Windows.

Contents

• About lstcsh
• Starting lstcsh
• Using lstcsh as Your Login Shell
• Host Redirection
• Task Control
• Built-in Commands
• Writing Shell Scripts in lstcsh

About lstcsh

The lstcsh shell is a load-sharing version of the tcsh command interpreter. It is compatible with csh and supports many useful extensions. csh and tcsh users can use lstcsh to send jobs to other hosts in the cluster without needing to learn any new commands. You can run lstcsh from the command-line, or use the chsh command to set it as your login shell.

With lstcsh, your commands are sent transparently for execution on faster hosts to improve response time or you can run commands on remote hosts explicitly.

lstcsh provides a high degree of network transparency. Command lines executed on remote hosts behave the same as they do on the local host. The remote execution environment is designed to mirror the local one as closely as possible by using the same values for environment variables, terminal setup, current working directory, file creation mask, and so on. Each modification to the local set of environment variables is automatically reflected on remote hosts. Note that shell variables, the nice value, and resource usage limits are not automatically propagated to remote hosts.

For more details on lstcsh, see the lstcsh(1) man page.

In this section

• Task Lists
• Local and Remote Modes
• Automatic Remote Execution

Task Lists

LSF maintains two task lists for each user, a local list (.lsftask) and a remote list (lsf.task). Commands in the local list must be executed locally. Commands in the remote list can be executed remotely.

See the Platform LSF Configuration Reference for information about the .lsftask and lsf.task files.

Changing task list membership

You can use the LSF commands lsltasks and lsrtasks to inspect and change the memberships of the local and remote task lists.

Task lists and resource requirements

Resource requirements for specific commands can be configured using task lists. You can optionally associate resource requirements with each command in the remote list to help LSF find a suitable execution host for the command.

If there are multiple eligible commands on a command-line, their resource requirements are combined for host selection.

If a command is in neither list, you can choose how lstcsh handles the command.

Local and Remote Modes

lstcsh has two modes of operation:

• Local
• Remote

Local mode

The local mode is the default mode. In local mode, a command line is eligible for remote execution only if all of the commands on the line are present in the remote task list, or if the @ character is specified on the command-line to force it to be eligible.

See @ character for more details.

Local mode is conservative and can fail to take advantage of the performance benefits and load-balancing advantages of LSF.

Remote mode

In remote mode, a command line is considered eligible for remote execution if none of the commands on the line is in the local task list.

Remote mode is aggressive and makes more extensive use of LSF. However, remote mode can cause inconvenience when lstcsh attempts to send host-specific commands to other hosts.

Automatic Remote Execution

Every time you enter a command, lstcsh looks in your task lists to determine whether the command can be executed on a remote host and to find the configured resource requirements for the command.

See the Platform LSF Configuration Reference for information about task lists and lsf.task file.

If the command can be executed on a remote host, lstcsh contacts LIM to find the best available host.

The first time a command is run on a remote host, a server shell is started on that host. The command is sent to the server shell, and the server shell starts the command on the remote host. All commands sent to the same host use the same server shell, so the start-up overhead is only incurred once.

If no host is found that meets the resource requirements of your command, the command is run on the local host.

Differences from Other Shells

When a command is running in the foreground on a remote host, all keyboard input (type-ahead) is sent to the remote host. If the remote command does not read the input, it is lost.

lstcsh has no way of knowing whether the remote command reads its standard input. The only way to provide any input to the command is to send everything available on the standard input to the remote command in case the remote command needs it. As a result, any type-ahead entered while a remote command is running in the foreground, and not read by the remote command, is lost.

@ character

The @ character has a special meaning when it is preceded by white space. This means that the @ must be escaped with a backslash \ to run commands with arguments that start with @, like finger. This is an example of using finger to get a list of users on another host:

finger @other.domain

Normally the finger command attempts to contact the named host. Under lstcsh, the @ character is interpreted as a request for remote execution, so the shell tries to contact the RES on the host other.domain to remotely execute the finger command. If this host is not in your LSF cluster, the command fails. When the @ character is escaped, it is passed to finger unchanged and finger behaves as expected.

finger \@hostB

Limitations

A shell is a very complicated application by itself. lstcsh has certain limitations:

Native language system

Native Language System is not supported. To use this feature of the tcsh, you must compile tcsh with SHORT_STRINGS defined. This causes complications for characters flowing across machines.

Shell variables

Shell variables are not propagated across machines. When you set a shell variable locally, then run a command remotely, the remote shell will not see that shell variable. Only environment variables are automatically propagated.

fg command

The fg command for remote jobs must use @, as shown by examples in Task Control.

tcsh version

lstcsh is based on tcsh 6.03 (7 bit mode). It does not support the new features of the latest tcsh.

Starting lstcsh

Start lstcsh

If you normally use some other shell, you can start lstcsh from the command-line.

1. Make sure that the LSF commands are in your PATH environment variable, then enter:

lstcsh

If you have a .cshrc file in your home directory, lstcsh reads it to set variables and aliases.

Exit lstcsh

1. Use the exit command to get out of lstcsh.

Using lstcsh as Your Login Shell

If your system administrator allows, you can use LSF as your login shell. The /etc/shells file contains a list of all the shells you are allowed to use as your login shell.

Set your login shell

Using csh

The chsh command can set your login shell to any of those shells. If the /etc/shells file does not exist, you cannot set your login shell to lstcsh.

1. Run the command:

chsh user3 /usr/share/lsf/bin/lstcsh

The next time user3 logs in, the login shell will be lstcsh.

Using a standard system shell

if you cannot set your login shell using chsh, you can use one of the standard system shells to start lstcsh when you log in.

To set up lstcsh to start when you log in:

1. Use chsh to set /bin/sh to be your login shell.
2. Edit the .profile file in your home directory to start lstcsh, as shown below:

SHELL=/usr/share/lsf/bin/lstcsh
export SHELL
exec $SHELL -l 

Host Redirection

Host redirection overrides the task lists, so you can force commands from your local task list to execute on a remote host or override the resource requirements for a command.

You can explicitly specify the eligibility of a command-line for remote execution using the @ character. It may be anywhere in the command line except in the first position (@ as the first character on the line is used to set the value of shell variables).

You can restrict who can use @ for host redirection in lstcsh with the parameter LSF_SHELL_AT_USERS in lsf.conf. See the Platform LSF Configuration Reference for more details.

Examples

hostname @hostD
<< remote execution on hostD >>
hostD 
hostname @/type==linux
<< remote execution on hostB >>
hostB 

@ character

@	@ followed by nothing means that the command line is eligible for remote execution.
@host_name	@ followed by a host name forces the command line to be executed on that host.
@local	@ followed by the reserved word local forces the command line to be executed on the local host only.
@/res_req	@ followed by / and a resource requirement string means that the command is eligible for remote execution and that the specified resource requirements must be used instead of those in the remote task list.

For ease of use, the host names and the reserved word local following @ can all be abbreviated as long as they do not cause ambiguity.

Similarly, when specifying resource requirements following the @, it is necessary to use / only if the first requirement characters specified are also the first characters of a host name. You do not have to type in resource requirements for each command line you type if you put these task names into remote task list together with their resource requirements by running lsrtasks.

Task Control

Task control in lstcsh is the same as in tcsh except for remote background tasks. lstcsh numbers shell tasks separately for each execution host.

jobs command

The output of the built-in command jobs lists background tasks together with their execution hosts. This break of transparency is intentional to give you more control over your background tasks.

sleep 30 @hostD &
<< remote execution on hostD >>
[1] 27568
sleep 40 @hostD &
<< remote execution on hostD >>
[2] 10280
sleep 60 @hostB &
<< remote execution on hostB >>
[1] 3748
jobs
<hostD>
[1]  + Running                sleep 30
[2]    Running                sleep 40
<hostB>
[1]  + Running                sleep 60 

Bring a remote background task to the foreground

1. To bring a remote background task to the foreground, the host name must be specified together with @, as in the following example:

fg %2 @hostD

<< remote execution on hostD >>
sleep 40

Built-in Commands

lstcsh supports two built-in commands to control load sharing, lsmode and connect.

In this section

• lsmode
• connect

lsmode

Syntax

lsmode [on|off] [local|remote] [e|-e] [v|-v] [t|-t] 

Description

The lsmode command reports that LSF is enabled if lstcsh was able to contact LIM when it started up. If LSF is disabled, no load-sharing features are available.

The lsmode command takes a number of arguments that control how lstcsh behaves.

With no arguments, lsmode displays the current settings:

lsmode
LSF 
Copyright Platform Computing Corporation
LSF enabled, local mode, LSF on, verbose, no_eligibility_verbose, 
no timing. 

Options

[on | off]

Turns load sharing on or off. When turned off, you can send a command line to a remote host only if force eligibility is specified with @.

The default is on.

[local | remote]

Sets lstcsh to use local or remote mode.

The default is local. See Local and Remote Modes for a description of local and remote modes.

[e | -e]

Turns eligibility verbose mode on (e) or off (-e). If eligibility verbose mode is on, lstcsh shows whether the command is eligible for remote execution, and displays the resource requirement used if the command is eligible.

The default is off.

[v | -v]

Turns task placement verbose mode on (v) or off (-v). If verbose mode is on, lstcsh displays the name of the host on which the command is run, if the command is not run on the local host. The default is on.

[t | -t]

Turns wall-clock timing on (t) or off (-t).

If timing is on, the actual response time of the command is displayed. This is the total elapsed time in seconds from the time you submit the command to the time the prompt comes back.

This time includes all remote execution overhead. The csh time builtin does not include the remote execution overhead.

This is an impartial way of comparing the response time of jobs submitted locally or remotely, because all the load sharing overhead is included in the displayed elapsed time.

The default is off.

connect

Syntax

connect [host_name] 

Description

lstcsh opens a connection to a remote host when the first command is executed remotely on that host. The same connection is used for all future remote executions on that host.

The connect command with no argument displays connections that are currently open.

The connect host_name command creates a connection to the named host. By connecting to a host before any command is run, the response time is reduced for the first remote command sent to that host.

lstcsh has a limited number of ports available to connect to other hosts. By default each shell can only connect to 15 other hosts.

Examples

connect
CONNECTED WITH              SERVER SHELL
hostA                       + 
connect hostB
Connected to hostB 
connect
CONNECTED WITH              SERVER SHELL
hostA                       +
hostB                       - 

In this example, the connect command created a connection to host hostB, but the server shell has not started.

Writing Shell Scripts in lstcsh

You should write shell scripts in /bin/sh and use the lstools commands for load sharing. However, lstcsh can be used to write load-sharing shell scripts.

By default, an lstcsh script is executed as a normal tcsh script with load-sharing disabled.

Run a script with load sharing enabled

The lstcsh -L option tells lstcsh that a script should be executed with load sharing enabled, so individual commands in the script may be executed on other hosts.

There are three different ways to run an lstcsh script with load sharing enabled:

• Run lstcsh -L script_name, or
• Make the script executable and put the following as the first line of the script. By default, lstcsh is installed in LSF_BINDIR.

The following assumes you installed lstcsh in the /usr/share/lsf/bin directory):

#!/usr/share/lsf/bin/lstcsh -L 

1. Start an interactive lstcsh.
2. Enable load sharing, and set to remote mode:

lsmode on remote

3. Use the source command to read the script in.