remote.tcl

::pwtk::remote

::pwtk::remote

SYNOPSIS

proc ::pwtk::remote {args} {

USAGE

   remote  ?OPTIONS?  USER@HOST  SCRIPT_CODE
 or    
   remote  ?OPTIONS?  USER@HOST  SCRIPT_FILE

 where OPTIONS are:
     -fg
     -rdir  DIRECTORY
     -rpwtk FILE

ARGUMENTS

USER@HOST ... either user@hostname.domain.name, hostname.domain.name, or hostname
SCRIPT_CODE ... the PWTK script code to run on the remote host
SCRIPT_FILE ... the PWTK script file to run on the remote host

PURPOSE

Run on the remote host either the script code contained in the 'SCRIPT_CODE' argument or the script file specified by the 'SCRIPT_FILE' argument. PWTK checks if the last argument is a file; if so, it consider it as a script-file, otherwise it considers the last arguments as a script-code.

The connection and file transfer to a remote host is managed via ssh and rsync. A user must be able to connect to the remote host without password (e.g., the ~/.ssh/id_rsa.pub key must be included in the ~/.ssh/authorized_keys file on the remote host).

The configuration of the PWTK remote execution is specified in the $PWTK/config/remote.tcl file and the user can tune it with the ~/.pwtk/remote.tcl file.

OPTIONS

-fg ... run the 'remote' in a blocking mode (aka foreground, default = background), i.e., with this option, 'remote' waits for script to finish on the remote host
-rdir DIRECTORY ... absolute pathname of the directory on the remote host where to run the script (default = mirror of the current local directory on the remote host)
-rpwtk FILE ... pathname of the PWTK launcher on the remote host (needed if pwtk is not on $PATH on the remote host)
-id ID ... ID number (this option is used internally by PWTK's tpool)
-fid FID ... kind of File-ID used for assembling the filename of the script (this option is used internally by PWTK's tpool)
-np ... do not use the propagate mechanism (used internally by PWTK; default = use propagate, see: ::pwtk::propagate)
-direct ... used internally by PWTK for "pwtk --remote=user@host"; use the script file directly without any "dressing" (implies -np)

BEWARE

::pwtk::remote is useful for Beowulf-type clusters without batch-queuing. On HPC supercomputers, consider using the PWTK batch-queuing instead (see section: 2.8 Batch Queuing Support).

EXAMPLE

    # run a script code on a remote host:

    remote -rpwtk ~/prog/q-e/pwtk/pwtk -rdir ~/calc/QE/myProject user@remotehost {
       import common.pwtk
       runPW relax.Cu111-1x1-4L.in
    }

    # run the script file 'calculate.pwtk' on a remote host:

    remote user@remotehost calculate.pwtk

DESCRIPTION

In the default "background" mode, PWTK submits the script to a remote host and returns immediately for further processing, i.e., the command is non-blocking. To wait for "background" remote process(es), use the ::pwtk::remote_wait command.

In the "foreground" mode (the -fg option), PWTK waits for the calculation to finish, i.e., the command is blocking.

By default, ::pwtk::remote runs the script on the remote host in the directory, which is a mirror of the current local directory. For example, if the current working directory on the local host is ~/path/to/project, then the calculation will be run on ~/path/to/project directory on the remote host (remote file-system); note that the value of $HOME on remote and local hosts can differ.

To request that the calculation on the remote host should run in an arbitary directory, use the "-rdir DIRECTORY" option.

BEWARE

The I/O files produced by remote calculations will not be copied to the current local directory (the only exception is the remote PWTK log file). It is the responsability of the user to copy the I/O files back to the local host.

RETURN VALUE

for foreground mode: an empty string
for background mode: ID of the remote background process (PWTK assigns a unique ID to each background remote process)

SOURCE

    variable remote_index
    variable remote

    # parse the options

    set options {
        {fg             "run the calculation in the foreground mode (default = background)"}
        {rdir.arg  {}   "directory on the remote host where calculation will be run (default = mirror of the current local directory but on the remote host)"}
        {rpwtk.arg pwtk "full pathname of the PWTK launcher on the remote host (default = pwtk)"}
        {id.arg    {}   "ID of this remote job, supplied by the caller"}
        {fid.arg   {}   "File-ID used for assembling the filename of the script"}
        {np             "do not use the propagate mechanism (default = use propagate)"}
        {direct         "use the script file is used directly (implies -np)"}
    }
    set narg 2
    set usage "?-f?  ?-fg?  ?-rdir DIRECTORY?  ?-rpwtk FILE?  ?-id ID?  ?-fid ID?  ?-np?  ?-direct?  user@host script"
    ::pwtk::parseOpt_
    
    # assign the last two arguments
    
    lassign $args user_at_host lastarg

    # get the hostname

    set host     [rhost_ $user_at_host]
    set hostonly [rhostonly_ $user_at_host]

    # get the remote directory
    
    #set rdir [::fileutil::relative $::env(HOME) [pwd]]
    set rdir [::fileutil::stripPath $::env(HOME) [pwd]]
    if { $opt(rdir) ne {} } {
        set rdir $opt(rdir)
    }
    set pdir [expr { [file pathtype $rdir] eq "relative" ? "~/$rdir" : $rdir }] ; # used for printing only

    # index

    if { $opt(id) ne {} } {
        set index $opt(id)
        set jobtxt " for job-$index"
    } else {
        set index [incr remote_index($hostonly)]
        set jobtxt ""
    }
    
    # findex

    if { $opt(fid) ne {} } {
        set findex $opt(fid)
    } else {
        set findex $index
    }    
    
    # lastarg (aka script)

    if { $opt(direct) } {
        fileMustExist $lastarg "PWTK script"       
        set scriptName $lastarg
        set scriptLog  [file rootname $lastarg].log
        set opt(np) 1
    } else {
        if { [file exists $lastarg] } {
            # the last argument is a file
            set scriptHead [file rootname $lastarg].
            set scriptCode [readFile $lastarg]
        } else {
            # the last argument is a script code
            set scriptCode $lastarg
            if { [set scriptHead [file tail [file rootname [info script]]]] ne {} } {
                append scriptHead .
            }
        }
        set scriptName $hostonly.${scriptHead}$findex.pwtk
        set scriptLog  $hostonly.${scriptHead}$findex.log
    }
    
    # make a script
    
    if { ! $opt(np) } {
        set scriptCode [::pwtk::propagate_dump]\n$scriptCode
    }
    if { ! $opt(direct) } {
        writeFile $scriptName $scriptCode
    }

    # printout
    
    print REMOTE "Running a script$jobtxt remotely on $user_at_host"
    print "
remote directory     :   $pdir
remote PWTK script   :   $scriptName
remote PWTK log file :   $scriptLog
"

    # check if the remote host is alive and user can connect to it

    #if { ! [host_is_up $user_at_host] } {
    #    ::pwtk::error "cannot communicate with '$host'\n\n$::errorInfo" 1
    #}
    if { ! [can_connect $user_at_host] } {
        ::pwtk::error "cannot connect to '$user_at_host'" 1
    }
            
    # create the remote directory and rsync the script there

    print "$remote(rsync)  $scriptName  $user_at_host:$rdir/\n"
    rmkdir $user_at_host $rdir
    rsync $scriptName $user_at_host:$rdir/$scriptName
    rsync_imports_ $user_at_host
    
    # check if $opt(rpwtk) is executable
    if { [rtclsh $user_at_host "puts \[auto_execok $opt(rpwtk)]"] eq {} } {
        ::pwtk::error "the specified PWTK executable '$opt(rpwtk)' either is not executable or does not exist" 1
    }
    
    # remote-exec the PWTK script

    print "Running remotely on $user_at_host:$rdir/
$opt(rpwtk) $scriptName > $scriptLog 2>&1\n"

    set pid [::pwtk::rexec_bg $user_at_host "cd $rdir; $remote(nohup) $opt(rpwtk) $scriptName > $scriptLog 2>&1 & echo \$!"]
    print "Remote PID = $pid ($scriptName)\n"; # N.B. do not use PID: as not to clash with the PWTK header PID: line

    # background or foreground ?
    
    if { $opt(fg) } {
        print "Waiting for the remote process $pid to finish...\n"
        remote_pidwait $user_at_host $pid
        
        print "\n$remote(rsync)  $user_at_host:$rdir/$scriptLog  .\n"
        rsync $user_at_host:$rdir/$scriptLog .
        return ""
    } else {
        # store the info about this background remote job

        set id [incr remote(ID)]
        
        array set remote [subst {
            ident,$id                      $hostonly,$index
            ID,$hostonly,$index            $id
            PID,$hostonly,$index           $pid
            user_at_host,$hostonly,$index  $user_at_host
            script,$hostonly,$index        $scriptName
            log,$hostonly,$index           $scriptLog
            rdir,$hostonly,$index          $rdir
            rpwtk,$hostonly,$index         $opt(rpwtk)
        }]
        return $id
    }
}

TABLE OF CONTENTS

::pwtk::remote