Ideas for Getting and Renaming files via SFTP

[gmalchow]

Hello,

I'm working on migrating processes that currently use FTP to start using SFTP. In this scenario we are the client and will be connecting to a trading partners SSHD.

I am trying to determine what will be the best way to get all files in a directory and then rename or delete them on the SSHD? I plan on using mget to get the files. I'm struggling with what is the be way to then delete or rename the files once I have them on my system.

I will be using Expect scripts and CL programs to automate the processes. My initial thought would be to get the files and then build an Expect script that will rename or delete the files I just got. For Example I would dynamically build a script what would look something like this:

#!/usr/local/bin/expect -f
set timeout 20
spawn sftp $env(SSH_USER)@$env(SSH_HOST)
expect {
default {exit 2}
"continue connecting (yes/no)?" {send "yes\n"; exp_continue}
"password:" {send "$env(SSH_PASS)\n"; exp_continue}
"sftp>"
}
send "rename test 1.txt /processed/test 1.txt\n"
send "rename test 2.txt /processed/test 2.txt\n"
send "rename test 3.txt /processed/test 3.txt\n"
expect {
default {exit 2}
"not found" {exit 3}
"sftp>"
}
send "quit \n"
exit 0

The number of files being renamed would be dependent on which files I just previously got.

I'm curious if there is a better way?

Thanks,
Grizzly

[gmalchow]

I'll update this by saying I know what I want to do, but I'm not sure how to achieve it.
Here's what I want to to:
Connect to remove server.
Get all files from a directory on the remote server.
Once I have the files on the local server, I want to get each file name so it can be used to rename the file on the remote server.

I'm trying to figure out how I can do this inside of an Expect script. I thought I could use ls and attempt to store the values in an array. Then loop through that and rename files. For example:

#!/usr/local/bin/expect -f
set timeout 20
set file ""
set files ""
set newfile ""
cd $env(MY_LOCAL_DIR)
# The following doesn't work because I don't really know the proper syntax. Error states "Cannot read ($(ls))"
set files=($(ls))
spawn sftp $env(SSH_USER)@$env(SSH_HOST)
expect {
default {exit 2}
"continue connecting (yes/no)?" {send "yes\n"; exp_continue}
"assword:" {send "$env(SSH_PASS)\n"; exp_continue}
"sftp>"
}
send "cd $env(REMOTE_DIR)\n"
expect {
default {exit 2}
"not found" {exit 3}
"sftp>"
}
# If I was able to populate the files array, I was going to attempt the following
foreach file $files {
set newfile = $env(PROCESSED_DIR)$file
send "rename $file $newfile\n"
expect "sftp> "
}
send "quit \n"
exit 0

I am ultimately trying to ensure I don't "GET" a file I've already gotten before.
The real issue is I lack the Shell and Expect scripting skills to make this work. Any help or ideas would be appreciated.

Thank you,
Grizzly

[gmalchow]

It took me a while to get there, but I believe I have solved my problem.

I was too focused on getting the result of ls into an array.
Using glob was the solution to my problem:

set files [glob -type f *]

I am then able to do the following:
foreach file $files {
send "rename $file $env(ARCHIVE_DIR)$file\n"
}

or
foreach file $files {
send "rm $file\n"
}