SFTP configuration
Secure File Transfer Protocol (SFTP) is a network protocol for
securely accessing, transferring, and managing large files and
sensitive data.
To use SFTP, an SFTP client and an SFTP server are required.
An SFTP Client is software that gives you the ability to connect
to an SFTP Server. This client software allows you to upload
files to be stored on the server, or to download files that are
already stored on the server.
An SFTP Server is basically a place to store files or a place
where you can retrieve files. For example, a software vendor
may store software updates on an SFTP Server and allow
customers to download the files with an SFTP Client.
SFTP Client Installation
SFTP usually runs on Port 22 (port numbers are part of the
addressing information used to identify the senders and
receivers of messages).
SFTP client are built in already preinstall on the windows 2019 and above.
Can verify this:
-
Go to Windows Settings–>Apps.
Click on “Manage optional features” Under Installed Apps.
Look for OpenSSH Client, check if it’s already installed, if not click on “Add a feature” to install
SFTP Server Installation
To configure SFTP Server using built in windows
This feature only available to windows 2019 and above.
Installation with Internet access.
-
Go to Windows Settings–>Apps.
Click on “Manage optional features” Under apps and features menu.
Look for OpenSSH Server, check if it’s already installed, if not click on “Add a feature” to install
Then make sure the OpenSSH Server Services are running in Automatic (Delayed)
Configure Windows Firewall for SFTP Server
In Powershell (Runs as Admin), add a new firewall script rule below:
New-NetFirewallRule -Name sshd -DisplayName 'OpenSSH Server (sshd)' -Enabled True -Direction Inbound -Protocol TCP
-Action Allow -LocalPort 22
Once firewall successfully added, can verify communication either by “cmd telnet” or “powershell tnc”
CMD Telnet
Powershell TNC
Set Up “Keygen” on SFTP Client
Launch Powershell with Administrative privileges. Generate the public private key pairs:
“ssh-keygen”
Two files are created in C:\Users\<USER>\.ssh (where <USER> is the username for the currently logged
in user):
- RSA private key: id_rsa
- RSA public key: id_rsa.pub
Transfer “Keygen” to SFTP Server
Copy the file id_rsa.pub to the SFTP Server directory “C:\ProgramData\ssh” rename file id_rsa.pub to
“administrators_authorized_keys”. Do not add a file extension.
Or can transfer and rename using “scp” function on powershell.
“scp C:\Users\<User>\.ssh\id_rsa.pub > <user>@<ip address>:C:\ProgramData\ssh\administrators_authorized_keys”
“scp C:\Users\System1Admin\.ssh\id_rsa.pub System1replication@192.168.0.33:C:\ProgramData\ssh\administrators_authorized_keys”
Verify “Keygen” Binding
From SFTP Client run “SFTP” command to verify keygen binding successful. If request for password the
binding was unsuccessful. Verify again the keygen and file name.
Configure SFTP script on SFTP Client
Use files and paste it to SFTP Server Client directory “C:/Windows/System32/OpenSSH”
SFTP_Client_Script_L3.
ps1
Open and edit the file above for item below:
# SFTP User Account
$SftpUserAccount = "<User that generate keygen>"
# SFTP Server Computername
$SftpServerComputername = "<SFTP Server>"
# Location on System 1 Evolution Server where replicated files are being created
$S1EvoDataFileDirectory = "<source file>"
# Folder on SFTP Server where System 1 Evolution Data Files are to be placed
$SftpServerFolderPath = "<destination folder>"
###############################################################################
# SCRIPT:
SFTP_Client_Script.ps1
# DESCRIPTION: The purpose of this PowerShell script is to automate the process
#
of calling the SFTP Client (sftp.exe) with the list of System 1
#
Evolution data files that are to be moved to the SFTP Server.
# TODO:
# - Automate Script through Scheduled Tasks
# - Secure Script
# - Add logging to Script
###############################################################################
# SFTP User Account
$SftpUserAccount = "<User that generate keygen>"
# SFTP Server Computername
$SftpServerComputername = "<SFTP Server>"
# Location on System 1 Evolution Server where replicated files are being created
$S1EvoDataFileDirectory = "<source file>"
# File filter criteria for System 1 Evolution related files
$S1EvoAllFilter = "*.S1*"
# SFTP Executable
$SftpExecutable = "sftp.exe"
# Folder Location of SFTP Client (sftp.exe)
$SftpExeFolderPath = "C:/Windows/System32/OpenSSH"
# Filename for SFTP.exe commands
$SftpCommandFileName = "sftp_file_transfer_script.txt"
# Folder on SFTP Server where System 1 Evolution Data Files are to be placed
$SftpServerFolderPath = "<destination folder>"
# Log files created from call to sftp.exe
$SftpProcessStandardOutputFileName = "sftpProcessStandardOutput.log"
$SftpProcessStandardErrorFileName = "sftpProcessStandardError.log"
# Sleep Delay (seconds) for SFTP processing loop
$SleepDelay = 1
# SFTP Max Files to Batch per call
$SftpBatchMaxFileCount = 500
###############################################################################
# FUNCTION: CreateSftpBatchFile()
# INPUTS: $ListOfS1EvoDataFiles - This list contain the S1 Evo Data Files
#
that are to be pushed to the SFTP Server.
# OUTPUTS: $SftpCommandFileName - This is the text file containing the commands used
#
by sftp.exe.
# This function is used to construct the text file used as input to sftp.exe
# There are basically three parts to this text file:
# - Add a command to change to the appropriate directory on the SFTP Server computer.
# - Looping function that will take each S1 Evo Data File passed into the function and
# append a "put" command before the file.
# - Call to "bye" to finish the SFTP client session with the SFTP Server.
###############################################################################
Function CreateSftpBatchFile{
param([string[]]$ListOfS1EvoDataFiles)
$SftpBatchFileContent = $null
$PutCommand = "put"
$ByeCommand = "bye"
$SftpBatchFileContent += "cd $SftpServerFolderPath `r`n"
foreach($file in ($ListOfS1EvoDataFiles | sort))
{
$SftpBatchFileContent += "$PutCommand " + [char]34 + "$S1EvoDataFileDirectory$file" + [char]34 + " `r`n"
}
$SftpBatchFileContent += "$ByeCommand `r`n"
}
New-Item -Path $S1EvoDataFileDirectory -Name $SftpCommandFileName -Value $SftpBatchFileContent -ItemType file -Force
Write-Host "Starting SFTP Transfer Script..."
while($true)
{
# --- STEP 1: Retrieve the current list of S1 Evo replication files
# Currently just finding all data files produced by S1 Evo
$FilteredDirectory = "$S1EvoDataFileDirectory$S1EvoAllFilter"
#Write-Host $FilteredDirectory
if(Test-Path $FilteredDirectory)
{
$ListOfS1EvoFilteredDataFiles = $null
$ListOfS1EvoFilteredDataFiles = Get-ChildItem -Path $FilteredDirectory -file -name | sort
if ($ListOfS1EvoFilteredDataFiles.Count -gt 0)
{
if ($ListOfS1EvoFilteredDataFiles.Count -gt $SftpBatchMaxFileCount)
{
$ListOfS1EvoFilteredDataFiles = $ListOfS1EvoFilteredDataFiles[0..$SftpBatchMaxFileCount]
}
# --- STEP 2: Construct the SFTP script file
#Write-Host "Constructing SFTP Script File"
$SftpFilteredBatchFile = CreateSftpBatchFile $ListOfS1EvoFilteredDataFiles
# --- STEP 3: Call SFTP with script file
Write-Host "SFTP Transferring File(s)"
Start-Process -FilePath "$SftpExeFolderPath/$SftpExecutable" `
-ArgumentList "-b $S1EvoDataFileDirectory/sftp_file_transfer_script.txt $SftpUserAccount@$SftpServerComputername" `
-Wait `
-NoNewWindow `
-WorkingDirectory $SftpExeFolderPath `
-RedirectStandardOutput "$S1EvoDataFileDirectory/sftpProcessStandardOutput.log" `
-RedirectStandardError "$S1EvoDataFileDirectory/sftpProcessStandardError.log"
if ((Get-Item $S1EvoDataFileDirectory/$SftpProcessStandardErrorFileName).length -gt 0kb)
{
Write-Host "ERROR: There was an issue calling sftp.exe`r`n"
}
else
{
# --- STEP 4: Remove the list of S1 Evo Data Files that have been transmitted
Write-Host "Removing transferred File(s)"
foreach($remFile in $ListOfS1EvoFilteredDataFiles)
{
Remove-Item $S1EvoDataFileDirectory/$remFile
}
}
}
else
{
#Write-Host "No System 1 Evolution Files found during this run"
}
}
else
{
#Write-Host "No System 1 Evolution Files found during this run"
}
#Write-Host "Sleeping script for $SleepDelay seconds"
Start-Sleep -s $SleepDelay
Configure bootable file
Open new notepad and use following command.
powershell.exe -file " C:/Windows/System32/OpenSSH SFTP_Client_Script_L3.ps1”
Save the notepad as “start up.cmd“ and paste on the same directory as sftp transfer script
“C:/Windows/System32/OpenSSH”.
Configure Task Scheduler
Launch the Windows Task Scheduler app.
Select Task Scheduler Library.
In the Actions pane, select Create Task. The Create Task dialog box is displayed.
In Name, enter SFTP Client Script.
Enter a description, for example, “Automated script to transfer System 1 Evolution data files to File
Server using SFTP”.
In Security options, modify When running the task, use the following user account. Select a user
account with permissions to run scripts. Make sure is the same user that are generated the keygen on
SFTP client.
Select Run whether user is logged on or not.
Select the Triggers tab and click New.... The New Trigger panel is displayed.
a.
b.
c.
d.
e.
In the drop-down list box Begin the task, select On a schedule.
In Settings, choose Daily.
In Advanced Settings, select the Repeat task every drop-down list box and choose 1 hour.
In the for a duration of drop-down list box, choose Indefinitely.
Click OK. The Create Task dialog box is displayed.
Select the Actions tab and click New.... The New Action panel is displayed.
a. In the drop-down list box Action, select Start a program.
b. In Settings, Program/script, enter C:\Program Files\OpenSSH\start up.cmd.
c. Click OK. The Create Task dialog box is displayed. 19.
Select the Settings tab. The Settings panel is displayed. Enable these settings:
a. Run task as soon as possible after a scheduled start is missed.
b. Select the If the task fails drop-down list box and choose 1 minute.
c. Click OK. The Create Task dialog box is displayed. 20.
Select Task Scheduler Library.
Select the SFTP Client Script task in the central pane.
In the Actions pane, select Export.... The Save As dialog box is displayed.
Choose a folder in which you want to save the script.
In the File name field, enter SFTP Client Script.xml.
Click Save.
Using Windows File Explorer, access the script file.
Open the folder location in which you saved SFTP Client Script.xml.
a.
b.
c.
d.
Open the file in a text editor (like Notepad).
Find this code: <Priority>7</Priority>. The number may be different.
Change 7 to 1. This causes Task Scheduler to run the process as high priority.
Save and close the file.
Launch the Windows Task Scheduler app.
Select Task Scheduler Library.
Delete the task named SFTP Client Script you previously created.
a.
b.
c.
d.
In the Actions pane, select Import Task.... The Open dialog box is displayed.
Navigate to the folder where you previously saved the script.
Select the SFTP Client Script.xml file.
Click Open. The Create Task dialog box is displayed.
Click Save.