Automated FTP Operations from PowerShell 7 using WinSCP .NET Assembly and WinSCP PowerShell Module.

This is the simplest method of automating ftp operations from PowerShell that I can come up with, having explored MANY incredibly convoluted alternatives.

It is an absolute minimum viable product that can be built upon, consisting of two downloadables that compliment one another and eight commands that probably do everything you need, and do it in a single command.

A link to the wiki for all the cmdlets is given in step 10 below.

Once the WinSCP module is installed, interfacing with an FTP server is as easy as this…
CerberusFTP Server displaying the inbound session from PowerShell using WinSCP cmdlets that call WinSCP .NET Assembly winSCPnet.dll
Don’t forget to close the ftp session when you’re done….
….and the session disappears from Cerberus FTP Server.

0. Download Software

Download matching versions of the Assembly and Cmdlets (5.17.10.0). The most recent version of the Automation .NET Assembly is 5.19.6.0 but you may have issues talking to WinSCP 5.19.6.0 using version 5.17.10.0 Cmdlets such as New-WinSCPSession where it complains about the winscp.exe version not matching the winscpnet.dll version.

WinSCP Automation .NET Assembly

https://sourceforge.net/projects/winscp/files/WinSCP/5.17.10/

WinSCP PowerShell Cmdlets (PowerShell Module)

https://github.com/dotps1/WinSCP/releases/download/WinSCP-PowerShell-v5.17.10.0/WinSCP.zip

1. Install WinSCP FTP Module in PowerShell 7

Install-Module -Name WinSCP
Import-Module -Name WinSCP
Add-Type -Path "WinSCPnet.dll"

2. Create encrypted credentials .xml file for use when connecting to ftp server automatically

$credential = Get-Credential
$credential | Export-Clixml ftpcredentials.xml
$ftpcredentials = Import-Clixml ftpcredentials.xml

3. Establish a session with an FTP Server

$ftpsessionoptions = New-WinSCPSessionOption -Credential $ftpcredentials -HostName "ftp.cyberfella.co.uk" -Protocol ftp

New-WinSCPSession -SessionOption $ftpsessionoptions

4. See if a file exists on the ftp server

Test-WinSCPPath -Path "/*.done"         


                False

5. Receive a file

Receive-WinSCPItem -RemotePath /ftptest.txt


                   Destination: C:\Users\matt\Documents\WindowsPowerShell\My Code

                IsSuccess FileName
                --------- --------
                True      ftptest.txt

6. Send a file

 Send-WinSCPItem -LocalPath "ftptest.txt"


                   Destination: \

                IsSuccess FileName
                --------- --------
                True      ftptest.txt

7. Send a folder and its entire contents (recursive by default)

Send-WinSCPItem -LocalPath "FTPTest"


                   Destination: \FTPTest

                IsSuccess FileName
                --------- --------
                True      {FTP-Functions-MB.ps1, ftptest.txt, ListFTP-MB.ps1}

8. Receive a folder and its entire contents (recursive by default)

Receive-WinSCPItem -RemotePath /FTPTest


                   Destination: C:\Users\matt\Documents\WindowsPowerShell\My Code\FTPTest

                IsSuccess FileName
                --------- --------
                True      {FTP-Functions-MB.ps1, ListFTP-MB.ps1, ftptest.txt}

9. Close Session to FTP Server

Remove-WinSCPSession

10. Wiki doc for all cmdlets here

https://github.com/dotps1/WinSCP/wiki

Additional commands required for Secure FTP (SSH Hostkey Fingerprint)

The example above was kept as simple as possible to demonstrate the minimum number of steps in order to “get things working”. Now we can build upon those steps and establish an sftp connection to the FTP Server.

Additional commands to set up an sftp session

Import-Module -Name WinSCP
Add-Type -Path "WinSCPnet.dll"

$ftpcredentials = Import-Clixml ftpcredentials.xml

$ftpsessionoptions = New-WinSCPSessionOption -Credential $ftpcredentials -HostName "10.0.2.15" -Protocol sftp

$sshHostKeyFingerprint = Get-WinSCPHostKeyFingerprint -SessionOption $ftpsessionoptions -Algorithm SHA-256

$ftpsessionoptions.SshHostKeyFingerprint = $sshHostKeyFingerprint

New-WinSCPSession -SessionOption $ftpsessionoptions

Cerberus FTP Server showing the sftp connection

Send-WinSCPItem -LocalPath "FTPTest"

Cerberus FTP Server Log entries showing the connection and the recursive transfer of a folder containing multiple files.

Receive-WinSCPItem -RemotePath /FTPTest

Recursively downloading a folder containing multiple files over sftp

Remove-WinSCPSession

The session disappears from Cerberus

My Powershell Code

# Pre-requisites
# EXECUTION POLICY SET TO BYPASS (TEST ENV)
# https://docs.microsoft.com/en-gb/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.2
# Set-ExecutionPolicy -ExecutionPolicy ByPass

# WinSCP Automation .NET Assembly
# https://winscp.net/download/WinSCP-5.19.6-Automation.zip    #WINSCP AUTOMATION .NET LIBRARY DLL
# https://winscp.net/eng/docs/library_powershell#example      #NOTES ON INSTALLATION OF DLL
# https://dotps1.github.io/WinSCP/                            #WINSCP CMDLETS MODULE

# Install (Run PowerShell as Administrator)
# Install-Module -Name WinSCP

# Import Module
Import-Module -Name WinSCP

# Cmdlets
# Get-Command -Module WinSCP

# Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"

#Create Credential
#Set credentials to a PSCredential Object.
#$credential = Get-Credential
#Export credential to xml file
#$credential | Export-Clixml ftpcredentials.xml
#Import credential from xml file
$ftpcredentials = Import-Clixml ftpcredentials.xml

#VARIABLES SECTION
$ftphost="10.0.2.15"
$ftproot="/"
$ftpdirectory="FTPTest/"
$SendItem = $ftpdirectory + ""
$ReceiveItem = $ftpdirectory + ""
$FileFilter = "*"
#END OF VARIABLES SECTION

#FUNCTIONS SECTION
function Connect-FTP {
    #Connects using sftp to ftp server
    Write-Host "Connecting to FTP Server $ftphost..."
    $ftpsessionoptions = New-WinSCPSessionOption -Credential $ftpcredentials -HostName $ftphost -Protocol sftp
    $sshHostKeyFingerprint = Get-WinSCPHostKeyFingerprint -SessionOption $ftpsessionoptions -Algorithm SHA-256
    $ftpsessionoptions.SshHostKeyFingerprint = $sshHostKeyFingerprint
    New-WinSCPSession -SessionOption $ftpsessionoptions
    Write-Host "Connected Successfully to FTP Server $ftphost."
}
function Disconnect-FTP {
    #Disconnects session to ftp server
    Write-Host "Disconnecting from FTP Server $ftphost..."
    Remove-WinSCPSession
    Write-Host "Disconnected."
}
function List-FTP {
    #Gets a list of the files matching the filter in the specified ftp directory only (1 level)
    Connect-FTP
    Write-Host "Listing files in $ftpdirectory..."
    $fileitems = Get-WinSCPChildItem -Path $ftproot$ftpdirectory -Depth 1 -File -Filter $FileFilter
    Disconnect-FTP
}   
function Send-FTP {
    #Sends a specified local file or recursive directory's contents to ftp server
    Connect-FTP
    Write-Host "Sending everything in $SendItem to $ftphost..."
    Send-WinSCPItem -LocalPath $SendItem
    Write-Host "Finished sending."
    Disconnect-FTP
}
function Receive-FTP {
    #Receives a specified remote file or directory on the ftp server recursively by default
    Connect-FTP
    Write-Host "Receiving file(s) in $ReceiveItem from $ftphost..."
    $fileitems = Receive-WinSCPItem -RemotePath $ReceiveItem
    Write-Host "Finished receiving."
    Disconnect-FTP
}
#END OF FUNCTIONS SECTION

#MAIN CODE
List-FTP
Receive-FTP
Send-FTP
#END OF MAIN CODE
Executing the script above to list files on an ftp server over sftp, download a folder and upload a folder
Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:

Missing sidHistory attributes on migrated accounts

Log onto a server and open a command prompt as Administrator.

Issue the following dsquery to create a four column, comma separated text file of all users names, logon names, primary object SID and if applicable, sidHistory SID.  Then open this .csv file in Excel and Auto Filter the sidHistory column to show all blanks.  This is the list of accounts that have NOT been subject to an inter-domain user account migration.

dsquery * “OU=Groups,OU=MigratedGroups,OU=Cromford,OU=UK,OU=DEV,OU=VMFARM,DC=cyberfella,DC=co,DC=uk” -filter “(&(objectClass=User))” -attr samAccountName cn ObjectSID sidHistory -limit 20000 > missing-sidhistories.txt

 

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:


Deleting inaccessible data such as users homedirectories

Users Home Directories are often hardened such that even Domain Administrators have problems migrating them and subsequently deleting them.  A way to deal with that is already documented here so this post is really just about the subsequent cleanup of the stubborn source data.

You can sit in Windows Explorer taking ownership and rattling the new permissions down each users tree if you like, but it’s a laborious process when you have 2000 users.  It doesn’t always work out 100% successful either.

This is my way of clearing out all users home directories that begin with the characters u5 for example.  You can adapt or scale it up it to suit your own requirements easily and save yourself a lot of time and effort.

First, make a list of the directories you want to delete.  Whether you have access to them or not is irrelevant at this stage.

dir /ad /b | findstr ^u5 > mylist.txt

dir /ad /b findstr ^U5 >> mylist.txt

Create an empty folder if you dont have one already.

mkdir empty

Now mirror that empty folder over the top of the users in the list, exploiting the operating backup right in robocopy that conveniently bypasses the NTFS security

for /f %F in (mylist.txt) DO robocopy empty %F /MIR /B /TIMFIX

This will leave empty folders behind but the security on them will have been overwritten with that of your empty folder, giving you the permission to delete it.

for /f %F in (mylist.txt) DO rmdir %F

Done.

Note: Use /TIMFIX with /B to correct non-copying of datestamps on files, resulting in 02/01/1980 datestamps on all files copied with /B Backup rights.

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:


Robocopy folders with ampersands in the name

Don’t use & in file and folder names.

With that little pearl of wisdom out of the way, what about when your users have used ampersand characters in their folder names and you’re trying to robocopy the data to it’s new home, only to have the copy fail?

Try this…

SET “source=dogs & cats”

SET “destination=dogs & cats”

or if you can get away with it without breaking links…

SET “destination=dogs and cats”

robocopy.exe “%source%” “%destination%” /MIR

For more robocopy wisdom, check this post here

In real-world practice, I have found that robocopy is woefully unreliable when it comes to copying permissions (using the /e /sec /xf * switches).  I recommend using emcopy to copy folder structures and their NTFS permissions.  Similar to the robocopy commands above, these emcopy commands worked almost* perfectly for me

SET “source=dogs & cats”

SET “destination=dogs and cats”

emcopy “%source%” “%destination%” /secfix /xf * /lev:1

*Note how I’ve changed the destination folder to not include the ampersand character.  In practice, permissions were not copied to folders with ampersands in the name using robocopy or emcopy – in fact robocopy didn’t copy permissions at all!

If you’re copying a subset of data from a bigger source set of data, then never use /MIR or you will run a high risk of loosing data.   Oh yes you will.  Use the above emcopy commands one folder at a time to get your destination folder structure in place, before finally syncing the subfolder you want into the new destination.  This saves a potentially troublesome cleanup exercise later, deleting superfluous data, e.g.

 

SET “source=dogs & cats”

SET “destination=dogs and cats”

emcopy “%source%” “%destination%” /secfix /xf * /lev:1

Followed by…

SET “source=dogs & cats\spaniels”

SET “destination=dogs and cats\spaniels”

emcopy “%source%” “%destination%” /secfix /xf * /lev:1

Followed by…

SET “source=dogs & cats\spaniels\springer”

SET “destination=dogs and cats\spaniels\springer”

emcopy “%source%” “%destination%” /secfix /xf * /lev:1

and finally sync your file data into the new secured folder structure…

SET “source=dogs & cats\spaniels\springer”

SET “destination=dogs and cats\spaniels\springer”

Synchronise all file data using your preferred robocopy or emcopy command here.

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:


Export all users in ActiveDirectory

If you’re tasked with generating a list / creating a spreadsheet of all user accounts in AD but are worried you might miss out an OU when manually going through and exporting the list using the Active Directory Users and Computers MMC Snap-in, then use Powershell to generate a list instead, safe in the knowledge it’ll find everything.

If you’re really keen you can subsequently use GNUWin32 to give you neat command line tools usually only available to a bash command prompt on a Linux/UNIX OS to chop columns out of the exported csv file using cut, awk, sort and uniq.  Or just use Excel to achieve it.  More on GNUWin32 here.

Open a Powershell and type the following to export all users in the directory to a csv file…

Import-module activedirectory

get-aduser -filter * | Export-Csv c:\myusers.csv

Since the OU Path’s are themselves comma separated, it throws the keys in the csv out of alignment, making it challenging to extract the columns to the right of it that contains the samAccountName  “Logon Name”.  To get over this hurdle, go back to PowerShell and be more specific about the exact key (or Label) you want, e.g. if you just want a list of Logon Names for all users in AD, then this command works…

get-aduser -filter * | select-object @{Label = “Logon Name”;Expression ={$_.saMAccountName}} | Export-Csv D:\ADUsers\ADUsers.LogonNames.csv

Some other useful Labels you may want to use are shown below for your convenience (including a neat If statement for extracting Disabled Accounts).

@{Label = “First Name”;Expression = {$_.GivenName}}
@{Label = “Last Name”;Expression = {$_.Surname}}
@{Label = “Display Name”;Expression = {$_.DisplayName}}
@{Label = “Logon Name”;Expression = {$_.sAMAccountName}}
@{Label = “Full address”;Expression = {$_.StreetAddress}}
@{Label = “City”;Expression = {$_.City}}
@{Label = “State”;Expression = {$_.st}}
@{Label = “Post Code”;Expression = {$_.PostalCode}}
@{Label = “Country/Region”;Expression = {if (($_.Country -eq ‘GB’) ) {‘United Kingdom’} Else {”}}}
@{Label = “Job Title”;Expression = {$_.Title}}
@{Label = “Company”;Expression = {$_.Company}}
@{Label = “Description”;Expression = {$_.Description}}
@{Label = “Department”;Expression = {$_.Department}}
@{Label = “Office”;Expression = {$_.OfficeName}}
@{Label = “Phone”;Expression = {$_.telephoneNumber}}
@{Label = “Email”;Expression = {$_.Mail}}
@{Label = “Manager”;Expression = {%{(Get-AdUser $_.Manager -server $ADServer -Properties DisplayName).DisplayName}}}
@{Label = “Account Status”;Expression = {if (($_.Enabled -eq ‘TRUE’) ) {‘Enabled’} Else {‘Disabled’}}}
@{Label = “Last LogOn Date”;Expression = {$_.lastlogondate}}

You can combine the Labels above in a single command with a comma in the select-object section, for example to extract all logon names and whether or not the account is disabled…

get-aduser -filter * | select-object @{Label = “Logon Name”;Expression ={$_.saMAccountName}},@{Label = “Account Status”;Expression = {if (($_.Enabled -eq ‘TRUE’) ) {‘Enabled’} Else {‘Disabled’}}} | Export-Csv D:\ADUsers\ADUsers.LogonNames.csv

I had some trouble with the LastLogon Label, so have included the working example used to obtain this information below.

get-aduser -filter * -properties * | select-object @{Label = “LogonName”;Expression = {$_.saMAccountName}},@{Label = “LastLogonDate”;Expression = {$_.LastLogonDate}}| Export-Csv D:\ADUsers\ADUsers.LastLogon.csv

 

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:


How to cable up VNX SP Ports (Dual Fabric topology)

So your VNX has two SP’s and you have two fabric switches.  You already know you have to connect each SP to each fabric for resilience, but you’re still a bit confused.  Fear not.  Use this as a guide.  It can be used no matter how many front-end port modules and SFP’s you have so that you get it right first time for all your designated Storage Ports, Mirrorview Ports, Sancopy Ports required for your project.  The FC Switch Ports you choose are not set in stone, but keep it the same on each side at least.

 What’s important is that the correct SP port, goes to the correct switch.

When you know you’ve got it right, you can get verify the WWN in Unisphere corresponds with the WWN logged in on the FLOGI database on the switch to check before you create the requisite fcaliases etc.

cablingvnx

It’s much easier to build it right first time than sort it out afterwards.

 

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:


Inject Administrators/Full Control permissions into inaccessible folders.

Note:  This can also be used to inject Everyone/Full Control, or a specific user, using the username or SID.  The Administrators Group SID is always S-1-5-32-544.  Other well-known SIDs are listed here.

Download the command line version of SetACL.exe from here.  Like all the best things in life, it’s free.

Open a command prompt as Adminstrator (right click cmd.exe, run as admin)

setacl -on “C:\Private No Entry” -ot file -actn ace -ace “n:Administrators;p:full” -rec cont_obj -ignoreerr

The “Private No Entry” folder should now have Administrators, Full Control Permissions.  If not, don’t fret, read on…

The following command gives Administrators the “dream ticket” to accessing all data by setting ownership to Administrators on all folders and files and forcing subdirectories to re-inherit inheritable Administrators:Full Control permissions from the parent.

setacl -on “C:\Private No Entry” -ot file -actn setprot -op “dacl:np;sacl:nc” -rec cont_obj -actn setowner -ownr “n:S-1-5-32-544”

If you still receive “Operating System Message:Access Denied” or similar, then you’ll need to take a robocopy of the “inaccessible” data using the /B switch to exploit OS Backup Right, leaving permissions behind using /COPY:DAT (instead of /COPY:DATSOU or /COPYALL) then repeat the process above on the copied data instead.

robocopy “C:\Private No Entry” “T:\Cracked Data” /B /COPY:DAT /E /NP /R:1 /W:1

Now view the Inherited permissions on the copied data…  You’ll see it has a whole bunch of new, open permissions that it’s got from the parent folder T:.

cacls “T:\Cracked Data”

The cracked data could be robocopied back over the original inaccessible source data using /MIR /COPYALL /SEC /SECFIX switches if required.  If it doesn’t allow it, then note that I have successfully robocopied an empty folder over the top of an inaccessible folder before using just /MIR  (in order to delete it), then robocopied the cracked data back into place, e.g.

robocopy “T:\Empty Folder” “C:\Private No Entry” /MIR /B

robocopy “T:\Cracked Data” “C:\Private No Entry” /MIR /SEC /B

Finally, if you want to re-harden the folder whilst retaining the access you’ve granted Administrators, then use the following commands…

Presently, access has been attained via inherited permissions so before removing inheritance, first inject a non-inherited ACE that allows administrators access, i.e.

setacl -on “C:\Private No Entry” -ot file -actn ace -ace “n:S-1-5-32-544;p:full” -rec cont_obj

Verify the Administrators:Full Control permissions are present on the folder

cacls “C:\Private No Entry”

Finally it is safe to remove inheritance without losing access (strictly speaking, you are “protecting the child object from inherited permissions on the parent object”)

setacl -on u567149 -ot file -actn setprot -op “dacl:p_nc;sacl:p_nc”

This sequence of commands can be used to copy users home directories that are typically hardened to only permit the user themselves access to the data contained within.  If you are using it to migrate home dierctories, there is a loop to re-apply user-specific permissions to each homedirectory afterwards here

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:


Deleting Windows data where the path length exceeds 260 characters

After migrating Windows data, it can be a royal pain cleaning up the source data using del *.* /s /q /f, especially when the path length exceeds 260 (or thereabouts) characters.  You can manually shorten the folder names and keep trying, but this may be time consuming, tiring and ultimately futile.

The simplest way I’ve found to reliably delete data, irrespective of path length, is to use robocopy.

  1. cd into the directory that you want to empty
  2. create a new empty subdirectory called empty
  3. rename all other adjacent folders 1, 2, 3, 4 etc if possible
  4. robocopy empty 1 /mir /r:1 /w:1
  5. repeat for each adjacent folder, 2, 3, 4 etc.

 

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:


Robocopy leaves some NTFS permissions behind?

And so does emcopy and icacls /save & /restore doesn’t work either…

Googling doesn’t help – every solution that promises to work, doesn’t.

Solution?

ALWAYS USE THE /B SWITCH!

Sorry for shouting, but I’m really rather excited to have cracked this major show stopper for my clients data migration.  The /B switch uses the Backup right to perform the copy.  That’s presumably running with system level privs, rather than my meager admin account in cmd run as administrator.  Magic.  Data integrity restored!  Professional reputation saved!

Note: Use /TIMFIX with /B to correct non-copying of datestamps on files, resulting in 02/01/1980 datestamps on all files copied with /B Backup rights.

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:


Find ACL’s that don’t match parent with AccessEnum

When data begins its life, the permissions are invariably set at one of the top two or three levels, Some examples of some types of data you wouldn’t want everyone to get their hands on in a given company might be…

X:\Payroll\Cost Centre 1\Salaries

X:\Human_Resources\Cost Centre 2\Disciplinaries

X:\Pensions\Cost Centre 3\Compulsory Redundancy Quotes

All users would have to have access to the top level share, then all users might still have access to the Human Resources, Payroll and Pensions folders too.   There may or may not be a cost centre folder to aid the business in understanding what parts of what department are consuming the most storage space (for internal billing), then there will be the “parent” folders (top level in terms of the point where specific permissions are set) whereby only certain individuals in a global group will have access.  ABE (Access Based Enumeration) might be enabled also, preventing users who don’t have access to certain folders from even seeing that the folder exists in the first place.

For the purposes of data migration and reporting, the IT department should have some kind of data admins global group that has full control from the top, all the way down.  Backup admins will also need modify access in order to perform file recoveries back to their original locations.

Over time, chunks of data get moved about as a result of departmental “tidy ups” and restructuring.  The effect this has on permissions is that when data is “copied” it inherits the security of the parent folders on the destination side, however when it is “moved” it takes its permissions with it – sort of.  I say “sort-of” because although it takes the security ACE’s with it, it also takes the attribute that says those ACE’s were originally inherited from the parent.

I say “sort-of” because although it takes the security ACE’s with it, it also takes the attribute that says those ACE’s were originally inherited from the parent.  And therein lies a problem for subsequent data migrations.

The effect this has, is to cause the folders on the migrated side (where a copy operation has been used) to subsequently re-inherit their permissions from their new parent.  Only non inherited permissions and inheritance attributes are ever copied, so these permissions that were once inherited but can no longer be inherited are likened to “ghosts” and are lost during any kind of copy operation (data migration operation using emcopy/robocopy or some other method).

I’d like to use “orphans” to describe these children with no matching parent permissions but the term is already taken to describe SIDs in ACEs that no longer resolve to a group in AD (occurs when a folder is secured then the group subsequently deleted), so I’ve come up with “ghosts” to describe them.  I can’t use “zombies” as that is taken to refer to a child process on a UNIX system that has completed execution but remains in the process table so until its parent process can read its exit status.  “Ghosts” is quite apt as they are not tangible and vanish when you migrate the data.

Technically, the copy operation is correcting anomalies but in the real world, that means loss of access, or worse, open access, depending on the nature of the change of permissions on the new parent.

The best way to deal with these ghosts is at source, but you need to know about them first, in order to deal with them.  Sysinternals accessenum GUI tool is a neat way to scan filesystems looking for children where the security differs from that of its parent.  Be sure to set the options accordingly.

accessenum

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash: