Error: “The requested operation could not be completed due to a file system limitation (mscorlib”

Background

Experiencing an error when trying to run an embedded process.  Part of the functionality that is bundled in the process, is a file copying module.

Error Message

Here is the error message :-

*** Error: The requested operation could not be completed due to a file system limitation (mscorlib) ***

Trouble Shooting

Application Message

Thankfully the particular file that is copied is logged, as well.


Copying log backup file to temporary work file. Source: '\\LABDB\Backup\hrdb_20180703104507.trn', Destination: '\\LABMirror\backup\HRDB_20180703104507.wrk'

Fast Copy

Attempted same file copy in FastCopy.

And, again thankfully it reported an error message; along with an error number.

Image

FastCopy_20180705_0712PM_v2.png

Text


WriteFileWait(The requested operation could not be completed due to a file system limitation665) : \\LABMirror\backup\HRDB_20180703104507.wrk
TotalRead  = 40,414 MB
TotalWrite = 40,289 MB
TotalFiles = 0 (0)
TotalTime  = 26:50
TransRate  = 25.0 MB/s
FileRate   = 0.00 files/s 

Explanation

  1. Error
    • Error Operation :- WriteFileWait
    • Error Description :- The requested operation could not be completed due to a file system limitation
    • Error Number :- 665

 

Error Causation

There are a few probably causes for error number 665.

Compression?

File

Finally, we checked the compression setting on others files in the targeted folder.

File Property

Image

file_20180706020000_trn_brushedup.png

Textual
  1. Size :- 2.07 GB
  2. Size on Disk :- 760 MB
Explanation

We can see that our file is compressed.

Folder Settings

Reviewed the Folder’s Advanced Attributes.

Image

AdvancedSettings_20180705_0717PM.png

Explanation

  1. Compress Contents to save disk space ( is enabled )

 

Remediation

Compression

Setting

We turned off compression on the targeted folder.

Fast Copy

Retried Fast Copy and it worked successfully.

NoError__20180705_1014PM_v2.png

 

Summary

Wished the original application displayed the OS’s error number and not just a simple error text that read “Error: The requested operation could not be completed due to a file system limitation (mscorlib)“.

Thankfully, there was enough diagnostic data in regards to the original file and destination folder that allowed us to replay via a more illuminating tool.

 

 

 

 

 

Icacls – Usage – Scheduled Tasks

Background

In an earlier post, Raimund Andrée – NTFSSecurity, Link, we spoke of having considered using Microsoft’s icacls to review NTFS permissions.

icacls

Let us try using Microsoft’s icacls.

 

Sample Script

Scenario

In our use case we will be using icacls to review NTFS Permissions on the C:\Windows\System32\Tasks\ folder.

Windows saves metadata on scheduled tasks in this folder.

 

 

Code Overview

  1. Issue “setlocal” to initiate localize environment settings
  2. Set environment variables
    • set target folder to C:\Windows\System32\Tasks\
    • set appPgm to icacls
    • Set local variables to indicate entries that we will like to discard from showing in the output
  3. Issue forfiles
    • /p
      • set Target folder
        • “%_folder%.”
          • Notice the period that prefixes the enclosing quotes
          • It is needed to handle Windows batch treating the ending backslash as an escape
    • /c
      • Set command to invoke
        • cmd /c
        • if @ISDIR==FALSE
          • Skip folders, work only actual files
        • if @ext==\”\”
          • Tasks are saved without extensions and so limit files we work on to those files that do not have extensions
        • Issue “%_appPgm% @PATH /q”
          • _appPgm is previously set to icacls
            • Parameters
              • @PATH
                • Forfiles is asked to substitute the current full filename
              • /q
                • Tells icacls to suppress success when displaying messages
  4. Issue “endlocal” to revert localized environment settings

Code Actual


setlocal

set _folder=C:\Windows\System32\Tasks\
set _appPgm=icacls

set _skipInherited=
set _skipSP=

set _skipInherited=find /V "(I)"
set _skipSP=find /V "Successfully processed"

REM Added period (.) after folder name for /p argument

forfiles /P "%_folder%." /c "cmd /c  if @ISDIR==FALSE  if @ext==\"\"  echo @PATH | %_appPgm% @PATH /q"  | %_skipInherited% | %_skipSP%

endlocal

Output

Listening

Talking about traps and ‘ving to escape them.

Xscape – Who Can I Run To
Link

 

References

  1. Command Line Reference
    • Icacls
      • Management and Tools > Command-Line Reference > Command-Line Reference
        Link
    • Setlocal
      • TechNet Archive > Windows XP > Command-line reference A-Z
        Link
  2. QandA
    • StackOverflow
      • Forfiles Batch Script (Escaping @ character)
        Link
      • How to use forfiles to delete files without extension
        Link
  3. Blogs
    • SS64
      • FORFILES.exe (Native command in Vista/Windows7/2008, via Resource Kit for XP)
        Link
    • Windows Command Line
      • Srini

Raimund Andrée – NTFSSecurity – Usage Scenario – Day 1

Background

Now that we have downloaded and installed Raimund Andrée’s NTFSSecurity in one of the standard PowerShell Module’s folder, we are ready to write a little test code and see how well it works.

Code

Script

getNTFSPermissions.ps1


param (
      [string]$file
    , [string]$folder
    , [string]$fileExt
 )
Set-StrictMode -Version 2.0
 
#Import NTFSSecurity
Import-Module NTFSSecurity

# Declare variables
[string] $CONST_FILEMODE_DIRECTORY = "d-----";
[boolean] $fileCheck = $false;

if ([string]::IsNullOrEmpty($file))
{
    $fileCheck = $false;
}
else
{
    $fileCheck = $true;

}

if ($fileCheck -eq $false)
{

    if ([string]::IsNullOrEmpty($folder))
    {
    
        $folder = Get-Location
        
    }
    
}

function getNTFSFile([string] $_fileLocal)
{

       
    Get-NTFSAccess -Path $_fileLocal

    
} #getNTFSFile()    

function getNTFSFolder([string] $folderLocal, [string] $fileExtLocal)
{

    #Declare Local variables
    [string]$_file = $null;
    [string]$_fileFullName = $null; 
    [string]$_fileExt = $null;
    [string]$_fileMode = $null;   
    [boolean]$_fileExtMatch = $true;

    # Get files
    Get-ChildItem $folderLocal | foreach {

       $_file = $_
       $_fileFullName =  $_.FullName 
       $_fileExt = $_.extension
       $_fileMode = $_.mode    
       
       <# #$_file #$_fileFullName #$_fileMode #>
       
       $_fileExtMatch = $true;

       <# If we are matching on file extensions let us see whether it matches #>
       if ([string]::IsNullOrEmpty($fileExt ))
       {
            $_fileExtMatch = $true;
       }
       else
       {
       
            if ( $_fileExt -eq $fileExtLocal )
            {
                $_fileExtMatch = $true;
            }
            else
            {
                $_fileExtMatch = $false;
                
                #"file extension $_fileExt does not match $fileExt "
            }
       
       }
       
       
       if (`
                 ($_fileMode -ne $CONST_FILEMODE_DIRECTORY )`
            -and ( $_fileExtMatch)`
          )
       {
       
            Get-NTFSAccess -Path $_fileFullName -ExcludeInherited

       }
        
    }
    
} #getNTFSFolder()  

if ($fileCheck -eq $true)
{

    getNTFSFile $file 

}

elseif ($fileCheck -eq $false)
{

    getNTFSFolder $folder $fileExt

}


 

Sample

Get Permissions for Excel files

Code


powershell .\getNTFSPermissions.ps1 -folder C:\temp -fileExt .xlsx

Output

Get Permissions for Scheduled Tasks ( Local to machine)

Code


powershell .\getNTFSPermissions.ps1  -folder C:\Windows\System32\Tasks

Output

Raimund Andrée – NTFSSecurity

Introduction

As  a quick follow-up to our last post, “Task Scheduler – The user account is unknown, the password is incorrect, or the user account does not have permission to modify the task” ( Link ), googled for available utilities that list NTFS permissions.

Utilities

Here are some available options:

  1. Microsoft
    • icacls
  2. Raimund Andrée – NTFSSecurity
    ( powershell module )

Raimund Andrée – NTFSSecurity

We settled on “Raimund Andrée – NTFSSecurity“, we will discuss the reasons later.

Repository

URL

  1. GitHub
    • Releases

Image

Tabulated

Version File Size
NTFSSecurity 4.2.3 NTFSSecurity.zip 183 KB

 

Installation

Prepare Downloaded File

Once downloaded please unblock file…

Obviously to unblock, please click the “Unblock” button.

 

Identify Install Folder

In Powershell parlance the files are delivered as modules and need to be placed in one of the folders listed in the PSModulePath environment variable.

Command


set PSModulePath

Output

Image

Tabulated

  1. C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\
  2. C:\Program Files\WindowsPowerShell\Modules\
  3. SQL Server
    • C:\Program Files (x86)\Microsoft SQL Server\110\Tools\PowerShell\Modules\
    • C:\Program Files (x86)\Microsoft SQL Server\120\Tools\PowerShell\Modules\
    • C:\Program Files (x86)\Microsoft SQL Server\130\Tools\PowerShell\Modules\
  4. Baseline Configuration Analyzer
    • C:\Program Files\Microsoft Baseline Configuration Analyzer 2\Modules\

 

Vendor’s Installation Guideline – Location

Image

How to install
Link

 

Explanation

  1. Development
    • During development I think it is best to place in the contextual user’s documents\windows\powershell folder
  2. Production
    • In production, depending on your Version of Power
      • <= v4 C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules
      • >= V4
        • C:\Program Files\WindowsPowerShell\Modules

Deploy

Script

Sample


set _folderSrc=C:\downloads\raandree\NTFSSecurity\NTFSSecurity

set _folderDestUserSpecific=C:\Users\%username%\Documents\WindowsPowerShell\Modules\NTFSSecurity
set _folderDestPowerShellModule=C:\Program Files\WindowsPowerShell\Modules

set _folderDest=%_folderDestPowerShellModule%\NTFSSecurity

if not exist "%_folderDest%" mkdir "%_folderDest%"

xcopy "%_folderSrc%" "%_folderDest%" /s /D

Output

Validation

Script

Sample


# Import NTFSSecurity
Import-Module NTFSSecurity

#get help on Get-NTFSAccess
get-help Get-NTFSAccess

Output

Dedicated

Dedicates to MSFT’s own Raimund Andrée.

References

  1. Tutorial
    • NTFSSecurity Tutorial 1 – Getting, adding and removing permissions
      Link
    • NTFSSecurity Tutorial 2 – Managing NTFS Inheritance and Using Privileges
      Link
  2. Script Center
    • File System Security PowerShell Module 4.2.3
      Link

WinOS – Identify Large Files Using WinDirStat

Introduction

This is the second in a series of posts on identifying the largest set of files on Windows Computer.

Lineage

  1. WinOS – Identify Large Files Using SpaceSniffer
    Link

 

Download

Please download the portable version of WinDirStat from here.

 

Usage

Launch

As it is a portable app, no need to install, just launch it.

 

Results Sorted By Percentage

 

windirstat_sortedbypercentage_20170214_1002am

 

Results Sorted By Percentage ->Folder – Personal

Here we dig in deeper into the Folder \ personal\work folder and located backup files from one of our vendors

windirstat_sortedbypercentage_backupfiles_20170214_1009am

 

WinOS – Identify Large Files Using SpaceSniffer

 

Introduction

As I left the house of worship on a couple of days ago, someone asked me what is the best way to free up storage on a hard drive.

I said I will send him some notes in the days thereafter.

 

Downloaded

Downloaded SpaceSniffer from Uderzo Software.

The product folder is here.

And, the Portable version is here.

fosshub

The latest version is 1.3.0.2 and file size is 1.58 MB.

 

Usage

Again, no need to install, just run as it is a portable software.

 

Outline

  1. The first thing to do is to choose your targeted Drive
    • In our case, we only have one drive and that is our Local Disk ( C: )
  2. Scan Starts
    • The Application starts scanning our targeted disk
  3. Scan Results
    • Found Files includes
      • hiberfil.sys
      • pagefile.sys
  4. Filter
    • It is beneficial to narrow on big files or specific file types
    • Filtering is accomplished by entering tags such as
      • Size
        • >10mb
          • Files greater than 10 Megabytes
      • Size and File Types
        • >10mb; *.mp*
          • File Sizes greater than 10 Megabytes
          • File Types *.mp* – *.mp will include mp3 & mpeg
  5. Errors & Warnings
    • Access Denied
      • There are some folders that are locked down.  The list includes
        • c:\windows\system32\LogFiles
        • c:\windows\system32\spool

 

Screen Shots

Choose your Space

chooseyourspace_20170213_1215pm

Master Status : Scanning…

masterstatusscanning

 

Filtering

Filtering by File Size

Filtering by File Size > 10MB

filteringbyfilesizes_10mb_20170214_0940am

Filtering by File Size & File Types

Filtering by File Size > 10MB and file Types *.mp*

filesizesandfiletypes_20170214_0947m

 

Errors and Warnings

Access Denied

accessdenied_20170214_0930am

NTFS Permissions – Via “Net Share”

Background

Have a need to create a few shares and grant permissions to them.

We will use NetShare to automate the Share Creation and permissioning.

 

Use NetShare To Create And Grant Permissions

Using NetShare grant permissions to Share SQLErrorLog

Code


set "_share=SQLErrorLog"
set "_folder=M:\Microsoft\SQLServer\SQLErrorLog"

set "_user_everyone=everyone"


set "_networkUserQA=LAB\DBQA$
set "_permission_read=Read"
set "_permission_full=Full"
set "_permission_change=Change"

if not exist %_folder% mkdir %_folder%

net share %_share%=%_folder% /GRANT:"%_user_everyone%",%_permission_read% /GRANT:"%_networkUserQA%",%_permission_change% 

Explanation

In the sample above, we did the following:

  1. Created a new network share by the name SQLErrorLog
  2. It is sourcing M:\Microsoft\SQLServer\SQLErrorLog
  3. We are granting the following permissions
    • Read to everyone; that is all local users
    • Change ( Read and Write) to the Network Service Account on machine DBQA.  Btw, the Domain name is LAB

Review Share Permissions

Code


net share SQLErrorLog

Output

netshare-qa

 

Use NetShare to drop Share

Code


set "_share=SQLErrorLog"
set "_folder=M:\Microsoft\SQLServer\SQLErrorLog"

if not exist %_folder% mkdir %_folder%

net share %SQLErrorLog%

if %errorlevel% EQU 0 Echo Deleting Share %_share% ... | net share %_share% /DELETE