DiskBoss Logo
Flexense Data Management Software

Bulk File Delete and Data Wiping

DiskBoss provides advanced bulk file delete and data wiping operations allowing one to securely wipe confidential information, schedule periodic file delete operations or automatically trigger policy-based file delete operations according to user-specified rules while saving file delete logs or exporting lists of deleted files to an SQL database.

File Delete Dialog

Users required to securely wipe confidential information are provided with a large number of different data wiping algorithms ranging from a simple, single-pass wipe to a powerful 7-pass, DOD 5220.22-M compliant data wiping algorithm. In addition, DiskBoss provides the ability to apply different types of delete operations for different types of files. For example, all types of documents and image files may be securely wiped while all other types of files just deleted.

File Delete Progress

Enterprise customers and IT professions are provided with the ability to implement fully automated file retention policies using rule-based periodic file delete operations preconfigured to delete files based on file creation, last modification and/or last access dates. Finally, DiskBoss Server can be used to automatically enforce PCI and HIPPA compliance policies and perform file management operations on specific types of files and directories matching user-specified rules.

File Delete and Data Wiping Options

In order to open the advanced file delete and data wiping options dialog, press the 'Advanced Options' button located on the file delete dialog. The delete options dialog consists of the 'General' tab, 'Advanced' tab, 'Rules' tab and the 'Exclude' tab. The 'General' tab provides the ability to control the file scanning mode, the performance mode, the number of processing threads and a number of additional advanced file delete options explained later in this document.

File Delete Options
  • Performance Mode - Sometimes, in order to minimize a potential performance impact on running applications, it may be required to intentionally slow down a long running file delete or data wiping operation. The 'Performance Mode' option allows one to set an appropriate performance level for the file delete operation.
  • Dir Scan Mode - DiskBoss is capable of deleting files located in multiple directories/disks in parallel thus improving the speed of the file delete process on multi-CPU/multi-core systems. Select the 'High-Performance' file scanning mode if you wish to delete files in all input directories simultaneously.
  • Dir Scan Threads - this option sets the number of parallel directory scanning threads for the file delete operation.
  • File Delete Threads - DiskBoss is optimized for multi-core and multi-CPU servers and can use a number of CPUs to effectively delete millions of files located on one or more servers or storage systems. The 'File Delete Threads' option allows one to set the number of CPUs or CPU-cores to use for the file delete operation.
  • Delete Input File Lists - DiskBoss can delete files listed in user-specified text files with a full file name (including a full path) per line. In order to configure a file delete operation to delete files listed in one or more text files, set the 'Read Inputs From Files' mode on the delete command inputs dialog. In this mode, DiskBoss will read the specified text files and delete files listed in the user-provided text files. Select this option, in order to automatically delete the input text files after each delete operation.
  • Always Delete Empty Directories - DiskBoss provides the ability to perform rule-based file delete operations and delete files matching one or more user-specified rules and policies. In the rule-based file delete mode, DiskBoss will delete only files matching user-specified rules while keeping all directories in place. Select this option, in order to always delete empty directories in the rule-based file delete mode.

The 'Advanced' tab provides the ability to control a number of advanced file delete options including the data wiping mode, delete file logs, delete file logs mode, delete files logs history, and delete files logs directory.

Secure Data Wipe
  • Wipe Data From Disk - DiskBoss allows one to securely wipe confidential information from the disk using a large number of different data wiping algorithms ranging from a simple, single-pass data wipe to a sophisticated 7-pass DOD compliant 5220.22-M data wiping algorithm. In order to enable data wiping for a file delete operation, select this option and select an appropriate data wiping algorithm.
  • Save Delete Log Files - DiskBoss provides the ability to save full file delete logs to plain text files, Excel CSV files or an SQL database via the ODBC database interface. In order to enable delete file logs, select this option, select an appropriate delete log format (text, CSV or SQL database) and specify a log files directory or an SQL database table name prefix for the SQL database logs format. In addition, for the SQL database delete file logs, the user needs to configure the ODBC database interface to use to save file delete logs to the SQL database.
  • Log Files Mode - Use this option to save an individual log for each file delete operation, consolidated hourly logs, consolidated daily logs or consolidated monthly logs for all executed file delete operations.
  • Log Files History - Use this option to specify the number of delete files logs to keep in the log files directory or in the SQL database. For example, if the log files mode is set to 'Consolidated Daily Log Files' and the logs history is set to 10, DiskBoss will keep in the logs directory delete file logs for the last 10 days. For the SQL database delete logs, DiskBoss will keep in the database delete log tables for the last 10 days.
  • Log Files Directory - Use this option to specify a directory to save delete log files. For each log file, DiskBoss will generate a unique file name with the date and time of each file delete operation.
  • SQL Table Prefix - This option is displayed when the delete file logs format is set to the SQL database. Use this option to specify an SQL table name prefix to save file delete logs to the SQL database. For each SQL database table, DiskBoss will generate a unique table name, which will include the specified table name prefix and the date and time of each file delete operation.

Deleting Specific File Types or File Categories

The 'Rules' tab provides the ability to delete files matching one or more user-specified rules. For example, in order to delete all types of documents that were last accessed more than 3 years ago, select the 'Rules' tab, press the 'Add' button and add a rule matching files categorized as 'Documents, Books and Help Files' and then press the 'Add' button again and add a rule matching files that were last accessed more than 3 years ago.

File Delete Rules

DiskBoss provides a large number of different types of file matching rules allowing one to precisely select which files to delete. In addition, DiskBoss Server, which runs in the background as a service, may be used to schedule periodic rule-based file delete operations to be executed at user-specified time intervals or at a specific time of day on selected days of week allowing one to implement automatic, rule-based retention policies required for Sarbanes-Oxley, HIPPA and PCI compliance purposes.

Excluding Directories From File Delete Operations

Sometimes, it may be required to exclude one or more subdirectories from a rule-based file delete operation. In order to exclude a directory from a file delete operation, open the advanced file delete options dialog, select the 'Exclude' tab and add one or more directories to be excluded from the file delete process.

Exclude Directories

User-Defined File Delete and Data Wiping Operations

One of the most powerful capabilities of DiskBoss is the ability to pre-configure advanced file delete operations as user-defined commands and then execute these commands in a single mouse click using the DiskBoss CUI application. In addition, the user is provided with the ability to execute pre-configured file delete operations using the DiskBoss command line utility or schedule periodic file delete operations using DiskBoss Server, which runs in the background as a service and allows one to implement fully automated file delete and data wiping operations.

File Delete Commands

In order to add a new user-defined file delete operation, press the right mouse button over the 'Commands' pane, select the 'Add New - File Delete Command' menu item and enter a unique file delete command name. On the command inputs dialog, specify one or more directories or files to delete and press the 'Next' button.

File Delete Inputs

In order to execute a user-defined file delete command, just double-click on the command item in the 'Commands' pane. Another option is to create a desktop shortcut for the file delete operation and then click on the shortcut to execute the file delete operation without starting the DiskBoss GUI application. Finally, in order to execute the file delete operation using the command line utility, type the following command:

diskboss -execute <Command Name>

Saving Delete File Logs

DiskBoss provides the ability to save full delete file logs to the pain text format or the Excel CSV format. In order to enable logs for a file delete operation, open the advanced file delete options dialog, select the 'Advanced' tab, enable the 'Save Delete Log Files' option, select an appropriate log file format and specify a directory where to save all log files.

File Delete Logs

DiskBoss provides the following log file modes:

  • Individual Log File For Each Delete Operation - in this mode DiskBoss will save an individual log file for each file delete operation with the log file name containing the exact date and time of each file delete operation.
  • Consolidated Hourly Log Files - in this mode DiskBoss will save consolidated hourly log files for the last X hours according to the specified log files history.
  • Consolidated Daily Log Files - in this mode DiskBoss will save consolidated daily log files for the last X days according to the specified log files history.
  • Consolidated Monthly Log Files - in this mode DiskBoss will save consolidated monthly log files for the last X months according to the specified log files history.

According to the selected log files format, log files mode and the logs history, DiskBoss will keep a history of log files in the specified logs directory with each log file containing a date, time, status and a full file name for each deleted file.

File Delete Logs Directory

Exporting Delete File Logs to SQL Database

Corporate customers and IT professionals are provided with the ability to export full delete file logs to a centralized SQL database. In order to enable SQL database logs for a file delete operation, open the advanced file delete options dialog, select the 'Advanced' tab, enable the 'Save Delete Log Files' option, select the SQL database logs format and specify an SQL table name prefix. In addition, open the 'Options' dialog and configure an ODBC data source to use to export delete file logs to the SQL database.

File Delete SQL Database Logs

DiskBoss provides the following SQL database log modes:

  • Individual Log Table For Each Delete Operation - in this mode DiskBoss will create an individual SQL database table for each file delete operation with the table name containing the specified table name prefix and an exact date and time of the delete operation.
  • Consolidated Hourly Database Log Tables - in this mode DiskBoss will keep consolidated hourly SQL database tables for the last X hours according to the specified delete file logs history.
  • Consolidated Daily Database Log Tables - in this mode DiskBoss will keep consolidated daily SQL database tables for the last X days according to the specified delete file logs history.
  • Consolidated Monthly Database Log Tables - in this mode DiskBoss will keep consolidated monthly SQL database tables for the last X months according to the specified delete file logs history.

According to the selected delete file logs mode and the logs history, DiskBoss will keep a history of SQL database tables with each SQL database table containing a date, time, status and a full file name for each deleted file.

File Delete Excel Logs

Deleting Files Listed in User-Specified Text Files

DiskBoss provides the ability to delete files listed in user-specified text files allowing one to perform fully automated or periodic bulk file delete operations. DiskBoss can read file names from user-specified text files and delete all files listed in one or more text files or in all text files located in a directory.

Delete Files Listed In Text Files

In order to configure DiskBoss to delete files listed in user-specified text files, create a user-defined file delete command, select the 'Read Inputs From Files' mode and add one or more directories or text files to the input list. The specified text files should contain full names of files or directories to be deleted with a single name per line. During runtime, DiskBoss will read the specified text files and delete all files listed in the text files.

Another option is to specify a directory which may contain a number of text files with lists of files to be deleted. In this case, DiskBoss will parse the directory, read all text files located in the directory and delete all files listed in the text files.

Periodic File Delete Operations

DiskBoss allows one to perform bulk file delete operations periodically according to a user-specified time interval. In order to perform a user-defined file delete operation periodically, create a new periodic job and select the user-defined file delete command to be executed according to the specified time interval.

In addition, DiskBoss provides the ability to automatically execute a user-defined file delete command when one or more text files with lists of files to be deleted are created in the specified inputs directory. In order to enable automatic execution of file deleted operations, add a disk change monitoring command, configure the real-time disk change monitor to monitor the directory with the text files listing files to be deleted, enable actions for the real-time disk change monitor and select the user-defined delete command to be executed after each newly created text file.

Real-Time File Delete Operations

During runtime, DiskBoss will continuously monitor the configured directory, detect newly created text files, read file names from these files and delete all listed files. Finally, in order to automatically delete text files with lists of files after each delete operation, open the user-defined file delete command, open the 'Options' dialog, select the 'General' tab and enable the 'Delete Input File Lists' option.

Delete External File Lists

Fully automated bulk file delete operations, performed without any user intervention, may be configured using DiskBoss Server, which runs as a service in the background, and is capable of continuously monitoring a directory for newly created text files and automatically delete all files listed in the created text files. Once DiskBoss Server is fully configured, the user just needs to copy one or more text files to the specified directory and DiskBoss Server will automatically perform all file delete operations in the background.

File Delete Performance

DiskBoss is capable of deleting vast numbers of files very fast using multiple CPUs to delete a number of files simultaneously. The performance of file delete operations depends on the type of the storage device and the hardware platform DiskBoss is installed on. For example, when deleting files from a fast NVMe SSD drive, DiskBoss reaches up to 28,000 Files/Sec with a single processing thread and scales up to 49,000 Files/Sec with 4 parallel file delete threads allowing one to delete up to 3 million files per minute.

File Delete Performance NVMe SSD

When deleting files stored on a SATA SSD drive, DiskBoss reaches up to 18,000 Files/Sec with a single processing thread and scales up to 32,000 Files/Sec with 4 parallel file delete threads allowing one to delete almost 2 million files per minute.

File Delete Performance SATA SSD

Deleting files stored on a mechanical SATA HDD drive is much more time consuming and less scalable operation and in this case DiskBoss reaches up to 3,600 Files/Sec with a single file delete thread and scales up to 5,300 Files/Sec with 4 parallel processing threads.

File Delete Performance SATA HDD

When deleting files stored on a NAS server via the network, the performance of file delete operations will be significantly affected by the network speed and more importantly the network latency. If the user needs to delete a large number of files located in a remote office via a long-distance, high-latency network, the performance will be slow and in order to improve the performance of file delete operations the user should open the delete command options dialog and configure up to 8-16 parallel file delete threads.

File Delete Performance NAS Server

When deleting files located on a fast NAS server via a local, low-latency Gigabit Ethernet network, DiskBoss reaches up to 1,300 Files/Sec with a single processing thread and scales up to 4,600 Files/Sec with 4 parallel file delete threads.

Deleting millions of files stored in one or more NAS servers may be a very time consuming operation and one of the ways to speed-up the file delete process is to use a 2.5 Gigabit Ethernet network. With the 2.5 Gigabit Ethernet the performance of the DiskBoss file delete operations continues to scale up to 7,000 Files/Sec with 4 parallel file delete threads, which represents a 76% improvement compared to the standard Gigabit Ethernet.

File Delete Performance 2.5 Gigabit Ethernet NAS Server

Due to a very wide adoption of laptops and NAS servers with built-in WiFi network interfaces, many users may consider deleting files stored in NAS servers via the wireless network. But, the latency of the wireless network is much higher and therefore it will take much more time to complete the file delete operation via the wireless network. The question is how much longer the user will need to wait and if it will save any significant amount of time to delete files via a wired network.

File Delete Performance WiFi NAS Server

Based on our benchmarks, via a 5 GHz wireless network, DiskBoss deletes up to 102 Files/Sec with a single file delete thread and scales up to 421 Files/Sec with 4 parallel file delete threads, which is approximately 10 times slower compared to the standard Gigabit Ethernet and almost 15 times slower when compared to the 2.5 Gigabit Ethernet. So, if the user needs to delete 100,000 files or more from a NAS server, a low-latency Gigabit Ethernet or 2.5 Gigabit Ethernet is required.

Deleting Files Using the DiskBoss Command Line Utility

In addition to the GUI application, DiskBoss Ultimate and DiskBoss Server provide a command line tool allowing one to perform file delete operations from batch files and shell scripts. The DiskBoss command line tool is located in the <ProductDir>/bin directory.

Command Line Syntax:

diskboss -execute <Command Name>

This command executes the specified user-defined file delete command.

diskboss -delete <Input 1> [ ... <Input X> <Options> ]

This command deletes the specified files and directories.