Shadow FS

Unstructured data is growing at unprecedented rates. The thanks to Retention requirements the lifecycle of data often exceeds that of the storage hardware that hosts it. The result is a nightmarish situation for IT professionals where ever more data is constantly being move forward in technology refreshes. Migrating file systems from a legacy storage system to new storage system is complex and challenging operation for even the most adventurous IT organizations.

Migration Issues:

  • Very time consuming
  • High data change rates
  • Can have thousands of shares on a single IP address (data mover)
  • Scheduling outage windows with buisiness owners
  • Access and Permission issues
  • Error prone made worse by ad hoc error handling approaches.
  • Cross vendor incompatibilities
  • Preserve Maintain original ACLS
  • Maintain original time stamps
  • Transitioning file level Retention to new system
  • Compliance needs and required reporting complications

Experience Matters

DataTrust Solutions has provided Professional Services to leading OEMs, Integrators and VARs for years and fully comprehends the issues, risks and pains that IT organizations experience. Leveraging our first-hand knowledge, we have developed a third generation file migration and verification application called Shadow FS. It was designed to operate in the most demanding environments where scale and Compliance requirements are primary IT concerns.

Shadow FS was a clean slate design that takes advantage of advances in virtualization and policy based automation to minimize risk, user downtime, cut-over windows, and other disruptions related to file storage migrations, rebalancing, consolidations, and tech refreshes. Shadow FS application integrates into Windows Server 2008 or 2012 and creates a migration appliance. This can be in a virtual or physical environment..

Shadow FS transfers file system content, file meta-data, extended attributes, streams, time stamps, permissions and ACLS – to a new target exactly as they were on the source, and proves it with standard reports. The file content, meta-data, permissions and ACLS have to be physically moved, and it must be done in a manner that allows the customer to prove the data was not changed or tampered with during the transition. In addition Shadow FS collects retention parameters from the source and converts into the correct format for the target, ensuring that the Compliance and Governance requirements are continuously maintained. Shadow FS provides verification for every file and chain-of-custody reports, but this is not what makes the migration software so special.

CIFS / NFS / Mixed Mode Protocols

Shadow FS supports source systems that use CIFS, NFS and mixed mode. Mixed mode or native mode is where data is accessible from both CIFS and NFS.  Shadows FS also writes files to CIFS, NFS or mixed mode targets and correctly passes all permissions and ACLS.

Policy driven Automation

Policy driven data migrations use rules to automate complex processes and free administration resources to do more valuable tasks. Why have storage Administrator spend hours tracking progress when the migration software can send a daily or weekly progress report? Shadow FS uses a Policy engine to automate the following areas:

  • Migration Filters (inclusive or exclusive)
  • Passing, extending or adding Retention
  • Permission cross-mapping and adjustment
  • Progress update emails are automatically sent nightly or weekly
  • Alerts

A Filter Policy can be established to enable Administrators to have fine grain control over a migration project by using file, meta-data, users, time stamps or path to include or exclude content from a migration.

In-Band or Out-of-Band

Shadow FS provides maximum operational flexibility and supports both Out-of-Band and In-Band file migration projects.

Out-of-band is a phased file system migration process in which a base line copy of the files are made to the new target. Then one or more incremental copies are made to capture the activity, including new files, deleted files and modified files, since the completion of the base line copy. A final incremental copy is made just before the final cut-over. Then there is an outage window to take the legacy system off-line and replace it with the new system. The advantage of the Out-of-band or phased approach is that there is no access disruption until the final cutover and no latency is introduced to the data path. The disadvantages of this approach are: it is lots of extra work to insure file consistency by resynchronizing edits to files already copied, consume more storage on legacy system.

A virtualization layer is inserted in the data path between the users/ applications and the storage systems. The virtualization layer, in the case of DTS is a virtual file system, that proxies both the existing file system/source storage as well as the new target. Shadow FS does NOT use stubbs, links, reparse points or old-school anchors in the OS file system. The primary advantages or our methodology are:

  • Simplicity - it is a lot less work
  • Users are assured that they are always working with the latest version of a file
  • Project duration is short
  • Cut-overs are a 5 minute task.

The two disadvantages to this approach are: 1) There is a five minute outage to insert the virtualization layer into the data path 2) The virtualization layer adds a few milliseconds of latency for processing.

Link to download the >> Shadow FS datasheet >>.

To learn more about how Shadow FS can simplify file system migrations for your organization contact DTS sales.

©2018 DataTrust Solutions