Migrating to Azure NetApp Files

Azure NetApp Files is a first party file service on Azure that is primarily designed to run most demanding Linux and Windows file based applications on Azure. To know more about Azure NetApp Files refer the documentation

Data migration can be quite challenging for large-scale efforts. Major concerns include minimizing any downtime to users and maximizing data transfer rates without adversely affecting network performance. It’s also vitally important to verify the accuracy of all the data after the migration. These tasks are critically important to the success of a reliable data migration and to continuation of business operations.

Data migration is about moving data quickly and reliably from one storage platform to another while minimizing disruption to the users and applications that use the files. A phased approach to migration does an initial copy of all files that aren’t locked or open. When most of the data synchronizes to the target system, a short disruption is necessary for the final cut over.

Good planning is the key to success in a NAS data migration project. Generally, most migrations take a similar approach that involves assessment, planning, resource acquisition and provisioning, pre-migration, migration, and verification.

Some data migration projects involve the use of utilities that run on either the host, network, or storage tier. Commonly, the data is either copied or mirrored. If the volume of files/data is large or distributed, it may make sense to evaluate and purchase an automated NAS migration tool—or a comprehensive solution.

Here we will look at some of the popular and frequently used data migration tools to migrate the data to Azure NetApp Files from on-premise NAS systems or File Servers or from AWS/Google Cloud file shares to Azure NetApp Files and also from Azure Files to Azure NetApp Files.

Robocopy

Robocopy (Robust File Copy) is a command-line file copy utility that comes with Windows Vista / Windows 2008 or newer. Until Vista, Robocopy was a part of Windows Resource Kit Tools as a free download (http://aka.ms/robocopydownload) Unlike normal copy commands, Robocopy is designed for reliable copy or mirroring while maintaining the permissions, attributes, owner information, timestamps and properties of the objects copied.

Robocopy Syntax

ROBOCOPY <source> <destination> [file…] [options]

<source> Source Directory (local or network path)

<destination> Destination Directory (local or network path) and 

[file…] Specifies the file or files to be copied. You can use wildcard characters (* or ?), if you want. If the File parameter is not specified, *.* is used as the default value.

Example:

ROBOCOPY \\sourceserver\share  \\anf-smb-server\anf-smb-share /zb /e /copyall /sec /purge /tee /fft /r:3 /w:5 /eta

For more examples please refer

rsync

rsync (or “remote sync”) is a versatile file copying utility for unix based systems. For most admins, this is their bread and butter. Rsync can be used to quickly move large amounts of data to both local and remote destinations. For this reason, rsync is often used to copy data, make backups, migrate hosts, and bridge the gap between site staging and production environments.

Rsync Syntax

rsync options SOURCE DESTINATION

Example: rsync -aarvh /nfslocalmount /anfvol1

Note: Be careful how you use the trailing slash (/) in the source path when syncing directories. The trail plays an important role. If you enter the trailing slash on the source(/source/), the rsync command does not create the source folder on the destination; it only copies the directory’s files. When you do not use the trailing slash(/source), rsync also creates the original directory inside the destination directory.

NetApp Cloud Sync

Cloud Sync has been designed to solve all issues related to data transfer and data synchronization. As part of NetApp’s Cloud Data Services, Cloud Sync makes moving data between any source and destination a simple task while keeping it synchronized in a robust and secure manner.

With Cloud Sync, you can ingest data into Azure NetApp Files from any NFS share, whether hosted on Cloud or On-Premises, as well as work with other types of storage, such as SMB shares and object store(Blob, S3, etc.). Cloud Sync also facilitates data synchronization in the reverse direction out of Azure NetApp Files back to NFS systems.

Cloud Sync is a SaaS (Software-as-a-Service) solution for data migration between any source and destination platform. After performing an initial baseline copy of the full data set, Cloud Sync will incrementally synchronize only the data that has changed, which makes it very efficient, especially when working with large datasets.

The Cloud Sync service works by performing data migration and synchronization operations through the use of a data broker instance. This instance can be created in the cloud, using an on-premises or Microsoft Azure virtual machine. In either case, the Cloud Sync UI simplifies the process by helping you create the data broker. For documentation refer the link.

Steps for migration using Cloud Sync:

• Register for Cloud Sync service by signing up at NetApp Cloud Central
• Create one or more data broker instances
• Setup connectivity between source, data broker and destination
• Configure Sync Relationship with various settings in the Cloud Sync GUI

NetApp XCP

NetApp XCP is client-based software that uses all the available system resources to handle high-volume datasets and high-performance migrations.

With XCP you can fully utilize available CPU, network, and storage resources to scan, scope, copy and verify large file trees at maximum speed. With logging, reporting, subdirectory granularity plus three levels of verification (stats, structure and full data), XCP offers unique capabilities to accelerate and improve file tree processing and data migration.

In the NFS world, traditional tools like rsync/find/dd/du use a simple back-and-forth style of serial IO operations so that when there are millions of files to process, waiting for all the round-trips to finish can take weeks. On the other hand, all the XCP features benefit from a built-in NFS client engine that generates parallel streams of asynchronous IO requests. XCP NFS engine significantly mitigates the effects of latency by keeping servers and networks busy all the time, making millions or even billions of files a lot easier to manage.

Steps for migration using XCP:

• Register a NetApp NSS account
• Download XCP to a host
• Login and download a license(free)
• Deploy the license on XCP host
• Setup sync relationship using the syntax below

./xcp copy <source NFS export path> <destination NFS export path>

XCP provides number of switches that you can specify when configuring a migration from Source to Destination depending on what requirements you have, it is typically used to migrate NFS file shares with millions of small files.

Comparison

Let’s compare how these migration tools place against each other in terms of key factors you need to consider when choosing a data migration solution,

	Robocopy	Rsync	Cloud Sync	XCP
Support	Community Support	Community Support	NetApp	NetApp
Typically used for	SMB Shares	NFS Mounts	SMB and NFS	NFSv3 with millions of small files
Need VMs/hosts	No	No	Yes (data broker)	Yes (XCP Hosts)
Interface	CLI	CLI	GUI	CLI
Cost	Free	Free	Free*	Free

Summary

Data migration would depend on various factors like available bandwidth, type of data i.e. the data with large files and data with millions of small files, amount of data, connectivity between source and destination, consideration of downtime during final cutover, the speed of data transfer, based on such and many other factors you can choose one of the above migration tools to migrate your NAS data to Azure NetApp Files. Azure NetApp Files has three different service levels you can choose from based on your requirement, consider using Premium/Ultra service level when migrating data to Azure NetApp Files to aid in faster migration.