File replication software for data migration

Introduction

EDpCloud is a file replication software that allows you to synchronize data between servers, laptops and desktops.

You can use the synchronization for backup, data migration, data distribution and data aggregation. The data replication software moves data between different operating systems, different geographic sites, different networks etc. In this article I will discuss how you can use EDpcloud to migrate a large volume of data from one server running Solaris to another server running Linux. A data migration is a major undertaking. A significant number of companies and projects fail to achieve the migration on time and on budget. This results in major costs runaways, delays and frustration of customers, suppliers, management, employees and creates major opportunity costs.

Before your start Data Migration and file syncing: Make a plan and inventory everything

In order to have a transparent and successful data migration (to the users), Inventory all the business processes, risks, reporting, applications, scripts, source code, file systems, unstructured data, system and application users, inputs, outputs and all the dependencies. You may want to start by creating data flow diagrams and then ranking the order of migration and creating a task schedule. For example, you may find out that before migrating the data, you will need to migrate and port any  source code your operations depend on. Compiling and porting the source code will depend on how it is written and what language it is written in, professional engineers will achieve that in a cost effective manner. A great deal of coordination between business units, systems staff, developers and database staff will be required. A change champion and leader within management will also have to be involved to remove all barriers, resistance to change and to provide moral and material support to the migration team during the change phase. A review of staff availability during the migration and post migration must be examined carefully. Please remember that a data migration seems to be only a technology change but in reality it will involve a deep structure change (culture and people involved). This is what is called revolutionary change. Once you undertake such a change, there is no going back. I will post another article about revolutionary (as opposed to evolutionary) change (and its rules) in the coming weeks.

Now to the easy part: moving data, scripts, databases, etc.

Configure EDpCloud file replication software to synchronize Solaris to Linux

Lets assume the two hosts are called solaris and linux. We want to migrate /data1 /data2 /home/svn/trunk/src /home/scripts and /databases. You want to put data in /alldata

You can use the GUI or vi to create the following configuration to sync from solaris to linux:

<?xml version="1.0" encoding="UTF-8"?>
<config name="Network_Configuration" >
    <link name="l1">
        <sender hostname="solaris" alias="*" password="foo"/>
        <receiver hostname="linux" numstreams="6" storepath="/alldata" password="foo"/>
    </link>
</config>

Once you created the configuration, you will need to restart edpcloud as root:
. /usr/local/enduradata/edpcloud/bin/edpcloud.sh restartall

Now queue the data for the transfer and to replicate the files and directories.

edq -l l1 -r linux -n /data1
edq -l l1 -r linux -n /data2
edq -l l1 -r linux -n /databases
edq -n /home/svn/trunk/src
edq -n /home/scripts

You will need to create a schedule to continue to synchronize the data, scripts and write your test scripts
to compare the output and the performance of the new system  with the old system on a continuous basis until everything ties. Continue the exercise before your cut-over time to the new system

You may want to do the following to dump mysql on solaris and reload it on linux

mysqldump --user=root --password=Yourdbpassword yourdatabase > yourdatabase.dump
edq -n yourdatabase.dump

When the transfer is done, you will need to upload the database on linux:

If the database does not already exist on linux, you may want to create it and grant all permissions, etc.

mysql -u root -p yourdatabase < yourdatabase.dump

This is in no way complete, please contact EnduraData if you need assistance with your data or system migration. Additional information about EDpCloud can be found on the main site.

Thanks for reading. Check other related posts below.

–e