Linux real time data synchronization and file replication software

Linux Real Time File Replication Installation and Configuration

In this article, I will discuss how to configure EDpCloud Linux Real Time File Replication and Synchronization to move, transfer and sync files automatically from one or multiple Linux servers, laptops, or desktops to a remote site for data protection or other workflow purposes (The remote site can be Linux, Windows, Solaris, etc.).

Linux real time replication between countries


Download A free Demo Copy of Linux, UNIX or Windows Replication Software

Download EDpCloud real time data replication software and a free demo license

First, run uname -a. If you have a 32-bit OS, download the 32-bit package. Otherwise, download the 64-bit package

Extract the tar file and run the real time file replication software install script:

First, install the Linux real time file replication:

tar xvf packagename.tar.gz
cd enduradata_edpcloud
./install.sh

You will be prompted for some parameters, such as the install directory. Use the defaults for now, but don’t start the file replication services/daemons yet. We will do that after we configure Linux, real-time file replication software.

cd to the edpcloud bin directory (under the install directory/edpcloud/bin) and source in the script to set up the environment (this will be done automatically later)

. enduradata_env Please note that once you source in the env, your paths are set and some aliases are created for you. IMPORTANT: INSTALL FUSE ONLY IF YOU DO NOT have it Install one of the Linux real file replication components: ./edinstallfuse.sh

If your linux-utils is older than 2.18, then touch the file below in the bin directory to make sure the correct mount is used (To find out the version of your linux-utils: run mount –version to find out the version):

. enduradata_env touch ${ED_BASE_DIR}/bin/has_old_mount_version If you had to do that, you must update your operating system. Set auto start after reboot: $edpcloud/bin/autostart/sys5debian_autostart

Install the real time file replication license by copying it to:

${ED_BASE_DIR}/etc/edlicense

Configure

cd to the edpcloud/etc directory and add the directories you want to replicate in real-time to edfsmonitor.cfg

 
Example:
/data1
/data2

/data1 and /data2 entries must exist in the file system since these will be mount points.

cd to the edpcloud etc directory and add the top directories( you want to replicate in realtime or in scheduled mode) to the includes file (use regular expressions)

 
Example of $edpcloud/etc/includes
^/data1/.*
^/data2/.*

You will need to add other directories if you want to replicate them (in real-time, on-demand, or in a scheduled manner).

cd to edpcloud/etc directory and create eddist.cfg

Example: To replicate data in real-time from any localhost to 192.168.32.50 and put all data in /media/bob/backup/%ip%”. The “%ip%” macro will be converted at run time by the receiver to use the sender’s IP. Make sure the directory exists and that the “.” in the IP is an underscore and not a dot. You will see the receiver log complaining if the directory does not exist (see example below)

mkdir -p /media/bob/backup/192_168_32_2 #if the sender is 192.168.32.2

 <?xml version="1.0" encoding="UTF-8"?>
<config name="enduradata" password="zooo" >
    <link name="or" >
        <sender hostname="localhost" 
            alias="*" 
        />
        <receiver  hostname="192.168.32.50" 
                storepath="/media/bob/backup/%ip%"  
        />
    </link>

</config>

Your server’s Linux real time file replication configuration is done.

Start edpcloud services now using the following command:

 
cd $installdir/edpcloud/bin
. edpcloud.sh startall

Verify that edpcloud file replication software is running

 
edstat
df

edstat will show you the status, the number of files in the journal, the bytes on the move, etc.

Examine the output of df, and you will see that edpcloud mounted both /data1 and /data2.

If for any reason, edstat is showing any errors, then you need to
examine eddist.log under the edpcloud logs directory (other useful logs are edfsmonitor*log, ed_receiver*log, ed_sender*log).
Test that file replication is taking place in real time and that your local data under /data1 and /data2 is being synchronized as it changes on the localhost to the remote server:
Now go to /data1 and touch a file or write to it, rename it, etc., and you will see your changes replicated to the remote host. Please note that deletes are not propagated by default.

Propagating deletes to the remote host

By default, to protect against misuse, deletes are not propagated.

If you want deletes to be propagated, then add the deletes=”1″ to the configuration as shown below.

 <?xml version="1.0" encoding="UTF-8"?>
<config name="enduradata" password="zooo" deletes="1" >
    <link name="or" >
        <sender hostname="localhost" 
            alias="*" 
        />
        <receiver  hostname="192.168.32.50" 
                storepath="/media/bob/backup/%ip%"  
        />
    </link>

</config>

By default the top directory as indicated by storepath="/media/bob/backup/%ip%" must exist unless you
edited enduradata_env to add:

ED_CREATE_STOREPATH=1
export ED_CREATE_STOREPATH

Another directive to create the top directory for storepath is to create file createpath under $edpcloud/etc:

touch $edpcloud/etc/createpath

You will need to perform the initial synchronization to copy the local files and directories first:

edq -l or -r 192.168.32.50 -n /data1

edq -l or -r 192.168.32.50 -n /data2

You may also use edmfq for better control of the file queueing for replication and synchronization.

You will need to perform the initial sync only once, and after that, all file changes on /data1 and /data2 on the localhost will be sent to the remote host i in real time.

If you want to synchronize and replicate files from multiple servers to the same remote host, install and drop the files you created in the edpcloud/etc directory on each of the servers, run edpcloud.sh restartall, and you are on your way to consolidating your backups, aggregating your content or whatever you want to achieve (Don’t forget to get the right package and to run the following command (IF and only if your mount version is older than 2.18)

touch ${ED_BASE_DIR}/bin/has_old_mount_version

Your Linux real-time file replication and synchronization software are installed and configured. EDpCloud keeps your many servers in sync to automate your business processes and data protection using file replication for online backup. Make sure you visit again and read about creating file archives.

Next time, we will talk about archiving file changes. Until then, be safe, but above all, protect your data.

If you like this article, please share it. We would also love to hear from you.

–a

Contact support for free help

Additional articles you may find interesting:

See Details for eddist.cfg

https://www.enduradata.com/file-replication-software-a-cross-platform-solution

Linux real-time file replication and synchronization software was last modified: August 14th, 2024 by S. Dimitri