Data Replication 2
Results 1 to 2 of 2

Thread: How many tables can a replication task handle for a single CloudBeam AMI setting?

  1. #1
    rangga.satya is offline Junior Member
    Join Date
    Nov 2017
    Posts
    6
    Rep Power
    0

    How many tables can a replication task handle for a single CloudBeam AMI setting?

    How many tables can a replication task handle for a single CloudBeam AMI setting?


    I'm having problem with replicating Oracle DB to a Redshift Cluster. We've tried 2 configurations, and neither work so far.


    Config 1:
    Single Task consiting all 125 Oracle tables
    1 Attunity Replicate Server (Windows Server 2012 (64-bit); CPU INTEL® Xeon® CPU E5-2460 @2.50 GHz, 2.49 GHz (3 processors); RAM 8 GB)
    1 Attunity CloudBeam AMI (m4.large)
    dedicated 10 GB network bandwidth between Replicate Server and CloudBeam AMI


    Config 2:
    2 Tasks; tables split across the 2 tasks
    1 Attunity Replicate Server (Windows Server 2012 (64-bit); CPU INTEL® Xeon® CPU E5-2460 @2.50 GHz, 2.49 GHz (3 processors); RAM 8 GB)
    2 Attunity CloudBeam AMI (m4.large)
    dedicated 10 GB network bandwidth between Replicate Server and CloudBeam AMI


    The Result for Config 1, only 90% of replication task finished, the rest shows Fatal Error message. Config 2, all full load tasks finished, but some of tables in Task 2 stuck in Queue forever.


    Do you have recommendation how we should set-up or configure this for our use-case? Neither configuration we tried seem to work.


    Thank you.

  2. #2
    reza.khan is offline Junior Member
    Join Date
    Jun 2013
    Posts
    25
    Rep Power
    0
    Hello Rangga,

    Thank you for contacting Attunity CloudBeam support.

    The default number of tables that are loaded in parallel is 5. When loading data to Redshift, the task will open 10 TCP connections per table to load the CSV files to S3 and so if you have 5 tables being loaded in parallel, that means 50 TCP sessions will be opened per task.

    Depending on the speed of the HDD of your source Replicate server, getting above 100 TCP sessions will cause issues during the data loading process and so we recommend keeping the total amount of tables between 5 and 10 tables to load in parallel.

    In order to understand why the other tables are stuck, we would need to review the error message for those specific tables. I did however want to point out that for the tables that didn't load successfully, you don't need to reload the entire task to get those tables to load.

    Instead, you can select the failed table and click the Reload button. Please note that option is only available if the Task is still running.

    Please lower the number of tables being loaded in parallel to 10 or less and for all failed tables, please try reloading them individually.

    Thank you.
    Reza K

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •