Get Data Moving 1
Results 1 to 7 of 7

Thread: View/query latency status

  1. #1
    mtaluc is offline Junior Member
    Join Date
    Jun 2015
    Posts
    2
    Rep Power
    0

    View/query latency status

    We're running close to 20 replication tasks and monitoring them is challenging. Though we have alert emails for latency set up, they only email out once when the latency goes beyond the threshold so it's easy to miss an email and discover later that latency is two days behind. In order to check latency we're having to open up each task and view apply latency individually. Is there any way to add the latency to the main "Task" page so we can see them all at once or is there some way to query the backend through repctl or sqllite?

  2. #2
    Hein is offline Senior Member
    Join Date
    Dec 2007
    Location
    Nashua, NH - USA.
    Posts
    151
    Rep Power
    12

    Maestro!

    Quote Originally Posted by mtaluc View Post
    We're running close to 20 replication tasks and monitoring them is challenging. Though we have alert emails for latency set up, they only email out once when the latency goes beyond the threshold so it's easy to miss an email and discover later that latency is two days behind. In order to check latency we're having to open up each task and view apply latency individually. Is there any way to add the latency to the main "Task" page so we can see them all at once or is there some way to query the backend through repctl or sqllite?
    I see your 20 tasks and raise you 11 servers and 100 more tasks.

    Being an old fart I wrote a perl script, driven by a CSV file, to execute REPCTL - GETTASKSTATUS, parse the JSON output and Report out.
    Here is some sample output:


    Code:
    Z:\>perl \scripts\latency.pl -E cae
    State  DB Status    Latency   Apply Incoming       Tables Task @ Wed Jul 22 17:28:04 2015
    ------ --- --------- ------- ------- -------- ------------ --------------------------------
    CAE_RPT Src RUNNING     0.6hr    18/f     1165    413/0/0/0 TSK_CAE_ATT15_PRD_OLTP
    CAE_RPT SQL RUNNING     2.6hr   184/h  8594536    413/0/0/0 TSK_ATT15_OLTP_CAERPT
    CAE_CUS SQL RUNNING     2.5hr   260/h  7656469    413/0/0/0 TSK_ATT15_OLTP_CUSTOMREPORTS
    The coarse CSV file is of course generated by an other script parsing EXPORTREPOSITORY output and manually fine-tuned.
    The latency script supports tasks selection by regexp and much more.

    But wait, there is more, for larger configuration you may want to consider a GUI product MAESTRO to visualize (and control) replicate tasks and other processing modules.
    One pane of glass to manage it all.
    Unfortunately I could not inline a picture just now.
    I added it as attachment. Hope that worked.

    Running out of time now, but will let you know more about the 'latency' script.
    It's a bit specific to one customer's configuration right now.
    Time permitting I'd like to make a slightly more general version.

    Cheers,
    Hein
    Attached Images Attached Images  

  3. #3
    mtaluc is offline Junior Member
    Join Date
    Jun 2015
    Posts
    2
    Rep Power
    0
    Thanks a bunch. That's totally what I needed. Wondering where you found documentation that led you to the GETTASKSTATUS option of REPCTL? I suspected there was more to it than the user guide shows, but haven't been able to find those details.

  4. #4
    Hein is offline Senior Member
    Join Date
    Dec 2007
    Location
    Nashua, NH - USA.
    Posts
    151
    Rep Power
    12
    Quote Originally Posted by mtaluc View Post
    Thanks a bunch. That's totally what I needed. Wondering where you found documentation that led you to the GETTASKSTATUS option of REPCTL? I suspected there was more to it than the user guide shows, but haven't been able to find those details.

    Try: repctl help option=1

    Actually, try: repctl help option=1 > repctl_commands.txt
    Next use searched like : findstr status repctl_commands.txt

    Or: In the GUI, hover around in the top right next to the WEB/SERVICE connection depiction.
    Watch that cursor carefully... tada... list of commands, indication of parameters, filter.
    Before you ask, no I do not know how to execute from there.

    For usage examples of repctl/json/perl, Google for " site:attunity.com hein json "
    Toss in "hein" for the good stuff. Just kidding. :-)

  5. #5
    Hein is offline Senior Member
    Join Date
    Dec 2007
    Location
    Nashua, NH - USA.
    Posts
    151
    Rep Power
    12

    replicate_latency.p - script to report latencies

    Quote Originally Posted by Hein View Post
    Try: repctl help option=1

    For usage examples of repctl/json/perl, Google for " site:attunity.com hein json "
    Toss in "hein" for the good stuff. Just kidding. :-)
    I found a bit of time to reduce a specialized reporting script to a more generic one.
    See attachment replicate_latency.pl (or send me an Email ... firstname @ attunity dot com)

    Sample run output:

    Code:
    Z:\scripts>perl replicate_latency.pl -n "_CA|_WA"
      Name Status    Latency   Apply Incoming       Tables Zero Error  LastUpdate  LastHistory Task @ Sat Aug  8 11:47:04 2015
    ------ --------- ------- ------- -------- ------------ ---- ----- -----------  ----------- -------------------------------
        CA RUNNING      52s     13/f     1229   1031/0/0/0  213 17672    8/8 4:00  08/08 02:57 TSK_xxxxxxxxx_CA
        WA RUNNING      18s      6/f      458   1045/0/0/0   30    31    8/8 4:00  08/08 02:59 TSK_xxxxxxxxx_WA
    2 tasks
    The numbers under the Tables column represent counts of Completed/Loading/Queued/Suspended

    The 'f' under apply is short for 'per second over the last Five minutes interval'
    Could be an 'h' for 'over the last hour, or 'o' for overall or 'c' for current if so requested.

    -h for 'help'

    Driver file, edited to protect the innocent (high error count in example :-):

    Code:
    Z:\scripts>type replicate_latency.csv
    #Name, Task, Server, Target
    CA,TSK_xxxxxxxxx_CA,DC01Aserver01,DC01DBserver01/xxxxxxxxx_CA
    NH,TSK_xxxxxxxxx_NH,DC01Aserver02,DC01DBserver01/xxxxxxxxx_NH
    WA,TSK_xxxxxxxxx_WA,DC01Aserver02,DC01DBserver01/xxxxxxxxx_WA
    Attached Files Attached Files
    Last edited by Hein; 08-08-2015 at 03:21 PM. Reason: add attachment.

  6. #6
    Hein is offline Senior Member
    Join Date
    Dec 2007
    Location
    Nashua, NH - USA.
    Posts
    151
    Rep Power
    12

    Maestro superseded with Attunity Enterprise Manager (AEM) which includes a RestAPI.

    Please note that "Maestro" ended up not being productized and essentially became a prototype for the much more powerful Attunity Enterprise Manager (AEM)
    AEM comes with a task overview similar to the one the scripts here present (no coincidence :-) ! ).
    AEM also offers a RestAPI to get tasks detailed status in much more robust and efficient way.

  7. #7
    Hein is offline Senior Member
    Join Date
    Dec 2007
    Location
    Nashua, NH - USA.
    Posts
    151
    Rep Power
    12
    Quote Originally Posted by Hein View Post
    Please note that "Maestro" ended up not being productized and essentially became a prototype for the much more powerful Attunity Enterprise Manager (AEM)
    AEM comes with a task overview similar to the one the scripts here present (no coincidence :-) ! ).
    AEM also offers a RestAPI to get tasks detailed status in much more robust and efficient way.
    The AEM RestAPI is fully documented with the AEM installation.
    The default location for the developpers guide is : C:\Program Files\Attunity\Enterprise Manager\clients\help\Attunity_Enterprise_Manager_D evelopers_Guide.pdf

    Please find attached a sample AEM Powershell script to get latency for a specified task in a loop, using AEM installation provided .dotnet client DLL's
    For Linux one would typically want to use Curl as per AEM Developers guide.

    Sample run for script:

    Code:
    PS > C:\scripts\AEM_gettaskdetails.ps1 Snowflake               
    Time                   Source   Latency  Incoming  Applying Ins/Upd/Del/DDL
    -------------------  --------  --------  --------  -------- ---------------
    2019-04-17 11:54:05  00:00:00  00:00:00         0         0 0/0/0/0
    2019-04-17 11:54:10  00:00:00  00:00:00         0     10001 0/0/0/0
    2019-04-17 11:54:15  00:00:00  00:00:06         0     10001 0/0/0/0
    2019-04-17 11:54:20  00:00:00  00:00:12         0     10001 0/0/0/0
    2019-04-17 11:54:25  00:00:00  00:00:00         0         0 0/10001/0/0
    2019-04-17 11:54:30  00:00:00  00:00:00         0         0 0/10001/0/0
    2019-04-17 11:54:35   STOPPED
    Hein
    Attached Files Attached Files
    Last edited by Hein; 04-17-2019 at 01:05 PM. Reason: attach file

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •