Creative Communities of the World Forums

The peer to peer support community for media production professionals.

Activity Forums Storage & Archiving Slow verify with BRU-PE

  • Slow verify with BRU-PE

    Posted by Neil Sadwelkar on February 20, 2016 at 12:41 pm

    With BRU-PE 3.1.17 on a Mac under OS X 10.10.5, SAS LTO-6, I’m seeing excruciatingly slow verify speeds particularly with drives with a huge number of files. For eg. .ari, .dpx, .exr file sequences

    Write is fine, though. Exact same setup I don’t remember such slowness prior to this version. Will try one more tape with the same data and a prior version.

    From the report, timings are like…

    Example 1
    20160220 11:26:17 – write start
    20160220 12:29:17 – write end
    20160220 12:30:10 – verify start
    20160220 16:54:27 – verify end

    wrote 262479872 blocks (524959744 KBytes) – 139025 KB/sec
    read 262479872 blocks (524959744 KBytes) – 33105 KB/sec

    So, 1 hr to write 500 GB, and 3.5 hrs to verify

    Example 2

    20160219 10:59:57 – write start
    20160219 15:03:52 – write end
    20160219 15:05:32 – verify start
    20160220 10:50:18 – verify end

    wrote 1058849792 blocks (2117699584 KBytes) – 144701 KB/sec
    read 1058849792 blocks (2117699584 KBytes) – 29791 KB/sec

    So, 4 hrs to write 2.1 TB, and 19 hrs+ to verify

    During verify there’s no drive access. In fact, we can even unmount the drive during verify.

    Anyone else observe this?

    ———————————–
    Neil Sadwelkar
    neilsadwelkar.blogspot.com
    twitter: fcpguru
    FCP Editor, Edit systems consultant
    Mumbai India

    James Vorley replied 9 years, 3 months ago 4 Members · 6 Replies
  • 6 Replies
  • Bjorn Frodason

    February 20, 2016 at 2:51 pm

    Experiencing the similar thing, but only slower. 100GB verify in 3 hours.
    I’m testing BRU PE and it seems to be nice piece of software in many areas.
    But this morning I started to do a verify on a 1.6 GB data on LTO-6 tape in IBM TS3100 lib and it is very slow. Writing was normal where the bottleneck was the network.
    This is just normal BRU verify, i.e. not comparing to the actual files.
    I’m testing BRU because of a problem I ran into with Retrospect, when upgrading a failed LTO-5 drive to LTO-6. I think I will make a specal post with that one, so I will not hijack this one.

    I’m “hoping” that the slow verify is connected to some underlying problem, SAS controller, failed LTO-6 drive, OS X or the HW. But I’m not sure what the best method is to diagnose.

    System information
    MacPro 1.1 Intel – 32 bit HW
    OS X 10.7.5
    6GB RAM –
    Sys HD Mirror w. SoftRaid v. 4.53
    SAS Controller Atto Express H380 driver 2.02, firmw June 2011
    Library IBM TS3100 – Firmw. D.00 / 3.20e – 4 years Old
    Drive-ULT3580-HH6 – Firmw. F9A1 – 6 months old

    ——————————
    Bjorn Frodason
    Technical department
    ODDI Printing & packaging.

  • Tim Jones

    February 22, 2016 at 2:22 am

    Hi Guys,

    We’re looking into this, however, it’s odd since the back-end bru engine component hasn’t changed in quite some time.

    What is the BRU version (not BRU PE) that you have?

    In a Terminal type:

    bru -h

    Tim

    Tim Jones
    CTO – TOLIS Group, Inc.
    https://www.tolisgroup.com
    BRU … because it’s the RESTORE that matters!

  • Bjorn Frodason

    February 22, 2016 at 8:04 am

    Hi there

    BRU -h results

    release: 18.1
    variant: 0.25
    serial number: UNSERIALIZED
    bru id: mac-osx
    config: Jan 4 2016 10:19:21
    capabilities: DEMO Edition
    archive: ntape0
    buffer size: 2048k bytes
    device table: /Library/Application Support/BRU PE/etc/brutab[ BRUTAB ]
    incl/excl file: /Library/Application Support/BRU PE/etc/bruxpat[ BRUXPAT ]
    max warnings: 1000 [ BRUMAXWARNINGS ]
    max errors: 500 [ BRUMAXERRORS ]
    copyright: Copyright (c) 1985-2016, TOLIS Group, Inc.

    ——————————
    Bjorn Frodason
    Technical department
    ODDI Printing & packaging.

  • Bjorn Frodason

    February 22, 2016 at 8:27 am

    Here are the Results from the Verify of the 1.6 TB data set, it was error free, but not fast.
    As I mentioned Writing speed was normal, and controlled by the network speed.
    N.b. Can the reason be that I stopped the verifying in the Archive stage?
    This is just “solo” verify, wich I ran afterwards.

    serial_number: UNSERIALIZED
    device: ntape0
    user: root
    group: wheel
    system: Darwin mubarak. 11.4.2 Darwin K i386
    bru: mac-osx
    command_line: /Applications/BRU PE/BRU PE.app/Contents/MacOS/bru -c -m -vvvvvvvvv -j -O -A -QB -f ntape0 -QX -L 2
    **** end info ****
    archive ID = 56c3517a35fe
    label = 2016-02-16 PG_Lok22 Backup
    buffer size = 2048k bytes
    media size =
    í málfríði.doc
    í tölum.docX
    089_i2.jpg
    **** bru: execution summary ****
    Started: Sat Feb 20 11:04:11 2016
    Completed: Mon Feb 22 05:32:23 2016
    Archive id: 56c3517a35fe
    Messages: 0 warnings, 0 errors
    Archive I/O: 0 blocks (0KB) written
    Archive I/O: 875259904 blocks (1750519808KB) read
    Files written: 0 files (0 regular, 0 other)
    Files read: 457065 files (406117 regular, 50948 other)
    Files skipped: 0 files
    Volumes used: 1
    Write errors: 0 soft, 0 hard
    Read errors: 0 soft, 0 hard
    Checksum errors: 0

    ——————————
    Bjorn Frodason
    Technical department
    ODDI Printing & packaging.

  • Neil Sadwelkar

    February 23, 2016 at 3:00 am

    Mine says…

    release: 18.1
    variant: 0.25
    serial number: ____-____
    bru id: mac-osx
    config: Jan 4 2016 10:19:21
    capabilities: PE TOOLS
    archive: ntape0
    buffer size: 2048k bytes
    device table: /Library/Application Support/BRU PE/etc/brutab[ BRUTAB ]
    incl/excl file: /Library/Application Support/BRU PE/etc/bruxpat[ BRUXPAT ]
    max warnings: 1000 [ BRUMAXWARNINGS ]
    max errors: 500 [ BRUMAXERRORS ]
    copyright: Copyright (c) 1985-2016, TOLIS Group, Inc.
    All Rights Reserved.

    There is a eight digit serial number which I’ve blanked above.
    I can’t remember if I uninstalled the older 3.1.12 version before installing this 3.1.17 version.

    ———————————–
    Neil Sadwelkar
    neilsadwelkar.blogspot.com
    twitter: fcpguru
    FCP Editor, Edit systems consultant
    Mumbai India

  • James Vorley

    January 19, 2017 at 12:18 pm

    I’m seeing what appears to be a similar issue intermittently. Did anyone ever get to the bottom of this?

    For example 167 MB/s write but only 53 MB/s verify:

    ++++++++++++++++++++++++++++++++++
    Last BRU PE Archive Execution summary:

    20170111 13:19:52|3855|root|[L163] START (r 18.1.0.25), CMD = '/usr/local/bin/bru -c -m -vvvvvvvvv -j -O -A -QB -f ntape0 -QX -L STV-6-0006 -'
    20170111 13:19:52|3855|root|[L167] device = ntape0, buffer = 2048K bytes, media size = , archive id = 587630f80f0f, (Op)
    20170111 17:25:28|3855|root|[L182] wrote 1108950016 blocks (2217900032 KBytes) on volume [1], 4:05:36, 150508 KB/sec
    20170111 17:26:36|3855|root|[L165] FINISH - 0 warnings, 0 errors, exit code = 0
    ++++++++++++++++++++++++++++++++++
    Last BRU PE Verify Execution summary:

    20170111 17:26:54|5487|user|[L163] START (r 18.1.0.25), CMD = '/usr/local/bin/bru -ivvvvjf ntape0'
    20170111 17:26:56|5487|user|[L167] device = ntape0, buffer = 2048K bytes, media size = , archive id = 587630f80f0f, ()
    20170112 05:09:51|5487|user|[L182] read 1108950016 blocks (2217900032 KBytes) on volume [1], 11:42:57, 52585 KB/sec
    20170112 05:09:51|5487|user|[L165] FINISH - 0 warnings, 0 errors, exit code = 0

    release: 18.1
    variant: 1.28
    serial number:
    bru id: mac-osx
    config: Jul 27 2016 13:47:11
    capabilities: PE TOOLS
    archive: ntape0
    buffer size: 2048k bytes
    device table: /Library/Application Support/BRU PE/etc/brutab[ BRUTAB ]
    incl/excl file: /Library/Application Support/BRU PE/etc/bruxpat[ BRUXPAT ]
    max warnings: 1000 [ BRUMAXWARNINGS ]
    max errors: 500 [ BRUMAXERRORS ]
    copyright: Copyright (c) 1985-2016, TOLIS Group, Inc.
    All Rights Reserved.

    The issue appears randomly and often not at all on the same data and tape set. When the issue appears it seems to be on restore and verify operations.

    I had the issue disappear for days and ~20 TB of read operations, then randomly reappear.

    System is Tolis Argest LTO-6 (HP drive), ATTO H680, Early 2008 MacPro, OS X 10.10.5. I have also tested on an early 2013 Mac Pro with HighPoint RocketStor 6328 and see the same thing.

    When the system is in whatever mode it gets in to that results in slow read/verify times if I switch to demos of other LTO software (YoYotta, PreRoll Post) I still see the slow read/verify times.

    TOLIS are suggesting possibly the Mac’s memory was fragmented at the time of that job resulting in lots of small memory reads during the tape operation. But a reboot doesn’t always seem to fix the problem. All I’m running on my system apart from BRU is Media Encoder which is used a handful of times a day for the odd transcode of ~ 1 minute long commercials.

    TOLIS also got me to export an L&TT support ticket which didn’t show any hardware faults with the drive.

    After previous nightmare issues with LTO (a hand-me-down NovaBackup and LTO-4 system) this system is *almost* perfect, but would love to get to the bottom of these random slow read times.

We use anonymous cookies to give you the best experience we can.
Our Privacy policy | GDPR Policy