Creative Communities of the World Forums

The peer to peer support community for media production professionals.

Activity Forums Storage & Archiving BRU PE to LTO workflow, share your methods?

  • BRU PE to LTO workflow, share your methods?

    Posted by Oleng Tjioe on July 13, 2013 at 7:15 pm

    Hi guys,

    I’ve been trying to wrap my head around BRU PE workflow especially the part where i can document and retrieve the files inside the LTO archive, and it eludes me.
    I wonder how you guys do it, how do you manage to keep track of what’s archived to which tape? What did you prepare beforehand? Do you use any library/catalogue softwares?

    Thanks
    Oleng

    Prodigi House
    Jakarta

    Paul Katzman replied 10 years, 7 months ago 5 Members · 21 Replies
  • 21 Replies
  • Oleng Tjioe

    July 13, 2013 at 10:56 pm

    I just remember that i probably should ask Tim Jones first..

    Hi Tim…
    Is there any kind of walkthrough in using BRU PE, like a video tutorial, that explains what to do and what to avoid?
    I usually left the archiving overnight, and I found that i get confused on a lot of details, like how to verify if the tape we used has the data (i remembered that in BRU Server there’s an information block about the tape ID & related infos of the loaded tape), or if the archiving actually succeeded.

    What i do is I usually waited to see that the job is running, when it started to writing, i just left the job and check the following day. When i come back & see that the archiving is done, i exported the pdf to check the job details and i can’t find any list of files in that pdf, which is weird because i saw that BRU reported the file name & what size being archived in the progress window.
    But yeah, there’s a lot of things that i can’t wrap my head around, i think that a video tutorial or a walkthrough might help answer some of them so i can find out if there’s some real problems that i encountered instead of problems caused by my own dumb mistakes.

    Thanks.
    Oleng

    Oleng,
    Prodigi House
    Jakarta, Indonesia

  • Tim Jones

    July 15, 2013 at 2:50 pm

    What are you specifically trying to achieve? Are you trying to create a DAM-like environment?

    Tim

    Tim Jones
    CTO – TOLIS Group, Inc.
    https://www.productionbackup.com
    BRU … because it’s the RESTORE that matters!

  • Tim Jones

    July 15, 2013 at 2:54 pm

    Ah – we are actually working on a couple of different scenarios – Archival and Long Term Storage, Archival of projects, and Archival for delivery.

    We’re heads-down on BRU PE for Windows right now, but the videos will come shortly after that release.

    Does anyone have a specific scenario that they would like to see covered?

    Tim

    Tim Jones
    CTO – TOLIS Group, Inc.
    https://www.productionbackup.com
    BRU … because it’s the RESTORE that matters!

  • Oleng Tjioe

    July 15, 2013 at 3:45 pm

    I’ll just recycle this post

    To put it short, i want to know the methods to match the content of the tapes with the files archived in the job. I think several things that confused me are:

    – At first i thought that the saved job definitions is used to compared the content of the tape (for Restore). Now i realised that the catalog tar is the document that BRU PE look at when restoring files.
    – When i create a job definition in Advanced View, there’s a base job definition, which correlation i don’t understand
    – I have several jobs that resulted in 0 files written, i even uncompressed the TapeInfo and open the catalog file, but there’s no file records there.

    But overall i haven’t get the whole picture how to make BRU PE works for us after archiving, the parts of verifying, documenting and restoring the tapes.

    Oleng,
    Prodigi House
    Jakarta, Indonesia

  • Oleng Tjioe

    July 15, 2013 at 8:03 pm

    Hi Tim

    so basically we’re archiving projects and HD clips/image sequences, the projects are archived up to a year, then we would reuse the tape for new projects.

    I don’t really have that much problem with using the software up to archiving part, but when it’s about double checking the job, verifying existing tape, it’s not so easy to grasp. (I used BRU server before, but that’s some time ago)
    I struggled with the terms (like tape set, tape name/id, archive id) and how the informations of the files inside the tape or about the tape are presented. Not to mention i can’t really see inside the tape to check if the files are there if the catalog records that there’s no files written in that job.

    And on top of it all i also haven’t have a good workflow for documenting the archives. Do i use another cataloging software, so that all the tapes & archives are listed in tabular data, search-friendly & sortable without spending much time doing it manually (and risking bigger probability of human error)

    I’m not sure if this is the right place to suggest a software feature/changes, but these questions might include some in here

    1. Verify archive in the tape
    I understand that it actually verify the archive structure or integrity, but i’d like it if there’s a notification of what archive is verified in the tape. Now it’s just put out a notice that the tape is verified successfully. And to get the information about the tape, i have to go to Restore section which was a bit counter intuitive i think?

    2. Do you think i should change to LTFS, for more ease of use? I mean in the future, someone else might have to do all the archiving, doublecheck, restoring and documenting the project files, and i know for sure 90% chance is that guy wouldn’t be as knowledgable at this as me. I’m not bragging, but it’s a fact of life here, it’s hard to get a technical savvy personel. I just happened to end up in this industry myself and stuck around.

    3. Is there a way to get the file path in the pdf catalog shorten, or maybe formatted like for every files in the same parent folder they would be grouped like for example:

    (parent folder as head)
    – (list of the files inside)

    /Volumes/Archives/ProjectFolderwithLongNames/DepartmentNameFolder/2013/06/16/
    – image0000001.dpx
    – longnameoftheframeimage0000002.dpx
    – longnameoftheframeimage0000003.dpx
    – longnameoftheframeimage0000004.dpx
    – longnameoftheframeimage0000005.dpx
    – longnameoftheframeimage0000006.dpx
    – longnameoftheframeimage0000007.dpx

    Now what i see is a really long list of truncated path names
    /Volumes/Archives/ProjectFolderwithLongNames/Dep…/…/…oftheframeimage0000001.dpx
    /Volumes/Archives/ProjectFolderwithLongNames/Dep…/…/…oftheframeimage0000002.dpx
    /Volumes/Archives/ProjectFolderwithLongNames/Dep…/…/…oftheframeimage0000003.dpx
    /Volumes/Archives/ProjectFolderwithLongNames/Dep…/…/…oftheframeimage0000004.dpx

    PS:
    I read the user guide today, and i got a better understanding of the terms, like archive id (in separate sessions), etc. but i still have some confusions. maybe i should divide this questions into several shorter posts

    Oleng,
    Prodigi House
    Jakarta, Indonesia

  • Oleng Tjioe

    July 15, 2013 at 8:42 pm

    I think it would be better for common people like me if the tutorial divided in 3 parts, Setting up the job, Archiving, and Post-Archiving.

    In the screenshot i uploaded here 6266_brutapeinfo201307153.41.44pm.png.zip, i find the terms in that notification a bit confusing, especially if the archive isn’t in separate sessions.

    I created a naming system, it’s ArchiveDate_ArchiveOrder_ProjectNumbers. This system doesn’t contain any archive id since i couldn’t tell what it will be beforehand.

    That screenshot is of a tape that contains several projects in a single archive, hence [date]_[archive#]_[projectnumbers] 20130608_001_9-13-21-24-35.
    I believe the Label is for archive/job label, and that i wasn’t really noticing why there’s two labels there when i ran the job, lol.

    Now with separate sessions archive, i get a better idea of what archive id and how to name them, but i hope you can see my confusion here, i’m still not clear on what i should do

    Oleng,
    Prodigi House
    Jakarta, Indonesia

  • Cody Foltz

    January 19, 2015 at 4:22 am

    After I complete a project, I typically try to keep everything within one root directory folder to make it easier for archiving. I am using BRU-PE 3.1.7 and LTO 5 tape/drive.

    I do have a question as far as when it is safe to delete assets from the system drive after it has ben written to tape. If I am archiving a full LTO 5 tape, and the verification process has begun (typically about 3hr. or more) at that point is it safe to delete the data from the system drive since it has been written to tape but hasn’t completed verification?

    Thanks.

    __________________________
    Cody Foltz
    http://www.leftdigitalmedia.com
    Editor | Producer

    MacPro V.3,1
    2.8 GHz Quad-Core Intel
    16GB 667 MHz RAM
    OSX 10.6.8

  • Rumen Mitov

    August 31, 2015 at 1:44 pm

    Hi there,
    I am Rumen from Bulgaria. I am using LTO4 in HP drive SCSI since 3.5 years with BRU PE last vers. We using to archive TV production sources audio and video. I have a 2 questions:

    1. Is there have some way to compress data so that it will useful after that. We have not any additional hardware for that and should be use software compression. What kind?
    2. Is it normal on Maxell LTO Ultrium4 tape 800/1600 to collect only 500GB. I did quite testing but without success.

  • Tim Jones

    August 31, 2015 at 3:02 pm

    HI Rumen,

    The compression in LTO drives is automatic. It’s also lossless, so you get back exactly what you put in. There is no further compression in your situation because the data is already compressed by either your camera or your CODEC during transcode.

    As for capacity, BRU does incur around 18% overhead to provide the reliability factors that are included in its container format, but loss of 300GB is 38%, so something is going on at the drive level. A couple of questions:

    When was the last time you ran a cleaning cartridge through the drive?
    Have you downloaded HP’s Library and Tape Tools and run a diagnostic test on the drive?
    (depending on the number of tapes you’ve used, the drive may be exhibiting head wear with will reduce tape capacity).
    What version of OS X are you using and which pSCSI host bus adapter are you using?

    Tim

    Tim Jones
    CTO – TOLIS Group, Inc.
    https://www.tolisgroup.com
    BRU … because it’s the RESTORE that matters!

  • Rumen Mitov

    August 31, 2015 at 5:50 pm

    Hi Tim! Thank you for the answer!

    If there no compression what is mean 1.6TB on the tape and which way can I fill in?

    Your Questions:
    1. Never clean
    2. Didn’t download and run diagnostic.
    3. Usualy run about 1.5 TB on 3 tapes.
    4. MAC OS X 10.8.5
    Areca Technology Corporation
    Product: ARC-1320
    Revision: version 2.0.3
    Initiator Identifier: 255

Page 1 of 3

We use anonymous cookies to give you the best experience we can.
Our Privacy policy | GDPR Policy