Bryson Jones
Forum Replies Created
-
That’s a great idea. To share a tip back, we build all our systems on our dev server and then populate them to the client in much the same way. That way we don’t get caught up in hardware deliveries and configuration issues while programming and planning.
Thanks for keeping this flowing.
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com
-
Scott, we have a client that brought up that scene detect “feature” as well. We’re waiting on an official response. Hope it becomes a selection.
As to the XML, yes, you can export Excel as text or XML and then do an XSLT to create a CatDV XML that could be “injected”.
One thing to note, a regular xml import into CatDV creates a new asset, even if the clip exists. To do what you’re describing you’ll need to use the command line interface.
The steps I see are: 1) Excel export 2) XSLT that output to CatDV format. 3) Use the CLI to write the data to the clip record.
Also as a pre-built solution, NSA has a product called XML-Update that reads an XML and updates specified fields in a clip. So you can drop XMLs into a watch folder and have CatDV updated.
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com
-
Have you confirmed with an outside tool that these files have source tc or perhaps is the Avid putting them out without it?
I’d try to verify it with another tool (maybe even re-importing them into Avid again) and see about that first.
Barring that, you could post a couple of short (short!) test clips for folks here to DL and test independently.
Hope you sort it, inconsistency is maddening.
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com
-
That CLI trick is nice, Rolf.
We’re actually working on some new transcoding workflows that write back to CatDV whether they pass or fail. Stay tuned to this space…
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com
-
I did a little “tour” of some servers based on your question and here are three db status reports.
Client A:
clips: 943,830
thmbs: 1,326,924
data size: 6.5GBClient B:
clips: 191,931
thmbs: 284,368
data size: 1.2 GBClient C:
clips: 27,310
thmbs: 14,972
data size: 85MBThey seem fairly linear from what I can see, but know that some folks go crazy with thumbnails. (Scene Detect creates a thumb for every mark so an hour can have hundreds of thumbs for a single clip!)
As to proxy sync, you are on the right track. I’ve not used rsync over a wan but tons of times on a LAN. But we’ve used “cheaters” like Chronosync and other apps. I’ve even had clients with non-secure (and very small) systems use cloud sync services with decent results.
If you want to get bigger, you can use StorageDNA to sync servers quickly and their DNASync license can be bought cheaper if you get it with their LTO solution as an add-on.
The king of them all is of course Aspera but that’s true Enterprise with a price to match. SDNA is cheaper and chronosync is $40 and then rsync is just admin time and your brain. 😉
A buddy of mine and I ran a CatDV db on Amazon years ago just for fun. (Wow that sounds geeky.) It was fine but I think you are onto the true solution. The timing is crazy, as this am I was on a call with one of our dev guys and he was discussing running a db clustered on a vm with iscsi sharing the data files. A bit much for CatDV maybe but it’s truly amazing how far you can scale with database technology.
Also, I believe we might see some db replication and sync coming in the future, as larger clients choose CatDV, so stay tuned.
Hope these numbers help.
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com
-
I’ve not done one in linux but we do a lot of our dev on vm’s with OS X and Win 2008 server. Should be fine and way easier to manage a linux box that way.
Scott, I’ve been really considering beefed up linux servers for our larger clients as well. I’m assuming that we can get a far stronger server config (SSD raids, 10Gb enet, etc…) for less money, and leave the Workers on whatever OS the client wants.
Thanks for paving the way on this.
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com
-
I will, along with a few members of the NSA team.
We will be visiting a few booths of our partner vendors and dealers and also have a demo suite in the Renaissance. We now have dealer partners in most regions of the country as well and they will all have show floor contact info.
Our schedules will be posted on our site the week before NAB and you will be able to check in at a few booths to reach us as well.
Stand by for a full schedule as we get closer to the show.
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com
-
The following is an opinion and offers some things to consider beyond price. If you’re choosing on price, then it’s easy. Find the cheapest one and deal with it. There is really little to consider if that’s your limitation. However, if you are looking to match the solution to your workflow and production, here are some thoughts.
Take with a shaker full of salt grains.
Evaluate a few. Dealers can demo them all for you and also discuss the real-world transfer times which vary WIDELY depending on your environment.
I’ve installed all of the systems you listed and all work well. The difference is all in their convenience, automation capabilities and speed, which depends a lot on your SAN, storage or network.
Consider the real-world use case. Will you actually pull a single asset back or will you most likely restore a group of assets? I’m about to make everyone mad here but in a lot of operations I’ve seen people tend to restore groups or folders or archives in bulk and so knowing “what tape is that on?” often trumps the actual file by file granularity.
(WARNING: Again this is based only on my companies experience and my opinion (!!)others may see more granularity in their deployed systems!!)
Pro Tip: Be careful that you don’t spend thousands of dollars implementing a solution that you cannot adequately staff and support.
I personally like all of the solutions presented here. But they are vastly different and depending on your particular workflow, they may all fail or succeed in a specific environment.
Case in point, I did a consult on a project with a large SAN that was planning to use an unnamed system to do a “backup” of the whole system. I stopped the project when I noted on a conference call that the operation would take 6 months of 7 day weeks to complete on their current infrastructure. (No fault of the archive system.) They had to reconsider the whole plan after that.
For example, let’s say a SAS connected system will pull 600MB per second theoretically. If you have a NAS that’s only connected via gigabit ethernet, none of these systems will get full pipe speed (SAS) since the network’s your bottleneck. You can use most anything and they will perform similarly. Choose by convenience price or another factor. BRU will be similar in speed to a Pro Cache or SNDA since they will all be using that gigabit pipe to see the storage. This is not literally true but you won’t see anything like SAS speeds.
However, if you have a fibre SAN that can transfer 300MB per sec to an archive machine, you might want to ensure your archiving system can connect at that same speed either by direct connection or an upgraded network connection (10Gb PowerCache in the case of Cache-A for instance, or BRU and SDNA’s SAS connection) if you need absolute raw speed in your rig. Most don’t actually, but you may.
The box you think you might buy may or may not be suited to your environment, even within a single product line, so evaluate a few options with all variables considered. All of these solutions vary from a single drive to multiple drives and loaders.
If you have under 10TB, don’t sweat it, buy anything (Not literally of course,.) But if you have a significant amount of data and need to secure that quickly, evaluate carefully and speak to your reseller and users in the real-world who will have accurate transfer times based on specific storage systems and network topologies.
Also, never forget that adding a MAM to your archive saves time on the restores, but adds greatly to time on the archive due to cataloging, proxy creation and tagging operations. I’ve seen people take months to archive data waiting on a slow proxy creation system to complete the proxy files. Plan accordingly.
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com
-
Your answer is in the variables.
CatDV works on paths, and the proxy is in an equivalent path structure. Therefore if you have the path to an asset as a variable and you have the path to the proxy share as an absolute value you can create the path to the proxy through combining variables and the proxy root path.
Contact support, they can easily demonstrate this concept for you on your machine as it’s way easier to show than type. It’s also very dependent on how you have your proxy mapped so you may have to do some mucking with that if your proxy path mapping is more complex. Hence my reluctance to give you an exact path. It’s environmentally dependent.
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com
-
Bob, that’s a pretty good .02, and worth way more than the price. 😉
I do think it’s critical to keep in mind the 2 sides of this post.
1. Want to buy an application to manage a few assets and create some proxy? Then a download will be a fair demo.
2. Want to implement a Digital Asset Management system? That’s a whole other project and has little to do with the standalone download. For instance, if you’ve been looking at Cantemo Portal, Levels Beyond, Media Beacon or any of the other mid-sized to enterprise DAM products, the demo download will likely leave you unimpressed, if only because all of those other products involve days to weeks of consulting and deployment, as does a fully funcional implementation of CatDV Server with all the trimmings.
CatDV Pro is a great little app that you can download and use instantly. Done.
CatDV Pro, with Server and Worker and an archiving system and maybe 2 or 3 other systems is a solid DAM/MAM that you can run a complete facility on. That one might take a few days to weeks to months to evaluate properly, just like any other DAM.
Be sure you are clear as to what you are evaluating when you click that download button but by all means, do it!
bryson
bryson “at” northshoreautomation.com
northshoreautomation.com