[Kevin Duggan]

Hi Justin

the very general query is a sledgehammer to crack a nut as opposed to the scalpel blade precision of an indexed query. I like to call it a query of last resort. This is why you see it is an option to display it in the Simple Query fields. It is the worst possible search in that it has to trawl the whole of the database for a match and the bigger the database the longer you have to wait. Usually you can help the speed by giving the system hints… ie what year , what series , something to get the indexed searching working . In my teaching this stuff to users you notice that we are all fundamentally lazy, so the any logging field option seems to mean that we will get what we want , so lets use that.. However we all want speed as well.So a well constructed logging schema will help your users find stuff faster. Explore so called Smart folders , they are really queries with more than one criteria , a mixture of these and teaching your users how to “GOOGLE” this stuff will make them all happy campers… There is a lot of traffic about this subject in this forum..

Kevin Duggan
Catdv

[Kevin Duggan]

HI Kevin
you wrote .”My assumption has been that once metadata is added to a video file, it stays with that file and can be searched and located by any program that reads metadata.”

This assumption is incorrect. What CatDv can do is extract the embedded metadata if present and write it to a Database along with any additional fields you may have added within CATDV plus the file path to the asset. The reasoning for this approach is simple. Imagine you have a million files , are we to open up this number of files to find out “whats inside them ” this would be in-efficent and cause the OS to probably crash. What CatDv does in its server version , is check with the MYSQL database for all information it has on this particular file or files. SQL databases are optimised for this type of activity and the results of a query are returned very quickly. This approach means that files do not have to be opened and the network traffic is reduced to text and thumbnails .
With premiere you can drag and drop assets from the CatDv interface to premiere if you wish, however if your copy of Premiere supports FCP XML import you can also take the metadata over to Premiere as well.
This is a rather simplistic explanation but you will find a ton of other information within this forum.

Happy New Year

Kevin Duggan
CatDv/MME

[Kevin Duggan]

Hi Bryan
you basically have the workflow correct. On the details you will need Calbrated Software to extend Qtime so that it plays mxf wrapped files. CATDV MXF option to read and stitch together these files, then MME will take the filepaths and the metadata created in CATDV and make an AAF file which you drag into the Avid. As i posted previously.. if you already have the P2 volume or instance of Volume already mounted in the Avid , then the clips imported into the Avid will auto relink…ie go Yellow as per normal AMA workflows… if you have not the right click and relink to the AMA VOLUME again the clips will go yellow as AMA does its magic but importantly you will have the metadata logged and then extracted from CATDV by MME in the Avid bin…

Takes longer to explain than to do it … :-)…
MME is designed to accomadate three possible workflows. NATIVE avid files in the the usual file structures . In MME select link by AVID MEDIA option. Generate AAF for clips that you wish to transcode but retain the metadata … and finally Qtime files via AMA.. select the link by qtime option . All three options import the metadata into the Avid..

Hope this helps

Kevin Duggan
CatDv/MME

[Kevin Duggan]

Hi Bryan
and yes I am listening Matt… “MME” METADATA MEDIA EXCHANGE has become known during its development as the ‘widget” . It is a much more sophisticated than its nickname would imply but its design goal is to retain the simplicity of the “drag and drop” metaphor that is familiar to editors.
So what is it you ask? Basically using FCP or CATDV to produce an XML file this is submitted to MME which parses the information and creates an AAF batch file . This file is directly dragged and dropped into an AVID bin and metadata information from the XML is brought into the Avid world. This metadata information can include , ten fields of information ..( a FCP limitation ) , markers , ( both single events and range markers ) , sequences and sequence markers. In short pretty much most of your primary metadata needs.
With the addition of Catdv, tighter intergration between the applications can be accomplished. Simply use Catdv to log your assetts with the advantages of global tagging and the ability to read additional metadata from the files automatically , mapping this to whatever FCP fields you require , then select all the logged assets and utilise the right click command to SEND TO MME command.

In the background MME creates an AAF file which you drag from its interface to the Avid Bin. In the case of P2 if you use AMA and the Avid has seen the AMA VOLUME previously ie the assets have been already added to the project the clips will auto relink and the metadata will be displayed.

Provision is also made for the importing of native Avid mxf clips ,which auto relink because the Avid already knows the path to these files.

MME is also cross platform so your PC AVIDS are not orphaned from this type of workflow. In short using MME with CATDV you can have one central database that can take metadata both to and from FCP and currently to the AVID….. We are currently beta testing and putting the product in the hands of our partners for feedback etc. and hope to ship very soon..

Kevin Duggan
Catdv/MME

[Kevin Duggan]

Hi jeff
just like an NLE CatDv can update its media location via the Media Menu or even attach media , similar to FCP , if you know that something is the correct file , even though some of the attributes do not match. The file paths are then updated and you are good to go. Catdv as I mentioned before can also move your assets for you. The advantage of this is that it updates it records to reflect the new location. As a general rule its not a good idea to do stuff behind your Asset Managements Systems back. Use the tools contained within it and life will be easier, however if someone does the tools are there to fix matters

Kevin Duggan
CatDv

[Kevin Duggan]

Hi Jeff
I have written about these types of “issues” elsewhere within this forum about the Catalog restriction, so do a quick search here. In truth the first thing you should realise is that on CatDv Server all queries are global across all catalogs. A query for “India” will return all records that match this keyword no matter how many records this maybe. However you then have to think if this returns 20,000 hits how useful in practice is this too me. So lets try and break things down a little further . If you look into the prefs you can see that you can restrict the number of records that Catdv loads from its database , to maybe 3000 or maybe 50000 its up to you , do you really want to wait whist Catdv loads 50,000 records into RAM? If a search or query returns this amount of hits then perhaps you should give it a little more specific information. Why not use a date field for instance and search within a COUNTRY field and a DATE field that you may specify and do an indexed search for INDIA and 2008 , this will ignore all values that fall outside of these two criteria and return results for these criteria incredibly quickly and load the metadata and thumbnails from the server into RAM. Then the fun can begin by using GROUPINGS , a drop down list of other available fields to add maybe another field to Filter further what we have already loaded into the CATDV browser. Maybe we can now filter on a field called CAMERAMAN …Selecting this field in GROUPING you will see a list of every cameraman who shot anything in INDIA in 2008 , by selecting each name in turn you may turn up what you are looking for. I try to explain this as a more human edit centric way of searching, . I have lost track of the number of times you have the conversation with a director or producer and the conversation is along the lines.. ” We shot it in India in May with Fred , in Mumbai. So translate that this into how you log and consequently search in CATDV. You then realise then that the catalog is just a convenience much like a folder is on a computer.You can give it a name but it is not a fundemental orginisational tool. The key is the schema or list of fields. CatDv with its query mechanism will present you with a list of these fields and values. So you can easily search on the same criteria with which you logged your footage. Taking care with the selection of these metadata fields is the key to success. I advocate using as few “freeform” fields as possible . Instead use Picklists. Groupings and Multi`s to provide a consisten logging interface. Utilise Global tagging to apply quickly values to all selected clips in the browser and then do a second pass of the clips for discrete and unique data about individual clips. You say you are using P2 , then you are in luck because CATDV can harvest metadata from within these clips if you utilise this ability within your camera. So for instance CAMERAMAN data and memo information can be gathered automatically .
You are probably familiar with the P2 viewer but if you use CatDv and then select all the assets you can copy them to any part of your SAN and CATDV can place all clips from a project within a single CONTENTS folder , try it SELECT all assets then right click to copy to a specified destination . All cards can be placed in one folder.
So in short don`t think catalogs , think metadata tags..

Hope this helps

Kevin Duggan
Catdv

[Kevin Duggan]

Hi Andy.
glad your getting on okay but may I make some suggestions. You are running the standalone version of Catdv and you have made several catalogs , one for each day of your shoot. Now this seems like a logical way of going about things and if you where running our server version of CatDv then this would present no problems with searching across all the catalogs and also give you a human type of metadata to search and filter your material on. You additionally state that you have about ninety clips in each catalog.
Well the best tip I can give you is to create a new catalog and drag and drop all of the previous content into a new GLOBAL catalog. CatDv can easily deal with thousands of clips in a catalog , it will maybe take a second or two to build the window but from on its all gravy as everything is loaded into RAM and its super fast. If you do this you will then realise that you have two friends in finding your stuff. One is what I call GLOBAL tagging. So you could for instance put your catalog into List view and filter on creation date. This will group all the shots from all your shoots dates into either ascending or descending order .. its your choice . Select say all your shots from one day with SHIFT /Select then start to fill in your form ie the User fields that you have created for logging. Now everything you type or select is effecting all of the selected assets . These are the global attributes , think cameraman director , producer , location, job number , job title etc etc … Then you can deselect the clips and do a second pass and add what is unique or dicreet to the shots… in this case you could tag specific locations , wideshot river , B+roll … downtown sunset , sunrise. etc etc..

All these keywords or tags will allow you to create searches across all clips (remember we dragged and dropped them into one single catalog ) so we could if the logging schema permits do a search for Everything that was shot by Jim in Texas for a specific show on the 21/10/2010 …. Start to think like normal editing folk. You know I am looking for a sunset shot of Dallas . I am sure you get the idea.. But it requires a bit of work to design your logging schema and to add the metadata . CatDv tries to make it as simple and fast as we can but someone has to view and log the footage. You will probably find in a long form documentary shot over an extended time frame that it will pay dividends as you move forward.
I have written here before as to the limitations of the catalog and the fact it is not actually required but understand that it appears attractive initially as a mechanism to organise you media , but search this forum for other ideas as to how others have structured their logging. Pretty soon you will be using filters and quick searches and indexed searches to find your stuff. The short hand I have to describe this is that initially Catdv presents to the user like a SPREADSHEET…. hey why would anyone want one of those , its just a blank page with a grid… Its only when you plug in the formulas that it becomes useful. CatDV is similar out of the gate its that blank page , think of how you are going to search and then log in that manner and build CATDV out to mimic you thought process would be a good start. Its easy to add additional fields as you go . The second concept is , think GOOGLE . Use the query panel to type in keywords and group the tags together as in my earlier examples. CatD will only present you with the candidates that match your search criterial, this is the filtering effect. Click on in the tree on the interface element “all clips ” to get back to your home position. However with the right metadata tags you can then use this mechanism to help build your bins.. So for instance if you tag all your interviews with name of the interviewee and a tag INTERVIEW . you could do a simple search for INTERVIEWS … which will return all interview….. then fileter on the name of the interviewee eg Bryson Jones to get just these takes. . Go through these interviews , they will most likely be very long if Bryson is involved but you can add Event Markers at the good bits. Also because you are using transcriptions have a look at the HTML summary tab as you will be able to search for key works in the transcripts. Bryson came to mind because this is a stock technique in reality shows of which he has done many.. He might be tempted to share some of his knowledge but he is a kindly but timerous animal and you may have to be patient , to see if he will appear..
As those that know me can testify I could go on , but maybe this will get you thinking and help your workflow.

Anyway hope this helps a little

Kevin Duggan
CatDv

[Kevin Duggan]

HI Bruce
I think what you propose raises some interesting points. What you are proposing is what I see many times at smaller shops that have shared storage. They have figured out rightly that the key to success is a well understood Folder structure that is enforced across the facility.
So what you want the Worker to do is to automate this to some extent by having a single drop folder and using Worker conditional testing create and place the files in the correct places on the SAN based on the CREATOR field. I have implemented something similar in Televisa in Mexico City where multiple ingest machines place capture to a central folder and based on naming conventions ( as Bryson suggests ) move stuff to the correct folders . This has been operating so smoothly over the last 12 months that I have just returned from building a further two Xsans to accomodate expansion and I did not have to change a single script.
You say you are using the Sony software so I am presuming you are using a LOG and TRANSFER methodology and re-wraping the SONY files to Qtime you reason that this gets embedded in the clip and that this is a good thing. So firstly I would ask you too look at these files in CATDV in the OTHER TAB of the interface which presents all of the metadata that is embedded inside the clips that CATDV can read.. You may be suprised just how much their is . However it should be pointed out that the re-wrapping process may throw a lot of metadata that is present in the source media, which is why I am a fan of native workflows which are possible using MXF4Mac or Calibrated Softwares quicktime extenders. If the Creator field is present you may be able to use this to create scripts in the WORKER to do what you desire . However you further write .. “we don’t have time or see value in describing each clip uniquely” This I think is a missed opportunity. We have clients with many many petabytes of data and I have seen that essentially folder based operations for editors/producers days are numbered as Workgroup type workflow practices such as software like CatDv enable become more widespread. In short we actually do not want people going anywhere near the folders on the SAN. A Smart Folder in CATDV can give you precisely the same functionality plus the harvested and the added metadata and due to its filtering mechanisms very fast pre-edit functionality due to the ability to do global tagging and Speed logging . In addition scripts can be triggered from within CATDV that to ftp and transcoding and other sundry day to day tasks that are the moment falling to Editors to do but could be de-skilled by the addition of automations. So I would urge you to think beyond just the folder approach to asset management and how CatDv can move and archive files under its control , which means the database is updated to reflect these changes without intervention as this leads to consistency which is after all your desired outcome. I make these remarks to you and also with a desire to spark off general discussion about these areas of interest within this forum. Those that know my patter will tell you I have a simple dictum .. “Where is it and What is it?” as applied to asset management. You correctly have identified the Where part of the puzzle I want to stimulate discussion further about the “What is it” part. I think I will leave the discussion here for the moment or the post starts becoming a “War and Peace” edition

Hope this helps

Cheers

Kevin Duggan
Catdv

[Kevin Duggan]

HI Stuart
I think the clue maybe using “MXF4Mac on Macs and Calibrated on PC clients.” .Greg from Calibrated informs me that the two products use differing approaches to 24p.

“For instance – a 1080i29.97 video file with 24 Advanced Pulldown – that has a reported 30fps NDF timecode of 7:35:41:15 – we would remove the advanced pulldown from the video and report timecode as 24fps 07:35:41:12

Does this ring true to what you are seeing? Could you provide further details of your workflow. ie are you generating proxies . Are you logging on the PC`s or the Macs or both etc

Cheers

Kevin Duggan
CatDv

[Kevin Duggan]

Hi Stuart

Firstly you need to enable ADVANCED menus in the General Prefs , this will give you the additional menu options . If you need to bring metadata from FCP. Open the desired project in FCP. Then goto the OTHER tab in CATDV and right/click (ctrl/click on a mac ) the BOLD FCP project name . This will give a load of options. I would start with Add to catalog . All of the clips will come over from FCP complete with metadata and a project file. You could then select all of the clips and do GLOBAL tagging ie add additional metadata to all of the clips in one go… then save it as a named catalog to the server if you wish. We have clients who have pulled whole series into CATDV this way. OR make an EMPTY clip from the EDIT menu add all of the metadata to this from your drop downs etc… GOTO EDIT> COPY to place all of this info on the Macs CLIPBOARD then select your asetts and GOTO EDIT>PASTE METADATA … this will give you a dialog .. whether to overwrite /merge or only write if blank to the fiields for the asetts… You should be able to figure out what you want to do.. Finally if fields or values are not in the correct place for you .. GOTO BULK EDIT and this will allow you to copy stuff to other fields to provide consistency with the your CATDV schemas

Hope this helps

Kevin Duggan
CatDv