[Matthew Hughes]

I wish I could edit my original post above since it is a grammatical nightmare but I will just continue down here and hope to do some cleanup in the future.

All of this discussion and the above deals mainly with Quick Clustering but Compressor uses the same methods for Administrative Clustering and for Computer Plus though some minor points will be different for each.

I can say now that without a doubt the Compressor that is set up to be the Cluster Controller does create an NFS share for both the temporary Cluster Storage and the source location containing your videos. It does them at different times. The process that Compressor uses can be shown in the following sequence:

1. You turn on sharing in Qmaster menu and set it “as QuickCluster with services” and hit ok. Immediately this Compressor creates an NFS share for the temporary Cluster Storage. The Default location is /Users/Controller/Library/Application Support/Apple Qmaster/Storage/xxxxxxxx-xxxxxxx/shared. Each time you turn off sharing and turn it back on it will create a new share location below the …/Storage/ folder. If you set a temp storage location different than the default then it will use the same child directory structure. For example if you create a new folder on your main HDD root called ClusterStorage then the location would be /ClusterStorage/xxxxxxxx.xxxxxxxx/shared. You can verify the share is created by typing into terminal “showmount -e” which prints to the screen the contents of the /etc/exports folder (the NFS shares).

2. You set up your nodes and turn sharing on and they can be seen by Qmaster – you can verify this by seeing them in the AppleQmaster/Administer Cluster display. If other computers are sharing services on the network they will appear in the bottom window known as the Qmaster Service Browser.

3. You submit a job to the Quick Cluster. At this point Compressor creates a new NSF share to where your source files are stored. You will see this added to the exports file as well. It also communicates to the other nodes on your network, that you shared services on, and tells them the share locations. They in turn use them to create two mount points on their local hard drives which they use to transfer the files back and forth. To help troubleshoot where the process is breaking down, turn on “show connected servers” on desktop in your Finder preferences. The first step is for the node to mount the share to the temp cluster storage. If it cannot do this it does not bother to attempt to mount the share to your source files. You will see the mount points flash a few times. If you only see the mount called “shared” then you will be trying again.

This is why it does not matter if you have anything shared on your computers. Yes, they need to be able to see each other on the network but do not need to have the source file location or the temp Cluster Storage locations shared in the Preference Sharing Pane or via Finder – these are using Apples’ AFP file sharing system by default. Most of the jobs I submit have file sharing turned off on the Cluster Controller.

If everything works correctly, then your job will start and transcoding will begin shortly.
A good tool to help troubleshoot Clustering is to turn on hidden files by typing into terminal “defaults write com.apple.Finder AppleShowAllFiles YES”. This will make it much easier to navigate to and delete the folders to reset the clustering mechanisms and to see the hidden shares that get created. I turn on ShowAllFiles on all of the machines I am going to use to Cluster.

Here is the output of a 2 machine Quick Cluster with one machine acting as the client and Cluster Controller, and one node with Compressor services turn on. They are using the defaults for all options.

NFS shares on Control Cluster created by Compressor:
Controller:~ Controller$ showmount -e
Exports list on localhost:
/Users/Controller/Library/Application Support/Apple Qmaster/Storage/F893BC82-0E954C8E/shared Everyone
/Volumes/DeepDiveVoRenders/DeepDiveVoRenders/fcp/.qmsubexp Everyone

Mounts on MacMini02 during cluster job:
server:~ macmini02$ mount
/dev/disk0s2 on / (hfs, local, journaled)
devfs on /dev (devfs, local, nobrowse)
/dev/disk1s2 on /Volumes/Macintosh HD2 (hfs, local, journaled)
map -hosts on /net (autofs, nosuid, automounted, nobrowse)
map auto_home on /home (autofs, automounted, nobrowse)
map -fstab on /Network/Servers (autofs, automounted, nobrowse)
Controller.local:/Users/Controller/Library/Application Support/Apple Qmaster/Storage/F893BC82-0E954C8E/shared on /Users/Shared/Library/Application Support/Apple Qmaster/Storage/F893BC82-0E954C8E/shared (nfs, nodev, nosuid, mounted by macmini02)
Controller.local:/Volumes/DeepDiveVoRenders/DeepDiveVoRenders/fcp/.qmsubexp on /Users/Shared/Library/Application Support/Apple Qmaster/Storage/subexp/B5DD365B-F91B5449 (nfs, nodev, nosuid, mounted by macmini02)

For each node the mounted folder name to the NFS share for the source files will be different. For the NFS share of the temporary Cluster Storage, the mounted folder name is the same.

The .qmsubex folder was a hidden folder created within my source file folder. I was using the Compressor option of “Copy Source Files as needed”. Though it never copied source files to the cluster storage it did make some type of copy into the .qmsubex folder (which could be short for Qmaster Sub Expression) which the nodes appear to be using. They could not be real files though they acted like them. No alias tag, they played and previewed in Finder, but they also were created beyond capable speed (30 GB mov files being created in 2 seconds) and while the drive said 380 MB free of a 3 TB HDD the folder showed 3.6 TB – obviously impossible. The transcoding started immediately.

So that is what you are looking for to know that it is working or not working correctly. Here is my outline of Apple’s Compressor 4 Cluster procedure so you can make sure you are following the basics steps.

Method 1 for distributed processing (Computer Plus)
Create Apple Qmaster service nodes in Compressor on networked computers
In Compressor, choose Apple Qmaster > Share this computer
Select “Share this computer” and “as Services only”
Select Compressor in the Services area.
Make sure “require these services to only be used in managed clusters” is NOT selected.
Click “OK”
When submitting Compressor Batch for processing
Leave the Cluster menu set to “This Computer”
Select the “This Computer Plus” checkbox.

Method 2 for distributed processing (Quick Clusters)
Configure the Cluster Controller
In Compressor, choose Apple Qmaster > Share this computer
Select “Share this computer”
Select “as QuickCluster with services” option
Rendering should not be selected.
Compressor should be selected.
Make sure “require these services to only be used in managed clusters” is NOT selected.
“Identify this Quickcluster as:” textbox should be used to name your Quick Cluster
“Include unmanaged services from other computers” should be selected.
“Require password” should not be selected/
Click “OK”.
Configure the Service Nodes
In Compressor, choose Apple Qmaster > Share this computer.
Select “Share this computer” and “as Services only”.
Rendering and Compressor should both be selected.
Make sure “require these services to only be used in managed clusters” is NOT selected.
All other settings are disabled.
Click “OK”
When submitting Compressor Batch for processing
In the Compressor Batch Window, click Submit
In the resulting dialog, use the Cluster menu and select your Quick Cluster name.
Click Submit

Method 3 for distributed processing (Apple Qadministrator)
Create a new cluster
In Compressor, choose Apple Qmaster > Administer Clusters
Click the Add (+) Button
Select Untitled Cluster and rename to what you want.
From the Controller menu choose a cluster controller from those available on the network.
Add Service nodes to the cluster
Click the disclosure triangle if the Qmaster Service Browser is not displayed
Add service nodes to the new cluster by dragging them from the QSB list into the cluster service nodes list
Click apply.

When Things go wrong these are things that you can do to make it work:

Procedure before getting the current working cluster:

On Cluster Controller and on the single node (MacMini02)
Deleted all Caches in ~/Library/Caches (Dragged all folders to Trash)
Deleted the Apple Qmaster folder located in ~/Library/Application Support/
Deleted the Compressor folder in ~/library/Application Support/ (will lose all settings and custom settings/dest)
Deleted all Compressor preferences in ~/Library/Preferences
Deleted all Qadministration preferences in ~/Library/Preferences
Deleted all Saved Application States in ~/Library/Saved Application State
Empty trash. (some Application states and/or caches will be in use so will need to just click through them)

On Cluster Controller
Deleted everything inside /private/var/spool/qmaster

Restarted both machines.

Controller Compressor set up
Share this computer
as QuickCluster with services
No Services enabled
Everything else is defaults
Preferences left as defaults including “Copy Sources files as Needed”

MacMini01 / MacMini02 / MacMini03
Share this computer
as Services only
Compressor Services with 2 instances
Everything else as defaults
Preferences left as defaults including “Copy Sources files as Needed”

I am unable to think of any reason deleting the preferences would do anything but when things seem to randomly work when attempting the same identical thing it is best to do much more than do less. I keep my custom Compressor settings and destinations on my desktop anyway, so just drag them in if I kill the preference folders. I can see the value of deleting the application state if it contains anything related to the settings used in the programs though I am not sure if they are the cause of issues so I just kill them. They will be recreated when that program starts back up. Though I think deleting the Application States and caches for Compressor, Qmaster, Qadministrator, and Share Monitor would be sufficient.

Deleting the Apple Qmaster folder located in ~/Library/Application Support/ is the Apple recommendation to get rid of a batch that is resisting being canceled along with deleting the spool contents.

I currently have been running the Single Quick Cluster Controller (Mac Pro Tower) with three nodes (MacMini2011 Server transcoders – Sandybridge chips) with the source files attached to the Cluster Controller vie eSata dock. The Cluster Controller is also acting as the client (submitting the jobs).

I hope this helps you get there.

Compressor Clustering is frustrating for me and I have a large knowledge pool to draw from in my coworkers and boss ( Quad CCIE for help in the network issues, programmer with Masters in CS, a Final Cut / video production expert) plus I am a CCNA myself. It is a complicated piece of software and it doesn’t help that the Compressor front end is still the chunky 32bit legacy code while the compressord transcoding backend is the new 64bit code.
At least you don’t have to worry about creating clusters to get the most out of your multi-core processor anymore. I have noticed very little difference between running nodes with 8,4,2, or 1 instances of compressor (it is compressord). It automatically multithreads to take full advantage of your processor. Running the different number of instances just changes the number of videos being simultaneously transcoded but the overall CPU percentage stays almost identical. You can transcode 4 files on the same CPU at the same time and take 60 minutes or transcode 1 at a time at 15 minutes each for a total of 60 minutes. I prefer the one at a time in case something breaks.

[Matthew Hughes]

Hi,

I am having the same issue you are having and have collected a plethora of research to find a solution but have yet to pinpoint the exact issue point and results capable answer. The problem is that Compressor is finicky and what gets clustering to work once will not always work every time.

What I do know.
Compressor creates a share location for temporary files and tells all of the machines on your network where it is either using nfsd protocol or some internal Qmaster communication. But it does create a share to this location via a NFS share. Does it create a NFS share as well for the location of the source files? I am not sure yet. You can verify the location that is being shared via NFS by looking at the /etc/exports file. Also showmount -e(verify option) in terminal.

I currently am using quick clustering so I have one machine that I am using as the client and cluster controller and three identical macmini2011s that I am using as my nodes, so my situation is slightly different than yours but at this point I am 99% sure this is a NFS share issue. For quick clusters on the cluster controller will have a share in the /etc/exports file. The nodes will not since they are just offering unmanaged services to the controller so do not have the Qmaster cache folder (/var/spool/qmaster) or source files that need to be shared to the network. In your case though they all might create a share, but more likely only the machine submitting the job will.

So it is not doing something correct right? Yes. It is not sharing the right location. This is going to be a super quick post but I will expand on it greatly when I get back to the office this week especially if I get a response to this thread, but things I have done that have sometimes allowed the batching to be submitted and to have all of the computers work as a cluster. We need the nfsd process to reset and Compressor to communicate with it the right locations when you submit the batch.

1. All computers must have the exact same version of OS, Compressor and Qmaster.
2. Unshare in Compressor, close it down, and delete the ~/Library/Application Support/Apple Qmaster folder. Make sure you delete the one in your home directory and not in system. This folder will be remade when you turn on sharing in Compressor. Shut down, and restart and start making the cluster again in Compressor. (Restarting the OS may or may not be necessary – NFS services are started automatically when you boot up so this might help clear it out.
3. Make sure all computers can see the location of the source files across the network.
4. Share that specific location in file sharing on the machine where they reside.
5. Click of file sharing, wait a few seconds, turn sharing back on.
6. Verify that you can see the unmanaged services being advertised by the other machines by clicking on the last option in the Qmaster menu in Compressor.

Sorry hve to go,

This was my syslog message when I got Quickclustering to work last time,

On Controller

Submit Job
9/10/11 11:31:25.750 AM authexec: executing /Applications/Compressor.app/Contents/PlugIns/Compressor/CompressorKit.bundle/Contents/Frameworks/Qmaster.framework/Versions/A/Resources/NFSSharingTool
9/10/11 11:31:25.822 AM nfsd: /Volumes/DeepDiveVoRenders UUID conflict with /Volumes/Deep Dive_voice renders, B8D77742-3239-3867-94BC-8E34EC20401A
9/10/11 11:31:25.822 AM nfsd: /Volumes/DeepDiveVoRenders using UUID 12F67EF8-E481-403A-B032-97F3FC0BAE22
9/10/11 11:31:58.234 AM rpc.statd: Invalid hostname to sm_mon: server.local
9/10/11 11:32:03.237 AM rpc.statd: Invalid hostname to sm_mon: server.local
Clustering working

Matt.