-
Promise SanLink2
-
I will have my two SANLink2 adapters later today so will start testing then. Will report back with results.
-
Hi Barry –
as you have discovered, you cannot rely on Thuderbolt bridge networking. In my opinion it is not even low cost, because even though it “appears” to be free, if you have two Mac’s in two different offices that are more than 6′ away from each other, you need an expensive Corning Optical Thunderbolt 2 cable, and this costs as much as two Promise SanLink 2 boxes. Bottom line, I have discussed this with the Intel folks for the past 2 years at the NAB show. Both years they have told me that while data transfer can be achieved using thunderbolt network (as you have seen yourself), the performance is inconsistent, and they do not recommend it. Their Thunderbolt guy said that “possibly with thunderbolt 3 this might work” but he could not assure me that this would be the case. All Thunderbolt development and research is done in Tel Aviv, Israel, not here or Japan, or China.With that said, any thunderbolt to 10G adaptor requires drivers. Nothing is native to the Mac OS. This means that if you purchase a thunderbolt to 10G adaptor from Small Tree, ATTO Technology, Promise, Sonnet, or Akitio, you must go to this companies website, and install their proprietary driver, and reboot your computer. As Mr. Duffy has pointed out, when you go into System Preferences> Advanced> Hardware, you change from Automatic to Manual, and change the 10G selection to Autoselect. Once you do this, you will be able to select an MTU of 9000. This applies to all products on the market today.
I must urge you to stop going to the Apple store to ask for advice. The employees at the Apple store know nothing about this subject. If you are insistent on dealing with Apple directly, your only hope is to speak with the Apple Enterprise group, which has no affiliation with any store. I am surprised, that since you have dealt with Mr. Duffy in the past, you did not deal with Small Tree right away, who would have resolved this for you quickly.
You will find that if disable SMB signing (create a nsmb.conf file in the /etc folder), your SMB networking will work very well, and you will not see the ramp up performance you are seeing with AFP. SMB has it’s own issues (random disconnects) but it is currently native to macOS Sierra 10.12.
Since you have the SanLink 2’s on order, I would be very surprised if you do not see dramatic performance improvements instantly between your two computers (once you load your Promise driver and reboot your computers).
Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com -
Well, it is nice to know Zelin made it thru the hurricane a few weeks ago ?
Hope to hear more from Bob now ?
Duffy
-
Hi Bob:
Thanks for you reply and advice.
1) Non optical TB cables can be had for $39 for a 6.6-foot cable, $48 for a 9.8-foot cable. Beyond that one has to go for the Corning optical cables that start at around $245 for a 33-foot cable.
2) If Thunderbolt 3 improves this for the ‘Bridge’ solution that will be very welcomed.
3) Yes, I have the SANLink2 Adapters now and started my testing yesterday using them. I did preload the Promise driver software which was done without issues.
4) Yes, as soon as connecting the SANLink2 adapters I was able to set MTU=9000 on my MBP. when configuring “Manual”.
5) I did discuss File Servers and 10G connectivity with Chris Duffy several years ago but at that time the cost of the minimum Small Tree file server would not fit our office budget at the time. The cost then was close to $10,000 including file server, switch, 10G adapters for the Macs (pre MP6,1 model) and cables. We limp along using 1GbE bonded pairs for our Mac for a while and then switch them all out for the Late 2013 MP6,1s.
6) All of the office MP6,1 and iMacs are situated at workstations in a circle around a enclosed central office. In the central office we have a maxed out MP6,1 12-core with as much RAM as we could ram into it along with a the largest internal SSD. This MP6,1 has all of our high-speed RAID-0, RAID-5 and backup units which are all connected to this MP6,1. We call it our File Server although strictly speaking it’s not in the technical sense. The cable distance between any Client Mac and the central inner office Mac is no more than 9 feet with a small amount of slack. In some cases the cable distance is 6 feet with some slack. Our Client Macs outside of the office pull/read from the inner office MP6,1 File Server their need project data, crunch on this data and eventually return results to the File Server Mac6,1. Each Client does this and grab their piece of the Project work and return their specific results. The File Server MP6,1 can then when all Clients have completed their workload and delivered results pull all the results together and produce final results.
7) The Clients use a Thunderbolt Bridge to the central MP6,1 File Server. Total cost was no more than $130.
8) The ‘Bridge’ works for us as the Clients ONLY read data from the File Server in the inner office for the most part to get ONLY their share of the Project work and deliver ONLY their result files. Because of this there’s no collision of data being written back to the File Server in the inner office.
9) Our solution is limited as there are only so many Thunderbolt ports on the MP6,1. But so far we can deal with having four Clients connected via Thunderbolt Bridge to the inner office File Server MP6,1. To go beyond we would need to obtain a ‘real’ file server such as SmallTree offers and use 10G cables, 10G switch and 10G adapters such as the SANLink2.
10) It should be noted that the 10G adapter all start off having to connect to the Mac via a Thunderbolt cable. For this reason, I expect the i/o patterns, dat flow, choppiness and stalling I see using the Thunderbolt Bridge to still be apparent.
11) Yesterday I ran my first set of tests using my home test rig of two Macs. I employed AJA for the testing as well as a few ‘dd’ execution for good measure. I used the old AJA version rather than the newest AJA version, but will also use the newer AJA over next few days to ensure the results are consistent (or not).
12) So here’s my test report
Summary
1) The Thunderbolt Bridge was far superior in both performance and cost vs. using the SANLink2 Adapters.
2) Let me say up front that using two SANLink2 Adapters for moving data between two Macs is likely an anathema to some people.
3) The SANLink2 Adapter in all fairness is a device for allowing a Mac to access a storage device (local or a NAS through a switch) that has 10GbE port(s).
4) My purpose for wanting to perform this evaluation was to see if the data transfers would be ‘smoother’ using the SANLink2 Adapters vs. the Thunderbolt Bridge which are seen to be ‘choppy’ at times. I really don’t expect the i/o rates to be better, but hopefully on a par with the Thunderbolt Bridge.
5) The storage device used was a Promise Pegasus2 R6 (RAID-0 across six 7200rpm 2TB Toshiba disks) connected directly via Thunderbolt-2.In order to establish a baseline for the best performance and to prepare for reviewing how well a pair of PROMISE SANLink2 Thunderbolt 2 to 10 Gb/s Ethernet Adapters will compare to using a simple Thunderbolt Bridge between two Macs; a Mid 2012 13-inch MBP9,2 with Thunderbolt-1 and a Late 2015 27-inch iMac 5K Retina with Thunderbolt-2, I first ran AJA on the iMac to sweep File Sizes from 128MB to 16GB to the iMac’s connected Promise Pegasus2 R6. The R6 unit was 25% full.
Both Macs were running the latest macOS Sierra 10.12.1 (Build 16B2553a). The File Sharing was set for Apple’s AFP.
Topology: iMac <–> Thunderbolt Cable <–> Promise Pegasus2 R6
As a further sanity check I decided to quickly run the newest AJA….
Woweeee, this new AJA is very colorful and is showing much higher write/reads for my iMac using the Pegasus2 R6 directly.
Just tested writing and reading a 64GB file….. (not sure what all the settings are at this time)
At first blush using the new AJA System Test the results for Case 2) and 3) (Bridge vs. SANLink2) are close for writes and reads.
I’ll need more testing using the new AJA to ensure of my findings are correct. However, at this time the TB Bridge still is a better choice when you compare the Bridge cost of a mere $60 for a TB cable vs. a pair of SANLink2 Adapters costing $1,070 there really is no issue in deciding which to use.
Here’s Case 2) using TB Bridge
Here’s Case 3) using SANLink2
-
Hi Bob:
Thanks for you reply and advice.
Question: Should the /etc/nsmb.conf have any contents or is it sufficient to just define/create it as a zero length file ?
1) Non optical TB cables can be had for $39 for a 6.6-foot cable, $48 for a 9.8-foot cable. Beyond that one has to go for the Corning optical cables that start at around $245 for a 33-foot cable.
2) If Thunderbolt 3 improves this for the ‘Bridge’ solution that will be very welcomed.
3) Yes, I have the SANLink2 Adapters now and started my testing yesterday using them. I did preload the Promise driver software which was done without issues.
4) Yes, as soon as connecting the SANLink2 adapters I was able to set MTU=9000 on my MBP. when configuring “Manual”.
5) I did discuss File Servers and 10G connectivity with Chris Duffy several years ago but at that time the cost of the minimum Small Tree file server would not fit our office budget at the time. The cost then was close to $10,000 including file server, switch, 10G adapters for the Macs (pre MP6,1 model) and cables. We limp along using 1GbE bonded pairs for our Mac for a while and then switch them all out for the Late 2013 MP6,1s.
6) All of the office MP6,1 and iMacs are situated at workstations in a circle around a enclosed central office. In the central office we have a maxed out MP6,1 12-core with as much RAM as we could ram into it along with a the largest internal SSD. This MP6,1 has all of our high-speed RAID-0, RAID-5 and backup units which are all connected to this MP6,1. We call it our File Server although strictly speaking it’s not in the technical sense. The cable distance between any Client Mac and the central inner office Mac is no more than 9 feet with a small amount of slack. In some cases the cable distance is 6 feet with some slack. Our Client Macs outside of the office pull/read from the inner office MP6,1 File Server their need project data, crunch on this data and eventually return results to the File Server Mac6,1. Each Client does this and grab their piece of the Project work and return their specific results. The File Server MP6,1 can then when all Clients have completed their workload and delivered results pull all the results together and produce final results.
7) The Clients use a Thunderbolt Bridge to the central MP6,1 File Server. Total cost was no more than $130.
8) The ‘Bridge’ works for us as the Clients ONLY read data from the File Server in the inner office for the most part to get ONLY their share of the Project work and deliver ONLY their result files. Because of this there’s no collision of data being written back to the File Server in the inner office.
9) Our solution is limited as there are only so many Thunderbolt ports on the MP6,1. But so far we can deal with having four Clients connected via Thunderbolt Bridge to the inner office File Server MP6,1. To go beyond we would need to obtain a ‘real’ file server such as SmallTree offers and use 10G cables, 10G switch and 10G adapters such as the SANLink2.
10) It should be noted that the 10G adapter all start off having to connect to the Mac via a Thunderbolt cable. For this reason, I expect the i/o patterns, dat flow, choppiness and stalling I see using the Thunderbolt Bridge to still be apparent.
11) Yesterday I ran my first set of tests using my home test rig of two Macs. I employed AJA for the testing as well as a few ‘dd’ execution for good measure. I used the old AJA version rather than the newest AJA version, but will also use the newer AJA over next few days to ensure the results are consistent (or not).
12) So here’s my test report
Summary
1) The Thunderbolt Bridge was far superior in both performance and cost vs. using the SANLink2 Adapters.
2) Let me say up front that using two SANLink2 Adapters for moving data between two Macs is likely an anathema to some people.
3) The SANLink2 Adapter in all fairness is a device for allowing a Mac to access a storage device (local or a NAS through a switch) that has 10GbE port(s).
4) My purpose for wanting to perform this evaluation was to see if the data transfers would be ‘smoother’ using the SANLink2 Adapters vs. the Thunderbolt Bridge which are seen to be ‘choppy’ at times. I really don’t expect the i/o rates to be better, but hopefully on a par with the Thunderbolt Bridge.
5) The storage device used was a Promise Pegasus2 R6 (RAID-0 across six 7200rpm 2TB Toshiba disks) connected directly via Thunderbolt-2.In order to establish a baseline for the best performance and to prepare for reviewing how well a pair of PROMISE SANLink2 Thunderbolt 2 to 10 Gb/s Ethernet Adapters will compare to using a simple Thunderbolt Bridge between two Macs; a Mid 2012 13-inch MBP9,2 with Thunderbolt-1 and a Late 2015 27-inch iMac 5K Retina with Thunderbolt-2, I first ran AJA on the iMac to sweep File Sizes from 128MB to 16GB to the iMac’s connected Promise Pegasus2 R6. The R6 unit was 25% full.
Both Macs were running the latest macOS Sierra 10.12.1 (Build 16B2553a). The File Sharing was set for Apple’s AFP.
Topology: iMac <–> Thunderbolt Cable <–> Promise Pegasus2 R6
As a further sanity check I decided to quickly run the newest AJA….
Woweeee, this new AJA is very colorful and is showing much higher write/reads for my iMac using the Pegasus2 R6 directly.
Just tested writing and reading a 64GB file….. (not sure what all the settings are at this time)
At first blush using the new AJA System Test the results for Case 2) and 3) (Bridge vs. SANLink2) are close for writes and reads.
I’ll need more testing using the new AJA to ensure of my findings are correct. However, at this time the TB Bridge still is a better choice when you compare the Bridge cost of a mere $60 for a TB cable vs. a pair of SANLink2 Adapters costing $1,070 there really is no issue in deciding which to use.
Here’s Case 2) using TB Bridge
Here’s Case 3) using SANLink2
-
Barry, Barry, Barry –
I see that you have joined Creative Cow just three days ago. I see that you have retired from Boeing recently. I am sure that you are a very smart guy. If you have searched the internet about me, you have probably seen that I am quite a jerk. This is a long thread, and I am going to reply with a non technical answer that is not becoming of a professional forum like Creative Cow. I am an “old guy” – almost 61. And Chris Duffy ain’t no youngster. There are a lot of great people on these forums.I am sorry to say that I am not going to help you for free, so that your son’s company can have the luxury of wonderful network performance, so that he can buy his family lots of new wonderful things, so that the rest of us can suffer. It is my advice to you that you say to your son “son, we should hire one of these companies – like Small Tree – or anyone else that you see here advertise on Creative Cow – so that we can get our stuff working properly”. We do this for our living,
and not that we are smarter than you – you seem to be a very very bright guy, but you seem like you need some help with this. Just like you might need some help getting your transmission fixed, or your leaky toilet fixed, or your leaking roof fixed. And someone – perhaps your son – needs to pay SOMEONE to assist you to get the rest of this working.
$10,000 for a complete shared storage system from Small Tree (or other manufacturers) is NOT a lot of money. You just spent over $1000 for the two Promise SanLink2’s and you got some free advice. If someone chimes in here, and says “look – I know how to get thunderbolt 2 bridge networking working” – are you going to return those SanLink2 boxes, and screw the dealer and Promise, so that your son can have more profit for his company, and buy his family more stuff ? WE WANT THE MONEY TOO ! We have families, we have medical insurance. And we don’t have a pension from Boeing.You have purchased wonderful SanLink2 boxes. As I mentioned before, there are other products on the market, from Small Tree, Sonnet, ATTO, and Akitio in addition to this, that would have worked as well. Each have drivers. Each have tuning files. Each have a unique way of setting them up. You and your sons company needs to HIRE SOMEONE to proceed with this process at this time.
Thunderbolt bridge is a terrible choice, compared to ANY of the hardware solutions on the market. I don’t know exactly what your son’s company does, but if it involves video editing, those “kids” doing the editing will HATE what you have setup, if you continue with Thunderbolt bridge networking. You are not saving your son any money. The #1 expense of any business is LABOR, and if they can’t work efficiently, then YOU are costing them money, just because you thought you found a solution for 60 bucks, which all of us can tell you, will not work in real life applications, when you use software editing or graphics products from Davinci, Adobe, Apple or AVID.
You know what to do next.
Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com -
Hey… No need to apologize for your age of for Chris Duffy. ? I’m several years ahead of you both. I’ve known Chris for quite awhile related to HPC. I’m pretty sure Chris knows my background.
I was unaware that Creative COW was not a forum for sharing information freely as is done on many other sites. My bad for not realizing this. I’ve been using/reading Creative COW for quite some time and yes, my account probably expired at some time and had to re-sign in a few days ago.
From my SANLink2 testing I do find it’s a waste of money for my son’s office environment. The Thunderbolt Bridge performs better and quite honestly the use of SANLink2 for joining two MP6,1s together is completely unnecessary. Thunderbolt 2 provides 20 Gbps and the 10G provides 1/2 of this. So really it’s no surprise to me that the use of SANLink2 cannot improve data transfer rates. My whole point was to see if the i/o transfers would be smoother over the hybrid TB/10G connection vs. the pure TB connection. My testing shows no change in this respect when using the TB/10G connection and of course the transfer rates are less than the pure TB connection.
As in my previous posting I did say the use of SANLink2 for connecting two MP6,1s was an anathema because the real purpose of the SANLink2 is for connecting to a 10G File Server storage system. If ever we were at a point at the office where we would need this type of File Server then the SANLink2 would be a good/proper choice and I would be in touch with Chris.
The one disadvantage we have at the office today for not employing a proper File Server such as SmallTree has, is that if our pseudo File Server crashes it affects all the Client Macs. Having a proper File Server would not cause this, unless of course the File Server crashes. ?
I will make a final test run using the SANLink2 adapter at the office over the weekend to satisfy myself with what I’ve learned so far on my home testing rig. At the office it will be that Server and Client system both employ Thunderbolt 2 whereas my home rig uses TB-1 on the Server and TB-2 on the Client. I do not expect the office testing to show a different set of results to what my home rig as shown so far.
I’m sure that at some time in the future I will call Chris again for his help/advice when we require a bone fide File Server storage system.
My SANLink2 adapters can be returned within 14 days for full refund, so their cost is not a factor for me.
Finally, I apologize if I’ve abused this forum thinking I can obtain free knowledge/help/advice. Personally, I like to share my knowledge, just as I’ve done in this thread. Just keep in mind that the older you get the more you find you don’t know, but do realize how much you do know. The trick/hope while aging is to not forget what you do know.
Take care and thanks for the discussion. ?
-
Hi Barry –
no offense – and please remember – it’s ME that is the jerk ! Creative Cow is an open forum for exchange of information, but for me (personally) there is a limit of how much I will give for free.If we want to be offended by anyone, it should be Apple, that has promoted Thunderbolt networking, which in real life, doesn’t work very well. The smaller companies (don’t you like to support small companies ?) make “third party products” (like Promise and Small Tree) that actually DO WORK !.
you state – –
“The trick/hope while aging is to not forget what you do know.”My problem is that most of the information that I know, from being a video engineer since age 20 is completely worthless today. No one told me 30 years ago that Linux would be “all the rage” today !
Bob Zelin
Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com -
Bob: I was not offended in any way. I understand the need for people like yourself needing the income or revenue stream as some call it.
My goal in all of this issue is to provide my son’s office with a cost effective solution for his office Mac workflow environment. He and his hired freelancers are indeed video professional people that use the Adobe and Apple products on Macs exclusively. For the past 3 years using Apple’s Thunderbolt Bridge for File Sharing has proven to be VERY beneficial and obviously low cost. I compare this to a similar but larger business partner (many more video editing workstations than we had) who decided to go to employing 10G throughout their office. They ended up paying around $100K when all said and done. The same freelancers they used were same as we used in our office and they were astounded by how well our small shop was able to get the same kind of work done using the Thunderbolt Bridge networking for file sharing.
Personally, I have no real issues with Apple’s implementation of Thunderbolt for simple networking and/or file sharing between Macs. It’s a very easy and understandable and low cost. I’m also aware that Apple’s networking software isn’t on a par with the likes of Linux and other proprietary UNIX systems. They do make changes that negatively affect networking as I use it at home and in the office. As mentioned earlier, one has to pay attention to the use of SMP and AFP in the macOS File Sharing configuration panel. Sometimes SMB is best and other times AFP is best. At this time with El Capitan and Sierra I find AFP is the best choice by a long shot.
Back in 1998 I was on a panel with several other HPC-centric people at Manchester University in UK and was asked if Linux was suitable/reliable/robust-enough etc for aircraft design engineering problems. This was being asked by some of the USA Govt. folks in the audience who presumably were at the time using costly proprietary systems from SGI, Cray, IBM, HP, Govt. home-grown systems, and so on. Around that time Linux was a low-cost ‘child’ OS and viewed with suspicion by a large community of IT folk. My response was that as Linux matured it would be the right choice because of its low cost and eventual wide-spread use. Some of the audience was astounded with my response as they knew Boeing was using Cray and SGI systems for the aircraft design engineering problems. It’s a pity you were not in that audience some 20 years ago. ?
Yes, fore sure, ones technical baggage can be harmful (or gets in the way) when using it for todays changing technologies. The need for keeping an open mind is essential and there should be no hesitation in trying to learn the ‘new stuff’.
Ethernet technology has been around for a long time and IMO will last for quite some time as its cheap, mature, easily deployed when comparing it to other more exotic and vastly more expensive networks such as Infiniband.
When I left Boeing I left behind an enormous shared file storage system I helped to design and implement. Access to it was from thousands of nodes in several large compute Clusters. It was a Panasas system and is still being employed and expanded today with much success. It was based on ethernet as well. You can imagine the number of large switches used to bring all that together. It was a challenge but fun to work on. The IT Networking Dept. were absolutely astounded at the amount of data we were pushing around the network attached/servicing the Panasas storage system. It overloaded their accounting data fields because the numbers were so large. ?
-
I forgot to mention that on the Client side when using the SANLink2 the Client’s kernel cpu use was 20% higher than when using the Thunderbolt Bridge.
Using the Thunderbolt Bridge the kernel_task cpu hovered around 98% while using the SANLink2 it hovered around 120%.
To ensure I was not seeing things I had my Mac Client system request a 30 GB file (that took some 120 secs) from the Mac Server and monitored the cpu use for using the SANLink2 and then for using the Thunderbolt Bridge.
20% is significant IMO.
Reply to this Discussion! Login or Sign Up