Creative Communities of the World Forums

The peer to peer support community for media production professionals.

Activity Forums Square Box CatDV Worker node ftp automation

  • Worker node ftp automation

    Posted by Pål henrik Wagner on August 17, 2010 at 12:07 pm

    I have tried to follow the guidelines in the CatDV Worker read me pdf to no avail.

    This is what we want to do:

    Watch a folder for video files.
    When a file has been dropped in the folder;
    1 compress this file to a low rez format
    2 place the processed file in a local folder
    3 move the original file to another local folder
    4 upload the processed file to an ftp server
    5 send an e-mail telling us about the success of the process.

    Everything except for the ftp-ing works.

    By taking the arguments listed in the log file of Worker Node, and pasting them in Terminal, I get these results:

    This is what it looks like in the log:

    Conversion finished after 95.749s
    [12:04:47.238]
    root=/Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_UPLOAD origPath=/TEST11.mov mode=0 newRoot=/Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_original newPath=/TEST11.mov
    Renaming /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_UPLOAD/TEST11.mov to /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_original/TEST11.mov
    setMovieFile /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_original/TEST11.mov on SourceMedia[‘/Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_UPLOAD/TEST11.mov’,start=0:17:26:13,end=0:17:48:11,video=XDCAM HD422 1080i50 (50 Mb/s) (1920×1080 25.0fps),audio=24-bit Integer (48.0 kHz, stereo, 16 bit),still=false,numFrames=548,importer=QuickTime (Movie),startFrame=0,movieStart=0,movieFrames=548,movieDuration=548,qttc=0:17:26:13]@f80bf7
    Import as Movie, resulting in 1 clip(s) duration 0:00:21:23
    Exported to /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_output/TEST11.mov, took 95.749s
    Moved file to /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_original/TEST11.mov
    [12:04:47.334]
    All change
    Completed Task 8 ‘/TEST11.mov’
    arg[0]: /usr/bin/ftp
    arg[1]: -u
    arg[2]: xx**@**********bc.no/4_stj_REISE/til_LYDETTERARBEID/'>ftp://nnnn:xx**@**********bc.no/4_stj_REISE/til_LYDETTERARBEID/
    arg[3]: /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_output/TEST11.mov
    [12:30:36.535]
    Date: Mon, 16 Aug 2010 12:30:36 +0200 (CEST)
    From: workernode_fish@xxx
    To: support@xxx
    Subject: test_ftp
    X-Mailer: CatDV Worker Node 3.1.0

    The task list details are as follows:

    But the server is not uploading anything to the ftp server.

    I’ve used this command as stated in the Worker Node notes:
    /usr/bin/ftp -u xx**@**********bc.no/4_stj_REISE/til_LYDETTERARBEID/'>ftp://nnnn:xx**@**********bc.no/4_stj_REISE/til_LYDETTERARBEID/ $l

    (This is what the manual suggests: /usr/bin/ftp –u ftp://user:password@host/public/upload/ $l)

    When I try the same arguments as stated above from the log into Terminal, where $l is substituted by the file path of the processed file, I get this result:

    fishrender01:TEST_output onlinereise$ /usr/bin/ftp -u xx**@**********bc.no/4_stj_REISE/til_LYDETTERARBEID/'>ftp://nnnn:xx**@**********bc.no/4_stj_REISE/til_LYDETTERARBEID/ /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_output/TEST11.mov
    Connected to nnnn.ftp.abc.no.
    220 Servers identifying string
    331 Password required for nnnn
    230-Last login was: 2010-08-16 12:47:14
    230 User nnnn logged in
    Remote system type is UNIX.
    Using binary mode to transfer files.
    200 Type set to I
    250 CWD command successful
    250 CWD command successful
    local: /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_output/TEST11.mov remote: /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_output/TEST11.mov
    229 Entering Extended Passive Mode (|||56181|)
    550 /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_output/TEST11.mov: No such file or directory

    When I cd to the correct local folder and only state the processed file, I have success.

    fishrender01:TEST_output onlinereise$ /usr/bin/ftp -u xx**@**********bc.no/4_stj_REISE/til_LYDETTERARBEID/'>ftp://nnnn:xx**@**********bc.no/4_stj_REISE/til_LYDETTERARBEID/ TEST11.mov
    Connected to nnnn.ftp.abc.no.
    220 Servers identifying string
    331 Password required for nnnnnnnn
    230-Last login was: 2010-08-16 12:48:34
    230 User nnnn logged in
    Remote system type is UNIX.
    Using binary mode to transfer files.
    200 Type set to I
    250 CWD command successful
    250 CWD command successful
    local: TEST11.mov remote: TEST11.mov
    229 Entering Extended Passive Mode (|||64405|)
    150 Opening BINARY mode data connection for TEST11.mov
    100% |*****************************************************************************************************************************************************************************| 3045 KB 929.94 KB/s 00:00 ETA
    226 Transfer complete
    3118882 bytes sent in 00:03 (887.92 KB/s)
    fishrender01:TEST_output onlinereise$

    This tells me that the $l command may be too informative, but I cannot find a different variable that I should use. I would appreciate any help that could enlighten me.

    Cheers,

    Pål Henrik

    Pål henrik Wagner replied 15 years, 8 months ago 5 Members · 8 Replies
  • 8 Replies
  • Bryson Jones

    August 17, 2010 at 3:27 pm

    Just a thought, but have you tried combining the variables to get the path and filename?

    I suspect that your shell connection may not be resolving the full path correctly. (meaning going back to root/Volumes may be throwing it off.)

    I know that it should be able to work from the full path, but since it’s not, can you see if perhaps you have a root directory set that it’s working from? It may also be that the ftp won’t create the “/Volumes/” part of the destination. I’m not sure, but I do have an idea to give you what you’re showing as your working path. See what you think of the following.

    /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_output/TEST11.mov is the full path, but I see that,

    4_stj_reise/Til_lyd/TEST_output/TEST11.mov seems to work. If I’m reading this correctly , this means that you are using the “grandparent” folder (third folder up) to correctly start your path. If so, then you are in luck. In the next release $R will give you the grandparent path, and so $R$f will give you the correct path for your ftp command.

    Would this help you?

    bryson

    bryson “at” hidefcowboy.com

    hidefcowboy.com

  • Pål henrik Wagner

    August 17, 2010 at 8:10 pm

    Hi,

    thanks for the reply.

    The thing is that I believe I have mixed up source and destination in this command.

    This command works:
    /usr/bin/ftp -u ftp://nnnn:xxxx@nnnn.ftp.abc.no/4_stj_REISE/til_LYDETTERARBEID/ TEST11.mov

    where TEST11.mov is the local file, and I already am in the correct local folder within Terminal.

    If I understand you correctly, I should try this:

    /usr/bin/ftp -u ftp://nnnn:xxxx@nnnn.ftp.abc.no/4_stj_REISE/til_LYDETTERARBEID/ 4_stj_reise/Til_lyd/TEST_output/TEST11.mov

    i get the same problem as earlier:

    Connected to xxx.ftp.xxx.no.
    220 Servers identifying string
    331 Password required for xxxx
    230-Last login was: 2010-08-17 14:22:26
    230 User xxxx logged in
    Remote system type is UNIX.
    Using binary mode to transfer files.
    200 Type set to I
    250 CWD command successful
    250 CWD command successful
    local: 4_stj_reise/Til_lyd/TEST_output/TEST11.mov remote: 4_stj_reise/Til_lyd/TEST_output/TEST11.mov
    ftp: local: 4_stj_reise/Til_lyd/TEST_output/TEST11.mov: No such file or directory

    From this, I think that I’ve been mixing up source and destination folders. My intention is to upload TEST11.mov to the ftp catalogue

    /4_stj_REISE/til_LYDETTERARBEID/

    The local path of TEST11.mov is
    /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_output/

    Not sure if I am able to explain very clearly, though…

    Pål Henrik

  • Marvin Holdman

    August 17, 2010 at 9:44 pm

    It appears from your error message that you are trying to log into a file.

    I assume that you wish to create the file “TEST11.MOV”. You won’t be able to upload a file there. You need to specify a folder to upload too.

    Use a FTP Client and see what your root file path is when you log in. This will be where you begin your upload folder description (folder hierarchy).

    Marvin Holdman
    Production Manager
    Tourist Network
    8317 Front Beach Rd, Suite 23
    Panama City Beach, Fl
    phone 850-234-2773 ext. 128
    cell 850-585-9667
    skype username – vidmarv

  • Pål henrik Wagner

    August 18, 2010 at 2:53 pm

    The funny thing now, is that I found a way that *should* work, but still doesn’t.

    If I put this command in the Worker Node

    cd /Volumes/Fish/Til/4_stj_reise/Til_lyd/TEST_output/ ; /usr/bin/ftp -u ftp://nnnn:xxxx@xxx.ftp.xxx.no/4_stj_REISE/til_LYDETTERARBEID/ $f

    I still can’t upload the processed file (the output file name is the same as the original).

    But if I out the same line (substituting $f with the file name TEST13.mov), I get a successful ftp upload.

    Is anybody else using the command ftp -u …… $l with Worker Node? Or something similiar for that sake.

    If someone has some examples of usage with automated ftp commands, I would love to see some.

    Cheers,

    Pål Henrik

  • Kevin Duggan

    August 18, 2010 at 8:21 pm

    Hi guys

    try using curl . Curl will get you any place you want to go . Either Google in or try man curl in the terminal then use this syntax..

    curl -T $b ftp://Username :Password@upload.Apple.com/SampleProject/$f

    So from the top … the Unix command curl
    -T upload the file specified
    $b this is the Catdv Worker variable.. Click on the HELP button to get a full
    list there are many preformed variables for you to utilise

    ftp:// USERNAME: Password@Apple .com
    example ftp://Kevin:letmein@CatDv.com/Upload Folder

    $f Catdv variable the name of the file plus extension ie MyMovie.mov

    hope this helps you all.

    Kevin

  • Kevin Duggan

    August 18, 2010 at 8:34 pm

    Hi guys

    try using curl . Curl will get you any place you want to go . Either Google in or try man curl in the terminal then use this syntax..

    curl -T $b ftp://Username :Password@upload.Apple.com/SampleProject/$f

    So from the top … the Unix command curl
    -T upload the file specified
    $b this is the Catdv Worker variable.. Click on the HELP button to get a full
    list there are many preformed variables for you to utilise

    ftp://kevin:catdv@Catdv.com/Upload_Folder/ $f

    $f the name of the file ie mypics.mov

    Hope this helps…

    Kevin Duggan
    Catdv

    apologies if this comes up twice it seemed to be truncated the first time I posted this

  • Rolf Howarth

    August 19, 2010 at 10:16 am

    Make sure you can do the operation from the command line first so you fully understand the command syntax, and only then work on doing it from the worker. Also, as Kevin suggested, you might find that curl is a more powerful alternative to the ftp command (it will also copy files by sftp, http, ssh, etc., not just ftp)

  • Pål henrik Wagner

    August 25, 2010 at 6:27 am

    Wow! Curl did it! Thank you guys!

    It was very fair and easy, when I just got the syntax sorted out.

    This is “Execute 3 Command” I use when the file is processed:

    curl -T $l ftp://xxxx.ftp.xxxx.no/4_stj_REISE/til_LYDETTERARBEID/ –user user:password

    where $l (that is the letter L) signifies the whole path of the processed file.

    This command will upload the Worker Node processed file to the ftp site and subfolder as stated above.

    Thank you again.

We use anonymous cookies to give you the best experience we can.
Our Privacy policy | GDPR Policy