I am working on a fairly simple job, but am having trouble with a few steps:

Basically what i am trying to do is wait for a trigger file to show up (trigger.txt) once I see the trigger file I will go to the folder, zip up the csv file that is in the folder, rename the csv to its current name plus the current date and time, move the new csv to an archive folder, delete the old file, and then upload the zip file to an ftp site.

I will always only have a single csv file that needs to be zipped, moved and uploaded, but the name of the csv file will always be different, and the zip file must always have the name of the csv file.

The trouble I am having is the the Archive Path. Unless I am mistaken you need to put a file name here or it will not work. Is that accurate? How would I be able to set the archive path, the same location as the csv file with the same name as the csv file only it is a zip? The other problem is uploading the file to an FTP site... it seems to work for me only if I put a specific file name to be uploaded.

Do I need to use variables? I Tried to create a currentfilename variable using the cron variables and choosing the output from a List Files Task. The List files task output has the full path and name of the CSV file, but the Archive Path in the compress task does not seem to like it. I could be adding variables all wrong though.

Thanks

mike

Can you get me an example file name for the CSV file? Also, is the Trigger file in the same folder as the CSV file? And, what version of VisualCron are you using? I have installations of 6.1.4 and 6.2.1 and have found 'some' variable stuff to be sketchy in loop situations on the 6.1.4 installation.

Brian

Also, just working out the details of your steps...


1. Trigger when \\path\folder\trigger.txt shows up
2. Create a zip file containing the CSV file in that folder.
3. Rename the csv to its current name plus the current date and time
4. Move the new csv to an archive folder (are you talking about the newly renamed, still only one file with new name, file ?)
5. Delete the old file (what file? - see above question. there would still only be one CSV file... with a new name according to your steps)
6. Upload the zip file to an ftp site.


So basically you want to zip the csv file and move the CSV file (you can rename during a move) into an archive folder. Then upload the zip (likely deleting after successful upload to avoid it sticking around) via ftp

Does the above sound accurate? Also, is there only one actual location you're looking for this Trigger file? or are there multiple subfolders that you're triggering off of and any of them subfolders could have a 'trigger.txt' dropped into it?

Brian

CSV is called BROWZxxxxxxx.csv (the x's are going to be a job number that could be up to 7 character long)

The trigger file is in the same folder as the csv file

1. Trigger when \\path\folder\trigger.txt shows up - Correct; Is working

2. Create a zip file containing the CSV file in that folder - Correct;This works if I hard code a name in the archive path \\servername\files\browz1234567.zip

3. Rename the csv to its current name plus the current date and time - Correct; Is working

4. Move the new csv to an archive folder (are you talking about the newly renamed, still only one file with new name, file ?) Move the renamed CSV file, and it is still only one file... this is working as part of the move task... moves file, renames file, deletes old file.

5. Delete the old file (what file? - see above question. there would still only be one CSV file... with a new name according to your steps) The original csv file is deleted from the working folder after it is moved and renamed

6. Upload the zip file to an ftp site - Correct; this is also only working if I hardcode a name browz1234567.zip


So basically you want to zip the csv file and move the CSV file (you can rename during a move) into an archive folder. Then upload the zip (likely deleting after successful upload to avoid it sticking around) via ftp Correct

Does the above sound accurate? Also, is there only one actual location you're looking for this Trigger file? or are there multiple subfolders that you're triggering off of and any of them subfolders could have a 'trigger.txt' dropped into it?

Trigger file is in one location, the working folder where the original csv file gets dropped.

I am using the latest version of Visual Cron 6.2.2

Thanks

Ok. I got it. :)


You have your file trigger. Great. Order will matter here... so do a List Files as you had mentioned.

For the FOLDER that the list-files is looking in, use the following variable, as its the folder where the TRIGGER.TXT was found:
{TRIGGER(Active|LastTrigger|File.Folder)}

Include file Mask: *.csv

========================
Then, Create an archive task that runs directly after list files and set the 'Archive Path' location to this:
{STRING(Replace,{TASK(PrevTask,StdOut)},csv,zip)}

What that does is take the output from your 'List Files' task (full path to the file including the csv extension) and replaces 'csv' with 'zip'.

On the CONTENT tab of the archive task, Add an item and use the same value as the list files path:
{TRIGGER(Active|LastTrigger|File.Folder)}

Include File mask: *.csv


For the FTP/SFTP upload, you'd also use this value for the folder and *.zip for the file mask:
{TRIGGER(Active|LastTrigger|File.Folder)}

I'm also assuming that you're deleting the zip file from that folder after it sends. Thats why I used the *.zip for the file mask to upload. If you need the specific file name, i can make a variable for that too.



That make sense?

Brian

I just changed my mind... and edited my previous post. Originally I said to use a specific task ID in one of the variables, but you don't have to so I changed it to

{STRING(Replace,{TASK(PrevTask,StdOut)},csv,zip)}

Brian

Awesome.

Everything up until the FTP upload is working. This was also one of my problems that could be resolved if I hard code the file and path name.

If I click the Test button on the FTP Upload job, it does see the file, but if I try to upload it, I get an impersonation error.

Any ideas on that one?

Thanks for the Help thus far.

mc

Can you send screenshots of the different tabs inside of your FTP task?

I've tested this on my system and it works great.


Brian

Sure

Sure

So you're not even using the variables yet. You have the UNC path hard coded and its still giving you that Impersonation error? It's gotta likely be related to your credential you're using for the task. I am not using a credential as my test files are all local paths (c:\temp\etc....) and the default ID of my service is 'SYSTEM' that has access to it already.

Is that the case for your system? Service running as SYSTEM and you're using the 'Credential' to actually allow you to get to the UNC path of your files?

Brian

You may want to turn on 'Extended debug logging' in the Server Settings.

Screenshot shows the option. On the 'Server' tab, click Settings. Then go to the 'Log Settings' tab and check the box for Extended Deug Logging. Apply settings and go run your job again. Then go back and immediately turn off the Extended debugging.


Then, go look at the bottom of this log file:
C:\Program Files (x86)\VisualCron\log\log_server20130411.txt

See if there is any usefull information surrounding the start and finish of that job.


Brian

I used the variables in the other tasks... on the upload I just want to take *.zip and upload.

The service on this server is running as the same user who owns the shared folder, and is an admin to this box. I will turn on the logging.

I used the variables in the other tasks... on the upload I just want to take *.zip and upload.

The service on this server is running as the same user who owns the shared folder, and is an admin to this box. I will turn on the logging.

In that case, why have the UPLOAD task using a credential from the dropdown list? Try it without it.

Heading out for the night. I'll check back later tonight or tomorrow when i get in.

Brian

Does not seem very helpful to me:

4/11/2013 2:35:55 PM Debug Calling GetNextTaskProcess()
4/11/2013 2:35:55 PM Debug Next Task is: Upload to FTP
4/11/2013 2:35:55 PM Debug Next action: ActionContinue
4/11/2013 2:35:55 PM Debug Calling StartTaskProcess() with Task: Upload to FTP (0)
4/11/2013 2:35:55 PM Debug Task (154) was added to processlist: Upload to FTP
4/11/2013 2:35:55 PM Info Task started: Upload to FTP (154)
4/11/2013 2:35:55 PM Debug Entering wait loop. Upload to FTP (154)
4/11/2013 2:35:55 PM Debug FTP Command Count: 1
4/11/2013 2:35:55 PM Debug Starting FTP command: 1 of 1
4/11/2013 2:35:55 PM Debug Connect->Start (0)
4/11/2013 2:35:55 PM Err Impersonation failed in FTP task.
4/11/2013 2:35:55 PM Debug Completing FTP command: 1 of 1
4/11/2013 2:35:55 PM Debug Commands done
4/11/2013 2:35:55 PM Debug Before dispose
4/11/2013 2:35:55 PM Debug Disposing timer
4/11/2013 2:35:55 PM Debug Trying to disconnect
4/11/2013 2:35:55 PM Debug Stopping watch
4/11/2013 2:35:55 PM Debug Before GetOutput
4/11/2013 2:35:55 PM Debug Before RaiseTaskCompleted
4/11/2013 2:35:55 PM Info Task completed: Upload to FTP (154)
4/11/2013 2:35:55 PM Debug Process status - About to RemoveTaskProcess (154)
4/11/2013 2:35:55 PM Debug Setting previous task in TaskProcessCompleted: Upload to FTP (304c0f7b-4b1c-4fd4-8607-f3478ba5dfac) in job: ZIP-Archive_FTPUpload
4/11/2013 2:35:55 PM Debug Process status - About to SendTaskProcess (154)
4/11/2013 2:35:55 PM Debug Process status - About to RemoveTaskProcess (154)
4/11/2013 2:35:55 PM Debug Task (154) was removed from processlist: Upload to FTP
4/11/2013 2:35:55 PM Debug Sleep ended because Task ended.Upload to FTP (154)
4/11/2013 2:35:55 PM Debug TaskWaitQueue.Dequeued.Upload to FTP (154)
4/11/2013 2:35:55 PM Debug Job (46) was removed from processlist: ZIP-Archive_FTPUpload
4/11/2013 2:35:55 PM Debug Next execution (2) for job 'ZIP-Archive_FTPUpload' is: 12:00:00 AM
4/11/2013 2:35:55 PM Info Job completed: ZIP-Archive_FTPUpload
4/11/2013 2:35:56 PM Debug Memory cleanup
4/11/2013 2:35:56 PM Debug Saving output files and clearing output memory.
4/11/2013 2:35:56 PM Debug Output stored in memory: 20
4/11/2013 2:35:56 PM Debug Calling CleanUp.
4/11/2013 2:35:57 PM Debug Saving Jobs

was the credential still in use during this? Or had you already removed it.

Sent from my phone
Brian

If upload is based on a File trigger then, to eliminate all problems with manual editing please use the Variables for both foldername and filename.

If there was a Credential access problem we would probably see an error in the log.

Thanks Brian for your answers!

---

EDITED: Phone screwed up the wording of my post. haha

Henrik,

What would cause the Impersonation error that MC is running to? Is that likely part of the FTP/SFTP Upload task or is it more likely related to the 'Connection' that the Upload task is configured to use?

I've not ran into impersonation errors before so i'm unsure at which point that would likely happen.


Brian

Sorry, I missed that part. The reason why it works in the Client is because you are running in the exact same environment as VisualCron.

The impersonation error could be a number of things;

- username, password or domain is wrong
- maybe you have checked "local login" - it should normally be unchecked when accessing network shares

---

- maybe you have checked "local login" - it should normally be unchecked when accessing network shares

Ok, This definately brings me back to the fact that a credential is being used at all. Since the service is running as an ID that has access to the location already, setting it to run the Upload task as a credential (that maybe hase this 'local login' checked) could very well be causing it.

MC, definately check on how that credential is configured. Ultimately though, i'd change it to NOT use a credential.

Brian

I was just about to write and let you know that was it. I switched the FTP task to use "No Credentials" and the FTP upload worked.

Woohoo. Thanks for the help.

mc

Awesome. Glad to help, MC. Anytime.

Brian