Running v9.6.5
A network folder contain multiple files, with .xml and .jpg extensions.

A VC job contains multiple tasks. 1 of the task contains a FTP download command, and 2 FTP upload commands (1 for *.xml and 1 for *.jpg), to upload files to different folders on a FTPs server.

The task would fail randomly on a *.jpg file - fail to delete source file after uploaded to FTPs server. It would continue to fail on said file in subsequent attempts with the same errors:

Exception in Task: System.Exception: Error uploading file(s): *****.jpg, err: System.IO.IOException: The process cannot access the file '\\***\***.jpg' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.File.InternalDelete(String path, Boolean checkHost)
at VisualCronService.ProcessTaskFTPClass.UploadFTPFilesMulti(ConnectionClientClass cc, FTPCommandClass fc, List`1 files, String destinationpath, Boolean bolDebugging, String& strError) in C:\git\code\VisualCronService\Jobs\TaskProcesses\Net\FTP\clsProcessTaskFTP.vb:line 1831

If uncheck "delete source file after copy" the task runs without errors.

If make a copy of the file, the copy can be uploaded and deleted without issues, but VS reports a null reference error when trying to upload the original file.

Exception in Task: System.Exception: Error preparing upload of FTP folder (2), targetpath: /***/Photos, CurrentPath: , log: 1, error: System.NullReferenceException: Object reference not set to an instance of an object.
at VisualCronService.ProcessTaskFTPClass.PrepareFTPUploadFolder(ConnectionClientClass cc, TElSimpleFTPSClient client, FTPCommandClass fc, String targetpath, Boolean bolDebugging) in C:\git\code\VisualCronService\Jobs\TaskProcesses\Net\FTP\clsProcessTaskFTP.vb:line 4116

On subsequent runs, VC (only dealing with 1 photo to upload) goes back to the IO exception posted at the top.

The file could be deleted manually. I couldn't figure out why the delete would fail. Any thoughts? Thanks.

Eventually the file cannot be manually deleted because "because the file is open in System". VisualCron CPU usage spikes to nearly 100% even when the jobs are all in "waiting" status.

The file may be in use by another user or software or system which you cannot control. If this is the case then you may have to limit access or create a "Cleanup" job as a separate event.
or
The file is still being held onto by VC. (this is very likely and has happened to me many times)

One way to test this would be to clone your job, change your file copy job to NOT delete after copying and create a few more tasks.
New Task #1 - Interactivity - Wait 90 Seconds (or maybe less)
New Task #2 - Delete the files you would have copied in an earlier task. You can use the same filter as the copy job or take the output from the copy job to determine which files to delete

Run your new cloned job to see if that solves your issue. If it does, that means that VC is holding onto your files too long.

You may also be able to tweak the settings on the ftp. My guess is that VC is not getting a reply back from the ftp server and that is what it is waiting for.

Thanks Joey

I've tried copying files from network directory to local directory first, moving file delete to a separate task, adding a wait task before deleting, etc. Checked and tried FTP connection/encryption settings like data/command channel encryption, enable sync data channel, (server requires Explicit, TLS 1.2). Couldn't eliminate the issue. When trying to delete the files that were stuck, I got "the file cannot be deleted because it's opened by VisualCron" even though the jobs are not running.

I ended up writing a console application with the FluentFTP package to replace the VisualCron FTP upload/file delete tasks. All it requires is to ensure the filestream object is disposed before deleting the file.

Thanks Joey

I've tried copying files from network directory to local directory first, moving file delete to a separate task, adding a wait task before deleting, etc. Checked and tried FTP connection/encryption settings like data/command channel encryption, enable sync data channel, (server requires Explicit, TLS 1.2). Couldn't eliminate the issue. When trying to delete the files that were stuck, I got "the file cannot be deleted because it's opened by VisualCron" even though the jobs are not running.

I ended up writing a console application with the FluentFTP package to replace the VisualCron FTP upload/file delete tasks. All it requires is to ensure the filestream object is disposed before deleting the file.

Sounds a bit strange, if we could get a test connection/account to the sftp to see if we can reproduce this exact issue we could troubleshoot it easier

---