/    Sign up×
Community /Pin to ProfileBookmark

FTP upload speed compared to other clients…

Copy linkTweet thisAlerts:
Dec 18.2012

Hi

I have a really simple script that uploads a file using ftp_put(). This script is about 10 times (!) slower than uploading the same file from the same server to the same server using Filezilla. The PHP script is as far as I can tell doing exactly the same as Filezilla, passive mode, utf-8 opt, etc. We have captured the traffic using Wireshart and the company responsible for the FTP server is currently investigating those, but I thought I’d just check in on this forum. Edit; the question is if anybody has seen this and has some advice ?

Thanks
Lubox

to post a comment
PHP

7 Comments(s) ↴

Copy linkTweet thisAlerts:
@eval_BadCode_Dec 18.2012 — You can use the "cp" command from ssh+[ba]sh. That would make the most sense and would be exceptionally faster than using FTP to copy files.
Copy linkTweet thisAlerts:
@LuboxauthorDec 20.2012 — Using Filezilla a FTP upload is as fast as the internet lines at the other end is (we have exceptional high bandwidth ourselves, so we're almost always maxing the other end). Of course, with PHP as FTP client we're not doing this since it is 10-20 times slower on all versions I've tried. If you have any examples of "cp" command from ssh+[ba]sh on Windows (!), I'll gladly check it out. I've tried the PHP copy(), but that is as slow as ftp_put() (probably using same methods).

Edit: Note that this only applies to uploads (ftp_put()). FTP downloads (ftp_get()) with PHP is as fast as Filezilla.

Thanks
Copy linkTweet thisAlerts:
@DerokorianDec 20.2012 — The reason its faster with filezilla is because it can create multiple streams. You may try forking the process, giving each fork a batch of files to process, this will allow the job to be running with multiple down streams and greatly decrease the total transfer time.
Copy linkTweet thisAlerts:
@eval_BadCode_Dec 20.2012 — Using Filezilla a FTP upload is as fast as the internet lines at the other end is (we have exceptional high bandwidth ourselves, so we're almost always maxing the other end). Of course, with PHP as FTP client we're not doing this since it is 10-20 times slower on all versions I've tried. If you have any examples of "cp" command from ssh+[ba]sh on Windows (!), I'll gladly check it out. I've tried the PHP copy(), but that is as slow as ftp_put() (probably using same methods).

Edit: Note that this only applies to uploads (ftp_put()). FTP downloads (ftp_get()) with PHP is as fast as Filezilla.

Thanks[/QUOTE]

wat...

ok you get putty, log in with your "SFTP" credential (SFTP meaning... ftp over ssh). So ssh will spawn a shell (probably bash or sh), and you can use "cp" the copy command,

cp <src> <dst>

it will be hundreds of times faster than trying to copy crap from the server to your computer and then back to the server again, lol.
Copy linkTweet thisAlerts:
@LuboxauthorDec 22.2012 — The reason its faster with filezilla is because it can create multiple streams. You may try forking the process, giving each fork a batch of files to process, this will allow the job to be running with multiple down streams and greatly decrease the total transfer time.[/QUOTE]

Multiple streams within the ftp-protocol? Is that even possible? Filezilla are able to transfer multiple files, but the protocol doesn't allow multiple threads for one and the same file (if I understood you correctly) as far as I know.

I'm already transferring many files at the same time in the system (40-100 simultaneous uploads depending on receiving server), so all in all the speed is not bad. But it could be much, much faster.

I've also tried to upload using cUrl, but that is equally slow. I will also try with socket operations and see if I manage uploads on that level.

Note that ftp downloads (ftp_get) is as fast as Filezilla, so there's a huge difference in ftp_get and ftp_put.

Could it still be the servers that are using som kind of limit if the connection is not from a known client or something? I'm trying to find servers that I do not see this on, but so far they've all been the same.

To "eval", any script examples would be nice. By the way, why do you think I copy crap from the server to my computer and back to the server?

Thanks

Lubox
Copy linkTweet thisAlerts:
@DerokorianDec 22.2012 — Multiple streams within the ftp-protocol? Is that even possible? Filezilla are able to transfer multiple files, but the protocol doesn't allow multiple threads for one and the same file (if I understood you correctly) as far as I know.

I'm already transferring many files at the same time in the system (40-100 simultaneous uploads depending on receiving server), so all in all the speed is not bad. But it could be much, much faster.
[/QUOTE]
Ah my misunderstanding. I thought you were uploading a file and moving on to the next when it was complete. So my suggestion was based on that assumption. If you are already doing multiple files at a time then I'm not sure I can give you any pointers to increase the speed. I just know my site back up started finishing much quicker when I spawned multiple processes to handle sets of files (aka pushing multiple files at once) and I thought maybe you were running into the same problem.

Sorry I'm less than useful sometimes.
Copy linkTweet thisAlerts:
@LuboxauthorDec 22.2012 — Ah my misunderstanding. I thought you were uploading a file and moving on to the next when it was complete. So my suggestion was based on that assumption. If you are already doing multiple files at a time then I'm not sure I can give you any pointers to increase the speed. I just know my site back up started finishing much quicker when I spawned multiple processes to handle sets of files (aka pushing multiple files at once) and I thought maybe you were running into the same problem.

Sorry I'm less than useful sometimes.[/QUOTE]

Hehe, no worries, I know I didn't make this clear in the first post, I just thought I'd stick to the real issue, the single transfer. So, I've tried ftp_put(), I've tried copy(), and I've tried the cUrl upload. All 10-20 times slower than the same upload from Filezilla. And it's not like there's a limit reached within PHP. I've tried it on fast servers and it may upload at 2500kB/s, I've tried it on slow servers and it may upload at 500kB/s. And Filezilla uploads 10 times faster on both servers. It just seems like there's a throttle somewhere saying "if PHP {10% of max speed}" ?

Thanks

Lubox
×

Success!

Help @Lubox spread the word by sharing this article on Twitter...

Tweet This
Sign in
Forgot password?
Sign in with TwitchSign in with GithubCreate Account
about: ({
version: 0.1.9 — BETA 5.26,
whats_new: community page,
up_next: more Davinci•003 tasks,
coming_soon: events calendar,
social: @webDeveloperHQ
});

legal: ({
terms: of use,
privacy: policy
});
changelog: (
version: 0.1.9,
notes: added community page

version: 0.1.8,
notes: added Davinci•003

version: 0.1.7,
notes: upvote answers to bounties

version: 0.1.6,
notes: article editor refresh
)...
recent_tips: (
tipper: @AriseFacilitySolutions09,
tipped: article
amount: 1000 SATS,

tipper: @Yussuf4331,
tipped: article
amount: 1000 SATS,

tipper: @darkwebsites540,
tipped: article
amount: 10 SATS,
)...