flifloo/BackupPC_archive_host_B2
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
48 commits 1 branch 0 tags 101 KiB
Commit graph

48 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ryan Tucker
e4fd5ca8ac Fix multiprocessing bugs, add unlink_worker 
This is still under some live development, for sure :-)

Among other minor (yet significant) fixes, I've added another
worker process to handle unlinking.  This verifies that the file
was transmitted properly (or at least with the right md5sum)
before unlinking it.
 2011-09-22 10:43:28 -04:00
Ryan Tucker
3892037351 Use SSL to talk to S3, and tweak send_file 
Note: We're now dependent upon Boto 2.0.
 2011-09-21 20:39:15 -04:00
Ryan Tucker
7bd57300cf Clean up process calls, renice to 10 where available 2011-09-21 20:37:59 -04:00
Ryan Tucker
3eeda045bd Logging, etc tweaks 2011-09-21 20:35:10 -04:00
Ryan Tucker
d07636a5ff Merge branch 'master' of github.com:rtucker/backuppc-archive-s3 2011-09-20 22:35:54 -04:00
Ryan Tucker
a888eab0c3 Remove SlowFile class 
This functionality is best provided by the upstream router and/or
underlying system.
 2011-09-20 22:26:19 -04:00
Ryan Tucker
7fa6dd0115 Adapt to use a multiprocessing queue 
This will let it GPG things in the background while S3ing things in the
foreground, hopefully helping throughput!
 2011-09-20 22:24:56 -04:00
Ryan Tucker
3af02240a4 Change "logging" to "logger" throughout 2011-09-20 22:22:56 -04:00
Ryan Tucker
c4cb3c8a7f Wrap main routine in if __name__ == '__main__' 2011-09-20 16:55:23 -04:00
Ryan Tucker
c3730f8719 markdown formatting fixes 2011-06-10 21:14:53 -04:00
Ryan Tucker
96e8cf12e4 add something that looks like documentation 2011-06-10 21:04:08 -04:00
Ryan Tucker
fb2aaeae0f make the backup-manager.py --help more useful 2011-06-10 20:55:59 -04:00
Ryan Tucker
45360b0b1a fixing hardcoded path (wow...) 2011-06-10 20:26:58 -04:00
Ryan Tucker
6360655bf8 Change exception to use comma format 
The "except foo as e" form, used on line 350, doesn't work with
older versions of Python; this change should allow it to work
with Python 2.x in general.

Thanks to Aaron Ciarlotta for the heads up!
 2011-01-25 22:47:51 -05:00
Ryan Tucker
01cc4abbf6 setting storage-class to REDUCED_REDUNDANCY 2010-05-19 08:52:37 -04:00
Ryan Tucker
a752c3c1dd fixing a logic bug (i think) 2010-03-18 09:09:39 -04:00
Ryan Tucker
7b1d0356e0 add --keep option to keep the n most recent backups when deleting by age 2010-03-16 11:26:49 -04:00
Ryan Tucker
2b5b57b888 respect the test mode when wanting to delete the COMPLETE file (oops) 2010-03-16 11:26:32 -04:00
Ryan Tucker
7b14ba9488 tweaking the congestion handler a bit 2010-03-16 11:25:50 -04:00
Ryan Tucker
a9262b201a oops, used the wrong variable name 2010-03-09 17:23:22 -05:00
Ryan Tucker
1fea10bc03 more accurate backup age handling; also nuking the .COMPLETE file on a delete 2010-02-18 15:34:58 -05:00
Ryan Tucker
9edeab8662 improve handling of interrupted uploads 2010-02-03 11:58:51 -05:00
Ryan Tucker
b471ad45e8 another tweak to the congestion mitigator 2010-01-24 10:29:19 -05:00
Ryan Tucker
f246975ce4 more delayfactor tweaks; also fixing a filehead problem on restarts 2010-01-20 22:54:52 -05:00
Ryan Tucker
e09b42797a oops, old values for targetspeed/lastblocktargettime are bad 2010-01-18 17:58:44 -05:00
Ryan Tucker
f87dbdafc9 some tweaks to speed rollback logic 2010-01-18 17:06:05 -05:00
Ryan Tucker
55e1a5cb4a trapping socket.error; retry old files on a restart instead of starting anew 2010-01-09 15:00:51 -05:00
Ryan Tucker
2581d85058 fix wrong filename for complete semaphore 2010-01-06 22:43:16 -05:00
Ryan Tucker
0694a2e8c7 stringio does not have a fastread... 2010-01-06 09:03:08 -05:00
Ryan Tucker
dfa9eff725 typo in exception handling; think i know the problem 2010-01-05 23:06:20 -05:00
Ryan Tucker
db7cea8a7e plan c for writing metadata 2010-01-05 17:42:14 -05:00
Ryan Tucker
d1256d08cc dupe file checking was not; adding more basename action to send_file 2010-01-04 21:04:18 -05:00
Ryan Tucker
5a5f190929 call update_metadata() to send the updated metadata 2010-01-04 16:09:34 -05:00
Ryan Tucker
8b3283aeb8 cleaning up timezone handling a bit 2010-01-04 16:09:08 -05:00
Ryan Tucker
1b1ad9445e adding code to indicate when the backup isn't finalized 2010-01-04 15:59:34 -05:00
Ryan Tucker
27a0be4b86 backup-manager now has list, delete, and script options 2010-01-02 21:18:09 -05:00
Ryan Tucker
def1f8bc4d initial commit of backup-manager 2010-01-02 19:32:27 -05:00
Ryan Tucker
cc73e77a3c more tweaks to the rate limiting code 2010-01-02 19:32:16 -05:00
Ryan Tucker
484922d4ef oops... removing bug in debugging code 2010-01-02 13:05:23 -05:00
Ryan Tucker
c6fc1f1d80 adding some more debugging output 2010-01-01 20:11:36 -05:00
Ryan Tucker
0b8a7b89d2 trying some adjustments to make it more responsive to ratty comms 2010-01-01 14:01:35 -05:00
Ryan Tucker
e80b23e036 fixing an infinite-pause error; sending some output to stdout to avoid timeouts 2010-01-01 13:59:54 -05:00
Ryan Tucker
5fe7494f15 some tweaks: easier retry, better maths 2009-12-30 09:35:15 -05:00
Ryan Tucker
f942a2763f increasing delayfactor punishment a bit; adding speed control 2009-12-29 18:53:14 -05:00
Ryan Tucker
dd0a141d5f adding rate limiting and retry logic 2009-12-29 14:41:50 -05:00
Ryan Tucker
07777a8628 moving bucket opening towards beginning 2009-12-29 08:48:39 -05:00
Ryan Tucker
c7a6d2a310 not using SSL for a little bit, due to stability issues -- payload still encrypted 2009-12-29 08:47:29 -05:00
Ryan Tucker
69c9fc4163 initial commit 2009-12-27 23:04:01 -05:00