flifloo/BackupPC_archive_host_B2
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
67 commits 1 branch 0 tags 101 KiB
Commit graph

67 commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ethanell 5461dec55c Use account_id instead of access_key for bucket names and trim it trim to 50 2022-10-19 13:38:03 +02:00
Ethanell 4cbaf0c52f Update backup manager script 2022-10-18 00:02:58 +02:00
Ethanell dca2dbdad8 Rewrite a lot of BackupPC_archiveHost_s3 to b2 2022-04-08 18:20:55 +02:00
Ryan Tucker e9ee689844 pyflakes / PEP8 compliance 2013-02-06 19:00:32 -05:00
Ryan Tucker 4f4574b727 Reorganize script to be less hacky 
A class for caching, less shady delete logic, etc.
 2013-02-06 18:02:07 -05:00
Ryan Tucker 59eedb580b Start the most needed backups first 
Uses a somewhat-hackish scoring algorithm to decide which backup
to start first, but it's better than alphabetical order!
 2013-02-06 16:21:45 -05:00
Ryan Tucker 5b4d78fea3 Don't encrypt COMPLETE files nor choke if they are 
May fix:
https://github.com/rtucker/backuppc-archive-s3/issues/2
 2012-11-12 16:52:02 -05:00
Ryan Tucker c2ddefb316 Update link to Boto source 
Fixes issue #1: https://github.com/rtucker/backuppc-archive-s3/issues/1
 2012-05-12 16:08:45 -04:00
Ryan Tucker a7c9100b7a autostart: Use ArchiveSplit from BackupPC's conf 
This is probably a bug; I ought to report it if it isn't
already fixed.
 2011-10-02 11:48:50 -04:00
Ryan Tucker 91296f1775 Clarifying license stuff. 2011-10-02 11:36:20 -04:00
Ryan Tucker 7ff0a277a6 Adding BackupPC_archiveStart from BackupPC source 
This is the unedited version from my copy of BackupPC.
I'm planning to edit this in order to fix a couple bugs...

-rwxr-xr-x 1 root root 4036 2011-06-30 04:41 /usr/share/backuppc/bin/BackupPC_archiveStart

backuppc:
  Installed: 3.2.0-3ubuntu4~lucid1
  Candidate: 3.2.0-3ubuntu4~lucid1
  Version table:
 *** 3.2.0-3ubuntu4~lucid1 0
        500 http://archive.ubuntu.com/ubuntu/ lucid-backports/main Packages
        100 /var/lib/dpkg/status
     3.1.0-9ubuntu1 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
 2011-10-02 11:30:34 -04:00
Ryan Tucker abbbbba970 backup-manager: add ability to start archives 
This will allow things to run completely on autopilot, creating
archives on S3 when necessary.
 2011-09-29 11:22:54 -04:00
Ryan Tucker b5533a843f Fix args for unlink_worker subprocess 
unlink_worker was not starting and the main process was
dying, due to args being the queue itself, not a tuple
containing the queue.
 2011-09-29 09:09:01 -04:00
Ryan Tucker d0ea15beff Move unlinks to unlink_worker; handle OSErrors 2011-09-24 18:27:42 -04:00
Ryan Tucker e74e89ede1 unlink_worker: don't verify the remote file 
send_file does this now, and odds are good that if it verified
once, it's going to verify a second time...
 2011-09-24 18:18:18 -04:00
Ryan Tucker e8f7ce4cf9 minor tweaks to exception handling and logging 2011-09-23 21:09:41 -04:00
Ryan Tucker 871613a6ea Various adjustments and bug fixes 
Fix: force integers to strs in Popen arguments

Refactored verify_file out of unlink_worker
If a file already exists on S3 and is identical, don't re-upload
Add some delays, etc, to allow things to settle
Put a STOP in the gpg_queue for each process
 2011-09-22 22:21:37 -04:00
Ryan Tucker 7bda017245 Update documentation to reflect changes 2011-09-22 10:57:50 -04:00
Ryan Tucker 5df1ce7594 Make logging and exceptions more consistent 
Mostly changed the format, added raises where things
are unrecoverable, and using a wider range of levels
than just "info".  Most output is now debug.
 2011-09-22 10:44:54 -04:00
Ryan Tucker e4fd5ca8ac Fix multiprocessing bugs, add unlink_worker 
This is still under some live development, for sure :-)

Among other minor (yet significant) fixes, I've added another
worker process to handle unlinking.  This verifies that the file
was transmitted properly (or at least with the right md5sum)
before unlinking it.
 2011-09-22 10:43:28 -04:00
Ryan Tucker 3892037351 Use SSL to talk to S3, and tweak send_file 
Note: We're now dependent upon Boto 2.0.
 2011-09-21 20:39:15 -04:00
Ryan Tucker 7bd57300cf Clean up process calls, renice to 10 where available 2011-09-21 20:37:59 -04:00
Ryan Tucker 3eeda045bd Logging, etc tweaks 2011-09-21 20:35:10 -04:00
Ryan Tucker d07636a5ff Merge branch 'master' of github.com:rtucker/backuppc-archive-s3 2011-09-20 22:35:54 -04:00
Ryan Tucker a888eab0c3 Remove SlowFile class 
This functionality is best provided by the upstream router and/or
underlying system.
 2011-09-20 22:26:19 -04:00
Ryan Tucker 7fa6dd0115 Adapt to use a multiprocessing queue 
This will let it GPG things in the background while S3ing things in the
foreground, hopefully helping throughput!
 2011-09-20 22:24:56 -04:00
Ryan Tucker 3af02240a4 Change "logging" to "logger" throughout 2011-09-20 22:22:56 -04:00
Ryan Tucker c4cb3c8a7f Wrap main routine in if __name__ == '__main__' 2011-09-20 16:55:23 -04:00
Ryan Tucker c3730f8719 markdown formatting fixes 2011-06-10 21:14:53 -04:00
Ryan Tucker 96e8cf12e4 add something that looks like documentation 2011-06-10 21:04:08 -04:00
Ryan Tucker fb2aaeae0f make the backup-manager.py --help more useful 2011-06-10 20:55:59 -04:00
Ryan Tucker 45360b0b1a fixing hardcoded path (wow...) 2011-06-10 20:26:58 -04:00
Ryan Tucker 6360655bf8 Change exception to use comma format 
The "except foo as e" form, used on line 350, doesn't work with
older versions of Python; this change should allow it to work
with Python 2.x in general.

Thanks to Aaron Ciarlotta for the heads up!
 2011-01-25 22:47:51 -05:00
Ryan Tucker 01cc4abbf6 setting storage-class to REDUCED_REDUNDANCY 2010-05-19 08:52:37 -04:00
Ryan Tucker a752c3c1dd fixing a logic bug (i think) 2010-03-18 09:09:39 -04:00
Ryan Tucker 7b1d0356e0 add --keep option to keep the n most recent backups when deleting by age 2010-03-16 11:26:49 -04:00
Ryan Tucker 2b5b57b888 respect the test mode when wanting to delete the COMPLETE file (oops) 2010-03-16 11:26:32 -04:00
Ryan Tucker 7b14ba9488 tweaking the congestion handler a bit 2010-03-16 11:25:50 -04:00
Ryan Tucker a9262b201a oops, used the wrong variable name 2010-03-09 17:23:22 -05:00
Ryan Tucker 1fea10bc03 more accurate backup age handling; also nuking the .COMPLETE file on a delete 2010-02-18 15:34:58 -05:00
Ryan Tucker 9edeab8662 improve handling of interrupted uploads 2010-02-03 11:58:51 -05:00
Ryan Tucker b471ad45e8 another tweak to the congestion mitigator 2010-01-24 10:29:19 -05:00
Ryan Tucker f246975ce4 more delayfactor tweaks; also fixing a filehead problem on restarts 2010-01-20 22:54:52 -05:00
Ryan Tucker e09b42797a oops, old values for targetspeed/lastblocktargettime are bad 2010-01-18 17:58:44 -05:00
Ryan Tucker f87dbdafc9 some tweaks to speed rollback logic 2010-01-18 17:06:05 -05:00
Ryan Tucker 55e1a5cb4a trapping socket.error; retry old files on a restart instead of starting anew 2010-01-09 15:00:51 -05:00
Ryan Tucker 2581d85058 fix wrong filename for complete semaphore 2010-01-06 22:43:16 -05:00
Ryan Tucker 0694a2e8c7 stringio does not have a fastread... 2010-01-06 09:03:08 -05:00
Ryan Tucker dfa9eff725 typo in exception handling; think i know the problem 2010-01-05 23:06:20 -05:00
Ryan Tucker db7cea8a7e plan c for writing metadata 2010-01-05 17:42:14 -05:00