[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / r / s / t / u / v / vg / vr / w / wg] [i / ic] [r9k / s4s / vip / qa] [cm / hm / lgbt / y] [3 / aco / adv / an / asp / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / qst / sci / soc / sp / tg / toy / trv / tv / vp / wsg / wsr / x] [Settings] [Search] [Home]
Board
Settings Home
/g/ - Technology


Thread archived.
You cannot reply anymore.



File: backup.jpg (68 KB, 1280x600)
68 KB
68 KB JPG
Hey /g/, I'm looking for a specific type of backup solution.
I'm going to be backing up from time to time to external HDDs. Ideally I want something automated or semiautomated to sync the new data when the HDDs are plugged in. The thing is, I will be backing up very large files and I want something smart like syncthing that will only add the changed parts of the files instead of coping the entire files again every time.
Something cross-platform is preferred because I currently use windows but plan on moving to GNU fully soon. If not possible, two similar solutions (one for each OS) is acceptable.
And last but not least, I would really rather a free software option, or at least open-source. Suggest proprietary software only as a last resort, please.
Any ideas? Also, does anyone here do backups in a similar manner?
>>
>>70566309
Duplicati
>>
>>70566801
Sounds good, but not exactly what I'm looking for. I want my backups to be fully accessible directly from the drives, without the overhead of restoring from the backup first.
Also, it seems like this would also bloat excessively with time. I want to record the changes made to the files, not additions that will change the files when restoring.
I might consider using this for cloud backup, though, so thanks.
>>
>>70566309

Absolutely no warranty, but if you feel lucky enough you can use my implementation of a backup tool for big files.

I use it to backup my 2 tb big files to external drives. The current implementation is windows only but i think wine should make it work on linux without any problems. Good luck in finding a solution.

https://github.com/Cloudnaut/HFFDC
>>
>>70567274
>tfw trusting all your data AND your backups to the hacker known as 4channel.
Seriously, though. It sounds interesting. Thanks for the share.
>>
GNU/Veeam
>>
>>70567536
It's open source, no need to worry
>>
>>70566309
A shell script.
>>
>>70567592
I'll take it if you have one.
>>
>>70567608
Why not write one yourself?
>>
>>70566309
>>70567274
* You want diff backup
* You want the files to be directly accessible
* You are storing large files
* You want to do this on Linux

A solution that werks on Loonix and isn't some gentooman's github is ZFS snapshots + send/recv. Have all your drives formatted as ZFS and use send/recv to push snapshots to your backup.
>>
>>70567548
lolwut?

>>70567671
If I knew how to do that I wouldn't need to ask /g/ for assistance. I don't even know what tools would be necessary for my use case. I will take nudges in the right direction if you have them.

>>70567684
Unfortunately zfs is not an option. My hard drives are already formatted as ntfs and they'll sadly have to stay like that for the rare occasion I might need to access them on windows. Also, on GNU I'll be using btrfs, but zfs. But thanks.
>>
Does anyone else have some ideas?
>>
>>70569218
I don't know, man. How big are the files that we're talking about here?
>>
>>70569386
Nothing extreme like the 2 TB this anon >>70567274 uses their program with, but some can be upwards of 20 GB which already would be very overkill to move back and forth every week on a USB 3 connection. Not to mention the unnecessary writes to the HDD. That's why I need to send only the changes to the files, which apparently is called delta.
>>
rsync --inplace --no-whole-file
>>
>>70569675
Thanks. After researching for a bit it does seem like rsync would probably be the logical choice. Unless anyone else has a better solution.
>>
>>70569675
This might be a solution if you don't expect any speed improvements. It will write less to the hdd but it will also take much longer than a regular copy, just saying it. Rsync has to read the section from the hdd before it can compare their contents. This will lead to increased copy times.
>>
>>70566309
cron and rsync - google it, it'll take about 2 min to setup if you've got half a brain.
>>
>>70571112
Why would reading from the external hard drive take longer than writing to it?
>>
>>70571192
Because if you copy over you just override the bytes without knowing if anything had changed.

Rsync has in addition to writing the bytes the overhead of reading all bytes, also the bytes which didn't change and calculate the checksum for each chunk
>>
>>70571135
I'm actually reading about udev and rsync now. Apparently with that I can make the backup happen when I plug the drive in, which is the level of automation that I wanted. Of course it's no a solution for windows right now, but I think it will come in handy when I move to GNU.



Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.