[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / r / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip / qa] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
Settings Mobile Home
/g/ - Technology

[Advertise on 4chan]


Thread archived.
You cannot reply anymore.


[Advertise on 4chan]


File: 1_5VhWdJERN_2PPLIQBgO4XQ.jpg (340 KB, 1350x1350)
340 KB
340 KB JPG
Discuss data hoarding and other home server projects

Previous thread
>>80312416

READ THE WIKI!
https://wiki.installgentoo.com/wiki/Home_server

HELP BY CONTRIBUTING!
Pages that need more work.
>Setting up your Storage
>Choosing an Operating System
>Single board computers

Nas Case Guide. Feel free to add to it.
https://wiki.installgentoo.com/wiki/Home_server/Case_guide

>IF YOU WANT PURCHASING ADVICE:
READ THE FAQS!
State your BUDGET.
List OS and software.
State storage/raid and # of disks.
List hardware features (low power draw, 10G Ethernet, etc)

>Why should I have a homeserver?
/hsg/ is about learning and expanding your horizons. De-botnet your life. Learn something new. Serving applications to yourself, your family, and your frens feels good. Put your /g/ skills to good use for yourself and those close to you. Store their data with proper availability redundancy and backups and serve it back to them with a /comfy/ easy to use interface.

>How should I get started?
Most people get started with NAS. It’s nice have a /comfy/ home for all your data. Streaming your movies/shows around the house and to friends. Know all about NAS? Learn virtualization. Spun up some VMs? Learn networking by setting up a pfsense box and configuring some vlans. There's always more to learn and chances to grow. Think you’re god tier already? Setup openstack and report back.

>What do I hoard?
Movies, music, books, porn, anything and everything you like. If you are looking for things to hoard check out >>>/g/ptg they have a list of various trackers for everything under the sun.

>Why Hoard?
Things that are online today might not be online forever. It's good to have a copy of something because you never know when it might get taken down due to copyright strikes or Big Tech censorship.
>>
>>80340718
Seagate is fine.
>>
>>80340730
wd is better.
>>
>Need help understanding Borg’s deduplication. As I understand it, only the first archive is a full back up and the subsequent ones just back up the changes.

>If there’s corruption/flipped bits in the first archive, won’t that ruin the whole repo? Why do people say borg is so reliable?
>>
>>80340718
Brandfagging drive manufacturers is even dumber than brandfagging CPUs or GPUs
>>
wd a shit
>>
>>80341296
This. Just accept that they will fail and deal with it with backups and RAID.
>>
>>80340718
>>80340753

I've always been a WD 'fan' in the sense that I always thought WD was better in terms of quality and reliability (honestly probably they once were way over seagate), but since I recently got a seagate, man; for the price they are quite good. I have 3 ironwolfs arriving today.

The ones I still not trust are toshibas
>>
If you intend to raid, just don't get SMR drives, get CMR.

https://www.servethehome.com/wd-red-smr-vs-cmr-tested-avoid-red-smr/
>>
>>80342455
>The ones I still not trust are toshibas
But they are the most reliable ones even I got a 4tb refurbished enterprise Toshiba from amazon for like 45$ and those are running fine 24/7 for 2.5+ years now.
>>
I want all traffic from a container to go through a commercial VPN, and set up an nftables based killswitch for it, how should I do this? Should I set up the firewall and VPN from the host or the container itself?
>>
>>80342767
some images have integrated vpn, you could also set a docker network with a vpn container on it, and put the openvpn image only on that net
>>
>>80340680
>30-50$ per HDD slot
>requires 5Gbps+ switch and distributed system
What's the appeal of these distributed boards for NAS?
>>
>>80342718
>samplecase of 1
product x is fine!
>>
>>80340680
What is the difference between a home server and 'just another drive?'
Or rather, what benefit do I get out of the server compared to 'just another drive?'
>>
>>80340680
Best firewall/router for 10g GPON/Fiber wan?
>>
>>80343610
1. it doesn't make noise near your desk, at 6+ HDD RAID the noise might be significant
1.1 if you wonder why would you get so many disks: redundancy often lets you avoid reaching for the backup (do make backups anyway). With 1Gbps speed (close to single HDD) restoring 6TB would take 13+ hours.
2. it's a separate point of failure, you aren't likely to wipe both your PC and your backups
3. you can specialize the case for holding many disks rather than obstructing airflow in your desktop
4. you can run things on it 24/7: torrent box, mail server, media streaming, whatever
5. you can access it from all devices, possibly from outside your home (probably doesn't matter now) (take care if you do expose it to the Internet)
>>
Do you guys put camera inside your house? How do you prevent it from recording you fucking some hookers or your lover?
>>
Why would you not want to tape yourself banging some hooker?
>>
>>80343610
Home server is useful for running server applications.

Even if you just get a mini PC with like 8 GB RAM, you can run all sorts of useful stuff on it that you then don't have to run on your desktop.

This keeps data and processes segregated, and segregation is good.
>>
>>80343619
As this is /hsg/ i would just use a server with a pfsense / opnsense VM.
>>
>>80344502
>>80344576
>>
>>80344764
It generally recommended to keep your router on a separate machine
>>
>>80340885
That's possible. You could test it yourself using a hex editor - tell us the results.
>>80342767
I use dperson/openvpn for that. You have to make other containers use its network stack, which sucks when two containers want to bind to the same port, but otherwise it's pretty ok. As long as your containers don't run as the same GID as the openvpn process inside that container - that would allow for leaks on other interfaces than tun0, because that's how the iptables rules are set there.
>>80343211
>you could also set a docker network with a vpn container on it, and put the openvpn image only on that net
How would you do that exactly? I'd prefer to use a VPN container as a Internet gateway for other containers, since the approach above isn't really ideal. I understand I'd have to do more port forwarding this way (VPN server -> VPN container -> e.g. rtorrent container).
>>
>>80340680
So... I'm still using an ivy bridge i7 laptop, considered an update but after thinking it through it's not worth it.
Altough I have around 2TB of media on external drives and plugging/unplugging it to access them is becoming annoying, add in another 1TB of family photos parents have on their pc (including scans of very old ones) and a surveilence camera we got recently and I've eventually decided to get a homeserver.
>Budget
I'd like to stay around 200euros (storage excluded) but I can easily spit out more if it means I'm not missing on something important
>OS
I'm leaning towards OMV, beside surveilence software I'd like to play with things like torrent box, jellyfin.
>Storage
Probably 3x2 or 4x2TB in raid5, don't know yet exactly what drives I will get, depends on the deals I would find
>Hardware
At first I've thought about building a full pc with something like i3-9100 but the more I've read the more I've downgraded realizing I don't need this or that (correct me if I'm mistaken) and would rather have something small, low power and low heat/noise. My only requirement might be gen7+ intel to cover transoding if needed but anything over that is probably overkill. I can go for amd but I haven't looked into that abd have no idea how it compares. I'm currently eyeing something like NUC or other mini pc with 4/4 Celeron but If I can build something in similar size/performane that could be an option. I also have 6gb ddr3 laying around but as far as I know I can't use those unless I go for older hardware. What would you recommend?
>>
I have a spare I7-4790K CPU that I'd like to use for a server to stream TV/movies, host files, and run Usenet services. I'm trying to figure out the best hardware, particularly the motherboard and hard drives. Any recommendations?
>>
Those of you who set up personal homeservers with actual server grade equipment, why do you do it?

I get those who let those machines for rendering, AI and those kind of workloads to set their PC free.

But those who do it just for storage and simple streaming and shit, why not using low power stuff besides repurposing? Not talking about raspi or similar, which is the lowest end, I mean stuff like the celeron J
>>
>>80342455
Toshiba has the lowest failure rate in the industry.
>>
>>80346971
Running multiple virtual machines on mine, that coupled with using SAS 12gb/s SSD's for spinning up containers etc faster.
>>
A friend of mine was asked by their parents to setup a home server for some media. If it was me I'd just do an rpi4 + a usb3 drive, but for him I suggested 3 options:

- NAS Zyxel NAS326
- WD My Cloud EX2
- Synology DS220j

Any suggestions? I heard synology is okay. I don't know much about hardware like that because I prefer custom built PCs or small devices like nanopc for my own tinkering purposes.
>>
>>80346998
prove it
>>
Other than Adguard Home, what's good to install on a raspberry pi for privacy/security?
>>
>>80347057
Odroid HC4
>>
>>80347156
>Odroid HC4
He's not a tinkerer, or that technical, he just wants a ready made solution, like the ones I listed. Not a board that requires command line arguments to configure.
>>
>>80347211
Just install NextcloudPi, no tinkering.

Everything except MyCloud, they got backdoor's on purpose.
>>
>>80347284
>NextcloudPi
He would have to install that, I'm not gonna do that for him, and I doubt he'd manage. As I said, not technical.

>Everything except MyCloud, they got backdoor's on purpose.
When will you autists learn that normal poeple don't give a shit about privacy.
>>
>>80347312
> When will you autists learn that normal poeple don't give a shit about privacy.

Everyone already knows that, otherwise i would be screaming at you to not get any of those, just telling you to not get the worst of it.
>>
File: 1468185966396.jpg (63 KB, 500x500)
63 KB
63 KB JPG
>>80347312
>When will you autists learn that normal poeple don't give a shit about privacy.
You're on a technology board.
>>
>>80347356
>not get any of those
You must be american with that reading comprehension. I'm looking for a recommendation for a ready-made plug-n-play solution for a friend, not myself.

>>80347369
Precisely, this is not /pol/.
>>
>>80347057
The Synology is pretty nice, I just set one up for a family member last weekend. I tried a Zyxel myself but the one I got was DOA and I had to send it back. Could just be bad luck though.

I'd probably do a NAS for the data and file hosting and a little mini PC for the streaming, something like a Xidu Philmac is capable and pretty cheap.
>>
>>80347375
Try google.
>>
>>80347381
Thanks, good to know. I was learning towards the Synology one.

>>80347394
Not practical for people with bad internet connections and larger media.
>>
>>80347375
Already told you, but you seem to be as dumb as your friend, just don't get the MyCloud ffs.

And actually pull your friend's head out of his ass, if he wants something decent, he has to put some work on it, otherwise just get fucking tresorit or some shit, it's not that hard to have nextcloud running, you'll own your data and actually learn something useful.
>>
>>80347421
You seem salty as fuck. Maybe go out for a walk or something.
>>
>>80347455
I just came back from running
>>
>>80347418
No I mean google it you entitled child.
>>
File: 1589632462768.jpg (2 KB, 108x124)
2 KB
2 KB JPG
>>80346971
it look cool
>>
>>80346529
>I7-4790K
Throw it away and buy something new.
>>
>>80348420
No. It works fine.
>>
>>80347534
ye, it do
>>
Anybody have any advice on where to troubleshoot from here?
Already tried to renew the DHCP, power cycle the computer, power cycle the router. The green led is on and the orange one is blinking and looking in network utility I'm getting sent and received packets, yet there's no internet and I can't ping 192.168.1.1 ... any advice?
Going to install windows 10 now to see if it works on that..
>>
hosting a minecraft server on an extra pc at home, I have it set to startup as a cronjob at boot, but when i ssh into the server how do i access the minecraft server command and do stuff like adding users to whitelist, restart server ect?
>>
>>80349018
tried factory reseting the router? can you log into it from another device?
>>
>>80346971
>Those of you who set up personal homeservers with actual server grade equipment, why do you do it?
You do realize that you can get a lot of old server equipment for pennies on the dollar right? The rest of your question is literally fucking answered in the OP.
>>
>>80349413
I’ve not reset it yet because my ISP is a pain and requires me to log into their account to access the control panel but I can access it fine on other devices and when connected to WiFi.
When I tried it on windows 10 at first it said ethernet no internet access, tried ipconfig/release and renew and it said there was internet access but it complains that no DNS address could be found when I do so.
I tried manually setting the DNS to 1.1.1.1/1.0.0.1 but when I do so internet access drops again.
It’s using the RTL8153 controller. I also tried to reinstall the driver but same issues persist.
>>
Unraid is kino. FreeNAStards can stay go.
>>
>turn old laptop into local file server
>start moving files on to it
>10 MB/s
Ok now I see why people buy used server equipment for their set ups
>>
>>80349769
>Unraid
What can it do that other servers wouldn't? Looks pricey for extremely mundane features.
>>
>>80349817
How old is the laptop? Is the lan gigabit?
>>
>>80349817
Do you only have a 10/100 NIC?
>>
>>80349871
It's plug n play, you can expand drives with minimal effort while maintaining parity, drives can be taken off the array and they'll still work on any Linux distro.

Also works as a VM and docker hypervisor.
>>
>>80349903
Interesting, thanks
>>
>>80349874
It's a Dell Inspiron 1545. Around 11 years old.
>>80349887
Specs say it's 10/100/1000.
Might be the hard drive just being slow. I'll have to take a look at it. Might be the one the laptop came with, can't remember.
>>
>>80350213
Check your switch aswell
>>
>>80350213
>Might be the hard drive just being slow.
I've had god awful 4200 RPM laptop drives that only write at 10-15mb and read at 20-30.
Wouldn't surprise me
>>
>>80350219
My "switch" is a shitty modem router from my ISP, it's complete crap and I'm trying to find a new cheap router to buy but all the WRT1900ACSs on ebay don't ship to my country and I don't know what to buy instead.
>>
>>80340680
why do you need like 4 raspberry pies hooked up? what good is it
>>
>>80350293
larping docker/k8s
>>
>>80350283
>>80350213
Well using another hard drive didn't change much. And it doesn't seem to be the router since I get better speeds moving shit from my PC to my work laptop.
>>
>>80351092
it's likely the cable
>>
I heard someone say a while back something about torrenting on a ZFS filesystem can cause issues, unless you do something. I forget what the issue was and the workaround.
Does anyone know of such issues though?

Also the wiki states expansion on ZFS can be a PITA, how so? I looked it up, it seems fairly straight forward?

One more thing, in a system with many drives, how do you determine which drive is which in software?
(Example: which physical drive == /dev/sda, which is /dev/sdb, etc.)
>>
>>80351317
Nope. I used that same ethernet cable that connected the PC to connect the server and still the same shit. I installed OMW on a SD card on the laptop's SD card port, dunno if that matters.
Now that I check both hard drives are 5400RPM. Are they really supposed to be that slow?
>>
>>80346529
i had a 4790K from my gaming pc from 2013, but i sold it (for a surprisingly high amount) and bought a 3 year old i3 sff instead, much lower power and quieter
>>
Anyone host an intranet page for themselves with links for all your self hosted services to use as a homepage? I mean I use bookmarks, but thats kind of a pain in the ass, as is remembering all IP addresses and ports.
>>
File: 1593523815873.jpg (47 KB, 894x773)
47 KB
47 KB JPG
>tfw still using plex because jellyfin still hasn't implemented summary view
>>
>>80352243
Well I tested the hard drive in my PC and I get 100MB/s so after all this testing I see that it's the laptop I'm using for the server that's bottlenecking it. Not sure why exactly tho.
>>
>>80347124
Not that anon but I think at least for the high capacity drives >>80346998 is the case, according to Backblaze at least.
>>
>>80344887
Why? I do plan on doing this btw but just because I don't think my CPU is powerful enough.
>>
Anons, is grafana/prometheus really that difficult to learn? I don't have any sql knowledge but I can do basic coding. Would this be like a full time job or could I do it in a few months spending like a few hours every other day or something
>>
>>80352550
After some googling it might be that my NIC is not operating at full speed and this might or might not be fixed with updating the driver.
>>
>>80350283
>WRT1900ACS
rubbish
do not buy
>>
>>80351655
>torrenting on a ZFS filesystem
Possibly something fragmentation related, fixed by enabling pre-allocation in the torrent client?
>the wiki states expansion on ZFS can be a PITA, how so?
Currently it's not possible to add drives to vdevs. Here's a good explanation: https://louwrentius.com/the-hidden-cost-of-using-zfs-for-your-home-nas.html
>how do you determine which drive is which in software?
I tend to use smartctl for it, checking the serial numbers.
>>80352449
I mostly rely on address bar autocompletion myself, but I also have a super barebones page (pic rel) just for occasional shits and giggles. There are things like Homer and Heimdall, the latter also providing additional stats of the services you run, e.g. count of active torrents, queued Ombi requests, etc.
>>80352478
That seems quite useful. Apparently there's a feature request already: https://features.jellyfin.org/posts/1011/summary-view
I wonder how difficult would it be to write another view mode, adapting the existing code from other views.
>>80353083
Well, I wouldn't want to e.g. accidentally shutdown the server and drop the Internet access for my flatmates.
>>80353107
Not really. You could read or watch some tutorials to get a glimpse of it.
>>80349903
It seems that you mostly pay for the ease of use here, as everything else is achievable with a mergerfs+SnapRAID setup. You wouldn't have a real-time parity calculation though - is anyone aware of a FOSS solution that allows for it without losing the other aforementioned advantages?
>>
>>80353386
Well if not that then what
>>
File: img7943.png (56 KB, 779x849)
56 KB
56 KB PNG
>>80353426
>That seems quite useful. Apparently there's a feature request already: https://features.jellyfin.org/posts/1011/summary-view
Yeah, I made that feature request.

Doubt it will be happening any time soon though, NPCs want this boring stuff more.
>>
>>80340680
Anyone have a rack mounted RAID system they'd recommend? Should I just get this? http://amazon.com/dp/B017NMX5KO/
>>
>>80353434
idk something with atheros chipset
marvell sucks, can't do wireless vlans
>>
>>80353386
>rubbish
literally what?
I hace this router and currently is rock solid with openwrt
>>
Is there any advantage in using ZFS for JBOD?
>>
Best way to setup some cheap storage for media? Raspberry pi and some external HDDs on a powered hub?
>>
>>80353198
Well I'm tired I'll have to take a crack at it tomorrow
>>
>>80353870
Odroid HC4
>>
>>80353870
I prefer the faster Odroid N2, but RPi4b would work, yes.
>>
>>80353816
Can't think of anything in particular, but JBOD mode isn't quite as resource hungry as RAIDZ or RAIDZ with deduplication even.
>>
>>80354224
So unless you're using RAIDZ, ZFS by itself doesn't do anything more, than say ext4, to keep data intact?
>>
How's Windows 7 for just network storage?
It's what I know and I'm too much of a brainlet for Linux.
>>
>>80354681
Pain in the arse, like all networking on windows.
Linux networking is decently easy, since you just use scp
>>
i want to upgrade my home network to gigabit and i dont know where else to ask
i need a router and a switch if the router doesnt have at least 8 ports. i have a rack but its not very deep so im not against used enterprise gear as long as it fits.

any good things i should scour ebay for?
>>
>>80354932
ICX7150-C12P
>>
>>80354932
Cheap $12 tenda or whatever switches from aliexpress are okay in most cases.
>>
>>80340680
Why do people even make these pi-clusters when you can get an old Optiplex with 8Gb on Craigslist for $80?
>>
>>80353083
There's advantages and disadvantages to hosting your router on a VM. I am of the opinion that you should use dedicated hardware because less complexity is better and if you have some kind of remote controlled outlet, you can physically power cycle without shutting your whole server off in the case of VM. VMs of course have their advantages in deployment, and ease of replacement and cost of course. Your server hardware might not have as many free physical interfaces as you want either for whatever you intend to do or not have expansion for additional ports. Maybe you don't want to give up a LAG group on a media server for bandwidth to your core switches to accommodate WAN failover with 2 ISPs. Probably not what you're interested in but I'm just giving examples of why some might prefer the dedicated hardware. Especially if you work in an IT job and get free shit now and then. Cost is king next to features and what you know how to use/want to learn how to use. UTMs can be expensive and have disgusting licensing models.
>>
Is there a way to prevent a free No-IP's subdomain from expiring without having to open the panel every so often and renewing it?
>>
>>80354681
Install Openmediavault. It's got an easy web GUI.
>>
>>80354321
I think there is no repair mechanism unless you have mirroring or RAIDZ redundancy.
>>
>>80354681
What you need is Unraid.
>>
Is there a faster way to transfer files with checksum?

Rsync checksum is great but it's so slow
>>
>>80355357

Optiplex is a power hug tho
>>
Syncthing or rsync?
>>
threadly reminder that sbc is for fags
>>
What hard drives are the best though, 5400 rpm 7200 rpm, 128 mb, 256 mb, 6tb, 8tb, WD, toshiba, or which seagate isnt shit?
They had 10gb seagate exos on sale yesterday for $204, but I passed on it because it was seagate.
>>
>>80356431

Seagate is the best.
>>
File: 1594782146086.png (186 KB, 1255x1403)
186 KB
186 KB PNG
good evening and I hate EIGRP
>>
File: file.png (139 KB, 731x970)
139 KB
139 KB PNG
im looking to virtualize my router (hardware failed) convince me not to configure my network like this. connecting bridged modem frames to an internal router even if its logically separated with vlans seems like a recipe for a bad time
>>
>>80356511
lol this brightened my day. now tell me where to plug in my router
>>80356513
>>
>>80356540
I thought you were joking when you posted that.
You have no perimeter router: it's going to be a hot mess every time you need to reorganize everything. The closest you could feasibly get to that is a NIC dedicated as your WAN interface going to a virtual router and even then you'll regret the added overhead eventually.
>>
>>80356652
>You have no perimeter router: it's going to be a hot mess every time you need to reorganize everything. The closest you could feasibly get to that is a NIC dedicated as your WAN interface going to a virtual router and even then you'll regret the added overhead eventually.
i was more concerned about security due to the slightest misconfiguration.

guess i'll probably shell out for hw. i was enjoying a pc engines w/freebsd until power surge took it out.
>>
>>80356689
I didn't even touch security because the logistics alone would be bad.
If you're concerned about security, get/build a pfSense firewall. They do routing in addition to traffic inspection and filtering.
>>
>>80346971
A lot of server equipment Just Werks™. It works now. It worked 5 years ago. It will work 5 years from now.

You don't need to buy actual server racks to get a good home setup either. You can buy a Dell H310 for like 30 bucks in perfect condition. That's enterprise equipment. Why buy that over just bolting a bunch of drives to an existing mobo you have from an old desktop? Because it's proven technology. If your mobo cooks, you unhook the card, pop out the board, put in a replacement, plug the card back in and continue on your merry way.

If noise/power consumption isn't a major concern for you, then you can get pretty cheap servers. However 'enthusiast' cases tend to be more open, so you can get lots of airflow over your drives and keep them cool and possibly even vibration isolated, and for most stuff, even enterprisey solutions like ZFS, an old quad core from a gaming rig is more than adequate.

A shitty raspi will probably cost you more than the power you save by using the raspi instead of a spare computer you have lying around, and if something breaks you may not have an easy line to get back online again.
>>
>>80351655
>>80353426
>torrenting on a ZFS filesystem can cause issues

There are a few issues with torrenting on ZFS. ZFS does not have a concept of preallocation. It's a copy on write filesystem, which means that anytime a block is updated, it copies that block over to a new section on the disk, then validates the data in that block, and then only after everything has been fully validated does it update the pointer tree (think of this as the filesystem) to point to that block. The original data is still there during this entire process, and if any part of it fails, it just reverts back to that known good state.

Torrents cause a lot of problems because they not only pick random blocks to download first, but they may be writing in block sizes that don't neatly align with what ZFS has. They also really like to dump data to disk frequently to conserve on system resources. This means that ZFS may be creating literally dozens of copies of your data, and this can result in pretty heavy fragmentation.

Depending on your cache setup, ZFS is reasonably intelligent about dealing with this, but it's still not pretty, especially on arrays that are nearly full. This gets really bad when you're array is getting to around 85% capacity or higher because it simply doesn't have many contiguous blocks of disk space to write the data to.
>>
>>80356838
>A shitty raspi will probably cost you more than the power you save by using the raspi instead of a spare computer you have lying around
Unlikely- in most cases the power difference amortizes a Rpi in like 2 years - maybe one if in a country that mostly compensates emissions.
>>
>>80356838
>>80357131
Also, all the enterprise fanboying does not actually translate to much at all. A Rpi or other ARM SBC also has a very good chance of working 5 years from now.
>>
>>80351655
>>80353426
>>80357121
Some torrent clients also allow you to only write complete pieces to disk, and that helps quite a bit. You'll still get more fragmentation on ZFS. In practice this doesn't actually matter much for home server use because you don't have dozens, let alone hundreds, of users slamming your system, and even if you did, you're bottleneck is almost guaranteed to be your Gigabit router rather than your drives. If you REALLY need the performance, either do a send:recieve of the files periodically (this essentially takes a snapshot and duplicates them so it requires disc space overhead), or just do the initial downloads to a random junk disk you have lying around, and then move them to your ZFS pool manually every so often.

>Also the wiki states expansion on ZFS can be a PITA, how so? I looked it up, it seems fairly straight forward?

You can't just slap a single drive into ZFS and press the rebuild button to redistribute the files across it. Expanding a vdev has all manner of overhead problems because it does not, cannot, and will not directly tamper with existing files because that will completely fuck up the pointer trees. vdev expansion is on the roadmap, but due to the paranoid way ZFS works, it's really not a trivial problem, and will ultimately involve a very lengthy resilvering process. No enterprise environment really wants this feature, so it's not likely to happen anytime soon.
>>
>>80357325
The `traditional' way to expand a pool is to add a bunch more disks and create an entirely new vdev. You can concatenate a bunch of RAIDZ vdevs together over time to get to some truly absurdly large pools, but again, existing data is not distributed across the new pools. Think of it as RAID5/6/7 pools that are jboded together. It's more complex than that, but that's sort of how it works. There's no requirement that the vdevs be the same size, or even have the same parity levels or physical disc sizes. As long as stuff is consistent within a vdev, ZFS is fine. This means that you can have 3(4+1) (15 disks, 3 5 disc vdevs with single parity each), and then bolt on a 1(6+2) (8 discs, 1 8 disc vdev with 2 bits of parity) and it will work. The drawback here is that performance may not be symmetrical across the array initially because the new disks are empty, and this can get even worse if the new vdevs have substantially different performance than the existing ones.

You can also 'expand' a ZFS pool by putting in a bigger drive and resilvering to that drive. Keep the old drive in (you can take it out and just resilver to the replacement, but why would you throw away potential parity information), and then once you've essentially mirrored the new drive you can take the old one offline and replace it. Once you've done this with every single drive in the array, you can then increase the size if the vdev to match the new drives. This process is time consuming, and read write intensive (meaning it shit's on performance while it's happening again and again).
>>
>>80357149
> Rpi or other ARM SBC also has a very good chance of working 5 years from now.

How much data are you willing to stake on that? Willing to bet that you'll be able to replicate the configuration with unknown hardware in 5 years? That's the thing about server hardware. It's been vetted, it provides useful diagnostic information, and there's a lot of it around.

If the H310 in my home server fails (which did happen when a motherboard I had died a few years ago and cooked a few other things along with it) I just drop in a replacement and I'm back in business in 20 minutes. It genuinely took me longer to find the spare card in moving boxes I had never unpacked than it did to install the new mobo/card/cpu and get back up and running.

Plug and play is worth a few dollars for the convenience to me.

>>80357131
I pay 7cents/kwH. Not including drives, my repurposed gaming rig box idles at like 30 watts thanks to modern processor's ability to underclock/volt themselves, so call it 5-7 cents a day. Maybe you can get basic stuff running on a bottom of the barrel pi that can pay for itself after a couple of years, but good luck running anything beyond the most primative stuff on there. It's just more effort than it's worth.

Doing stuff on a pi is for when your idea of a home media server is a 12TB external that's just attached to the network. If that works for you, great. The point I'm making is that a pi is severely limited when a lot of people have perfectly usable hardware just lying around.

>>80357325
>>80351655
>in a system with many drives, how do you determine which drive is which in software?

Pull info up with smartctl or whatever and then make liberal use of the labelmaker someone in your family probably has.
>>
How many sectors is too many bad sectors? A drive has just thrown up 8 bad sectors.
>>
>>80358217
>8

Is it a 512e drive? 8 would indicate that it could be a single 4k sector going bad on a 512e drive. 512e drives emulate 512bye sectors on 4k physical sectors. It's a compatibility thing.

One bad sector on a drive doesn't really mean anything. If that number starts to change, replace the drive. You may also wish to stress test it if you can conveniently do so.
>>
File: melty gentoo 2.jpg (222 KB, 800x1129)
222 KB
222 KB JPG
When the hell are we going to get proper rackmount ARM servers? I want dozens of cores per rack U without having to make RasPi compute module backplanes.
>>
>>80358430
Cheers for the reply. It one of these: https://www.toshiba-storage.com/products/toshiba-internal-hard-drives-x300/ a 4TB one in RAID1. Data integrity isn't really important on this RAID1, it's just my CD collection ripped to OGG. So if I can get away with leaving it until it completely shits the bed, I'd rather do that.
>>
>>80340885
Make regular snapshots or use ZFS
>>
>>80355785
Use duckdns
>>
>>80356284
Well, copy them over using another method, then checksum and compare the results yourself?
>>80356285
Not necessarily, the Micro ones can be super efficient.
>>80356294
These are quite different types of software. If you need real-time sync then Syncthing, rsync if not.
>>80356511
kek
>>80357121
>>80357325
Great writeup, thanks anon.
>>
>>80353682
so do i and it's rubbish
try actually using it then come back and tell me it's "rock solid"
>>
>>80358586
>https://www.toshiba-storage.com/products/toshiba-internal-hard-drives-x300/

I don't see anything from a quick glance at the spec sheets, but that's not unusual for non enterprise drives. I wouldn't say most, but a solid portion of them are 512e drives these days. A quick look at images makes it look like the 4TB model is thin, so that means very few platters (bigger models just get more platters), which makes the probability of it having 4k physical sectors higher. Not absolute, but if I had to bet one way or the other, I'd go with a single bad 4k sector on a 512e drive.

If there's no change it was probably just a random dud sector. It happens. I've had drives reallocate a couple of sectors in the first month of operation, and then happily run for 8+ years. Just keep an eye on it for a bit.

>Data integrity isn't really important on this RAID1

Fair enough. Again, just be wary if the reported numbers start to fluctuate in the next month or two. If they do, you don't have a good way of knowing how many errors the drive is failing to report. Pulling the disc in that situation can be better than letting it passively pollute your RAID, at least assuming you're not running something like ZFS with built in error checking/correction.

>>80358756
>Great writeup, thanks anon.

No problem. If you have other questions I can try to answer them. I'm far from the most knowledgeable about ZFS, but I've worked with it enough to understand some of it's shortcomings. It has a lot of annoyances, but it's unfortunately still the only good out of the box solution that doesn't require enterprise level budgets for data safety. RAID just isn't good enough because of write hole problems unless you have some other means of doing data checksums (like torrents do).
>>
>>80359366
Thanks again. It's in a Synology and ext4, so no ZFS. Makes sense about pulling it early to prevent pollution. Will monitor it daily for the next month or so and see if the sector count climbs.
>>
ClouDns, Namecheap or Jewflare?
>>
i just did put together a machine from old scraps and 4 almost new 3tib hd (plus 1 hd for the system), only to realize that btrfs has issues with >8tib filesystems on 32bit architectures.
what cheap mobo is there with at least 4 sata ports?
>>
>>80340680
Is there anything that looks like /synology enclosures but allows you to use your own hardware?
I'm currently waiting on email from U-NAS for the NSC-810, but it's uglier than sin.
>>
>>80359712
Orico do their own.
http://uk.orico.cc/goods.php?id=6902
CPU and that is shit, but it looks like it's just a mini itx system, so you could gut it and put something better in, if you can find one.

If I had to guess, it's probably the same chassis manufacturer that producers the U-NAS, but it's tarted up to look sleeker.
>>
>>80359712
>>80359832
And if you want a tower, Supermicro do a 4 bay that is used by Free/TrueNAS. They also do an 8 bay, but I don't think it's ever been offered retail (only iXsystems seems to have them) and I've seen people complain about managing temps in the 8 bay ones.
>>
>>80359712
buy old used syno case
>>
I have several 500GB drives in my workstation and I'm getting a 4TB drive. I want to move them all to a FreeNAS node. Do I just add them all to a single VDEV/pool and set it to RAIDZ1? Or do I need to do some kind of additional configuration because the drives are different capacity?
>>
File: 1603658097170.png (66 KB, 334x256)
66 KB
66 KB PNG
>>80354093
I suggest using something like iperf to check. That way you aren't limited to the speed of your disk.
>>
>IF YOU WANT PURCHASING ADVICE:

>State your BUDGET.
$1,000 USD not including hard disks. (I already have a 5950x.)
>List OS and software.
I'd like to use Proxmox/ZFS. For software, I plan on using this for Plex(or jellyfin, not sure) Blue Iris or Shinobi, Sonarr, Radarr, Mail in a box, Home Automation (without alexa, google, etc) any suggested software for home automation? I'd also spin up some VMs for linux learning, testing new distros. I'd like pfSense (how reliable is this in a VM than hardware?). I'd like to setup the ability to VPN into my house from my phone/tablet, so I was thinking OpenVPN? Other ideas I wanted to do was whip up a file server, running adblocker like pihole (would this interfere with pfSense..?)
State storage/raid and # of disks.
>List hardware features (low power draw, 10G Ethernet, etc)
I would like low power draw, however, I have a spare 5950x. (No, I did not buy multiple, I was gifted this by Amazon. They sent me two with one order. I did buy one for my personal pc) Should I not use this chip? If not, what chip should I consider?
I also would like a 10G ethernet connection to all my pcs from this server. How costly is this?
>>
>>80361860
>State storage/raid and # of disks.
I have nine 14TB drives, which, should net me 114 TiB (I want 120 TiB, so Ill probably grab another 14tb drive before this is all said and done). I wanted to do ZFS but I also plan on getting 1 or 2 m.2 NVME drives to use as cache/level 2 arc.

Are there any caveats that one needs to consider with this setup? I've read that intel chips are recommended for Plex for something but I forgot what. Would I be SOL going with AMD for Plex?
Also, read earlier about issues with ZFS and torrenting. Is this a true hurdle?
>>
File: file.png (146 KB, 1160x801)
146 KB
146 KB PNG
>>80361860
I can answer a few questions

>any suggested software for home automation?
Home Assistant, hands down. Run a VM with Debian/Ubuntu and install the supervised version (docker version). Do not use HassOS, it is a nightmare to work with and not worth it.

>I'd like pfSense (how reliable is this in a VM than hardware?)
Super reliable. I have my VM running pfSense since January 2020 and never had a problem

>I'd like to setup the ability to VPN into my house from my phone/tablet, so I was thinking OpenVPN?
pfSense has an OpenVPN server/client built in, really easy to configure. Also has Wireguard too.

>running adblocker like pihole (would this interfere with pfSense..?)
pfSense has a plugin called pfBlocker that acts exactly like Pihole and feeds directly from pfSense unbound DNS

>I also would like a 10G ethernet connection to all my pcs from this server. How costly is this?
The most expensive part is getting a switch that supports 10G. Look for switches with SFP+ ports (SFP ports aren't 10G)

> I've read that intel chips are recommended for Plex for something but I forgot what
Quicksync is the technology. However, if you have a GPU, Plex uses that instead (you'll need to buy Plex Pass for that)

Pic is my setup. It works flawlessly.
>>
Is it safe to buy my drives off amazon? I've had issues in the past, but now I don't want to go to the store and just want it shipped to my door again.
>>
>>80362022
Anon! Very nice!
What are you running on Home Assistant?

>all this info on pfsense
Dude, thank you. This is great.

Any pictures of your server rack?
>>
>>80361635
>>80354093
I realized the wifi card isn't being recognized and thought if I could get it working then that'd be fine. It's a Broadcom BCM4312. Tried following some tutorial to install the drivers but it didn't work. God damn it.
>>
>>80362150
>Any pictures of your server rack?
It is just a simple 4U case with a dual NIC, nothing out of the world.
I'm planning on getting a rack and a multilayer switch after renovating my home in about 3 months.
>>
>>80362512
What OS do you have?
>>
>>80362644
I just installed openmediavault 5.5.11 on it. Tried to follow this tutorial https://www.youtube.com/watch?v=JW0f3NOjWys but got some errors.
>>
>>80362712
Ok I just saw an error message when turning the thing on that tells me to go to https://wireless.wiki.kernel.org/en/users/drivers/b43 to get the correct firmware. According to the page my wireless is supported. Let me see if I can get this working. Wish I had seen that message earlier
>>
>>80357121
>>80357325
A bit confused, can you make it a little more brainlet friendly lol.

It sounds like in the end, it really doesn't matter. It's just a performance loss?
Or is it actually putting a LOT of wear on the drives?

If I recall what I read, it was something about pre-allocating as well as where you download the files. I thought I remember the solution being, download them to one folder, like a default download folder, then manually move them to your "Movies" or "Music" folder.

I can't entirely understand how that makes sense, as to why it would matter where it is being downloaded. So I may be remembering that wrong.

The pre-allocation seems like it may be of concern however as you also stated. This really isn't an issue though, because it can be turned off in the torrent client.

You also state cache setup, are you suggesting to download to the cache then move..? or?
And what happens in theory if you aren't using a cache setup or it's very small? Is it a matter of how much cache you have?

Idk do you have any reading material or videos about torrents on zfs specifically? I found a forum which is a bit confusing, but not much else.
>>
I want to do a home server, however, I am worried about a few things:
1) My entire house is carpeted except bathroom and kitchen.
2) The next alternative is my garage which is not heated but has great low temps.
Would a server rack be okay on carpet?
>>
>>80363283
>>80357121
>>80357325
Ok, ignore my previous reply for the most part.
I found what I had originally read.
Pic related, Michael W. Lucas - FreeBSD Mastery: Advanced ZFS

This seems to make a bit more sense, it seems to be basically suggesting download the torrent files to one directory that has different settings, and then afterwards you can move them.

Now I do question, if I move them out of that main directory, am I going to have problems while SEEDING those files from the other directory?

For example, if I download to Dir1, then move them to Dir2, Dir2 is where they will reside, but they will still be attached to my torrent client seeding from Dir2. is this fine? I see no reason why it should have issues, seeing it's transmitting not receiving, there's nothing to copy. But I wanted to ensure?

Because if seeding from "dir2" will be an issue, that's going to cause issues with my planned setup.

Also still open to more reading material or videos specifically about torrenting on zfs if you have any.
I would also suggest you add a little note about this into the wiki :)
>>
File: 1605528584467.png (33 KB, 594x564)
33 KB
33 KB PNG
>>80363430
Fuck I forgot the picture
>>
>>80361378
Anyone?
Can I even use any kind of RAIDZ if I have 1x4TB drive and 3x500GB drives? Do they have to be the same capacity? What are my options here?
Spoonfeed me pls.
>>
I am just starting to get into the home server hobby now that I feel fairly comfortable with the command line. I have repurposed an old gaming rig into an Ubuntu server. Something I've been thinking about recently is that it would be nice to be able to remote into the server and have a full desktop experience.

I like using Ubuntu but I've never been able to switch my main desktop entirely because I play too many games. I think the ideal solution would be remote desktop so that both windows/Ubuntu would be fully accessible at all times.

I was reading that all I would have to do is to install a desktop environment and then install xrdp and maybe open some ports on the firewall. Has anyone done this? Is it really that easy?
>>
>>80363119
Well the wlan interface shows up in the network tab now. How the hell do I actually use it?
>>
>>80363701

Try anydesk. No need to open any ports at all. It’s not open source tho.
>>
>>80362022
>Quicksync is the technology. However, if you have a GPU, Plex uses that instead (you'll need to buy Plex Pass for that)
The pass is required for QuickSync too, since it's also hardware based transcoding.
>>
File: cat.gif (1.92 MB, 368x368)
1.92 MB
1.92 MB GIF
Is plex future proof? Im starting to agree with the floss faggots on this one. The main issue with leaving plex is how good plex is compatibility wise, there is no good replacement. Jellyfin is not nearly as simple to setup remote sharing with the tech illiterate.


I just see this company selling out in the future in a bad way. Like fucking over the user base legally.
>>
File: muat turun.png (2 KB, 151x151)
2 KB
2 KB PNG
>>80340680
how do i
>>
>>80346971
Power is super cheap where I am, and there's not a huge difference between a big CPU and a small one when idle, anyway. And it is very nice to have resources in reserve if/when I need it. Compiling is really a lot faster on 24 threads, and restoring from backups over 10Gbe is a breeze. Never have to worry about networking a LAN party with 48 ports and a NAS cache. Kind of the same reason people buy sports cars even though you're not allowed to drive them at their potential, it's just fun. I do however have basic critical networking services staged on a low power device to take over an run on a UPS during power outages.

>>80349249
The easiest thing would probably be to run it as a Docker container and attach to it whenever you wanted to enter commands.

>>80352449
I use Heimdall which is just a standard links page but is prettier with icons/tiles.

>>80356284
Use something based on torrents, like Syncthing, which was much faster for me than rsync. Checksumming is built into the protocol and it is designed for massive bandwidth.

>>80356294
Depends on what you need it for, but see above.

>>80358438
They've existed for a while...just pretty expensive and I can never seem to find them 2nd hand.

>>80361860
>I also would like a 10G ethernet connection to all my pcs from this server. How costly is this?
I got a 48 port 10Gbe (copper) enterprise switch for $700 a couple years back. You can get new 10Gbe PCIe cards for under $80, and second hand for pretty cheap. Fiber is more expensive for the home when you consider needing to rerun wiring, and the fact that even 2nd hand transceivers are still pretty pricey. Alternatively, if you have just a few PCs to connect to, you could buy a couple 10Gbe NICs and directly attach them to the server and have it route as well.

>>80363305
Carpet shouldn't be a problem. Unless it gets really cold/hot in your area the garage should also be fine, might consider if pests could become a problem though.
>>
File: 61JgZIqDFNL._AC_SL1100_.jpg (95 KB, 1100x1100)
95 KB
95 KB JPG
12 5.25" slots on the front. IDK if i trust "Anidees" but might be worth considering for those that want to stick a bunch of hotswap bays on the front. Seems easier than trying to source old pc cases.

https://www.amazon.com/dp/B081M3JJ5C
>>
>>80361966
>intel chips are recommended for Plex for something but I forgot what.
Probably QuickSync, which is a hardware x264 en/decoder built into the CPU. Not a necessity but you'll need to make sure your AMD can support the number of simultaneous transcoding streams that you need it to.

>>80362040
Really depends on what exactly your fear is, but it's reasonably safe, as much as anything else.

>>80363701
It is really that easy, and there's lots of different software packages for remote desktop that will work for you. As an aside, using SteamPlay/Proton/Wine on Linux allows you to play a great many number of Windows games without any issues and might be interesting to you.

>>80364246
>Jellyfin is not nearly as simple to setup remote sharing with the tech illiterate.
How so? I just give them my Jellyfin URL and that's all they have to know. Or do you mean having them host themselves? Then, yeah, sure, but have them pay you in money/pizza/beer to set it up for them.
>>
File: std.png (32 KB, 785x835)
32 KB
32 KB PNG
>>80352449
if you use traefik and a single host this can be pretty useful https://github.com/ms-jpq/simple-traefik-dash kinda just werks
>>
>>80364734
>https://github.com/ms-jpq/simple-traefik-dash
ty, I was trying to find a dashboard that does exactly this. Surprised none of the big ones have implemented automated Traefik parsing.
>>
>>80364616
>you will never be early 2000s chad burning CDs with pirated games 12 at a time to sell on the local market
>>
I got a cheap DS380 and want to chuck a low power system in it that will basically just be for storage and maybe some really basic media streaming, but all my local options for ITX mobos whether server or not, used or not, are rather expensive. Would I be retarded to get a chink board off Aliexpress?
>>
Difference between NAS drives (Red, Ironwolf, N300)
vs
'Enterprise' drives (Gold/Ultrastar, Exos, MG)
vs
'Performance' drives (Black, X300)

for performance
>a) in a NAS RAID
>b) as a standalone drive
?
>>
>>80365375
Probably, aliexpress doesn't have that many recent boards. If you can find a current ryzen apu or Pentium or something like that, that's good of course.
>>
>>80365393
It deoends. But just shuck large externals for good cost per tb/drive and Linux md RAID6 or equivalent.
>>
Reading from the wiki as a beginner, and I still am not sure what my beginning steps are, nor what all the acronyms entail. It seems like a prebuilt NAS isn't efficient, so the best way to start would be to repurpose an old PC to use as a server before trying to actually make a server from scratch, correct?
>>
>>80365521
Just get an odroid n2 and connect drives/run services if you have no particularly large needs. Or a current power efficient pentium/i3/ryzen apu or such.

YMMV if you have a not too old still power efficient machine but I'd not seek out one myself.
>>
>>80365521
yeah, it's really just that you want to have a computer that's running a bunch of SERVER programs whereas the computers that you typically use are running mostly CLIENT programs.

you could try and find a tutorial on youtube to walk through getting something very basic set up.
>>
>>80349249
You can use screen or tmux to run the server in a detachable terminal
I have no idea if this is good practice but it should work
>>
>>80349817
HDDs slow down over time, I have some 10 year old HDDs that do like 60% of their top speed, it craters much more if there's reallocated sectors
Also, old HDDs where quite slower than modern HDDs, back then 50 MB/s was real fast, laptop HDDs were somewhat slower too
>>
>>80364004
Thanks for the recommendation I'll try it out. Might be faster than xrdp.

>>80364663
Thanks for the reply. I really wish I didn't have to work today, I want to try this now. The idea of having a remote Ubuntu desktop is really exciting for me. I had usually installed Ubuntu flavors, so the fact that I can now install a DE from scratch is extremely exciting. It would be so cool to try out different DE's like this, although I probably will run into complications when trying to move from Mate to something else. Maybe a VM would be better for trying out different flavors.
>>
>>80365375
What is your budget.
>>
>>80365393
Doubt you're gonna feel a performance difference in a RAID setup between any of those.
>>
>>80363849
Well I installed OMV on a less old laptop I had around just to check and even that one only has around 10MB/s transfer speed too
AAAAAAHHHHHHHHHHHH
it's not the hard drives.
It's not the cable
It's not router
it's not the NIC
Just fucking kill me
>>80365695
Both hard drives I tested work fine but only slow down when I put them on a laptop with OMV installed and transfer something onto it from another device. Happens with two different laptops.
>>
File: Capture.jpg (20 KB, 633x142)
20 KB
20 KB JPG
>>80365962
This is what I get from these commands on the OMV. Putting the hard drive on my PC lets me get 100MB/s transfer speed easily. Only thing it could be is some sort of config on OMV or something. I'm at my wit's end here.
>>
>>80365848
I don't really have one, essentially as little as possible while having a functioning unit that serves the purpose I want (storage, torrenting, basic media serving). I'm not like third world pressed for money or anything, just stingy and uneducated on NAS-oriented hardware choices.
>>
I've gotten a hold of an old ibm 7147H8U u1 server. Anyone know of a quick guide to reimage the thing. Been a while since I've mucked with rack mounted servers. Or should I just dig up the manual and go from there. The plan is just to slap ESXi on it.
>>
>>80366051
SuperMicro starts at $200.
>>
>>80366004
Wait a minute. Is the "100Mb/s" here referring to Mbps or MB/s?
>>
File: 1601586519599.jpg (50 KB, 960x667)
50 KB
50 KB JPG
>>80364616
>Tempered glass
I was onboard until this. I'm not buying a case with a fragile side panel I'm liable to put my foot through. Especially for a fucking server. Fuck TG, fuck zoomers, fuck case manufacturers, and fuck jannies.
>>
File: wayne.jpg (219 KB, 408x408)
219 KB
219 KB JPG
Friends I am trying to run a very simple barebones apache server off of my windows machine and it appears to serve files perfectly well over localhost but when I try and connect through my actual domain it lags a lot and only sometimes serves files, othertimes just times out and chucks an error. Can I get any guidance or advice? Not really sure why this is happening. It was working before I moved and so my router is different but I'm still not sure what the actual problem is, especially if it sometimes works and sometimes doesn't vs not working at all. I mean clearly my domain is routed correctly, right? And surely the router must be port forwarded properly since it intermittently works, right? And nothing changed in my server config...

Any ideas? Thanks.
>>
>>80367247
Anyone know? I'm so tired...
>>
>>80367429
b = bit
B = byte
>>
>>80367375
I have this same issue with my server on my local network. I think the issue is caused by the fact that it's on your local network. So for some reason it understands the local host path but sometimes fails to resolve the domain to the local path.

I never found a fix (please help anons) but my workaround was to connect to a VPN which fooled my computer into thinking I wasn't on my local network so that it would always resolve.
>>
>>80367496
Well hang on a minute, are you saying you WERE able to connect properly with external machines NOT connected to your same local network as the server? I have yet to try that, but I guess it might be worth a shot...? Either way fucking annoying.
>>
>>80367594
Yes, external machines were able to connect fine but on my local network it would intermittently fail. As soon as I connected to a VPN I was able to always get it to resolve. I would check to see if external machines can connect on yours, might be the same issue
>>
>>80367630
So I tried using my phone and disconnected from the network, surprisingly it actually DOES work... however, it's now consistently slow. In the sense that it loads fine with no hiccups but literal test PICTURES I can watch slowly load vertically like it's 1995 lol. What gives? Before you ask ya boiii has gigabit and this machine is hooked up over wire.
>>
>>80363283
>>80363305
OK this turned into a fucking novel.

>It sounds like in the end, it really doesn't matter. It's just a performance loss? Or is it actually putting a LOT of wear on the drives?

It's both.. sort of. Your typical home use case doesn't involve a shitload of stuff with their media server. If you're doing performance intensive stuff, this may start to become a problem. For raw bulk media storage it doesn't really matter that much in most cases unless you're living in a frat house or something and have dozens of people accessing stuff constantly. Even then, you'll probably run into network limits long before you run into drive throughput limits if you have more than a few discs in your pool.

It does massively amplify the writes to the drives, as well as reads, but ONLY while you are downloading the files. Unless you're constantly redownloading new stuff and deleting the old stuff, this shouldn't be an issue, but if that's your use case, just skip ZFS because you don't care about data retention anyways. Again, for typical home use, none of this is a problem unless you're buying really shitty drives.

>I thought I remember the solution being, download them to one folder, like a default download folder, then manually move them to your "Movies" or "Music" folder.

Yeah that's sort of a solution. The easiest way to see what happens is to try for yourself. Setup a simple ZFS system and limit its size to be like 50GB. Download 30-35GB of torrent data, and then try to copy that data elsewhere. You'll see fragmentation through the reduced throughput.

Now, delete the original on your ZFS pool, and copy it back. Try copying it off the ZFS pool again. You'll get more or less linear throughput.

>The pre-allocation seems like it may be of concern however as you also stated. This really isn't an issue though, because it can be turned off in the torrent client.
>>
>>80367803

Preallocation just tells the file system to reserve a chunk of drive space for data to be written to. Your typical file system will try to make this a contiguous, or mostly continuous chunk to prevent fragmentation. Your typical file system also allow you to directly manipulate a file with no form of error correction or validation.

ZFS doesn't behave like this. ZFS does not allow existing data to be overwritten until it has completely validated that the updated data has been written correctly. (As an aside, this is also why snapshots are so easy on ZFS. You can just tell it to take a snapshot, and it will preserve the current blocks and a set of block pointers to them. This means that you can "look back" to old versions of things by simply looking at and old map of the files) It does this in a manner that is transparent to the applications using the file system because ZFS is designed not to trust anything (except your RAM). It doesn't trust your drives. It doesn't trust the applications telling it to write data. It checks everything at every step of the way and if it finds an error, it tries to correct it.

Bittorent may write to a block literally dozens or even hundreds of times (if the block size in ZFS is big enough). This means that ZFS will silently and dutifully duplicate that block every time a write is requested, even if it's only 16k, and then validate the new block before updating the block pointer tree to point to the new location for the data.
>>
>>80367342
i mean you're right, but I still dont see any other options for modern cases with 5.25" bays.
>>
>>80367823
Copying the files is a linear in and out action. ZFS sees a stream of data coming in, looks for a place to put it, dumps it there, and validates it. It tries to be relatively linear when doing this. If you have compression turned on, it will try to compress the data as it comes in. If you have deduplication turned on, it will scan the block tree for data with identical hashes and simply add block pointers to point to the existing data, and then dump the rest in as linear of a fashion as it can. (Deduplication intrinsically results in some degree of fragmentation).

>Now I do question, if I move them out of that main directory, am I going to have problems while SEEDING those files from the other directory?

Some torrent clients will insist on revalidating the data if you point them to a new directory. This is fine, albeit redundant since ZFS validates the data too. Seeding is purely read operations. The data isn't being moved or copied around on the disk. ZFS will validate (and where possible correct) each block of data before giving it to bittorent. Again, this is entirely transparent to bittorent or any other application requesting the data. The fact that bittorent may want to validate that data itself is irrelevant and redundant. Seeding off of a ZFS pool is completely fine. Downloading to a ZFS pool is where problems can occur, hence the idea of copying stuff to a different dataset and routinely purging the contents of the temporary one.
>>
>>80367496
I think you need to activate NAT Reflection. Basically tell your firewall that you trying to connect to your domain name will always be through your local network (if you are in your local network, of course)
>>
>>80367844
>You also state cache setup, are you suggesting to download to the cache then move..? or?
I'm talking an SSD cache for L2ARC in ZFS. I'll be honest, I'm not entirely sure how much it actually helps with fragmentation, but unless you're short on cache space, ZFS seems to handle most of the rewrites to the cache, and then relatively intelligently evicts completed cached items to disk. Since I'm not writing huge amounts constantly, the cache lingers for long enough to prevent large scale fragmentation. But that's my use case, not yours.

I wouldn't recommend bothering with L2ARC, or at least not a large one, for just a bulk media server. I use one on my home box. I need it to get acceptable write performance with deduplication turned on for some of my data (I have a 14TB set of files that dedups and compresses down to about 3TB, along with some other stuff where I save a few TB). Dedup is something I would never recommend you turn on unless you're willing to throw hardware at the problem or accept horrible performance, and for a multimedia server it's mostly pointless since you won't have much, if any, identical data in a video/music library. Dedup has to do a full scan of the block pointers to look for ones with identical hashes to the block you want to write, so it either needs to store a couple of GB per TB worth of metadata in RAM, or it needs to scan through a fast SSD, which degrades the write speeds (not enough to matter for my use case). Scanning through several GB/TB off of a pool of standard hard drives makes dedup nearly unusable on large arrays. We're talking single digit MB/s writes. Read performance can still be excellent, but if you have to scan through 50GB of data just to write a couple of megs every single time, it's going to perform like shit.
>>
File: 1604958967726.jpg (164 KB, 518x448)
164 KB
164 KB JPG
>>80367841
It's a perfect storm of bullshit and I hate it.
>All the new old stock cases that had 5.25" bays are disappearing due to everyone and their mother getting into PC gaming because of chink flu lockdowns
>All the new cases are tempered glass meme accidents waiting to happen
>Any new case will have a minimal number of 5.25" bays because "optical media is dead lmao just buy twenty different subscriptions for $9.99 a month plus tip"
>Only real options going forward are external SAS devices and USB optical drives
Maybe one day when I get more confident with power tools and have some money to splash around on making fuck ups I can build my own perfect vision of a computer case to shove all my meme optical drives into so I can rip DVDs at light speed and make mdisc and old tape backups like the memelord I am.
>>
>>80363283
>>80363430
>>80367866
>Also still open to more reading material or videos specifically about torrenting on zfs if you have any.
I don't have anything specifically torrent related, but you can certainly look up optimizations for databases. The difference between torrented media and databases is that stuff like movies/music don't compress well/at all, whereas databases frequently compress quite heavily, but they overlap in that databases are frequently doing tons and tons and tons of small writes, which when coupled with their own internal management engines can cause gargantuan fragmentation on ZFS.

>>80365393
My experience with NAS drives is that they're just enterprise drives that failed some internal binning procedure and got a special firmware that gimps them so they don't destroy themselves as a result. Some of them are lower RPM which is more power efficient, but they're still more expensive.

You're paying for the "data recovery" service on a NAS drive. Which if you have a properly redundant array should never be a problem. That extra warranty is fucking worthless if you ask me. It takes forever, when any decent home server system can just rebuild an array. Enterprise drives are designed to be hammered constantly all the time. For home use you can probably get away with consumer/prosumer drives, and you'll probably save a few bucks in electricity over the years.
>>
>>80368003
Personally, I won't touch a drive that doesn't have a 5 year warranty. I've had too many drives die on me over the years. I also won't touch WD reds with a barge pole. The early ones were little more than rebranded greens you paid a premium for so they still made a profit when you RMAd the thing 7 fucking times under warranty, and the SMR bullshit recently means that you have to be incredibly careful what you buy. To me there's almost never a good reason not to buy enterprise drives unless your doing low end budget stuff like shucking externals. Yeah, a good system can rebuild as I already said, but I'd prefer to spend a few dollars more to reduce the odds.
>>
>>80367486
Alright so I was retarded after all. Both laptops are transferring files at the speed they're supposed to. Altho the first laptop's NIC for some reason was listed as 10/100/1000 in the manufacture's website. Assholes. An entire day wasted for nothing. Oh well.
>>
File: baskets.jpg (122 KB, 800x600)
122 KB
122 KB JPG
Is there such a thing as a cheap PC case with 4x+ 3.5 drive capacity?
>>
>>80368168
Used Fractal.
>>
>>80368192
Every time I go looking for NAS cases, it always keeps coming back to the Node 304.
I don't even need 6 drives, but fuck it no other design seems to acknowledge the concept of a small factor NAS box.
>>
>>80344502
Bringing a hooker to your house is terrible opsec.
>>
what are good access points to buy right now?
i'm staying away from ubiquiti since they're becoming infested with telemetry and there doesn't seem to be much more choice.
>>
Any Z97 boards with 10GbE?

Probably just going to put a card in, but if it's basically the same price as a used board on ebay, I may as well just grab a spare board.
>>
File: jut3dllgnui61.jpg (424 KB, 2592x1936)
424 KB
424 KB JPG
How much would it cost to buy/build a small factor low-power server like this?
Also, would it be a bad idea to also use my server as a router or should I just keep a dedicated box for it?
>>
>>80369020
>terrible opsec
Depends of what the 'op' is. If the op is having sex, then pretty much mission accomplished, no?
>>
File: joe hookers.jpg (202 KB, 1777x742)
202 KB
202 KB JPG
>>80369739
If you want to just have sex plenty of hookers offer incall. If you have one come to your home, pretty much every other security measure you use to mask your identity is useless because she has your address. By texting her your address, you have also let a ton of other people including but not limited to Google and the alphabet agencies that you're fucking hooker. You have also let a criminal (or, in the case of countries where prostitution is legal, a shady character) into your home. She might not steal anything then and there - although she might - but if you have expensive stuff laying around or you live in a nice place which looks like you have expensive stuff, she might send her thug boyfriend there afterwards, or let it slip that she went to a client's place in X location and he has expensive stuff. The chance that this will happen is low, but the potential downside is quite awful.

tldr don't fuck hookers at your own home
>>
>>80359522
I use cloudns, dns.he.net and 1984.is as slaves with a hidden bind master. Cloudns is a bit slow on zone transfers but except from that it works perfect.
>>
>>80369487
10Gbit dual port cards are 20$ on ebay e.g. Solarflare im not sure if you can get a board cheaper.
>>
>>80363468
>>80368003
Did you have any input in regards to seeding from the directory you move the file to?

>it seems to be basically suggesting download the torrent files to one directory that has different settings, and then afterwards you can move them.

>Now I do question, if I move them out of that main directory, am I going to have problems while SEEDING those files from the other directory?

>For example, if I download to Dir1, then move them to Dir2, Dir2 is where they will reside, but they will still be attached to my torrent client seeding from Dir2. is this fine? I see no reason why it should have issues, seeing it's transmitting not receiving, there's nothing to copy. But I wanted to ensure?
>>
Redpill me on raspberry Pi cluster
>>
>>80370761
Addressed in >>80367844

Reading does not cause fragmentation in ZFS. I mean it technically CAN, but that's if ZFS detects errors, corrects them, and then reallocates that block to a different spot on the disk. TL;DR, seeding off of a ZFS volume is perfectly fine.
>>
>>80371251
In most cases the question is: why do you cluster them? If you need the power, three clustered x86-64 will probably be better than the equivalent number of rpi.
>>
Whats the best raid setup for media storage for plex? Not overly concerned with redundancy since the drives will just be full of shit that can be downloaded again
>>
>>80340680
Why does no one post their rig/setup/programs their running? Cmon. I come here to be inspired.
>>
>>80371814
Well for most bang for your buck capacity with a bit of redundancy for error correction, Raid Z1 or Mirror. Only if this is for plex content though. For a general purpose jack of all trades server I'd go with Raid Z2. For me though I'd hate to re-download or re rip my movies/shows. I've got about 5TB. I'd prefer never to have to do it all again.
>>
>>80372766
My hsg is a Rpi 2B
>I come here to be inspired.
the load average is 0
>>
Can i use pi-hole to block youtube ads on my smart TV
>>
>>80372970
It doesn't block youtube ads.
>>
/hsg/ what can you run on a rpi that's good for home security and privacy?
>>
And yes you do want ZFS for your plex server's storage. All those media files. All pristine now (no playback errors). Give it a few years on a non ZFS system and then watch as random "errors" creep in.. Then your media library that was your pride and joy turns into a big headache cause now you've gotta stamp out all the errors. Go with ZFS, live life enjoying your movies and your data without headaches. Hell you can be an 80 year old dude still watching that rip of Avengers Endgame that you originally did way back in 2019 when you were 22.
>>
>>80373041
>Hell you can be an 80 year old dude still watching that rip of Avengers Endgame that you originally did way back in 2019 when you were 22
I doubt the hard drives would last that long though.
>>
How necessary is it to buy the 5.1V power supply for the Raspberry Pi 4B?

Does a regular 5.0V wall charger for a phone struggle to power it?
>>
>>80369220
Well, if you wanted you could flash them with OpenWrt.
>>80372766
>rig
Dell 3040 Micro + 4x WD My Book 8TB.
>setup
Docker on Debian. Separate compose files per service category.
>programs
Pic rel. Now post yours, anon.
>>80372890
nice
>>
How do I get into PTP
>>
>>80372766
> rig
Picrel. Storage is 2 12tb drives in raid 5 (so I can add more drives later)
>setup
Docker containers defined in different docker compose files
>programs
See pic >>80364734
>>
File: 1605493867385.jpg (308 KB, 463x2880)
308 KB
308 KB JPG
>order new 22U sysrack cabinet
>amazon says expect it next week
>fedex said expect it tomorrow
>sysrack still hasn't shipped it
>fedex updated to pending
this sucks.
>>
>>80340680
gross
>>
>>80369723
>How much would it cost to buy/build a small factor low-power server like this?
Too much.
>>
>>80372993
Privacy: A lot of self hosted services, the usual we run here
Home security: Probably nothing really. Try a few kilometers of layered defense perimeter, don't forget modern air space defenses.
>>
File: hobbyte.jpg (41 KB, 425x309)
41 KB
41 KB JPG
>>80372886
>5TB
>Needing 2 discs of parity.

The rebuild times on a 4x2TB array is measured in a few hours with a 3+1 RAIDZ1. RAIDZ2 is for when you have very large discs and the rebuild time would be enormous. 5TB is not that much to redownload in this day and age.

>>80373041
For most home media server setups, ZFS is honestly overkill. Yeah it's definitely better than not having it, but torrents can correct errors by simply redownloading bad data. ZFS also has a lot of design restrictions that you have to account for. You can't freely expand an array the way you can with software raid solutions on linux for example. The only way is to create new vdevs, which involves multiple discs being purchased at once.

For small scale media boxes, it's overkill, and the annoyances are often enough to justify not using it.

I say this as someone who uses ZFS. If I had to do it over again, I'd pick ZFS without hesitation, but I don't recommend it to people looking to cobble things together with some externals an a pi.
>>
Anyone know some wizardry with FreeNAS to fix permissions?

I changed motherboards and upgraded version and now two drives with Windows shares absolutely refuse to allow anything in. Tried going back a version and no luck. I could put the board back in and transfer those drives content to an external but I really want to know a fix in case it happens again.

Only difference in the drives having this issue is they had ACL set and the others ran on the old Linux style permissions. Stripping does nothing and it was a dataset at the root folder of the drive which the ACL Manager won't manage. (long running server from many generations ago)

FreeNAS has always had bitchy permissions. I remember in the past with the Linux style ones if I went in unchecked write saved, then rechecked it and saved it would fix most of the issue. But nothing seems to work. I doubt its anything in Windows being an issue since nothing on the Windows machine as changed and all other drives migrated without issue.

Been a zfs fan for a long time but holy fuck I'm near to switching to Windows server because this is rage inducing.
>>
File: 1596782000512.png (285 KB, 1346x365)
285 KB
285 KB PNG
>>80375599
[citation needed]
>For most home media server setups, ZFS is honestly overkill.
Literally how? it can run on basically anything, needing 256GB of ram is only true when you are looking at hundreds of active users with hundreds of TB sized zpools
>but torrents can correct errors by simply redownloading bad data.
"lmao just redownload it bro" is not a fix for a failing hard drive, ZFS allows a home user to build an array with a high level availability and redundancy with commodity components(no raid controllers, no need for enterprise class disks)
>ZFS also has a lot of design restrictions that you have to account for.
such as? yeah a massive raid0 NTFS volume is worlds better than individual datasets, per dataset recordsize, per dataset quotas, full posix permission compliance, snapshots, compression, deduplication, sync write and read caches, ARC, etc
>You can't freely expand an array the way you can with software raid solutions on linux for example.
you don't need to, nor should you want to, what are you expanding exactly? if you have completely filled every nook and crany of an array it's because you have exceeded the capability of the array, adding disks doesn't magically give you more space without significant compromises on data integrity, this applies to ZFS and any other raid solution.
>For small scale media boxes, it's overkill, and the annoyances are often enough to justify not using it.
again you haven't provided a single example and i am telling you, no, zfs is perfectly applicable to this use case, if you are dead set on using embedded devices like a pi, a mirror vdev is still worlds above anything else you could achieve on such a limited platform.
>>
>>80372993
>privacy
Pihole to block ads and their embedded tracking. Any streaming shit you want (eg music) that the commercial option would be tracking you.

>Security
A camera
>>
>>80373290
Fast chargers do it ok but some ancient one you've kept in a drawer forever will brownout with the pi4.
>>
>>80375827
>what are you expanding exactly?
I guess the anon meant the amount of terabytes available for use.
>adding disks doesn't magically give you more space without significant compromises on data integrity, this applies to ZFS and any other raid solution.
What are these?



Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.