from MTZ@lemmy.world to selfhosted@lemmy.world on 09 Oct 23:23
https://lemmy.world/post/37132643
UPDATE: To everyone who suggested YUNO, thank you so much. This seems like it is about to make my journey much easier. It is basically almost exactly what I was looking for, but I was unaware that it existed.
Thank you ALL for your suggestions, actually. It’s a bit overwhelming for an almost complete noobie but I an going to look into all of the suggestions in time. I just saw that there were several mentions of YUNO so I decided to make that one of the first things I investigated.
So, about two months ago, I had a very eye opening experience. As the result of a single misconfigured security setting on my Android, I was locked out of my Google Account on my phone AND all of my PCs. I had no access whatsoever to Google, or any of the literally hundreds of services that I get through Google.
This is when I realized that I relied entirely on Google/Android because those two days were actually very difficult, being cut off from media, services, passwords, everything, from the past almost twenty years of my life, could be taken away from me in an instant. The decades of my life that were locked away in my Google Account included hundreds of thousands of pictures, almost a hundred thousand audio tracks, several hundred books, several hundred apps, thousands of videos, etc. ad infinitum. Unfortunately, very little of this material was backed up at that point. That is my fault. Also, the misconfigured security setting was my fault as well.
The amount of data, media, memories, services, etc. that would have been lost is actually endless and it would have affected my life in several ridiculously negative ways.
Luckily, in the end, I was able to get my access back and then basically immediately grabbed all of the several terabytes of information and media of mine that they had, and that I was almost locked out of. I have it all in my house now on a drive in my computer, with a backup made on another disconnected disk.
I then decided that no corporation was ever going to have such an insanely high level of influence on and control over my entire life and my media ever again. That experience was actually very scary.
I’ve been trying to get into SelfHosting, but am finding it quite daunting and difficult.
There is a LOT of stuff that I have to learn, and I am mostly unsure of where to even begin. I know basically nothing about networking.
I need to learn the very basic stuff and work my way up from there, but everything that I’ve seen on the Internet assumes that the reader already has a basic to intermediate understanding of networking and the subjects that surround it. I do not, but I am going to learn.
I just need someone to show me where to start.
Thanks in advance for any assistance!
#selfhosted
threaded - newest
I’m fairly technical but I honestly don’t know where to begin either. Also trying to improve our personal security to an extent.
Hope you get some answers
Thanks! I hope you do, as well!
The FUTO guide is meant for total beginners.
Thank you! I will begin to look over it tomorrow!
I am a sysadmin with over 30 years of experience managing servers and networks for businesses of all sizes as well as for myself, friends, and family.
The FUTO guide is extremely detailed, accurate, and accessible. It does not always follow best practices, and it’s not a comprehensive guide to all of the possibilities for self-hosting. It’s not trying to be. It is a guide for someone with no technical expertise (but with basic technical ability) to degoogle/deapple themselves at a reasonable level of cost and effort.
You do not have to do everything in the list, you can pick and choose the parts you’re interested in. That said, I would recommend reading through the whole article as you have time, because it does a very good job of explaining the concepts involved in building a self-hosted setup, and understanding how everything works is the biggest step toward being able to effectively troubleshoot problems when they inevitably crop up.
If you have specific questions about things that aren’t answered in the guide or via a quick web search, post them here.
Thanks for the detailed explanation! I will definitely begin studying the document tomorrow!
Love the explanation. I’ve had a homelab for 20 years now and have never heard of FUTO. You’re explainer has made me bookmark the site now for future skimming.
This, so much. I remember when Louis told everyone about it, people (mostly Reddit) were so nitpicky over every minor detail.
I’d be curious seeing the discussion. Its healthy to have respectful competing opinions and see what rises to the top
It was a while ago, I don’t remember much of it unfortunately. Only the idiots.
I didn’t know I needed this. Thanks!
They use OpenVPN for some reason. Wireguard is superior in every way. In case you set up a VPN.
It may be for beginners, but this is not written by or for normal people! The "Why Build Your Own Sovereign Cloud" intro literally starts explaining how modems work and twisted pair wires...
Its very much written for somebody who wants to nerd out and fixate on the technical underpinnings of everything, not your average friend who wants to replace Google Photos.
Unless you have experience with ethernet equipment and such it is probably better to start with some hosted service of an open-source app like Nextcloud or Immich or (slightly more advanced) a VPS somewhere. Doing it immediately from home with your own server has a steep learning curve.
Thank you for the advice! After I get a firm grasp of the concepts at hand, I will look into NextCloud and Immich.
A single misconfigured thing can suck real bad as you’ve seen.
Selfhosting involves lots of things that can be misconfigured or go bad.
That’s not to scare you out of it out anything, merely to congratulate you in seeking knowledge first.
Disclaimer: I’m biased towards networks because I’m a network engineer, opinions may differ.
I would say… having at least a vague grasp of layers 1-4 of the traditional network model is a decent start.
You don’t need to understand everything, but knowing a minimum will help a lot imho.
It’s hard to point you in the right direction without knowing what you already know or not.
Plus, if you end up accidentally locking yourself out of your own system: boot access means root access (Secure your IPMI/iDRAC, folks!)
Yeah. I guess that is true. The part about not being able to point me in the right direction. I have a shaky grasp of several network protocols and things of that nature. Nothing deeper than surface level at this point.
Hello again, thanks again for the great advice. I have a question though. The traditional network model that you mention…is that also sometimes reffered to as the OSI Model? Are the two the same thing? I just want to make sure that I am studying the correct thing.
Yea, I was referring to the OSI model.
I think the very first step to building resiliency is to sign up for Proton’s cloud services. That will give you access to mail, both from Gmail via forwarding and a new inbox with a separate address. You’d also get a password manager and cloud storage. From there you can start self-hosting alternatives. Probably start with Immich as Google Photos is a big deal and it takes a ton of storage. Proton is a Swiss non-profit so the probability for enshitification is not nearly as high as with Google.
As soon as you have redundant storage, do a Google Takeout and download a full archive of your stuff. This feature may not be there for long given the current corporate climate.
Thanks, I will certainly look into this after I get some sort of basic understanding of the concepts at play.
Honestly? Don't do the whole switch, or even a big switch from a few services to another.
Start small. Very small. Try doing just one service you rely on, like your images or music. Immich just announced their first stable release. I use navidrome for my music. Make sure to test these on a copy of your data, not your actual data.
Once you've got one service working as you want it to do, then you can try your hand at another service. This way, you don't get stuck trying to do everything all at once.
It may be worth considering how much (if any) you want to spend at the start, too. That'll inform your next immediate task; setting up basic backups for your data. A spare drive is a good start, but it may be worth keeping another one at your parents house, or similar.
That is exactly my plan, to have this set as a long term goal with several incremental micro-goals, as opposed to attempting to do it all in a weekend. I figure making it a long term thing gives me much more of a chance to actually learn what it is that I am doing. Plus, at my level (no real networking knowledge to speak of) trying to do this in a weekend sounds like a nightmare, lol.
An “I Wish I Had …” addition to this: maybe start something like Wiki.js or Joplin that you will then use to document what you do as you go. Keep a record of your process so that you can work your way back if you need.
Great suggestion! I will definitely implement something like that.
Damn, that’s scary indeed! First of all, congratulations on your resolve to take control of your data. You have a long journey ahead of you, but don’t be discouraged, take one step at a time and don’t be afraid to ask for help.
As for where to start, I think you’ve already figured it out yourself: invest some time in learning the basics of networking. You don’t need to become an enterprise-level networking wizard, just learn the basics: learn what an IP address is, what a network mask is (sometimes also referred to as “prefix length”), what DNS is and does, how to change these settings on your home network and why you’d want to change them. Try stuff, break it, fix it, repeat. Also, if you’re not familiar with or already using it, it might be a good opportunity to pick up Linux. If you’re coming from Windows, a beginner friendly distribution like Linux Mint will do nicely. Try installing it on an old computer to see what it’s like, poke at it until you’re comfortable, then maybe make it your main operating system. Knowing Linux basics (command-line shenanigans in particular) will give you a big edge when you decide to start hosting your own services.
I’m not very familiar at all with Linux. I have used a few distros here and there where the majority of the work is done in a GUI. The CLI intimidates me, hahaha. I began exploring distros through WSL a little while back. I still haven’t “dove in” yet though.
As far as the networking terms that you used, I have at least a slight grasp on all of that and what the different network “components” do, but I do not yet know how to implement or configure most of the things that you mentioned.
This is an area where AI can be helpful. Tell the AI what Linux distro your on and what you want to do. Most of the time it will give you pretty good answer. If you don’t understand what it is telling you to do, ask it to explain the thing in detail. Most important thing though is to always verify what it tells you before you run stuff. Google search specific commands or use the “man” command to get documentation. The key thing is the AI can make you aware of CLI commands and tools more easily then trying to find what you need on your own.
Thanks! That is good advice and I will begin doing that to try to familiarize myself with the CLI.
There’s a GitHub repo called awesome self hosted. I would get the link but I’m mobile. It has a pretty exhaustive list of self hosted software with a link to their page. Might be worth loooking at.
edit: github.com/awesome-selfhosted/awesome-selfhosted
This seems to be an extremely valuable resource. I love the GitHub awesome lists, and I definitely starred this one as a reference point!
I had something similar happen with Google a few years ago. Even though I had my password and access to my email they decided I was trying to hack my own account and locked me out. Like you I immediately started to look for other solutions.
Syncthing file sharing is really easy to install and use. There are no ports to configure on your router and everything is encrypted in transit. I have my phone’s DCIM directory set up to sync to my home server and PC so new photos are backed up and available everywhere in a few seconds. I installed Syncthing intending to move to another solution eventually, but it works so well (aside from one or two files that occasionally don’t sync) that I’ve just stuck with it.
For passwords Keepass & KeepassXC work really well on just about every platform. I share the password file using Syncthing and in years of doing this I’ve never had a problem that I didn’t cause myself and those were minor.
You can get both of these up and running with very little effort and quickly limit your reliance on Google, then move to other solutions if you find they’d work better for you.
In the time since this happened, I have set up KeePassXC on all of my PCs as well as KeePassDX on my phone, and taken all of my passwords 100% out of Chrome.
I’ll absolutely look into SyncThing! I’ve heard of it many times, I just haven’t used it myself yet.
Thanks for the info!
Syncthing is incredible. I use it on my devices, and everything is also backed up to my NAS.
Second this, Syncthing rocks. Only ever have rare, minor problems in day-to-day use
That econdary drive I highly recommend you find a way to move that out of your house. For me I have a friend 8 hours away, we swap drives on occasion to keep each other’s backups in case of flood/fire/toddler or whatever other force of nature to save ourselves cloud backup costs
That’s a great idea. I’ve had a safety deposit box for years. I can just store it in there!
Safe deposit box is exactly the right size to hold a 3.5" HDD. Or several. I keep a backup Yubikey there too, because I love the physical token 2FA, but I’m pretty sure I’m going to lose it.
Check out YUNOhost - it’s pre-configured for you and designed for beginners. Mine’s been running for about three years on a VPS with no problems and I had no previous experience with self-hosting.
Definitely keep your files backed up locally though. No server is invincible.
I will certainly look into that. I’ve never heard of YUNOhost but I’m going to give it a look soon!
Do it! It’s madness that YUNOhost is not more well-known, considering what it does.
Install Virtualbox (or some software to handle virtual machines).
Install Debian (or some other OS of your choice, I won’t judge if you prefer Windows).
Update your OS (
apt update && apt upgrade -yon Debian).Take a snapshot of your VM’s current state after updating. Saves a lot of time if you mess up or want a clean slate.
Now you decide on what you want. Do you want to install n8n or Node-RED for automation? Do you want to use Immich for pictures? Paperless to save papers in a digital format? Audiobookshelf to listen on your books or podcasts? Jellyfin to stream your media? Set up a Minecraft or Factorio server?
Once you have decided on what you want to do, try to do it in your virtual machine.
Once you understand how to set it up and configure it to your liking, decide on how you want to host it. I took an office computer, added a few HDDs and replaced the case with a bigger one and it’s now my home server, but any old laptop will do. Just make sure to take backups.
I used to have a Dell R710 and a virtual machine for each service I hosted, but I have moved to docker because it as simple as taking the often provided compose file, tweaking it a bit (where to store data etc) and running it with
docker compose up -d.It’s a great idea to run it in a VM first, because I’m bound to make mistakes along the way. Awesome advice! I’m definitely going to be referencing your comment throughout the process! Saved!
I had a Raspberry Pi once and wanted to move file from the current folder to some other folder. I typed
mv /* /path/to/folder/and move everything in the root directory and down to this other folder.EDIT: Meant to say that snapshots are cheap backups. I ended up reinstalled the OS.
Correct would have been dot slash:
mv ./* /path/to/folder/Yikes. Before you dip into any of the self-hosting, take and get a WD Gold drive - from Western Digital directly (wd.com) - do NOT go through Amazon or NewEgg or any third party merchant. Send in the warranty that goes with it and register the drive (this is for covering the off chance it’s a DOA unit) Then get a good quality enclosure to pop the drive into and take your time and back up EVERYTHING onto that new HD.
Don’t use an SSD.
You want a spinning platter drive, as this is backup only, so once it’s full with all of your content, it gets dated and labeled and popped into a drawer for safe keeping. If you have countless terabytes of data, get more drives and swap them into the enclosure, date and incrementally fill. A fine tip sharpie to note what’s on the drive is fine, or if you’re obsessively anal about it, make a spreadsheet with that info… If your drives are kept dry and stored with care they will last for DECADES…
The truth if being honest here - I’m a data hoarder and most of the stuff I’ve tucked away since I first came online (in 1999) is now on drives that I maybe spin up once a year. I used to have the notion that it was critical that all my shit was accessible all the time and I ended up dropping money on networked storage… and over time, realized that as long as I knew where the files were, DID have the most important stuff - family photos and scans - tucked away not only in long term storage, but on multiple drives in multiple machines, (home, work, laptop) it was okay not have it served up instantly.
Just reading your post made me go cold inside - I can only imagine what you were going through until it got sorted. From a bonafide old school data hoarder… Please, back your shit up locally. Use enterprise drives.
Then sort a self-hosting soultion.
Which software do you use for backups?
I do my backups manually.
As I have run unsuported Mac installs for the last 20 years, I started a long time ago, automatically partitioning my OS drives and making storage volumes to work off of.
The storage volume in the computer will have subfolders for the type of data - music, video, photos, etc.
When my storage volumes fill, I will pull my latest backup drive out of storage, hook it up then go into each storage subfolder, sort by date and add everything that’s newer than what’s in the backup drive. (which is actually how Apple’s Time Machine backups work - incrementally sorted by date - but I’ve had this method since the start, so I just stuck with it)
I just make sure to take note of how many files/folders I’m adding to the backup drive and note what it has at the start, then at the end, as a double-check of it all, before I clear the storage drive on the computer. (I did not do this and lost almost a years worth of music rips, waay back in 2003. Rebuilt the music I lost then iTunes threw a wobbler and lost the library for me. FML…)
The longest backup will ALWAYS be the initial one if you’re dealing with a first time backup. The rest, once you work out how to organize your files, is academic.
What I’ve found is that your tastes will change, you grab content you think you’ll want to hold onto forever… and then years later, you realize it’s low-bitrate, low-resolution, too pixellated… whatever… and you decide to delete it.
With the software doing the backups for you - it’s too easy to just let it rip and go have dinner while it works and you end up with files that you’d otherwise get rid of. Part of being a data hoarder is not keeping everything forever. There’s a ton of garbage online. Tastes change as you get older… You want to curate that shit so you can keep what’s most important - like family stuff.
And really good porn.
Several detailed, easy to understand and very good pieces of advice! Thank you! I have definitely saved your comment for referencing throughout this process!
I really wish people would realize the level of dependency, and thus leverage, these companies have encouraged us to give them, before they learn it first hand.
Yeah, it was dumb. I should have thought about it long before what occured, but I didn’t. But, in the end, I definitely learned my lesson.
You’re not dumb, we are all being brainwashed into sticking our asses in the air and convinced we won’t get fucked.
get your password situation squared away! every time i spin something new up i am grateful to have a pw manager to keep it all unique and maximum character limit
don’t even have to memorize the user of a lot of em
That’s the only thing that I do have taken care of! I basically immediately grabbed them out of Chrome and put them in KeePassXC on my PCs and KeePassDX for my Android.
Baby steps!
@MTZ @SidewaysHighways You may wish to disable all browser password managers, on all devices and use an alternate method of password management that suits your needs.
This is a cautionary tale on browser password managers (amongst other facepalms) that saw about millions of people's personal details stolen - https://www.oaic.gov.au/__data/assets/pdf_file/0037/228979/Medibank-data-breach-alleged-timeline-infographic.pdf
I will probably get flogged by this answer but here it goes:
I’d throw you right into the deep end: get a spare machine (an old laptop or PC) and install proxmox on it. Play around, breaks shit, delete the container/VM and start over.
Grab stuff from the Community Helper Scripts and see new stuff, try alternatives, see what works for you and don’t be afraid of breaking stuff.
It takes a bit longer and some basic concepts might fly over your head, but the stuff you learn like this, you learn by heart.
It’s been a few years since I started tinkering with a laptop with a busted video output circuit. Now I serve NextCloud and Immich to my family, keep receipts and documents neatly organised on Paperless, have a decent arr stack and a bunch of extra goodies. All from “a PC without video? Might as well make a server” now with a proper machine with several drives on ZFS pools, health checks and redundancy.
Its a helluva rabbit hole.
I was unaware that those Community Helper Scripts existed! They should definitely be helpful at some point down the road!
Good luck!
Isn’t that how everyone does it? 😬
I didn’t start with a spare, so by the time I was semi-reliant on my self hosted stuff, a breakage was an issue. Also I started with bare Linux, then CasaOS. There was no easy rollback from snapshot/restore backup like on proxmox
I’m about 90% decoupled from Google, it’s been a journey.
I’m at the difficult stage of contemplating how to decom my gmail email, and the Google account itself.
I’ll throw my hat in the ring and offer any help if you need it. Similar to others here, I suggest you start with something discrete like photos.
I have a couple of domains at OVH, and for the last few years I’ve been using email aliases when making accounts, all redirecting to my Gmail. The other day I had enough, created a mailbox on one of the domains, changed all aliases to this new address, and installed Thunderbird mobile. So far it’s been going well.
The webmail runs roundcube, which is usable but not great. The biggest problem I’ve had is the inability to create automatic filters.
Edit: forgot about the calendar, I’ve moved that to my Nextcloud instance
I moved my calendar to Nextcloud, then radicale. My contacts too. Gmail is just a wean away.
My problem is how I’ll be able to deal with work apps like ms authentication. Even if I set up a 2nd “normal” phone for work only, I need to sign in to the play store to get the app… Its a chicken-and-egg problem.
@non_burglar @Damage Are they just TOTP authentication codes you need? If so, you there's alternatives
Certain apps do not allow one to use freeotp et al (o365).
Right, forgot about the contacts, those are on Nextcloud as well. What’s the advantage of radicale over NC?
I still run the play store, but I get what I can from F-Droid. I’m not familiar with the MS authentication app, but I use Ente Auth, idk if that’s comparable.
Functionally, they work the same. I got kinda tired of fixing NC every other upgrade, though. It was always some “occ add missing indices” or some similar garbage. Like just solve this, already. Make that part of the upgrade.
Hmmm, luckily I haven’t had issues so far, but I’m using the AIO docker container
In addition to what another poster said about getting an off-site backup hard drive, I would recommend looking into setting up a raid array for data redundancy with your online storage. You don't want one hard drive failure to make all of your data inaccessible.
I wanna say thank you for making this post OP. I've got a spare laptop that I want to try to turn into my own cloud server but I find the endeavour similarly hard as well. I'll be looking at the tips in the comments. Good luck OP!
Good luck to you, as well friendly stranger!
Hi @MTZ , #selfhosting could be a move in the right direction for you. I started managing my own servers over 10 years ago, locally, from my home, later VPS and finally again from my home. Eventually I moved toward @yunohost - it simplifies a lot of things! I documented some my experiences at https://wasi.ovh
Start small: setup file/photo sync (@nextcloud), calendars and contacts and gradually start adding data from old backups once you feel comfortable.
Have fun and good luck :)
Thanks for the suggestions! Someone else has mentioned YUNOHost and that is definitely on my radar and my mental list of things to look into!
@lpryszcz @MTZ woohoo! good luck, and in case you go for it, we have a really nice community here to help you out: https://help.nextcloud.com/
What exactly do you want to do? Just have storage that you upload all your media to, which is also backed up somewhere else?
Use DietPi, it’s a great OS
I don’t currently have a RPi or any SBC though.
if it seems daunting, which it is!, maybe it would be a good piont to ask yourself if you really need to run a server and these self hosted web apps in the first place. i did for about 10 years but i realised at some point i didnt need half of it.
if youre planning on having multiple users or want to share one of the services (like real time editing of files or passwords etc) then thats where self hosted stuff makes sense to me, but if not then syncthing can do a lot without needing any complicated setup
keepass is a good example. or note taking apps like jopin or obsidian where the data is store in plain text and where you can choose where the data is stored works great with syncthing
for about a year before i did any self hosted stuff i was running only syncthing on my laptop and phone without any server so its do-able and you can get started right now and worry about getting a server later.
tailscale is a huge help as well and is very easy to setup. say you repurpose an old laptop as a server for now and install whatever services on it, jellyfin for example, you will only be able to access that when you are on your home network but not when you are away, and thats where tailscale comes in. as long as its installed on each device you should be able to connect to your apps/services from anywhere.
basically you can start small and then over the coming months and years as you learn more you will get more confident about moving onto more complicated setups
I have set up KeePassXC and KeepassDX, but that’s about as far as I have gotten thus far. I’m currently trying to figure out Obsidian for notes and a general knowledge base.
Start with a nas, the rest will naturally come when you try to access your data for outside, or organize your data, or save more data types to your nas.
Your nas should be the central device and you build the rest around it.
Now, The question is, which nas? I would recommend synology, they are not too performance, a bit expensive and the company is lately doing suspicious moves, but the sw and the hw are rock solid and they are quite good for beginners from almost all angles. Extra point for how many howtos and tutorials are present in internet.
Once you are comfortable with them, you will realize the rest
since you’re so new to self hosting, don’t open anything up to the internet. You’re in for a world of pain if you do.
Write things down
You will break something - and that’s good, it’s the best way to learn - but you’ll want to make a note of what you did / went wrong / how you fixed it.
Future you will still break things and be grateful that you wrote that thing down
You’ll buy something and find next year it was the wrong thing (too small, too large, too old, too new), so just get second hand stuff until you know what you need.
Cabled networks are so much better than wireless, but then you’ll need switches and cables and shelves and stuff… so using today’s wifi is fine, but know where you’re heading.
You need to store you stuff - that’ll be in a NAS
You need something to run services on - that’ll be your server
These might be the same physical metal lump (your 2nd laptop?), they might be separate… play around, break something and work out what feels right for you… and then put your data on there
… and that’ll break too.
Just be aware… if sync files between devices. That’s not a backup. (Consider you’ve deleted / corrupted something - it’s now replicated everywhere)
Having a NAS with 10 drives in a RAID6 array, is not a backup. It’s just really robust against a drive failure, but a deleted file is still a deleted file.
Take a full copy of your data off your system - then restore it somewhere else.
Did it work? If so, that’s a backup.
I’ve always struggles with practicing restoring backups. Do you have to buy an identical 2nd machine to see if everything still works w/o messing up the first one?
Not really.
I keep my data backups (docs, photos, etc) separate from the OS backups.
So, depending on what you’re using to do the backup, often they can just simulate a restore and just check the backup’s not corrupted. Not really a restore, but at least you know it’s not trash.
If you’ve backed up your data with a simple copy / sync (ie not a “backup” program), then you can restore your data somewhere else (maybe even jist a part of it) and do a compare.
But, yeah, if you’re restoring the OS, then it might be ok restoring it in a VM to check it…
I’m slowly moving towards no OS backups and using Ansible to be able to recreate the system(s) from scratch… of course I need to backup the ansible files too 😉
I would like to backup and test restore app configs not sure how to do that yet :p
Maybe OS backup is the way somehow. Maybe I need an atomic distro …
Well, it depends on the app & how it’s installed.
Most bare-metal installed apps are in
/etc/or~/.config/… maybe under/opt/or/usr/- just copying those files out on a regular basis is usually good enough.Might be a little more tricky if you’re trying to backup a database / live files, but those apps usually have some documentation about backups.
If it’s a container or flatpak, etc. then it’s probably somewhere obscure.
Thanks for the advice. I am using Trilium to create a knowledge base as I go, and I am keeping meticulous notes on my progress, successes and failures.
At first, you have to decide what do you need. You can selfhost almost everything, but in my opinion there is no need to do so.
Second thing is hardware to host it. I saw a few comments recommending NAS. It is of course good thing, but my suggestion is just building your own NAS. You need only decent computer to do it.
The easiest way is just installing TrueNAS on it - with that you can setup file sharing and your apps via docker.
But what apps would you need/want? I can recommend a few from my stack:
As an alternative to file shares via SMB, nextcloud is really good option. It’s google drive on steroids. Also includes photo gallery with great app on android/ios with autoupload option.
Nice! Yes, photo storage and backup as well as note sync, reminder sync, calendar, etc. are all very important micro services to me.
I think Nextcloud is a really good option for you. It includes everything you mentioned.
Backup. I use Backblaze personal which is $179 for two years of ‘unlimited’ storage. All my important self hosted data is duped to some old 2.5" external drives connected to my work machine that then is backing up to Backblaze. I also have 1yr retention, so any deleted file is accessible for up to 1yr.
After backups are sorted, stick with the OS you know best. If Windows (I hope not), then HyperV for VMs is good. Try the official Nextcloud VM from Hanson IT. Nextcloud is a good catch-all, but it’s beaten by other specific tools. I now host all I need from specific Docker containers: photos, calendar, email backup etc etc
But I would say Docker. Docker desktop if Macos or Windows if your thing. Get to know docker and the world of self hosting is your oyster.
As what others say, keep it all to your home network and tread carefully when trying to remote access it all.
I’d recommend starting by hosting a nextcloud instance.
Then do some optional steps:
That’s pretty much what you need to start hosting your own files, then later on you can setup a email server, media server like Jellyfin, homepage and everything.
Just go one step at a time and when you hit an issue you can and should ask Google or ChatGPT. Remember, everything exposed to the Internet is vulnerable so take security seriously. Always have everything protected by a decently long password, pairing requirement with your server confirming adding a device or an API key.
@MTZ saving this for later! I aim to go the same path soon :)
Good luck on your journey!
If you want to start cheap, I can recommend you to use an old notebook. In my opinion it’s the perfect home server for beginners.
Most services don’t need much. So it’s just fine if your “server” is like 10 years old. My first notebook server had 2 cores and 4 GB ram and it run Proxmox with like 10 lxc containers just fine.
Awesome suggestion! Thank you.
The only thing to watch out for using a laptop that is plugged in 24x7 is the battery. Battery management systems are generally pretty good, but Li-ion batteries can fail catastrophically. As long as you make a point to check on it periodically it’s probably fine.
I’m using an old laptop as a local interface for my network setup, since its in my basement, and I actually pulled the battery out entirely since I have a beefy UPS powering everything. Paranoid, maybe, but a Li-ion battery sitting on top of my equipment rack could do a ton of damage if it were to fail someday.
Instead of self hosting, why not try better offerings?
Most anything you probably use Google for, you can do with a disroot account. Riseup is a great group, with many similar services (not all). The tildeverse also has myriad replacement services.
Just try to support them, financially.
I have no idea what Riseup, Tildeverse, or Disroot are but I will certainly look into it.
Tildeverse.org
Disroot.org
Riseup.net
Amazing, thanks for sharing. I understand Riseup and Disroot and the missions of each, but I’m having a bit of trouble with Tildeverse , likely because I am not well versed in *nix operating systems as of yet. I’m going to commence to reading up on all of these!
Oh, yeah, I guess I kinda tossed that out there, as they do host a ton of servives. However, its very welcoming as an onboard ramp to learn about *nix stuff. Just ease yourself into that, while exploring Disroot and RiseUp :)
You seem to imply you also want to selfhost some email service. But that’s sadly one of the few things that will always be better at a trusted third party email provider.
Besides that it seems the most important thing you want is pure data storage, and that kind of selfhosting is not hard. In many cases one would not even consider it as part of “selfhosting” as it can be as simple as a local NAS or external HDD.
So my question is what do you actually want to accomplish? Because I think for a lot of your concerns you don’t even need to go and host something.
I don’t want to replace email, per se. More so, the services that come along with it. Stuff like photo storage and sync, notes and reminder sync, calendar sync, and a lot of the hundreds of other small things that I have just been leaving up to Google that would cause a huge problem for me if I lost access.
I have the password situation handled with KeePassXC on my PCs and KeePassDX on my phone, but that’s about as far as I have come as of yet.
Note taking? Joplin. It’s been amazing for me and replaced Evernote after they started with their BS.
Photo storage. Been using Photo prism but I’m open to a better one.
Keepass is a great way of password management, I use keepass as well. I also use syncthing to sync my password database across all devices and then I have the server acting as the “always on” device so I have access to all passwords at all times. Works amazing because syncthing can also be setup so when a file is modified by another device, it makes a backup of the original file and moves it to a dedicated folder (with retention settings so you can have them cleaned every so often). Life is so much easier.
For photo access you can look into immich, its a little more of an advanced setup but, I have immich looking at my photos folder in syncthing on the server, and using that location as the source. This allows me to use one directory for both photo hosting and backup/sync
Do you know anywhere that I could find a guide on how to set that up correctly? I’ve used SyncThing in the very recent past and it ended up being not good because of my own ignorance. I have syncthing on my phone and my desktop, but neither of them are syncing anything to anywhere. I’m terrified that I will mess something up and possibly lose data, or end up with a gigantic mess of files that do not belong in the synced directory. That’s what happened the only time I ever did use Syncthing. For some reason, it basically transferred a gigantic directory of things that were not relevant at all from my PC to my phone, filling up all of the storage and it was a hassle to get rid of everything that was not supposed to be there. I REALLY want the KeePass databases on my phone and on my main PC to communicate and update each other though. Entering everything in twice has been frustrating.
I haven’t used a guide aside from the official getting started with syncthing page.
It should be similar to these steps though, I’ll use your desktop as the origin device.
Some things you may want to keep into consideration. Syncthing only operates when there are two devices or more that are online. I would recommend if you are getting into self hosting a server, having the server be the middle man. If you end up going that route these steps stay more or less the same, it’s just instead of sharing with the phone, its sharing with the server, and then moving to the server syncthing page and sharing with the mobile. This makes it so both devices use the server instead of trying to connect to each other. Additionally, if you do go that route, I recommend setting your remote devices on the server’s syncthing instance to “auto approve” this makes it so when you share a folder to the server from one of your devices, it automatically approves and makes a share using the name of the folder shared in the syncthing’s data directory. (ex. if your folder was named documents and you shared it to the server, it would create a share named “documents” in where-ever you have it configured to store data). You would still need to login to the server instance in the case of sharing said files to /another/ device, but if your intent was to only create a backup of a folder to the server, then it removes a step.
Another benefit that using the server middleman approach is that if you ever have to change a device later on down the road, you are only having to add 1 remote device to the server instance, instead of having to add your new device onto every syncthing that needs access to that device.
Additionally, if you already have the built in structure but it isn’t seeming like it is working, some standard troubleshooting steps I’ve found helpful:
That is so awesome. Thank you so very much for taking the time out to inform me on this matter.
If you have systems or services you’re dependant so strongly, always have an backup / emergency access. 3rd party or self hosted.
My 5c but I think you agree.
Point being as a decades old it professional I see design more important as the detail implementation.
Goddamned inflation. It used to be just 2 cents worth!
Ah crap that’s what I get for not being native English. But still, correct somehow.
I don’t believe inflation is a serious problem. But that is just my 9 cents.
Indeed, I do agree but I’ve never done anything close to this magnitude so it is kind of intimidating for me. It is a learning process though!
I’d recommend not to go containerized but that can start a flame war. I would think it easier. But best to stick to the recommended beginner tutorial that someone else posted and go along from there.
Then ask questions on the way.
Native vs containerized really depends on what it is going to be doing tbh. If it’s just downloading and/or moving files around, containerized is fine. And having your docker-compose.yml files saved somewhere external will make future hardware upgrades/recovery much easier.
There is certainly some learning curve to figure out the quirks of a compose file, but the nice part is that most services will post an example compose file for you to edit as needed. And that means learning it is basically just a matter of reading the example files and figuring out what the different fields mean; yaml is extremely easy to read, even for someone who has never looked at it before. You may have some fringe cases that need a deeper dive, but the vast majority of setups are basically just a matter of “copy the example compose, edit the volumes as needed, and fuckin send it.”
Yes indeed. However as a beginner I think it’s far easier just to install and run a daemon. But maybe that’s just me. And of course if the intended way of running it is only container by default.
I hard agree with this. I would NEVER have wanted to start with containerized setups. I know how I am, I would have given up before I made it past the second LXC. Starting as a generalized 1 server does everything and then learning as you go is so much better for beginnings. Worst case scenario is they can run docker as the later on containerized setup and migrate to it. Or they can do what I did, start with a single server setup, moved everything onto a few drives a few years later once I was comfortable with how it is, nuked the main server and installed proxmox, and hate life learning how it works for 2 or 3 weeks.
Do i regret that change? No way in hell, but theres also no way I would recommend a fully compartmentalized or containerized setup to someone just starting out. It adds so many layers of complexity.
Thanks, you put it better than I can.
As someone who went through this after trumps 2nd term and power grabs i can give you my process:
Fast forward 6 months: My router is now running OpenWRT. With a few necessary exceptions my network access is always through ProtonVPN. My external devices are connected via wireguard to the router when not on home wifi and only after that reach the www. I have 24/7 access to my services from everywhere. My main server is now an old office mini pc running about 10 services. Im using borg for nightly snapshots(its a bit like apple time machine) and after that everything is backed up to another server at a friends house via rsync and ssh. I have a third mini computer whose purpose is to be my tv’s UI with access to services like the national broadcasts web ui and my own jellyfin and invidious (adless youtube client) The tv does not have an internet connection anymore. I even made a custom land page that automatically opens full screen in a browser when open my tv.
The point is: this builds gradually and you have fun doing it. …until it breaks :D The most painful parts involved networking so you can settle for LAN only at first to keep things simple
Re-investing in a new platform full of tools (Proton suite) isn’t in my opinion a rational answer. My answer is self host vaultwarden, self host your file storage, and choose between Proton and Tuta for mail, and use your own domain name so you can take your email address with you should you move.
In my opinion No-one should ever store any form of personal data implicitly on someone else’s computer.
Oh i have vaultwarden as well nowadays
When you’re connecting to your local network, say from outside your home, you connect through proton vpn? I’m wanting to self host, but I thought I would have to switch between proton and whatever VPN the local machine is using? Say immich for instance
My phone is on a wireguard tunnel into my router which puts my wireguard vpn in the same forewall zone as my home LAN. Internet access is routed through the tunnel and then through another tunnel to protonvpn and from there to the www. It was a bit elaborate to set up but it works. Wouldnt really recommend the setup for everyone, it was a bit of a pain in the ass to get working. I used Openwrt and policy based routimg plus wireguard for the tunnels into and put of the router.
how well does invidious work on the TV? how easy are the controls?
I use a small wireless logitech keyboard-mousepad so it works very well. I had to make exceptions in the router for googles video severs to bypass the vpn though.
yeah not all invidiouses allow proxying the video, and at some it used to be off by default
Im running my own instance. But yeah, videos from google servers won’t load behind a proxy. Or they might work for a while but then wont and then you have to switch vpn servers which is very annoying.
Yunohost should be the software you’re looking for. Install stuff by clicking. Much less terminal stuff
yunohost.org
Sevral people have mentioned Yuno and I’m going to look into it shortly. Thanks for the input!
There’s Yuno, CasaOS is ridiculously easy to setup, manage and maintain as well. There’s UnRaid (not free, but very good), Proxmox is extremely versatile.
I am currently running light services (caldav, carddav, PW manager, and some other lighter stuff) on an N150 mini PC, and have a hefty server for heavier services running on Proxmox.
Of course, I follow the 3-2-1 backup rule, but only for data I could never get again. Movies, Series, music, I never back up.
+1 for YunoHost from the POV of a total self-hosting newbie (I’m now self-hosting my own GoToSocial, Pixelfed, PeerTube and NextCloud thanks to it… upgrades and backups are super easy, too)
Can I ask where you host your backup service without paying another cloud provider?
My VPS provider offers automated daily backups (stored in different locations for extra security) for 2.99 a month
I have a big super micro server i was given a while back but have yet to set it up. I was going to put proxmox on it. Would you recommend yuno over that?
Depends on what you want to do. For a small server, if you want to host multiple things, hosting them straight on the metal without putting a VM in between would be more performant. If your server doesn’t have much RAM and CPU to give, then getting rid of the emulation layer makes sense.
Can you tell me why you want to use proxmox and what for?
It has 256gb of ram. I don’t remember the CPU power but it has 2.
I want to do self hosted storage, currently have 12-16tb (I’ve forgotten which). I’ll also want to have other services running. Like game servers or things like immich and jellyfin. I’ll also want to have something for git and probably Jenkins (or similar), then also a place to host anything I create that needs hosting.
When i asked where to begin with such a server, pretty much all of the responses were to go with proxmox. I’m not a fan that it’s nagware though so I’m open to other suggestions.
256 GB of RAM? Wow. And game servers too? If that’s small, them I don’t know what you consider big…
Anyway, proxmox does fit your scenario well. Separating your hosted services into VMs or containers makes a lot of sense. And a few game servers also have installations specific to different distros, so instead of fumbling about with your specific distro, just creating a VM with the distro you need is way easier.
I never said it was small haha. I did say it was big.
I never planned to get a server like this initially. Just the perks of having a partner working in IT for a school that’s decommissioning old tech since they’re outsourcing some stuff soon.
Oh my god, you were right. Yuno is AMAZINGLY useful for exactly what it is that I am attempting to do!
Glad you like it! If it’s useful to you, don’t forget to donate or at least say thanks to the contributors once everything is up and running and stable.
Don’t forget backups! Restic is in yunohost and should be useful for that. Yunohost has a guide.
I haven’t actually began to use it yet because first I am trying to understand the framework and fundamental basics of what it is that I am attempting to do. When I get a grasp on that, I will definitely be using YUNO and overwhelmingly likely will donate a good sum of money to that project.
You can test it in a virtual machine like virtualbox or virt-manager. Then you can get a good feel for it.
Ahhh, good point! I definitely know about Virtualbox. I shall try that. Thanks!!
How secure is yuno? Is it actually secure plugging into your Ethernet for remote access to something like immich?
It’s super intimidating when the weight of the weight of the global hacking community is attacking you from the moment you expose a port.
Is their progress on a simple sustainable solution to security? Is this the primary roadblock to self hosting becoming more common place? Or am I way off
Yunohost is probably more secure than you figuring everything out yourself. More people have a vested interest in keeping it secure. They have a minimal page on security but they have fail2ban, unattended upgrades,and a secure SSH configuration. If something is discovered, you might be vulnerable but at least there will be knowledgeable people fixing it.
Security is always difficult and nothing is 100% secure. The three letter agencies around the world have been hacked and they are in the business of hacking others. Hackers themselves get hacked on the regular. Using yunohost as a noon probably reduces the chance of you getting hacked.
If you have something only you need to access, you can also host yunohost for yourself and make it accessible only via a VPN. Headscale, tailscale, maybe even your router provides a VPN service, or setup wireguard yourself. If others have to access it… I dunno. That’s a good question to ask on /c/selfhosted
I was a member of r/selfhosted before I abandoned reddit altogether, but I’m not familiar with /c/selfhosted. May I ask what the C is?
c stands for community. I didn’t know on which server it is but it’s on lemmyworld !selfhosted@lemmy.world
Just a heads up on what you are getting yourself into, if you fuck up your self hosted setup badly enough there is no recovery.
That isn’t necessarily intended to scare you off from self hosting, just that the first and most important lesson to learn is to have a good system of backups that are backed up automatically, are easy to recover from, and are separated enough from other copies of the data that if something goes terribly wrong one copy will survive.
Thanks for the advice. Currently, I have a backup of all of my important data on a high capacity HDD that is completely disconnected from any devices. There is no real way to automate backups with that setup, but it’s what I am working with at the moment.
I’d recommend setting your sights on the 3-2-1 rule. 3 copies of your data, 2 different mediums, 1 off-site. Hetzner Storage Box is a good cheap offsite option.
To piggy back off of Arcayne’s comment
I don’t backup everything to a 3-2-1. I at least have everything on a 2-1 backup scheme. Two copies, one offline. My personal photos and videos are on a proper 3-2-1. I can redownload my movies and music. I can’t retake pictures from my childhood. It saves a lot of money doing it that way. My personal media is less than 1 TB, whereas my whole collection is many TB and would be prohibitively expensive to 3-2-1.
I’m 100% self taught & was in exactly the same place. I’d never used Linux before I got my first Pi. I spent a bit of time trying to familiarise myself with & made some notes regarding command line (notes I still rely on).
There’a so many ways to achieve the goal, you’ll eventually find a way that works for you. My personal preference was Docker/Docker Compose deployed via Portainer.
Even that was confusing. Until I found this excellent video on how to read Docker requirements & apply them step by step into Portainer. He explains slowly & methodically exactly what he is doing & why.
Portainer is a method of handling Docker stacks/containers via a web UI. Both Docker & Portainer are simple to install.
It’s easier to use Docker Compose files and/or .env (environment variable) files (both are even simpler to deploy through Portainer) but this video taught me what was going on & gave me confidence to have a go. What attracted me to Docker is you can easily remove stacks/containers if/when you make a mess rather than wiping the drive & start again, which is how I went about things initially.
This gave me the tools to set up Nginx Proxy Manager & I never looked back.
As you’ve realised, a robust backup solution is essential (plus off site backup for particularly important stuff) as things will inevitably go wrong along the way (I see Borg, Restic mentioned often, I went for Kopia).
I can’t recommend highly enough making detailed notes along the way, I rely on Joplin.
If you start using Docker, dont fall into the trap of using the “latest” tag. If you know the version number you’re running its far easier to re-deploy if an update is bad.
Enjoy your new time consuming, teeth gnashingly frustrating …and yet rewarding hobby 👍
This post inspired me to audit my firewall rules, had put it off for too long
While YUNO is a great way to get started, I strongly encourage you to understand basic concepts, like docker, and maybe try to run something outside of it for fun. While not even remotely the same thing since YUNO is just the OS and “app store”, you would be very similarly tied to that ecosystem the same way you are to Google now. Not to mean that YUNO would have any control over your stuff, but you would be dependent on them for what you can self host.
Yes, I’m definitely trying to understand the “why” of what it is that I am doing. I’m just going to be using YUNO as a starting point.
Like you I also had a terrible experience, only with Apple and my Iphone where it didn't accept my (correct) pincode, and I couldn't log in to Apple ID due to not having access to my phone. I ended up having to reset my phone after much banging my head against the wall. I decided then that I would not be reliant on that thing for access to everything else. I've ordered a #JollaC2 Community Phone by @jolla@techhub.social to replace my Iphone. No more #bigtech for me.
Good luck to you! I had never heard of Jolla before. That’s very cool. Im in the US and am so far unsure if Jolla products are marketed or even supported here. More investigating!