I’ve been a Software Engineering Student for 2 years now. I understand networks and whatnot at a theoretical level to some degree.
I’ve developed applications and hosted them through docker on Google Cloud for school projects.
I’ve tinkered with my router, port forwarded video game servers and hosted Discord bots for a few years (familiar with Websockets and IP/NAT/WAN and whatnot)
Yet I’ve been trying to improve my setup now that my old laptop has become my homelab and everything I try to do is so daunting.
Reverse proxy, VPN, Cloudfare bullshit, and so many more things get thrown around so much in this sub and other resources, yet I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up.
Why is self-hosting so daunting? I feel like even though I understand how many of these things work I can’t get anything actually running!
You must log in or register to comment.
Part of working with tech is knowing what to search for und using the right keywords. If you could give an example what guide / information you are unable to find, someone could give you an example how to search for it. I personally know a few junior devs and junior devops which use llama2 / chatgpt since they dont know how to search for it or read the docs.
I actually hate GPT, dislike it’s answers and find myself knowing better than it most times.
I’ve been trying to setup a DNS server to create my own domains internally within my VPN but I keep finding info on how DNS servers work, and how to make a records on registrars, but nothing on what I actually need to install and run to have my own DNS for example. Same thing goes for many other services, but that’s the one bugging me for the longest time because it should be so simple.
I’ve found plenty of tutorials on how to make a cache DNS, just not an authoritative name server btw, and I’ve searched for both DNS and name server to no avail. If it was Linux I’d write some custom rules in my hostfiles and be done with it, but it’s so much harder to do on Windows and that’s my daily use OS for now…
“Selfhost authorative dns” returns this for example https://wiki.selfhosted.show/DNS/
And we have two guides for the most common dns implementantations. Depending on the vpn software you can push the dns ip as part of the configuration or by using DHCP.
I also found the pihole docs for unbound rather helpful https://docs.pi-hole.net/guides/dns/unbound/.
For me, I have that as the 4th result, after some Reddit and IBM which probably would’ve discouraged me from continuing my search. I’d have to read on it.
Also, TIL PiHole doesn’t necessarily need to run on a Raspberry Pi. I guess assumptions really do come back to bite me in the ass haha
Pi runs Raspbian which is just Debian with customisation applied. So of course it can run elsewhere. You don’t know as much as you think you do perhaps 😉
I seriously thought it was a product, rather than software tbf. The name always sounded so “corporate” I never considered it.
I definitely know more about the theory than the practice. I’m clueless as to what my options even are so I can’t argue with that.
But I did know about the Linux “inheritance” of distros if you wanna call it that, and I’m fully aware of what that entails.
Just honestly didn’t look at it twice cause I thought “there must be an FOSS option” without realizing what PiHole really is. Just a case of prejudice biting me in the ass I guess.
Sounds like the next step in your journey is combing through this list and seeing what’s out there: https://github.com/awesome-selfhosted/awesome-selfhosted
So much great stuff! But most of it has drawbacks, like missing features or less attractive UI. But it’s free and open source so we love it all the same.
I’ve read that repo a million times! My self-hosting needs are more esoteric and I mostly play around with it. I’ve no need for media services or 90% of what that repo offers yet!
I mostly want to end up self-hosting my own apps, but I need some foundational knowledge
There’s a lesson in here somewhere about patience. Get good at skimming. I was looking for how to do something I was unfamiliar with the other day and I had to sift through 15 results across four different search strings before I found the solution that was going to work for me. But because I’m good at skimming it only took me 1/2 hour to discover and implement. Google isn’t magic and this is why someone else recommended Chat GPT to help with some of the sifting, especially early on.
“self hosted DNS server” is what I searched for, but I also knew what I was looking for “Unbound” with PiHole is what I use.
Also, Windows has a host file that it uses. %windir%\System32\Drivers\etc\hosts
%windir% is the environment variable for your windows install directory. Usually C:\Windows
You can add your entries in there, just like Linux, though I am sure the syntax might be slightly different. I really haven’t played with host files on Linux, yet.
I hate the fact you gave your honest opinion and othets have to downvote you. I myself rarely use ChatGPT to find a direct answer, instead I use it to give me resources to find the answers for the questions I didn’t think about asking or knew existed. More like thinking outside the box or brainstorming. And always do your due diligence to verify any answer it gives.
I use Pihole for this in my home network. It has the capability to configure local domains for your internal network resolution as well as create whitelist/blacklist and use as a DHCP server alternative to your home router – you just need to configure your router to point to the Pihole instance for DNS on your LAN network configuration to get started with the DNS piece
Check YouTube. I haven’t found a lot of written out guides but someone has made a video on it, I bet. Christian Lempa, Techno Tim, Db Tech, NetworkChuck come to mind for self hosting tutorials on containers and new applications. I’m sure there’s a lot more as well.
Thanks for the suggestions! The algorithms keep feeding me people who just explain what stuff is and it drives me nuts. You wouldn’t believe the amount of videos, articles and blogs I’ve seen on setting up a DNS server just for it to be about either a cache or an explanation of how it works. I’ll look into these later!
Jeff Geelong is another good resource on youtube, especially if you ever want to get into infrastructure as a career.
Github is also an amazing resource and always RTFM.
Edit: spelling
You don’t have to do it, 99.9999%^([citation needed]) of the population don’t. If it doesn’t interest or excite you then it’s not worth it.
You’re kidding?
I’m an avid home-labber trying to get into software engineering and I’m finding the software dev work to be super daunting lol
I can setup a reverse proxy in my sleep - but write a JS function without double checking google? forget it!
It comes with experience I guess, I’ve got a bad habit of researching to the core and many times have a hard time grasping things like containers without understanding how it’s setup technically. Sometimes I find a decent explanation, but specially for libraries that do “magic” I gotta go diving into the source to understand what’s going on, else I have trouble understanding what I am doing and what I should be doing.
Which makes it so hard because networking is very low level and I’m very unfamiliar with this environment
NetworkChuck on utube
I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up.
This is big part of why I, even as an IT professional, don’t do much homelabbing or selfhosting. So often I’ll be scrolling through these subs and see something someone has done and I’ll decide I want to do that too, but so often I end up abandoning whatever it is because doing x requires googling for y and z because without y and z, x doesn’t work. And after a day of dealing with issues and tinkering on work stuff, do I want to spend more time troubleshooting? Often not.
Maybe you should read the documentation for the programs you try to install
Because most people who develop these things are, frankly, terrible at good documentation, or understanding the end-user perspective.
There’s also a downward spiral effect when you start getting into these things, because lots of them require dependencies, or ask you to do things but don’t explain why, and you’re just left wondering why you added that line to a config file somewhere, but if you don’t put it there, nothing works.
A vertical slice of the amount of knowledge you need passes through so many different disciplines, operating systems, GUIs, and programming languages that it would look like a Milhojas cake.
I’ve been a technical writer in the software industry for 17 years. The number one challenge in my work is extracting all of the information I need to write good documentation from the experts elsewhere in my company.
Each one of these things is a separate topic/project. There is of course, overlap, but concentrate on just one at a time. If you try to do everything, you’ll go mad. Application Administrator, Network Engineer, Sysadmin, Network Security, are often separate full time jobs. Just concentrate on your own needs, one at a time, then focus on how simple you can pare down that one thing you need/want to self host. What is the minimum set of capabilities you must have for this one thing? Then, expect to have as much research for this one thing as taking a 1 credit college course, at least. Maybe more, maybe less, depending what it is. That’s a more realistic set of expectations.
I’ve done a lot of web design and web hosting, and messing with Linux server stacks for around 30 years, now. You absorb a LOT over time, but it is all always changing. When I set up my first server, Cloudflare didn’t exist yet. VPNs weren’t a thing yet. The only people I knew with a network in their home were Computer Science professors. Wifi was not a thing yet. It isn’t you being inadequate, it’s you trying to do multiple jobs that each require constant professional development.
Senior DevOps Engineer here with 15+ years of experience in system engineering now, even with my skillset and experience I am improving my setup piece by piece.
When I look at my old docker-compose versions and general setup, I sometimes laugh how I could’ve been so dumb.
You don’t have to and won’t learn and set everything up at once, ever.
I’ve studied automotive and never really graduate, but I love reading manuals since I was a kid and always believe that even I don’t know what am I doing as long I can read It’s manual, guide, documentation and read support I could get a grasp of it even if it takes more time.
A part that made it easier for me to understand self-hosting is the raspberry pi. It let me thinker self-hosting without worrying about power bill, how can I easily reinstall its OS even if I make bunch of mistake, make different copies of my setup on different sd card without worries of hdd or ssd.
Then from that just upgraded stuff from what I’ve learned.
I wish I could afford a Pi. Would be so cool. Unfortunately I’m stuck with my gaming PC from 6 years ago that I recently updated from. Much more powerful hardware but I can’t just swap out the drive or not worry about power usage sadly haha.
Still, I’ve had to reinstall the OS about 8 times last year alone haha, but we’re still learning (most because I forgot the password tbh haha
As a computer science student you should know: just break it down into small chunks.
- Get service to run locally (IP / port)
- port forwarding, access remotely
- Set up a free DuckDNS domain, Lets Encrypt, use for a bit
- Buy a cloudflare domain, set up dynamic dns, Lets Encrypt (again), swap away from DuckDNS
- Set up Nginx reverse proxy, port forward to that instead, proxy to service’s internal IP + port
- Run more things!
That’s generally what I did… I’ve been self hosting for about a year now and I’ve got 6 services going now… once you get going, it’s quite easy!
On a certain level, most of those things don’t matter.
Yes, reverse proxying and being able to point a domain is nice
A VPN more secure
Cloudflare (either tunnels, DNS, or DDoS protection) is also good
You can always do better with more “best practices” but if you want to stream a movie to some friends? Throwing up a server, opening a port and providing an IP just works as well. The better you get the more fancy you will be.
Its very much a tinkering hobby. Find a reason you want to tinker, and if you enjoy the tinkering you will keep up with it. My current dashboard points to a direct IP. Most of my services are reverse proxied but for some reason I can’t get CORS to work well with dashy. At some point I’ll get around to fixing it, but it gets the job done now.
Invest some time in understanding Docker containers and then install something like Portainer (for example on top of Proxmox + VM, or ProxMOX + LXC, or some Linux or bare metal) and then installing new apps will become very trivial once you understand how to “create” the containers and how to map the volumes and ports.
I got hooked by Youtube videos from TechnoTim Christian Lempa and others. Videos include screen capture what they are doing so it bridges some things that may be missing from text guide because it may be ‘obvious’ if you are doing it all the time.
I keep to principal that I have two ‘sections’ in my lab ‘production’ and ‘testing’ Testing I am mucking about testing stuff it works and production has some stuff that I rely on.
Because it’s a complicated topic, you’re just feeling the actual responsibilities of doing it right that many are ignoring now.