Home

Your logging is probably down

$$11171
https://piefed.social/u/Ek-Hou-Van-Braai posted on Mar 13, 2026 09:12
https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down
Reply
$$11174
https://piefed.zip/u/squirrel posted on Mar 13, 2026 09:17
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Let’s tinker around and accidentally break something.

https://piefed.zip/comment/4233081
Reply
$$11177
https://lemmy.ml/u/wersooth posted on Mar 13, 2026 09:26
In reply to: https://piefed.zip/comment/4233081

and debug it until you have to reinstall your entire stack from scarch

https://lemmy.ml/comment/24509842
Reply
$$11178
https://slrpnk.net/u/Prunebutt posted on Mar 13, 2026 09:30
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

When’s the last time you checked if your backup solution works?

https://slrpnk.net/comment/21223483
Reply
$$11180
https://feddit.org/u/JetpackJackson posted on Mar 13, 2026 09:39
In reply to: https://slrpnk.net/comment/21223483

Yesterday! Switched my media server from freebsd to alpine and got the arr stack all set up using the backup zip files

https://feddit.org/comment/11996110
Reply
$$11182
https://lemmy.today/u/tal posted on Mar 13, 2026 09:42
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

logging is probably down

You do, of course have a dedicated rsyslogd server? An isolated system to which logs are sent, so that if someone compromises your other systems, they can’t wipe traces of that compromise from those systems?

Oh. You don’t. Well, that’s okay. Not every lab can be complete. That Raspberry Pi over there in the corner isn’t actually doing anything, but it’s probably happy where it is. You know, being off, not doing anything.

https://lemmy.today/comment/22854693
Reply
$$11183
https://lemmy.today/u/tal posted on Mar 13, 2026 09:49
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

All of your systems are set up, but are they capable of being redeployed using a configuration management software package? Ansible or something like that?

Oh. They’re not. Well, that’s probably okay. I mean, you could probably go manually reproduce configurations, more or less.

https://lemmy.today/comment/22854757
Reply
$$11184
https://lemmy.today/u/tal posted on Mar 13, 2026 09:52
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

You have an intrusion detection system set up, right? A server watching your network’s traffic, looking for signs that systems on your network have been compromised, and to warn you?

Oh. You don’t. Well, that’s probably okay. I mean, probably nothing on your network has been compromised. And probably nothing in the future will be.

https://lemmy.today/comment/22854793
Reply
$$11185
https://sh.itjust.works/u/neidu3 posted on Mar 13, 2026 09:53
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Barring any hardware issues or external factors, will it run for 10000 years? Any logs not properly rotated? And other outputs accumulating and eventually filling up a filesystem?

https://sh.itjust.works/comment/24264349
Reply
$$11186
https://feddit.uk/u/Egonallanon posted on Mar 13, 2026 10:00
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Buy a UPS and setup a NUT server on the spare raspberry pi you have lying around.

https://feddit.uk/comment/23818318
Reply
$$11187
https://lemmy.today/u/tal posted on Mar 13, 2026 10:02
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

All of those systems in your homelab…they aren’t all pulling down their updates multiple times over your network link, right? You’re making use of a network-wide cache? For Debian-family systems, something like Apt-Cacher NG?

Oh. You’re not. Well, that’s probably okay. I mean, not everyone can have their environment optimized to minimize network traffic.

https://lemmy.today/comment/22854901
Reply
$$11188
https://lemmy.today/u/tal posted on Mar 13, 2026 10:07
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

You have squid or some other forward http proxy set up to share a cache among all the devices on your network set up to access the Web, to minimize duplicate traffic?

And you have a shared caching DNS server set up locally, something like BIND?

Oh. You don’t. Well, that’s probably okay. I mean, it probably doesn’t matter that your devices are pulling duplicate copies of data down. Not everyone can have a network that minimizes latency and avoids inefficiency across devices.

https://lemmy.today/comment/22854949
Reply
$$11189
https://lemmy.world/u/Admax posted on Mar 13, 2026 10:09
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Then it turns out your monitoring system failed and FUCK IT’S BEEN A MONTH SINCE THE LAST PROPER BACKUP

https://lemmy.world/comment/22637259
Reply
$$11190
https://programming.dev/u/CameronDev posted on Mar 13, 2026 10:15
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Do your backups work?

https://programming.dev/comment/22694964
Reply
$$11191
https://lemmy.today/u/tal posted on Mar 13, 2026 10:15
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

You have remote power management set up for the systems in your homelab, right? A server set up that you can reach to power-cycle other servers, so that if they wedge in some unusable state and you can’t be physically there, you can still reboot them? A managed/smart PDU or something like that? Something like one of these guys?

Oh. You don’t. Well, that’s probably okay. I mean, nothing will probably go wrong and render a device in need of being forcibly rebooted when you’re physically away from home.

https://lemmy.today/comment/22855057
Reply
$$11192
https://leminal.space/u/probable_possum posted on Mar 13, 2026 10:17
In reply to: https://lemmy.today/comment/22854693

Ah. The approach that squirrel@piefed.zip suggested. ;)

Thanks for the tutorial though.

https://leminal.space/comment/21769181
Reply
$$11195
https://discuss.tchncs.de/u/Petter1 posted on Mar 13, 2026 10:20
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

You should use Arch, then you can update every 15 minutes 🤭

https://discuss.tchncs.de/comment/24499838
Reply
$$11196
https://lemmy.world/u/lemming741 posted on Mar 13, 2026 10:20
In reply to: https://lemmy.today/comment/22855057

Does a $12 Shelly plug count?

https://lemmy.world/comment/22637394
Reply
$$11198
https://lemmy.world/u/tychosmoose posted on Mar 13, 2026 10:30
In reply to: https://lemmy.today/comment/22855057

If you do have the smart PSU and power management server you probably also went down the rabbit hole of scripting the power cycling, right? Maybe made that server hardened against power loss disk corruption so it can be run until UPS battery exhaustion.

What if there is a power outage and NUT shuts everything down? Would be nice to have everything brought back up in an orderly way when power returns. Without manual intervention. But keeping you informed via logging and push notifications.

https://lemmy.world/comment/22637486
Reply
$$11199
https://piefed.social/u/halcyoncmdr posted on Mar 13, 2026 10:35
In reply to: https://slrpnk.net/comment/21223483

Backup? Psh… That’s what the lab is for.

https://piefed.social/comment/10512847
Reply
$$11200
https://lemmy.world/u/czardestructo posted on Mar 13, 2026 10:38
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

I havent messed much with my servers in 2 years. I think that means I’ll hit my RIO in another 5 :)

https://lemmy.world/comment/22637567
Reply
$$11201
https://feddit.org/u/truthfultemporarily posted on Mar 13, 2026 10:39
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Have you tried introducing unnecessary complexity?

https://feddit.org/comment/11996666
Reply
$$11202
https://ani.social/u/SpikesOtherDog posted on Mar 13, 2026 10:41
In reply to: https://lemmy.ml/comment/24509842

GET OUT OF MY HOUSE!

https://ani.social/comment/15650407
Reply
$$11204
https://lemmy.today/u/tal posted on Mar 13, 2026 10:48
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

You have all your devices attached to a console server with a serial port console set up on the serial port, and if they support accessing the BIOS via a serial console, that enabled so that you can access that remotely, right? Either a dedicated hardware console server, or some server on your network with a multiport serial card or a USB to multiport serial adapter or something like that, right? So that if networking fails on one of those other devices, you can fire up minicom or similar on the serial console server and get into the device and fix whatever’s broken?

Oh, you don’t. Well, that’s probably okay. I mean, you probably won’t lose networking on those devices.

https://lemmy.today/comment/22855480
Reply
$$11206
https://lemmy.dbzer0.com/u/TerHu posted on Mar 13, 2026 10:52
In reply to: https://lemmy.world/comment/22637394

if you can cycle your home assistant with the shelly plug whilst your home assistant is down, yes. from experience it’s really quite annoying to have a smart plug switch off HA…

https://lemmy.dbzer0.com/comment/24929730
Reply
$$11207
https://lemmy.decronym.xyz/u/Decronym posted on Mar 13, 2026 11:00
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

Fewer Letters More Letters
DNS Domain Name Service/System
HA Home Assistant automation software
~ High Availability
PSU Power Supply Unit

[Thread #161 for this comm, first seen 13th Mar 2026, 11:00] [FAQ] [Full list] [Contact] [Source code]

https://lemmy.decronym.xyz/comment/17600
Reply
$$11212
https://piefed.social/u/Ek-Hou-Van-Braai posted on Mar 13, 2026 11:28
In reply to: https://slrpnk.net/comment/21223483

But if my backups actually work then I miss out on the joy of rebuilding everything from scratch and explaining to my wife why non of the lights in the house work anymore.

https://piefed.social/comment/10513294
Reply
$$11215
https://lemmy.world/u/the_tab_key posted on Mar 13, 2026 11:33
In reply to: https://lemmy.today/comment/22854901

I set this up years ago, but then decided it was better to just install different distros on each of my computers. Problem solved?

https://lemmy.world/comment/22638176
Reply
$$11216
https://lemmy.blahaj.zone/u/varnia posted on Mar 13, 2026 11:34
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

I had a automatic reboot of all VMs and the hypervisor because of a kernel update at night. Nextcloud decided to start in maintenance mode and Jellyfin refused to start because the cache folder didn’t have enough space left. Authentik also complained about outdated provider configuration…

Need to investigate the Nextcloud and Authentic issue during weekend 🤗

https://lemmy.blahaj.zone/comment/19605544
Reply
$$11217
https://lemmy.zip/u/beegnyoshi posted on Mar 13, 2026 11:35
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

I haven’t messed with my raspberry pi in maybe a month… And I think one of my backups got corrupted because I receive an email saying that it failed along with tons of errors every night. Hmm, maybe I should get to that soon…

https://lemmy.zip/comment/25225394
Reply
$$11220
https://lemmy.world/u/AkatsukiLevi posted on Mar 13, 2026 11:49
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Do you have a spinning fish display in front of your homelab server, right? We all know the spinning fish improves performance and security, it is a indispensable part of homelabbing

https://lemmy.world/comment/22638356
Reply
$$11223
https://startrek.website/u/Kirk posted on Mar 13, 2026 11:59
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

https://www.youtube.com/watch?v=5Jls8KcGxTA

https://startrek.website/comment/21879748
Reply
$$11224
https://fedinsfw.app/u/Fedegenerate posted on Mar 13, 2026 12:01
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Going into spring/summer that’s ideal, I wanna go places do things. Mid winter, I’m feature creeping till something breaks.

https://fedinsfw.app/comment/271011
Reply
$$11227
https://lemmy.wtf/u/rosco385 posted on Mar 13, 2026 12:08
In reply to: https://lemmy.ml/comment/24509842

Are you implying it’s possible to debug without having to reinstall from scratch? Preposterous! 😂

https://lemmy.wtf/comment/20452849
Reply
$$11228
https://lemmy.world/u/lemming741 posted on Mar 13, 2026 12:11
In reply to: https://lemmy.dbzer0.com/comment/24929730

HA is on the same proxmox host as the router. So yeah I can end up locked out. Hasn’t happened yet tho! The relay is on my test machine, it’s always nvidia that crashes there.

https://lemmy.world/comment/22638700
Reply
$$11231
https://ani.social/u/Sabata11792 posted on Mar 13, 2026 12:16
In reply to: https://feddit.org/comment/11996666

If you know how your setup works, then that’s a great time for another project that breaks everything.

https://ani.social/comment/15651460
Reply
$$11256
https://fedia.io/u/Australis13 posted on Mar 13, 2026 12:18
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Honestly, that would be living the dream… I have too many other things I want to do!

https://fedia.io/m/selfhosted@lemmy.world/t/3590167/-/comment/14368599
Reply
$$11235
https://feddit.org/u/B0rax posted on Mar 13, 2026 12:26
In reply to: https://lemmy.dbzer0.com/comment/24929730

The Shelly can be configured to automatically turn back on after a certain amount of time. It has local scripting capabilities.

If they did that… I don’t know.

https://feddit.org/comment/11997870
Reply
$$11238
https://lemmy.world/u/Skullgrid posted on Mar 13, 2026 12:32
In reply to: https://lemmy.world/comment/22638356

J O E L

https://lemmy.world/comment/22638990
Reply
$$11240
https://lemmy.ca/u/panda_abyss posted on Mar 13, 2026 12:35
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Gotta be honest, my home lab chugs along quite happily.

Atomic fedora makes it hard to break, and then all the services are containerized and managed by configuration and just files only.

When there’s an update to a service: just pull service. Firewall needs configuring: just firewall-reset && just firewall-enable.

The only flaky thing is a vpn that I run through glutan and I’m thinking of dumping that provider.

https://lemmy.ca/comment/22188501
Reply
$$11241
https://piefed.social/u/tomiant posted on Mar 13, 2026 12:36
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Man I always get sad when I see this meme format because the story behind it is so fucking tragic… :(

https://piefed.social/comment/10514037
Reply
$$11243
https://lemmy.ca/u/panda_abyss posted on Mar 13, 2026 12:37
In reply to: https://lemmy.today/comment/22854901

You can forgejo with a container index enabled, I don’t know if there’s a way to use that as a proxy for downloading containers though.

https://lemmy.ca/comment/22188532
Reply
$$11248
https://sh.itjust.works/u/nonentity posted on Mar 13, 2026 12:48
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

If it’s stable, it’s not a lab.

That’s infrastructure.

https://sh.itjust.works/comment/24266447
Reply
$$11250
https://lemmy.world/u/FauxLiving posted on Mar 13, 2026 12:53
In reply to: https://lemmy.today/comment/22855057

Oh. You don’t. Well, that’s probably okay. I mean, nothing will probably go wrong and render a device in need of being forcibly rebooted when you’re physically away from home.

*furiously adds a new item to the TODO list*

https://lemmy.world/comment/22639309
Reply
$$11254
https://lemmy.dbzer0.com/u/greedytacothief posted on Mar 13, 2026 12:56
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Yeah, my home server was being a little too stable and I wasn’t really learning anything. So I switched from fedora to proxmox, now I’ve got a nixos vm I’m going to try to get all my services running in.

https://lemmy.dbzer0.com/comment/24931392
Reply
$$11255
https://lemmy.world/u/FauxLiving posted on Mar 13, 2026 12:57
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

The comments in this thread have collectively created thousands of person-hours worth of work for us all…

https://lemmy.world/comment/22639385
Reply
$$11257
https://lemmy.zip/u/possiblylinux127 posted on Mar 13, 2026 13:04
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

You need monitoring

https://lemmy.zip/comment/25226867
Reply
$$11258
https://lemmy.world/u/InnerScientist posted on Mar 13, 2026 13:09
In reply to: https://feddit.org/comment/11996666

Infrastructure diagram? No! In this homelab we refer to the infrastructure hyperdodecahedron.

https://lemmy.world/comment/22639568
Reply
$$11261
https://lemmy.world/u/InnerScientist posted on Mar 13, 2026 13:14
In reply to: https://lemmy.today/comment/22854949

That won’t work in most cases, all https traffic isn’t cached unless you mitm https which is a bad idea and not worth it.

Only cache updates those are worth it and most have a caching server option.

https://lemmy.world/comment/22639661
Reply
$$11262
https://lemmy.world/u/InnerScientist posted on Mar 13, 2026 13:15
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

If logging is down and there’s no one around to log it, is it really down?

https://lemmy.world/comment/22639675
Reply
$$11263
https://lemmy.dbzer0.com/u/Martineski posted on Mar 13, 2026 13:16
In reply to: https://piefed.social/comment/10514037

What story?

https://lemmy.dbzer0.com/comment/24931745
Reply
$$11266
https://aussie.zone/u/Kurroth posted on Mar 13, 2026 13:20
In reply to: https://piefed.social/comment/10514037

I did not know there was a story, assumed it was from that TV series about the cartel guy

https://aussie.zone/comment/21905964
Reply
$$11267
https://sh.itjust.works/u/jaschen306 posted on Mar 13, 2026 13:27
In reply to: https://lemmy.zip/comment/25226867

Kubernetes?

https://sh.itjust.works/comment/24267144
Reply
$$11270
https://lemmy.world/u/non_burglar posted on Mar 13, 2026 13:35
In reply to: https://feddit.org/comment/11996666

Haha too right mate

https://lemmy.world/comment/22640042
Reply
$$11271
https://lemmy.dbzer0.com/u/wizardbeard posted on Mar 13, 2026 13:39
In reply to: https://lemmy.zip/comment/25226867

I’m remembering a very not fun discussion my team had about “the monitoring system not sending any alerts doesn’t inherently mean everything is ok” after an outage that was missed by our monitoring system.

You need to make sure you’re monitoring connectivity as well as specific problem states. No data is a problem state often overlooked, and it’s not always considered for every resource type in these systems out of the box.

And you probably want a heartbeat notification. Yes, it’s noise, but if you don’t see anything from monitoring you need to question if monitoring is the thing that broke. It sending out a notification every so often going “yes I am online” is useful.

https://lemmy.dbzer0.com/comment/24932202
Reply
$$11272
https://lemmy.dbzer0.com/u/wizardbeard posted on Mar 13, 2026 13:42
In reply to: https://lemmy.world/comment/22637259

Hearbeat notifications man. “Yes I am online” email once a day or so. Yeah it’s more emails to delete but it can be a lifesaver.

https://lemmy.dbzer0.com/comment/24932234
Reply
$$11273
https://lemmy.zip/u/chaotic_ugly posted on Mar 13, 2026 13:42
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Me to my lab.

https://lemmy.zip/pictrs/image/adf13581-185b-4900-8ac9-04333a1f8e32.avif

https://lemmy.zip/comment/25227661
Reply
$$11274
https://pawb.social/u/EonNShadow posted on Mar 13, 2026 13:44
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

I wish it was stable

I had a drive die yesterday

https://pawb.social/comment/21128344
Reply
$$11277
https://sh.itjust.works/u/DownByLaw posted on Mar 13, 2026 13:50
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Have you already tried implementing an identity provider like Authentik, so you can add OIDC and ldap for all your services, while you are the only one that’s using them? 🤔

https://sh.itjust.works/comment/24267535
Reply
$$11280
https://lemmy.world/u/PumpkinEscobar posted on Mar 13, 2026 13:54
In reply to: https://sh.itjust.works/comment/24267535

Behind a traefik reverse proxy with lets encrypt for ssl even though the services aren’t exposed to the internet?

https://lemmy.world/comment/22640354
Reply
$$11282
https://lemmy.world/u/Coleslaw4145 posted on Mar 13, 2026 13:59
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

No try migrating all your docker containers to podman.

https://lemmy.world/comment/22640438
Reply
$$11284
https://feddit.nl/u/shane posted on Mar 13, 2026 14:04
In reply to: https://lemmy.dbzer0.com/comment/24932202

One alert daily reporting that there are no alerts is probably good for a home lab….

https://feddit.nl/comment/23859188
Reply
$$11286
https://lemmy.world/u/nucleative posted on Mar 13, 2026 14:10
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Never run:

docker compose pull docker compose down && docker compose up -d

Right before the end of your day. Ask me how I know 😂

https://lemmy.world/comment/22640637
Reply
$$11288
https://lemmy.world/u/AnUnusualRelic posted on Mar 13, 2026 14:13
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

That’s not a homelab, that’s a home server.

https://lemmy.world/comment/22640673
Reply
$$11292
https://feditown.com/u/exu posted on Mar 13, 2026 14:17
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

I test in my Homeproduction

https://feditown.com/comment/5129852
Reply
$$11293
https://feddit.org/u/mech posted on Mar 13, 2026 14:18
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Time to switch distros!

https://feddit.org/comment/11999649
Reply
$$11297
https://mander.xyz/u/fossilesque posted on Mar 13, 2026 14:21
In reply to: https://lemmy.world/comment/22640438

Don’t encourage me.

https://mander.xyz/comment/25848637
Reply
$$11301
https://piefed.ca/u/cenzorrll posted on Mar 13, 2026 14:27
In reply to: https://piefed.zip/comment/4233081

“Damn, I’ve got this Debian server shit down. I wonder how an opensuse server would work out” *installs tumbleweed *

True story

https://piefed.ca/comment/3833545
Reply
$$11305
https://mander.xyz/u/piranhaconda posted on Mar 13, 2026 14:30
In reply to: https://slrpnk.net/comment/21223483

What’s a backup solution..? (I’m only being half sarcastic, I really need to set one up, but it’s not as “fun” as the rest of my homelab, send suggestions)

https://mander.xyz/comment/25848789
Reply
$$11308
https://lemmy.wtf/u/diablomnky666 posted on Mar 13, 2026 14:34
In reply to: https://lemmy.world/comment/22640354

To be fair a lot of apps don’t handle custom CAs like they should. Looking at you Home Assistant! 😠

https://lemmy.wtf/comment/20455516
Reply
$$11309
https://programming.dev/u/epicshepich posted on Mar 13, 2026 14:35
In reply to: https://mander.xyz/comment/25848637

And then try turning on SELinux!

https://programming.dev/comment/22698836
Reply
$$11310
https://lemmy.sdf.org/u/SexualPolytope posted on Mar 13, 2026 14:35
In reply to: https://lemmy.world/comment/22640438

Just did that last weekend. Nothing to do anymore. 😢

https://lemmy.sdf.org/comment/26593446
Reply
$$11312
https://programming.dev/u/epicshepich posted on Mar 13, 2026 14:35
In reply to: https://sh.itjust.works/comment/24267535

Hey my wife uses some of them too!

https://programming.dev/comment/22698857
Reply
$$11315
https://piefed.ca/u/cenzorrll posted on Mar 13, 2026 14:39
In reply to: https://ani.social/comment/15651460

Saturday morning: “Incus and podman seem interesting. I bet I could swap everything over while the family is out this afternoon”

Sunday evening: “Dad, when will the lights work again?”

https://piefed.ca/comment/3833699
Reply
$$11317
https://feditown.com/u/exu posted on Mar 13, 2026 14:46
In reply to: https://lemmy.sdf.org/comment/26593446

Did you do Quadlets?

https://feditown.com/comment/5130056
Reply
$$11319
https://lemmy.world/u/sytone posted on Mar 13, 2026 14:49
In reply to: https://lemmy.today/comment/22855057

Tal just got the chaotic evil tag today.

https://lemmy.world/comment/22641256
Reply
$$11320
https://piefed.social/u/notabot posted on Mar 13, 2026 14:50
In reply to: https://programming.dev/comment/22694964

Have you tested your backups recently? Having them complete is one thing, having the data you need for recovery is another. Have you backed up your vm configurations and build scripts?

Go test your latest backup!

https://piefed.social/comment/10515790
Reply
$$11322
https://ani.social/u/Sabata11792 posted on Mar 13, 2026 14:52
In reply to: https://piefed.ca/comment/3833699

“Dad, when will the lights work again?

As soon as selinux decides I have permission.

https://ani.social/comment/15653850
Reply
$$11323
https://lemmy.sdf.org/u/SexualPolytope posted on Mar 13, 2026 14:53
In reply to: https://feditown.com/comment/5130056

Yes of course. Had to spend a couple of hours fixing permission related issues.

https://lemmy.sdf.org/comment/26593787
Reply
$$11324
https://piefed.ca/u/cenzorrll posted on Mar 13, 2026 14:55
In reply to: https://lemmy.today/comment/22854693

Hmmm. My pi{VPN,hole,dhcp,HA} has a little bit of overhead left…

https://piefed.ca/comment/3833877
Reply
$$11330
https://programming.dev/u/Avicenna posted on Mar 13, 2026 15:06
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

You can always configure your vim further

https://programming.dev/comment/22699345
Reply
$$11329
https://lemmy.world/u/Agent641 posted on Mar 13, 2026 15:06
In reply to: https://lemmy.world/comment/22639675

Who will log the loggers?

https://lemmy.world/comment/22641554
Reply
$$11331
https://programming.dev/u/shym3q posted on Mar 13, 2026 15:09
In reply to: https://lemmy.world/comment/22640637

compose up will automatically recreate with newer images if the new one were pulled. so there is no need for compose down btw

https://programming.dev/comment/22699422
Reply
$$11333
https://lemmy.world/u/erev posted on Mar 13, 2026 15:09
In reply to: https://programming.dev/comment/22698836

It’s not that difficult to get SELinux working with podman quadlets, especially if you run things rootless. I have a kerberized service account for each application I host and my quadlets are configured to run under those. I very rarely encounter applications that simoky can’t be run rootless but I usually can find an adequate alternative. I think right now the only thing that runs as root is one of the talk or collabora containers in my nextcloud stack. No selinux issues either.

https://lemmy.world/comment/22641628
Reply
$$11335
https://piefed.ca/u/cenzorrll posted on Mar 13, 2026 15:13
In reply to: https://sh.itjust.works/comment/24266447

I’ve moved my homelab twice because it became stable, I really liked the services it was running, and I didn’t want to disturb the last lab**cough**prod server.

My current homelab will be moar containers. I’m sure I’ll push it to prod instead of changing the IP address and swapping name tags this time.

https://piefed.ca/comment/3834134
Reply
$$11339
https://sh.itjust.works/u/DownByLaw posted on Mar 13, 2026 15:23
In reply to: https://lemmy.world/comment/22640354

Don’t forget about Anubis and crowdsec to make it even safer inside your LAN

https://sh.itjust.works/comment/24269107
Reply
$$11344
https://slrpnk.net/u/Prunebutt posted on Mar 13, 2026 15:40
In reply to: https://mander.xyz/comment/25848789

No mercy for you, then. ;)

https://slrpnk.net/comment/21228767
Reply
$$11353
https://feddit.nl/u/quick_snail posted on Mar 13, 2026 15:55
In reply to: https://lemmy.today/comment/22854793

Wazuh ftw

https://feddit.nl/comment/23861376
Reply
$$11356
https://mander.xyz/u/piranhaconda posted on Mar 13, 2026 15:59
In reply to: https://slrpnk.net/comment/21228767

I at least have external backups for important family pics and docs! But yea the homelab itself is severely lacking. If it dies, I get to start from scratch. Been gambling for years that “I’ll get around to a backup solution” before it dies. I wouldn’t bet on me :|

https://mander.xyz/comment/25850649
Reply
$$11357
https://lemmy.today/u/tal posted on Mar 13, 2026 16:00
In reply to: https://sh.itjust.works/comment/24267535

Probably a good idea to switch over to WPA-Enterprise using Authentik’s RADIUS server support and let all of the users of your wireless access point log in with their own network credentials, while you’re at it.

https://lemmy.today/comment/22861217
Reply
$$11359
https://lemmy.world/u/poolhelmetinstrument posted on Mar 13, 2026 16:03
In reply to: https://lemmy.sdf.org/comment/26593787

But did you run them as rootful or the intended rootless way.

https://lemmy.world/comment/22642583
Reply
$$11361
https://lemmy.today/u/tal posted on Mar 13, 2026 16:05
In reply to: https://lemmy.world/comment/22639568

It seems like a good time to learn graphviz’s dot format for the network layout diagrams, with automated layout.

https://blog.ipspace.net/kb/NetAutJourney/40-Network-Diagrams/

https://lemmy.today/comment/22861316
Reply
$$11364
https://programming.dev/u/epicshepich posted on Mar 13, 2026 16:13
In reply to: https://lemmy.world/comment/22641628

I use podman-compose with system accounts and I don’t have a ton of issues. The biggest one is that I can’t seem to get bluetooth and pip working on Home Assistant at the same time. Most of the servers I manage have SELinux and it works fine as long as I use :z/:Z with bind mounts.

A few years ago, I set up a VPS for my friend’s business; at the time, I didn’t know how to work with SELinux so I just turned it off. I tried to flip it back on, and it somehow bricked the system. We had to restore from a backup. Since then, I’ve been afraid to enable it on my flagship homelab server.

https://programming.dev/comment/22700649
Reply
$$11366
https://lemmy.world/u/PHLAK posted on Mar 13, 2026 16:21
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Time to start documenting it!

https://lemmy.world/comment/22642913
Reply
$$11367
https://mander.xyz/u/BuboScandiacus posted on Mar 13, 2026 16:21
In reply to: https://lemmy.world/comment/22642913

NEVER1!!!11!!

https://mander.xyz/comment/25851136
Reply
$$11371
https://lemmy.world/u/Dultas posted on Mar 13, 2026 16:28
In reply to: https://lemmy.ml/comment/24509842

Guess this is a good time to test my infrastructure automation.

https://lemmy.world/comment/22643080
Reply
$$11373
https://sh.itjust.works/u/WhyJiffie posted on Mar 13, 2026 16:28
In reply to: https://programming.dev/comment/22700649

are you sure it really bricked it? when turning it on, on next boot it needs to go over all the files and retag them or something like that, and it can take a significant amount of time

https://sh.itjust.works/comment/24270275
Reply
$$11375
https://sh.itjust.works/u/WhyJiffie posted on Mar 13, 2026 16:38
In reply to: https://lemmy.dbzer0.com/comment/24932234

but you probably won’t notice that some of the regular emails are not sent anymore

https://sh.itjust.works/comment/24270481
Reply
$$11377
https://lemmy.world/u/olafurp posted on Mar 13, 2026 16:42
In reply to: https://programming.dev/comment/22698836

I set my homelab up on Bazzite immutable with podman and SELinux. It took a while to work everything out and have it boot up into a valid state hahaha

https://lemmy.world/comment/22643355
Reply
$$11381
https://lemmy.world/u/sibannac posted on Mar 13, 2026 16:51
In reply to: https://lemmy.world/comment/22642913

Don’t look too closely you can jinx it.

https://lemmy.world/comment/22643517
Reply
$$11387
https://piefed.social/u/immobile7801 posted on Mar 13, 2026 16:58
In reply to: https://feditown.com/comment/5130056

I had problems getting apps with multiple containers working in quadlets (definitely a knowledge issue on my part, but didn’t feel the time learning it was beneficial, but will probably revisit during kubernetes learning) so went back to podman with docker compose.

https://piefed.social/comment/10517518
Reply
$$11388
https://lemmy.zip/u/rumba posted on Mar 13, 2026 16:58
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Nothing to install? Not with that attitude!

Start a 10” rack.

https://lemmy.zip/comment/25231725
Reply
$$11391
https://programming.dev/u/CameronDev posted on Mar 13, 2026 17:04
In reply to: https://piefed.social/comment/10515790

Restore is future me’s problem. Fuck that guy :D

https://programming.dev/comment/22701585
Reply
$$11392
https://lemmy.world/u/jeffep posted on Mar 13, 2026 17:08
In reply to: https://sh.itjust.works/comment/24270481

Couple it to your smart watch, backup every 10 seconds, and make it vibrate when successful

https://lemmy.world/comment/22643780
Reply
$$11393
https://piefed.social/u/notabot posted on Mar 13, 2026 17:10
In reply to: https://programming.dev/comment/22701585

Ah, that frission of excitement when you come to restore! Will it work? Does it contain that very important file? Is it up to date? How much will future you hate past you if it isn’t there?

https://piefed.social/comment/10517654
Reply
$$11394
https://lemmy.world/u/jeffep posted on Mar 13, 2026 17:11
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Can’t believe nobody here mentioned nixOS so far? How about moving all of your configs in a flake and manage all of your systems with it?

https://lemmy.world/comment/22643819
Reply
$$11396
https://lemmy.world/u/jenings posted on Mar 13, 2026 17:13
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Started running unmanic on my plex library to save hard drive space since apparently the powers that be don’t want us to even own hard drives anymore. So far it’s going great, it’ll probably take weeks since I don’t have a gpu hooked up to it

https://lemmy.world/comment/22643857
Reply
$$11399
https://piefed.zeromedia.vip/u/fleem posted on Mar 13, 2026 17:16
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

heck i really wish we could all throw a party together. part swap, stories swap. show off cool shit for everyone to copy.

help each other fill in the missing pieces

y’all seem like cool peeps meme-ing about shit nobody else gets!

time to test the backups!

https://piefed.zeromedia.vip/comment/1819614
Reply
$$11403
https://sh.itjust.works/u/WhyJiffie posted on Mar 13, 2026 17:18
In reply to: https://lemmy.world/comment/22643780

you are just making yourself learn to ignore that your smartwatch vibrates. It’s a bit like breathing and blinking, you are so used to it you can completely forget that its happening. if your smartwatch, or phone, or whatever, starts vibrating all the time, you will get used to it and not notice when it stops happening anymore, but also it will hide any actually meaningful notification.

https://sh.itjust.works/comment/24271123
Reply
$$11404
https://lemmy.dbzer0.com/u/dditty posted on Mar 13, 2026 17:20
In reply to: https://lemmy.today/comment/22855480

I just installed Debian on a decommissioned Chromebox for exactly this purpose + 4x usb-to-serial adapters.

https://lemmy.dbzer0.com/comment/24936141
Reply
$$11407
https://lemmy.world/u/FauxLiving posted on Mar 13, 2026 17:24
In reply to: https://lemmy.world/comment/22643819

I made a git repo and started putting all of my dot files in a Stow and then I forgot why I was doing it in the first place.

https://lemmy.world/comment/22644014
Reply
$$11408
https://lemmy.world/u/FauxLiving posted on Mar 13, 2026 17:25
In reply to: https://programming.dev/comment/22699345

or learn emacs

https://lemmy.world/comment/22644017
Reply
$$11410
https://lemmy.world/u/Ensign_Crab posted on Mar 13, 2026 17:29
In reply to: https://piefed.zeromedia.vip/comment/1819614

You just described a convention.

https://lemmy.world/comment/22644100
Reply
$$11412
https://programming.dev/u/epicshepich posted on Mar 13, 2026 17:36
In reply to: https://sh.itjust.works/comment/24270275

Honestly, I don’t know what happened, but it was unreachable via SSH and the web console. There shouldn’t have been a ton of files to tag since it was an Almalinux system that started with SELinux enabled, and all we added was a container app or two.

https://programming.dev/comment/22702065
Reply
$$11413
https://lemmy.today/u/tal posted on Mar 13, 2026 17:36
In reply to: https://ani.social/comment/15653850

The old lighting wasn’t that great anyway. If I were to just put lighting on a DMX512-controlled network, then all of it could be synchronized to whole-house audio…

https://lemmy.today/comment/22863144
Reply
$$11414
https://programming.dev/u/Zink posted on Mar 13, 2026 17:37
In reply to: https://feddit.org/comment/11996666

This is just as true in my non-computer hobbies that involve physical systems instead of code and configs!

If I had to just barely meet the requirements using as little budget as possible while making it easy for other people to work on, that would be called “work.” My brain needs to indulge in some over-engineering and “I need to see it for myself” kind of design decisions.

https://programming.dev/comment/22702089
Reply
$$11415
https://programming.dev/u/epicshepich posted on Mar 13, 2026 17:37
In reply to: https://lemmy.world/comment/22643355

Any reason you chose Bazzite for your homelab distro? First I’ve heard of someone doing that!

https://programming.dev/comment/22702090
Reply
$$11420
https://sopuli.xyz/u/Alaknar posted on Mar 13, 2026 17:45
In reply to: https://lemmy.world/comment/22644017

Then configure vim using emacs

https://sopuli.xyz/comment/22408751
Reply
$$11422
https://sopuli.xyz/u/Alaknar posted on Mar 13, 2026 17:46
In reply to: https://programming.dev/comment/22702090

Wouldn’t an immutable OS be overall a pretty good idea for a stable server?

https://sopuli.xyz/comment/22408763
Reply
$$11432
https://lemmy.world/u/irmadlad posted on Mar 13, 2026 17:59
In reply to: https://piefed.zeromedia.vip/comment/1819614

time to test the backups!

Always a white knuckle event for me

https://lemmy.world/comment/22644573
Reply
$$11433
https://lemmy.world/u/RizzRustbolt posted on Mar 13, 2026 17:59
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Time to expand.

https://lemmy.world/comment/22644576
Reply
$$11434
https://lemmy.sdf.org/u/SexualPolytope posted on Mar 13, 2026 17:59
In reply to: https://lemmy.world/comment/22642583

Rootless. The docker containers were rootful, hence the permission struggles.

https://lemmy.sdf.org/comment/26597531
Reply
$$11435
https://lemmy.world/u/irmadlad posted on Mar 13, 2026 18:00
In reply to: https://lemmy.world/comment/22642913

At 71, I have to document. I started a long time ago. I worked for a mec. contractor long ago, and the rule was: ‘If you didn’t write it down, it didn’t happen.’ That just carried over to everything I do.

https://lemmy.world/comment/22644597
Reply
$$11436
https://lemmy.sdf.org/u/SexualPolytope posted on Mar 13, 2026 18:01
In reply to: https://piefed.social/comment/10517518

I think it’s kinda better using quadlets, because I wrote some custom scripts, and quadlets made the process better. But podman compose is probably file too.

https://lemmy.sdf.org/comment/26597553
Reply
$$11439
https://lemmy.today/u/paequ2 posted on Mar 13, 2026 18:02
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Actually, one thing I want to do is switch from services being on a subdomain to services being on a path.

immich.myserver.com -> myserver.com/immich
jellyfin.myserver.com -> myserver.com/jellyfin

I’m getting tired of having to update DNS records every time I want to add a new service.

I guess the tricky part will be making sure the services support this kind of routing…

https://lemmy.today/comment/22863639
Reply
$$11438
https://lemmy.world/u/irmadlad posted on Mar 13, 2026 18:02
In reply to: https://feddit.org/comment/11996666

unnecessary complexity?

I can help with that. It’s a skill I have. LOL

https://lemmy.world/comment/22644623
Reply
$$11440
https://ani.social/u/Sabata11792 posted on Mar 13, 2026 18:03
In reply to: https://lemmy.today/comment/22863144

Don’t forget to integrate it into Home Assistant so you can alert the ISS when the mail man is on the porch.

https://ani.social/comment/15656891
Reply
$$11442
https://lemmy.world/u/irmadlad posted on Mar 13, 2026 18:05
In reply to: https://lemmy.world/comment/22640637

Right before the end of your day

Oh, gosh, I did this last evening. I didn’t check what time it was, and initiated an update on some 70 containers. I have a cron that shuts down the server in the evening, and sure enough, right in the middle of the updates, it powered off. I didn’t even mess with it and went to bed. Re-initiated the update this morning, and everything is up and running. Whew!

https://lemmy.world/comment/22644681
Reply
$$11444
https://lemmy.world/u/irmadlad posted on Mar 13, 2026 18:07
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

OP, totally understand, but this is a level of success with your homelab. Nothing needs fiddling with. Now, there is a whole Awesome Self Hosted list you could deploy on a non-production server and run that through the paces.

https://lemmy.world/comment/22644718
Reply
$$11446
https://lemmy.world/u/FauxLiving posted on Mar 13, 2026 18:08
In reply to: https://sopuli.xyz/comment/22408751

https://media3.giphy.com/media/v1.Y2lkPTc5MGI3NjExZjRrbWhyMm5heXQ1dDY2eDF2a2ZqcXN1d2NtbmVxOG5pb2FqNm5nbyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/Xn7mOX7VQDDOw/giphy.gif

https://lemmy.world/comment/22644735
Reply
$$11450
https://discuss.tchncs.de/u/shadowtofu posted on Mar 13, 2026 18:10
In reply to: https://lemmy.today/comment/22863639

I had the same idea, but the solution I thought about is finding a way to define my DNS records as code, so I can automate the deployment. But the pain is tolerable so far (I have maybe 30 subdomains?), I haven’t done anything yet

https://discuss.tchncs.de/comment/24508332
Reply
$$11452
https://sopuli.xyz/u/HK65 posted on Mar 13, 2026 18:11
In reply to: https://lemmy.today/comment/22863639

In Nginx you can do rewrites so services think they are at the root.

https://sopuli.xyz/comment/22409163
Reply
$$11460
https://sh.itjust.works/u/CorvidCawder posted on Mar 13, 2026 18:17
In reply to: https://lemmy.today/comment/22863639

Wildcard CNAME pointing to your reverse proxy who then figures out where to route the request to? That’s what I’ve been doing - this way there’s no need to ever update DNS at all :)

I find the path a bit clunky because the apps themselves will oftentimes get confused (especially front-ends). So keeping everything “bare” wrt path, and just on “separate” subdomains is usually my preferred approach.

https://sh.itjust.works/comment/24272011
Reply
$$11463
https://lemmy.today/u/tal posted on Mar 13, 2026 18:20
In reply to: https://lemmy.world/comment/22644014

So that when setting up a new system, you can migrate all your user configuration easily, while also version-controlling it.

https://lemmy.today/comment/22863996
Reply
$$11465
https://lemmy.world/u/FauxLiving posted on Mar 13, 2026 18:23
In reply to: https://lemmy.today/comment/22863996
git commit --message 'So that when setting up a new system, you can migrate all your user configuration easily, while also version-controlling it.'
https://lemmy.world/comment/22645033
Reply
$$11467
https://programming.dev/u/Bibip posted on Mar 13, 2026 18:23
In reply to: https://lemmy.ml/comment/24509842

Scarched arth

https://programming.dev/comment/22702867
Reply
$$11475
https://lemmy.world/u/Admax posted on Mar 13, 2026 18:31
In reply to: https://lemmy.dbzer0.com/comment/24932234

Oh but I have them !
Every day an email is sent out with the backup status.
Every day I got my email in the morning with the back up logs.
For years.
I associated email received to backup successful, until a month or so when my vpn broke and the emails where just “could not connect”, but it took me a while to bother actually opening the message body as it had always been the same for years.

So I’ll manage it differently, have the email subject be more explicit about a success or a failure amongst other things.
Always learning :^)

https://lemmy.world/comment/22645186
Reply
$$11497
https://lemmy.blahaj.zone/u/magic_smoke posted on Mar 13, 2026 18:57
In reply to: https://lemmy.today/comment/22863639

Alternatively if you’re tired of manual DNS configuration:

FreeIPA

Configures users, sudoer group, ssh keys, and DNS in one go.

Also lotta services can be integrated using LDAP auth too.

So far I’ve got proxmox, jellyfin, zoneminder, mediawiki, and forgejo authing against freeipa in top of my samba shares.

https://lemmy.blahaj.zone/comment/19611767
Reply
$$11523
https://lemmy.world/u/suicidaleggroll posted on Mar 13, 2026 19:25
In reply to: https://lemmy.today/comment/22863639

Why are you having to update your DNS records when you add a new service? Just set up a wildcard A record to send *.myserver.com to the reverse proxy and you never have to touch it again.

https://lemmy.world/comment/22646292
Reply
$$11528
https://lemmy.world/u/suicidaleggroll posted on Mar 13, 2026 19:34
In reply to: https://lemmy.world/comment/22640354

Who cares if it’s exposed to the internet?

  1. Encrypting your local traffic is still valuable to protect your systems from any bad actors on your local network (neighbor kid cracks your wifi password, some device on your network decides to start snooping on your local traffic, etc)

  2. Many services require HTTPS with a valid cert to function correctly, eg: Bitwarden. Having a real cert for a real domain is much simpler and easier to maintain than setting up your own CA

https://lemmy.world/comment/22646433
Reply
$$11532
https://slrpnk.net/u/moonshadow posted on Mar 13, 2026 19:35
In reply to: https://mander.xyz/comment/25850649

https://wiki.archlinux.org/title/Timeshift

https://slrpnk.net/comment/21232743
Reply
$$11534
https://lemmy.zip/u/yabbadabaddon posted on Mar 13, 2026 19:38
In reply to: https://lemmy.world/comment/22643819

I already have Ansible to manage my system and I like to have the same base between my pc and my server build muscle memory.

If I was managing a pc fleet I would consider NixOS, but I don’t see the appeal right now.

https://lemmy.zip/comment/25234822
Reply
$$11535
https://lemmy.world/u/Abbysimons posted on Mar 13, 2026 19:38
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

The rare moment when everything actually works. 😄

https://lemmy.world/comment/22646513
Reply
$$11541
https://lemmy.world/u/Vile_port_aloo posted on Mar 13, 2026 19:45
In reply to: https://lemmy.world/comment/22644597

Do you write down what you write down on the internet?

https://lemmy.world/comment/22646635
Reply
$$11542
https://lemmy.sdf.org/u/chM5tZ8zMp posted on Mar 13, 2026 19:46
In reply to: https://lemmy.today/comment/22861316

TIL. Thank you!

https://lemmy.sdf.org/comment/26599659
Reply
$$11547
https://programming.dev/u/epicshepich posted on Mar 13, 2026 19:52
In reply to: https://sopuli.xyz/comment/22408763

I honestly don’t know a ton about immutable distros other than that they let you front-load some difficulty in getting things set up in exchange for making it harder to break. I was just surprised that the distro of choice was Bazzite, since its target audience seems to be gamers.

https://programming.dev/comment/22704379
Reply
$$11567
https://programming.dev/u/moonpiedumplings posted on Mar 13, 2026 20:19
In reply to: https://piefed.zeromedia.vip/comment/1819614

https://infosecmap.com/

https://wiki.hackerspaces.org/List_of_Hacker_Spaces

Also check out meetup.com for linux user groups and other events.

https://programming.dev/comment/22704807
Reply
$$11579
https://lemmy.world/u/Tangent5280 posted on Mar 13, 2026 20:53
In reply to: https://piefed.social/comment/10513294

Carry around a candle in one of those old timey holders like Scrooge Mcduck

https://lemmy.world/comment/22647754
Reply
$$11596
https://lemmy.world/u/synapse1278 posted on Mar 13, 2026 21:14
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Living the good life

https://lemmy.world/comment/22648075
Reply
$$11612
https://lemmy.world/u/irmadlad posted on Mar 13, 2026 21:43
In reply to: https://lemmy.world/comment/22646635

As in a blog or wiki? I do not because I am not authoritative. What I know came from reading, doing, screwing it up, ad nauseam. When something finally clicks for me, I write it down because 9 times out of 10, I will need that info later. But my writing would be so full of inaccuracies that it would be embarrassing and possibly lead someone astray.

https://lemmy.world/comment/22648508
Reply
$$11623
https://lemmy.dbzer0.com/u/Scrath posted on Mar 13, 2026 22:02
In reply to: https://lemmy.world/comment/22646292

Not OP but a lot of people probably use pi-hole which doesn’t support wildcards for some inane reason

https://lemmy.dbzer0.com/comment/24940834
Reply
$$11639
https://lemmy.today/u/tal posted on Mar 13, 2026 22:18
In reply to: https://lemmy.sdf.org/comment/26599659

Sure. What that guy is using is actually not the most-interesting diagram style, IMHO, for automatic layout of network maps, if you want large-scale stuff, which is where the automatic layout gets more interesting. I have some scripts floating around somewhere that will generate very large network maps — run a bunch of traceroutes, geolocate IPs, dump the results into an sqlite database, and then generate an automatically laid-out Internet network map. I don’t want to go to the trouble of anonymizing the addresses and locations right now, but if you have a graphviz graph and want to try playing with it, I used:

goes looking

Ugh, it’s Python 2, a decade-and-a-half old, and never got ported. Lemme gin up an example for the non-hierarchical graphviz stuff:

graph.dot:

graph foo {
    a--b
    a--d
    b--c
    d--e
    c--e
    e--f
    b--d
}

Processed with:

$ sfdp -Goverlap=prism -Gsep=+5 -Gesep=+4 -Gremincross -Gpack -Gsplines=true -Tpdf -o graph.pdf graph.dot

Generates something like this:

https://lemmy.today/pictrs/image/c7fb0167-fbda-47f5-914f-a0daa3066c67.png

That’ll take a ton of graphviz edges and nicely lay them out, albeit not in that kind of hierachy shown. You can create massive network maps like this. Note that was last looking at graphviz’s automated layout stuff about 15 years ago, so it’s possible that they have better algorithms now, but this can deal with enormous numbers of nodes and will do reasonable things with them.

I just grabbed his example because it was the first graphviz network map example that came up.

https://lemmy.today/comment/22868426
Reply
$$11643
https://lemmy.blahaj.zone/u/emerald posted on Mar 13, 2026 22:23
In reply to: https://lemmy.world/comment/22640438

And then migrate all your podman containers to proxmox

https://lemmy.blahaj.zone/comment/19614417
Reply
$$11656
https://lemmy.zip/u/jjlinux posted on Mar 13, 2026 22:43
In reply to: https://lemmy.blahaj.zone/comment/19611767

Having a very similar infrastructure, I would love to know if you ever find anything that works for this. I’ve been maintaining a SnipeIT instance manually, but that’s a real PITA. Tried the same with ITSM-NG, but haven’t even lookid in it for months.

https://lemmy.zip/comment/25238124
Reply
$$11662
https://lemmy.world/u/kamen posted on Mar 13, 2026 22:50
In reply to: https://lemmy.world/comment/22646513

Quick! Break something!

https://lemmy.world/comment/22649490
Reply
$$11665
https://sopuli.xyz/u/mr_anny posted on Mar 13, 2026 22:56
In reply to: https://lemmy.world/comment/22649490

Maybe try this…

https://sopuli.xyz/comment/22413754
Reply
$$11671
https://sh.itjust.works/u/Croquette posted on Mar 13, 2026 23:04
In reply to: https://lemmy.dbzer0.com/comment/24940834

That’s my case. I send every new subdomain to my nginx IP on pi-hole and then use nginx as a reverse proxy

https://sh.itjust.works/comment/24276515
Reply
$$11681
https://sh.itjust.works/u/fruitycoder posted on Mar 13, 2026 23:20
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

How is the kubernetes (k3s/rke2) migration coming along?

https://sh.itjust.works/comment/24276777
Reply
$$11698
https://startrek.website/u/qjkxbmwvz posted on Mar 13, 2026 23:43
In reply to: https://lemmy.dbzer0.com/comment/24940834

I switched to Technitium and I’ve been pretty happy. Seems very robust, and as a bonus was easy to use it to stop DNS leaks (each upstream has a static route through a different Mullvad VPN, and since they’re queried in parallel, a VPN connection can go down without losing any DNS…maybe this is how pihole would have handled it too though).

And of course, wildcards supported no problem.

https://startrek.website/comment/21889851
Reply
$$11712
https://lemmy.dbzer0.com/u/Scrath posted on Mar 13, 2026 23:56
In reply to: https://sh.itjust.works/comment/24276515

That wasy exact setup as well until I switched to a different router which supported both custom DNS entries and blocklists, thereby making the pi-hole redundant

https://lemmy.dbzer0.com/comment/24942669
Reply
$$11723
https://feddit.org/u/Shayeta posted on Mar 14, 2026 00:09
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

One word: chaos engineering

https://feddit.org/comment/12008485
Reply
$$11728
https://sh.itjust.works/u/FreshLight posted on Mar 14, 2026 00:17
In reply to: https://piefed.zip/comment/4233081

My man person!

https://sh.itjust.works/comment/24277553
Reply
$$11737
https://lemmy.world/u/Bakkoda posted on Mar 14, 2026 00:30
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

I should do some breaking network changes… While tunneled in.

https://lemmy.world/comment/22650821
Reply
$$11742
https://lemmy.world/u/nucleative posted on Mar 14, 2026 00:38
In reply to: https://programming.dev/comment/22699422

You’re right. I got in the habit of doing that because I’m endlessly tweaking my .env files and I don’t think those reload unless you shut down first

https://lemmy.world/comment/22650902
Reply
$$11743
https://lemmy.world/u/jeffep posted on Mar 14, 2026 00:40
In reply to: https://lemmy.zip/comment/25234822

Okay, but why not create more work for yourself by rebuilding everything from scratch?

https://lemmy.world/comment/22650926
Reply
$$11765
https://lemmy.world/u/damnthefilibuster posted on Mar 14, 2026 01:51
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Backups. You’re forgetting them.

https://lemmy.world/comment/22651585
Reply
$$11772
https://lemmy.today/u/MonkeMischief posted on Mar 14, 2026 02:02
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Don’t worry, you’re one Docker pull away from having to look up how to manually migrate Postgres databases within running containers!

(Looks at my PaperlessNGX container still down. Still irritated.)

https://lemmy.today/comment/22871764
Reply
$$11778
https://lemmy.world/u/Fmstrat posted on Mar 14, 2026 02:26
In reply to: https://lemmy.today/comment/22855057

I built an 8 outlet version of those with relays and wall outlets for.. a lot less.

https://lemmy.world/comment/22651901
Reply
$$11781
https://lemmy.world/u/Fmstrat posted on Mar 14, 2026 02:27
In reply to: https://lemmy.world/comment/22638700

An 8 switch relay, old Pi, and 8 hardware store outlets can be had for not much more. I did that and let PiKVM control my outlets directly.

https://lemmy.world/comment/22651913
Reply
$$11815
https://sh.itjust.works/u/Croquette posted on Mar 14, 2026 03:52
In reply to: https://lemmy.dbzer0.com/comment/24942669

I run opnsense, so I need to dump pi-hole. But I don’t have the energy right now to do that.

Pi-Hole was pretty straightforward at the time and I did not look back since then. Annoying, but easy.

https://sh.itjust.works/comment/24279625
Reply
$$11833
https://lemmy.world/u/olafurp posted on Mar 14, 2026 05:07
In reply to: https://programming.dev/comment/22702090

At the start I just wanted a desktop machine that runs Steam through sunshine/moonlight so hardware support and gaming stuff such was very important.

My homelab used to run on my laptop when it could all fit within a couple 100s of GB and I was the only user but moving it was tricky. Since I’m a programmer I’m not afraid of this stuff so I just spent the hours to figure out one problem at a time.

I ended up figuring out adding HDD whitelist in SELinux, make it accessible in podman, manually edit fstab because tools didn’t work, systemd service for startup, logging in automatically where I already forgot everything and would have not had to do any of this on a bog standard Ubuntu server.

https://lemmy.world/comment/22653244
Reply
$$11835
https://lemmy.world/u/olafurp posted on Mar 14, 2026 05:07
In reply to: https://sopuli.xyz/comment/22408763

Good for stability, bad for flexibility for when the homelab grows more complex.

https://lemmy.world/comment/22653254
Reply
$$11892
https://programming.dev/u/epicshepich posted on Mar 14, 2026 08:39
In reply to: https://lemmy.world/comment/22653244

Respect! I too often take it for granted that it’s a privilege for my gaming rig and my homelab server to be separate boxes.

https://programming.dev/comment/22713117
Reply
$$11897
https://feddit.org/u/Hupf posted on Mar 14, 2026 08:55
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

No upstream bugs to fix?

https://feddit.org/comment/12012845
Reply
$$11901
https://sh.itjust.works/u/WhyJiffie posted on Mar 14, 2026 09:00
In reply to: https://programming.dev/comment/22702065

that started with SELinux enabled

that does not matter, it needs to go over all of them. I don’t know how long it takes with SSD, but with HDD it can take a half an hour or more, with a mostly base system. and the kernel starts doing this very early, when not even systemd or other processes are running, so no ssh, but web console should have been working to see what its doing

https://sh.itjust.works/comment/24282198
Reply
$$11903
https://programming.dev/u/epicshepich posted on Mar 14, 2026 09:03
In reply to: https://sh.itjust.works/comment/24282198

Good to know! I do hope to eventually re-enable SELinux on my flagship server, so I’ll keep this in mind. As for my friend’s server, I think he migrated to Alpine a while back.

https://programming.dev/comment/22713289
Reply
$$11914
https://lemmy.world/u/Puddinghelmet posted on Mar 14, 2026 09:31
In reply to: https://lemmy.world/comment/22649490

Wreck it Ralph!!

https://lemmy.world/comment/22655439
Reply
$$11919
https://lemmy.world/u/olafurp posted on Mar 14, 2026 09:36
In reply to: https://programming.dev/comment/22713117

I came to the same conclusion, Nobara for would have been best.

https://lemmy.world/comment/22655496
Reply
$$11935
https://feddit.uk/u/sunbeam60 posted on Mar 14, 2026 10:36
In reply to: https://lemmy.world/comment/22650821

“Yes, while connected to my wireguard server through port 123 here from my Chinese office, I should probably try to upgrade the wireguard server. That’s a great idea!”

Ask me how I know.

https://feddit.uk/comment/23840414
Reply
$$11940
https://lemmy.zip/u/Klajan posted on Mar 14, 2026 10:42
In reply to: https://lemmy.dbzer0.com/comment/24940834

It does support it, you just have to add it to dnsmasq. I have it Setup under misc.dnsmasq_lines like so:

address=/proxy.example.com/192.0.0.100
local=/proxy.example.com/

Then I have my proxied service reachable under service.proxy.example.com

https://lemmy.zip/comment/25246146
Reply
$$11946
https://sh.itjust.works/u/TheparishofChigwell posted on Mar 14, 2026 10:54
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Off topic, warning: this comment section is making me want to learn things

It’s been 2 days off reddit and my brain has opinions other than “aaaargh” or “meh”.

Proceed with caution

https://sh.itjust.works/comment/24283137
Reply
$$11950
https://lemmy.dbzer0.com/u/Scrath posted on Mar 14, 2026 11:51
In reply to: https://sh.itjust.works/comment/24279625

I use a MikroTik Router and while I do love the amount of power it gives me, I very quickly realized that I jumped in at the deep end. Deeper than I can deal with unfortunately.

I did get everything running after a week or so but I absolutely had to fight the router to do so.

Sometimes less is more I guess

https://lemmy.dbzer0.com/comment/24949020
Reply
$$11955
https://lemmy.ca/u/HugeNerd posted on Mar 14, 2026 12:12
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

Yes that does seem to describe modern computing, indeed, consumer electronics in general.

It’s no longer about solving actual problems, it IS the problem.

https://lemmy.ca/comment/22205957
Reply
$$11963
https://lemmy.ml/u/M4d_Ghoul posted on Mar 14, 2026 12:27
In reply to: https://lemmy.today/comment/22871764

I feel your pain. Had to fix my immich, NC and Joplin postgresdb. Turned out, DB via NFS is a risky life. ;D

https://lemmy.ml/comment/24532125
Reply
$$12115
https://lemmy.world/u/FauxLiving posted on Mar 14, 2026 22:14
In reply to: https://feddit.uk/comment/23840414

I stopped the tailscale service…

… while ssh’d through the tailscale interface.

Luckily, it was my home server and I had to drive there anyway.

https://lemmy.world/comment/22665382
Reply
$$12231
https://lemmy.world/u/InnerScientist posted on Mar 15, 2026 07:55
In reply to: https://lemmy.world/comment/22651585

Pro tip: If you’re using openwrt or other managed network components don’t forget to automatically back those up too. I almost had to reset my openwrt router and having to reconfigure that from scratch sucks.

https://lemmy.world/comment/22670857
Reply
$$12239
https://lemmy.world/u/Mr_Dr_Oink posted on Mar 15, 2026 08:35
In reply to: https://piefed.social/c/selfhosted/p/1873493/your-logging-is-probably-down

It makes me start looking for the next thing. Got my jellyfin, got my pi hole, my retro console and just recently home assistant set up. (Just a few more buts to add to that). Next i think i am going to look into self hosting a cloud storage solution. Like google drive/photos etc. Would be nice to make my own backups and have them offline

https://lemmy.world/comment/22671202
Reply
$$12521
https://lemmy.world/u/Vile_port_aloo posted on Mar 15, 2026 19:06
In reply to: https://lemmy.world/comment/22648508

It’s how cults start!

I’ve started to take a l lot more notes at work I guess there will be a time where I take notes of what month it is!

https://lemmy.world/comment/22679473
Reply
$$12642
https://lemmy.world/u/irmadlad posted on Mar 15, 2026 21:24
In reply to: https://lemmy.world/comment/22679473

I guess there will be a time where I take notes of what month it is!

You may jest, but there are times when I can’t remember what I had for breakfast. They say that you never truly forget anything, but that our recall mechanism fades over time. For a myriad of reasons, including age, my recall mechanism is shit.

https://lemmy.world/comment/22681608
Reply
$$12839
https://programming.dev/u/moonpiedumplings posted on Mar 16, 2026 05:30
In reply to: https://lemmy.today/comment/22871764

https://github.com/pgautoupgrade/docker-pgautoupgrade

Or if you are on k8s, you can use cloudnativepg.

https://programming.dev/comment/22745859
Reply
$$13452
https://lemmy.world/u/Vile_port_aloo posted on Mar 17, 2026 07:47
In reply to: https://lemmy.world/comment/22681608

Offt depends what you had and your version of health. I am hopeful that technology helps when I am that age, only a few years but ai agents seem to be a start. Just need to let go of those big data fears.

https://lemmy.world/comment/22707004
Reply
$$14493
https://lemmy.world/u/Bakkoda posted on Mar 18, 2026 17:44
In reply to: https://lemmy.world/comment/22665382

I used to make nginx changes while vpn’d into my network and utilizing guacamole (served via said nginx). I’m not a smart man.

https://lemmy.world/comment/22734606
Reply
$$15316
https://lemmy.today/u/MonkeMischief posted on Mar 20, 2026 19:29
In reply to: https://programming.dev/comment/22745859

I’m just using Docker on Proxmox, buuuut… I’m gonna look into this project. It looks like a LIFESAVER. Thank you for sharing this. You’re awesome! :D

https://lemmy.today/comment/23018196
Reply
$$16529
https://lemmy.today/u/paequ2 posted on Mar 23, 2026 16:58
In reply to: https://lemmy.world/comment/22646292

Because I’m an idiot. 🤦 Thanks!

https://lemmy.today/comment/23077540
Reply