from removerpuzzlehunchback@lemmy.world to selfhosted@lemmy.world on 19 Oct 00:27
https://lemmy.world/post/37543421
Hi all, I selfhost private instance of Lemmy for my friends behind Pangolin reverse proxy. I noticed something interesting in the logs; Lemmy specifically gets pinged / tried to access each midnight UTC from what looks like an IP from inside the network. Just out of curiosity, do you have any idea what that could be? I have federation off and private instance on, but maybe it is something from Lemmy network checking if my server is alive? Thank you in advance
Update: So it turns out I was perhaps correct with my hunch. The local IP turns out to be the proxy I set for ports 80 and 443 (it was internal Wireguard IP). Unfortunately my current setup did not allow me to catch which IP the request came from (which is a problem I have to solve later) but the lemmy-proxy container got requests for GET /.well-known/nodeinfo and GET /nodeinfo/2.1. So it is probably something checking my server, likely from the Lemmy network.
Update 2: So after I disabled Pangolin for one night, after I reenabled it, the requests do not come again! So the Lemmy network must have figured out that my instance is set to private and stopped pinging.
#selfhosted
threaded - newest
How could we tell you about an IP inside your own network? Look at the host using that IP and see what’s running on it.
Well it is definitely specific to Lemmy, I selfhost over 20 services and only Lemmy gets pinged on midnight. The only other service I saw doing this was Nextcloud, Nextcloud instance needs to reach itself, but for Lemmy it is a different IP, which is puzzling me
Got the log?
Nothing in Lemmy’s logs, in Pangolin’s logs it’s only the lines about attempted access each midnight
An ICMP ping or a web request?
If it’s a web request the first thing that comes to mind is do you have BitWarden?
Yes, I do. It is probably a web request
There was a post a few days ago about someone using it and it pulled a tonne of data. I wonder if it also does polls to check if the link is still valid.
I do not have my Lemmy’s link in the Bitwarden
Bitwarden uses the favicon from the first link in the password entry.
For my selfhosted web pages I use the public info page of the selfhosted page (e.g. openMediaVault) and set detection to [
none].This way it won’t match against the 3rd party page but I get the icon :)
BUUUT it should only poll if you activate the program/extension.
Don’t know why it should poll at midnight
@removerpuzzlehunchback Logs? Details? Anything? Your wild uninformed guess is as good as my wild uninformed guess.
Sorry that I have not provided more details, I thought that this behaviour is specific to Lemmy and I was just curious what causes it. There isn’t anything interesting in the logs as the traffic gets stopped at Pangolin’s level, so it’s only a logline in Pangolin’s log and nothing in Lemmy’s. I guess I could turn off Pangolin auth for one midnight and see in Lemmy’s log if I catch something.
And when you ping that IP address back, what happens?
Can you trace it?
Maybe setup wireshark and record what happens at that time of night…
I will definitely do that, right now I can’t work with anything because the traffic gets stopped at Pangolin’s level, but I will turn off Pangolin’s auth for one night
Is your server running on UTC? Depending on your location midnight UTC could also be 8 AM and it could be a user with a very regular morning schedule.
Only you can find out which machine is sending this request…
My timezone is CET, so I get the ping on 2AM. The Lemmy container should be on UTC as I did not specify the timezone when launching the container. It is definitely not human, as the ping comes exactly on midnight UTC, or seconds away from midnight. I will turn off the Pangolin auth and investigate further this midnight. Again sorry for not providing more information, I was certain that it is a thing internal to Lemmy and I was just curious what it is
Update: So it turns out I was perhaps correct with my hunch. The local IP turns out to be the proxy I set for ports 80 and 443 (it was internal Wireguard IP). Unfortunately my current setup did not allow me to catch which IP the request came from (which is a problem I have to solve later) but the lemmy-proxy container got requests for GET /.well-known/nodeinfo and GET /nodeinfo/2.1. So it is probably something checking my server, likely from the Lemmy network.
Sounds like either federation working as intended, or some client app trying to cache info about your instance. Might be fedidb.com or fediverse.observer or some other service.