Poking around my server logs
Every now and again I download and poke around my server logs to see if there's anything interesting in there. Usually what I'm interested in is if there are new apps or services that are crawling my site and also if there's any IP address that is doing bulk downloads of my content. Since I'm doing it right now, I'm gonna share some data. Maybe you're also running a blog and having more data points is always fun.
- The log file I have in front of me goes from
[30/Apr/2023:23:55:10]
to[10/May/2023:15:38:06]
and contains127790
lines. Those requests are generated by8536
distinct IP addresses so the requests to IP ratio is in the order of15:1
. 127790
in roughly 10 days means12779 req/day
,532 req/hour
and9 req/minute
93552
are requests to one of my RSS feeds. That is almost 75% of all requests which is wild to me.1182
are requests coming from a Mastodon instance and I can see714
unique ones in here. I have no idea why these requests are in here but it's interesting to see. I might start collecting all these URLs. Could be fun. There seem to be A LOT of Mastodon servers out there. Kinda cool.11261
, so around 9%, are requests containing keywords such as bot, spider, or crawler.
Next day edit: since I did a bit more digging promped by an email exchange, I'm gonna share some extra data related to the RSS requests.
4376
unique IPs have requested my RSS feed1790
have only made 1 request. That's 40% of the total unique IPs related to the RSS feed.4172
have made 100 or fewer requests. 100 might seem a lot but if you have an app set up to do a fetch every couple of hours you can easily generate 100 requests in 10 days.9
IPs are in the 4 digits territory. 2 of them are related to NNW, 2 more are tied to Miniflux and 1 is apparently a SimplePie installation.- These 9 combine for a total of
16431
which is 17% of the RSS requests and almost 13% of the total requests.
What is a resonable number of requests coming from an RSS reader app or service? Reeder for Mac, which is the one I use, can be configured to sync your feed every 5 minutes. 10 days is 14400 minutes so a request every 5 minutes means you can generate 2880 individual requests if you leave your app open asking for new content all day long. Quite a waste IMO.
Are these numbers meaningful in any particular way? I doubt. Still, it's fun to poke around and see what's happening at the server level. If there's anything specific you want to know about my logs let me know.