Poking around my server logs
Every now and again I download and poke around my server logs to see if there's anything interesting in there. Usually what I'm interested in is if there are new apps or services that are crawling my site and also if there's any IP address that is doing bulk downloads of my content. Since I'm doing it right now, I'm gonna share some data. Maybe you're also running a blog and having more data points is always fun.
- The log file I have in front of me goes from
127790lines. Those requests are generated by
8536distinct IP addresses so the requests to IP ratio is in the order of
127790in roughly 10 days means
93552are requests to one of my RSS feeds. That is almost 75% of all requests which is wild to me.
1182are requests coming from a Mastodon instance and I can see
714unique ones in here. I have no idea why these requests are in here but it's interesting to see. I might start collecting all these URLs. Could be fun. There seem to be A LOT of Mastodon servers out there. Kinda cool.
11261, so around 9%, are requests containing keywords such as bot, spider, or crawler.
Next day edit: since I did a bit more digging promped by an email exchange, I'm gonna share some extra data related to the RSS requests.
4376unique IPs have requested my RSS feed
1790have only made 1 request. That's 40% of the total unique IPs related to the RSS feed.
4172have made 100 or fewer requests. 100 might seem a lot but if you have an app set up to do a fetch every couple of hours you can easily generate 100 requests in 10 days.
9IPs are in the 4 digits territory. 2 of them are related to NNW, 2 more are tied to Miniflux and 1 is apparently a SimplePie installation.
- These 9 combine for a total of
16431which is 17% of the RSS requests and almost 13% of the total requests.
What is a resonable number of requests coming from an RSS reader app or service? Reeder for Mac, which is the one I use, can be configured to sync your feed every 5 minutes. 10 days is 14400 minutes so a request every 5 minutes means you can generate 2880 individual requests if you leave your app open asking for new content all day long. Quite a waste IMO.
Are these numbers meaningful in any particular way? I doubt. Still, it's fun to poke around and see what's happening at the server level. If there's anything specific you want to know about my logs let me know.