- 5 Posts
- 19 Comments
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish1·2 days agoFor now I asked chatgtp to help me to implement a simple return 403 on bot user agent. I looked into my logs and collected the bot names which I saw. I know it won’t hold forever but for now it’s quite nice, I just added this file to /etc/nginx/conf.d/block_bots.conf and it gets run before all the vhosts and rejects all bots. The rest just goes normally to the vhosts. This way I don’t need to implement it in each vhost seperatelly.
➜ jeena@Abraham conf.d cat block_bots.conf # /etc/nginx/conf.d/block_bots.conf # 1️⃣ Map user agents to $bad_bot map $http_user_agent $bad_bot { default 0; ~*SemrushBot 1; ~*AhrefsBot 1; ~*PetalBot 1; ~*YisouSpider 1; ~*Amazonbot 1; ~*VelenPublicWebCrawler 1; ~*DataForSeoBot 1; ~*Expanse,\ a\ Palo\ Alto\ Networks\ company 1; ~*BacklinksExtendedBot 1; ~*ClaudeBot 1; ~*OAI-SearchBot 1; ~*GPTBot 1; ~*meta-externalagent 1; } # 2️⃣ Global default server to block bad bots server { listen 80 default_server; listen [::]:80 default_server; listen 443 ssl default_server; listen [::]:443 ssl default_server; # dummy SSL cert for HTTPS ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem; ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key; # block bad bots if ($bad_bot) { return 403; } # close connection for anything else hitting default server return 444; }
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish2·2 days agoI already have LVM but I was using it to combine drives. But it’s not a bad idea, if I can’t do it with Docker, at least that would be a different solution.
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish41·2 days agoSadly that’s not the solution to my problem. The whole point op open-sourcing for me is to make it accessible to as many people as possible.
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish1·2 days agoHm, I’m afraid none of them really seems to cover the repo-archives case, therefor I’m afraid the size:all doesn’t include the repo-archives either.
But I’m running it in a container, perhaps I can limit the size the container gets assigned.
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish2·2 days agoI have monitoring of it, but it happened during night when I was sleeping.
Actually I saw a lot of forgejo action on the server yesterday but didn’t think it would go so fast.
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish3·2 days agoI need to look into it, thanks!
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish3·2 days agoYeah, I really need to figure out how to do quotas per service.
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish3·2 days agoBut then how do people who search for code like yours find your open source code if not though a search engine which uses a indexing not?
I also thought about it, but the custom domain feature only works on the $5 / month plan.
If you want free static hosting then probably: https://wasmer.io/
If you have the machine at home then you could set up port forwarding to it, but you would need to do everything yourself like:
- running a web server like nginx
- setting up ssl for it with certbot
- storing the static files in /var/www/html for example
- port forwarding from your router to that machine
- using some service like DuckDNS to point a domain to your dynamic IP at home
- pointing a CNAME to the DuckDNS subdomain on your domain
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish3·2 days agoIt does not, because that feature is usually used for scripts to download some specific release archive, etc. and other git hosting solutions do the same.
Jeena@piefed.jeena.netOPto Selfhosted@lemmy.world•Forgejo fills up hard drive with repo-archivesEnglish4·2 days agoI have nothing against bots per se, they help to spread the word about my open source code which I want to share with others.
It’s just unfortunate that forgejo fills up the hard drive to such an extend and doesn’t quite let you disable this archive feature.
No, I’m running everything on one server, there is sometimes a lot going on on PieFed and the load gets too much so it times out. I haven’t had the time to research it.
And it says on, just because I set it to retry some times.
I have 3 locations right now:
- Hetzner cloud (1 server)
- Home (my PC and a raspberry Pi)
- My parents house (a raspberry Pi)
I have most of those things on https://uptime.jeena.net/status/everything
Jeena@piefed.jeena.netto linuxmemes@lemmy.world•When you haven't told anyone you use Linux for a whileEnglish0·26 days agoI use Arch BTW.
Jeena@piefed.jeena.netto Programming@programming.dev•AWS deleted my 10-year account and all data without warningEnglish7·1 month agoI really need to start backups of my S3 bucket and my Hetzner Server to a local hard drive.
Jeena@piefed.jeena.netto Fediverse@lemmy.world•Can no longer access my old instance (lemmings.world) because I'm from the UK. I made several communities there. Is there any way I can mod them again or do I move them to this instance?English7·1 month agoTheoretically if you would host your own Lemmy/PieFed server then you would be able to access everything (as long as you host it outside of the UK I guess). And then you could keep being moderator on those communities with your new account on your own instance.
That only helps you out though, if the communities were UK specific ones and other people from the UK meet to participate then there is no other way than moving them to a UK friendly instance.
Jeena@piefed.jeena.netto Fediverse@lemmy.world•Can no longer access my old instance (lemmings.world) because I'm from the UK. I made several communities there. Is there any way I can mod them again or do I move them to this instance?English11·2 months agoWhat happened? Is it the instances fault or UK’s?
Hm, but this only works on tmpfs which is in memory. It seems that with XFS I could have done it too: https://fabianlee.org/2020/01/13/linux-using-xfs-project-quotas-to-limit-capacity-within-a-subdirectory/ but I used ext4 out of habit.