• 1 Post
  • 52 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle
  • Numbers from my instance, running for about a 1 year and with average ~2 MAU. According to some quick db queries there is currently 580 actively subscribed communities (it was probably a lot less before I used the subscribe bot to populate the All tab).

    SELECT pg_size_pretty( pg_database_size('lemmy') ): 17 GB

    Backblaze B2 (S3) reports average 22.5 GB stored. With everything capped to max 1 USD, I pay cents - no idea how backblaze does it but it’s really super cheap, except for some specific transactions done on the bucket afaik, which pictrs does not seem to do.

    According to my zabbix monitoring, two months ago (I don’t keep longer stats) the DB had only about 14G of data, so with this much communities I am getting about 1.5G per month (it’s probably a bit more as I was recently prunning stuff from some dead instances).

    Prometheus says whole lemmy service (I use traefik) is getting within about 5 req/s (1m average) though if I go lower it does spike a lot, up to 12 requests within a second then nothing for few.












  • I wouldn’t recommend putting ssh behind any vpn connection unles you have a secondary access to the machine (for example virtual tty/terminal from your provider or local network ssh). At best, ssh should be the only publicly accessible service (unless hosting other services that need to be public accessible).

    I usually move the ssh port to some higher number just to get rid of the basic scanners/skiddies.

    Also disable password login (only keys) and no root login.

    And for extra hardening, explicitly allow ssh for only users that need it (in sshd config).


  • I don’t use nginx proxy manager but websocket has to be enabled for apps that use websockets (duh) - you would have to dive into docs or example infra configs to check if the service uses it.
    Rule of thumb here would be to enable it for everything. Optionally you could check if the service works with/without it.

    E: Websockets are used when a website needs to talk in “real-time” with the servers - live views and graphs will usually use it also notifications, generally if the website does not reload/redraw fully but data seems to change then there is a high chance it uses websockets under the hood (but there are ways to do it without ws, ex. SSE).

    Example: Grafana uses websockets but qbittorrent web ui uses other means (SSE) and does not require ws.


  • borg backup with rsync.net

    Borg does de-duplication and compression, I’ve used it for multiple things like backing up minecraft servers and it can reduce the final backup size by a lot (like 1-2 TBs to a hundred of GB, though that was with content that was highly compressible and didn’t change much over-time so the deduplication did a lot too).

    There is also borgbase.com which looks a bit better and focuses only on borg repositories instead of also being compatible with just about any usual tools (eg rsync, rclone etc)


  • taaz@biglemmowski.wintoSelfhosted@lemmy.world[Solved] Sudden Issues
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    7 months ago

    I would try momentarily replacing the defined dns servers with nameserver 1.1.1.1 and see if stuff improves, though the pull error would hint that docker did resolve the name but somehow didn’t get an answer.
    Hard to guess what else could be a problem apart from some obvious stuff - check if the internet connection is healthy and stable (ping, watch for spikes in ms or drops, also any outgoing firewall filters?)




  • You might want to focus on what you can do the most right now (fullstack) and change tracks later (embedded).

    As a SWE in primarily python backends I can give you these ideas:

    • Basic CMS in Django, let’s say an eshop
    • Messenger-like app with FastAPI (with user management, chats, chat history, big plus would be fulltext search), this explores asynchronous events processing, this could include websockets with redis and some databases like Postgres or Maria, Mongo could work too or you could try Firebase (which is a valid trade experience but for me, experience with “real” databases/caches is more valuable).
      .

    Also last time I heard there is always space for Java backend engineers (Spring Boot).

    Focusing on fullstack you could try one of the big frameworks like Next.js (React), Nuxt (Vue) or Nest.js (backend framework) with static serving Vue/React (I might be wrong in these, I am not directly working with these tools).

    Either way, personally, personal projects are cool but the main thing that counts is if you can navigate the problem space of the given field, not necessarily the absolute skill in a given framework (but definitely being a plus), for frontend it’s often about UI designs, UX etc and backends are about efficient representation of data, the processing of them and then designing something like HTTP API around them.
    So choose any project you like, from websites to just processing-heavy HTTP backends (trading?), learn what problems you encounter and remember how to overcome them.


  • A bit of rant but I hope this can fix some of the discrepancies I have found between free Nginx and the Plus one:

    If I want to configure (minimum amount of) proxy connections to keepalive then I have to use upstream ... { ... } directive, but by using that nginx changes how upstream proxy url is DNS resolved. Within docker this can actually cause problems when the nginx container starts sooner then the upstream service - afair to fix it one needs Nginx Plus because in free upstream DNS resolution “mode” can’t be configured.