It definitely takes more effort to get started

You need something like stunnel/OpenVPN flag which masks your traffic as HTTPS I think. Even then DPI can probably detect it

Hey, it’s nice to talk to you. I’ve seen you around this community and I like your comments.

I said K8S because I work with it, but if OP doesn’t need HA I guess Podman is fine too. I don’t like Docker anymore after what they pulled a year or so back

Yes but if your instance holds a decent population then it’s a net benefit

Run K8S on a VM on Proxmox for this stuff

Thanks for this, I didn’t know Scaleway had a free service. I’ll definitely take a look

The reason to host your own instances is altruism. You help out the community with decentralisation and also absorb some of the bandwidth and storage costs from other instances. This is necessary for the Fediverse to survive.

Thanks man, that would be much appreciated

Thanks, looking forward to it

Thanks. I don’t see the content of the blogs in the feed, just the title - but maybe that’s a problem with my reader (I use Capy on Android). I’ll try a couple of other readers to see if it works

You got an RSS feed for me?

Your blog is awesome. I have always wanted someone to break down RF homelabbing for me and I think as your blog progresses I will find such content.

I’m also looking for blogs/material on OS hardening (Linux/*nix), do you plan to write on that (and any recommendations)?

That means it’s likely a problem with DNS.

Yeah I’m not going to run them on CPUs, that’s not going to be very good. I’ll buy the GPUs when I can.

Yes, just thought if you could check that the correct ports are opened. I.e. is port 443 open for NGINX on Unraid? Is NGINX forwarding traffic to the correct port to your backend? Is the backend configured to allow traffic on a certain domain/all domains if it is handling HTTPS?

Thank you, that makes sense. Yes, I will look to create templates using AI that I like. Thanks again for the help

Thanks for the edit. You have a very intriguing idea; a second LLM in the background with a summary of the conversation + static context might make performance a lot better. I don’t know if anyone has implemented it/knows how one can DIY it with Kobold/Ollama. I think it is an amazing idea for code assistants too if you’re doing a long coding session.

Better be AGPL or she’s never getting cloned on my PC, that’s for sure!

I see. Thanks for the note. I think beyond 48GB of VRAM diminishing returns set in very quickly so I’ll likely stick to that limit. I wouldn’t want to use models hosted in the cloud so that’s out of the question.

Absolutely. TheBloke’s fine-tuned models with their guardrails removed are the only conversational models I will run. I get enraged looking at AI telling me to curb my speech.

I do use Python but I haven’t touched AI yet so it’s going to be a learning-curve if I go down that route. I am hoping to get finetuned models OOTB for this kind of stuff but I know it’s a hard ask.

I was going to buy 2-3 used GPUs/new budget GPUs like the B580 but with the tariffs the prices of these are INFLATED beyond what I can afford to pay for them. Once something changes (financially speaking) I’ll probably throw enough VRAM at it to at least get the 8B models (probably not FP16 but maybe quantised to 4K/8K) running smoothly.

Thanks for the reminder. I have wanted to use character AI for so long but couldn’t bear to give away my thought patterns to them (look at my hypocrisy: I’m giving it all away anyway when everyone is free to scrape Lemmy). I guess I’m an idiot.