If you guys like hiking and stuff, there’s this cool open source app called trail sense on f-droid and it’s just so much feature packed…

I don’t hike, so I only use it for it’s pedometer capabilities and a hypothetical situation where “I might get really lost” but the amount of features it has for hiking and survival is crazy and so I think deserves to be more known.

I had a hunch that writing the actual Upload/download speed tather than mbps was probably wrong. My bad, my internet provider lingo is rusted.

I don’t have a jellyfin server but 1MB/s (8mbps) for each person watching 1080p (3.6Gb per hour of content for each file) seems reasonable. ~3MB/s (24mbps) upload and as much download should work.

Maybe this? https://genius.com/Sudden-death-wedgies-lyrics

Song sample : https://madmusic.com/song_details.aspx?SongID=44954

For me it’s the opposite. The song start and the music is a banger but as soon as the lyrics comes, it’s ruined.

Because the guy dying isn’t important, it’s the name of his ‘clients’ that matters.

The Russian brick video. Nothing gore involved and it’s an accident that can happen to anyone at anytime while driving.

No. Quantization make it go faster. Not blazing fast, but decent.

Completely forgot to tell you to only use quantized models. Your pc can run 4bit quantized versions of the models I mentioned. That’s the key for running llms on at consumer level hardware. You can later read further about the different quantizations and toy with other ones like Q5_K_M and such.

Just read phi-3 got released and apparently it’s a 4B that reach gpt 3.5 level. Follow the news and wait for it to be add to ollama/llama.ccp

Thank you so much for taking the time to help me with that! I’m very new to the whole LLM things, and sorta figuring it out as I go

I became fascinated with llms after the first AI booms but all this knowledge is basically useless where I live, so might as well make it useful by teaching people what i know.

The key is quantized models. A full model wouldn’t fit but a 4bit 8b llama3 would fit.

Yeah, it’s not a potato but not that powerful eaither. Nonetheless, it should run a 7b/8b/9b and maybe 13b models easily.

running them in Python with Huggingface’s Transformers library (from local models

That’s your problem right here. Python is great for making llms but is horrible at running them. With a computer as weak as yours, every bit of performance counts.

Just try ollama or llama.ccp . Their github is also a goldmine for other projects you could try.

Llama.ccp can partially run the model on the gpu for way faster inference.

Piper is a pretty decent very lightweight tts engine that can be directly run on your cpu if you want to add tts capabilities to your setup.

Good luck and happy tinkering!

Specs? Try mistral with llama.ccp.

It shouldn’t happen for a 8b model. Even on CPU, it’s supposed to be decently fast. There’s definitely something wrong here.

Sadly, can’t really help you much. I have a potato pc and the biggest model I ran on it was Microsoft phi-2 using the candle framework. I used to tinker with Llama.cpp on colab, but it seems they don’t handle llama3 yet. ollama says it does , but I’ve never tried it before. For the speed, It’s kinda expected for a 70b model to be really slow on the CPU. How much slow is too slow ? I don’t really know…

You can always try the 8b model. People says it’s really great and even replaced the 70b models they’ve been using.

Run 70b llama3 on one and have a 100% local, gpt4 level home assistant . Hook it up with coqui.Ai xttsv2 for mind baffling natural language speech (100% local too ) that can imitate anyone’s voice. Now, you got yourself Jarvis from Ironman.

Edit : thought they were some kind of beast machines with 192gb ram and stuff. They’re just regular middle-low tier pcs.

Russia used to be that powerful, but not today. It will put a decent fight but I doubt it’ll be able to deal against so many powerful military foes.

This post is about Ukraine mate.

And the comments I was answering to was talking about nato/us/eu starting a war with Russia. You should be less oversensitive.

Second, there will be no nuclear war escalation so calm down.

Yep, The rest of the world would greatly appreciate it.

Cope and seethe.

I geuss you didn’t read the last part of my comment then.

You’re making too much sense for their patriotic propaganda riddled brains. During the cold war, the media made the population fear the enemy and it’s nukes. Nowadays, they instead turned it into making a complete joke of Russia , even making fun of their nukes. So, no wonder the US population think it’s ok to start a war with them despite the thousands of nukes at their disposal. They think it’ll be as easy as invading yet another poor middle eastern country.

the NATO/EU/US may decide to become directly involved in the war

After their involvement, I doubt they’d do nothing if some of their soldiers get shredded by a Russian bomb. Let’s imagine Russia is beaten and it’s on the brink of complet defeat. Do you really not expect them to send nuclear warheads everywhere before collapsing as an ultimate fuck you ? 10% of the warheads are enough to turn earth into a post apocalyptic wasteland.

Get off your high horses. I know Ukraine is a very sensitive subject for some but I didn’t mention anything about it. We’re fucking talking about not escalating it into nuclear war here. You know Ukraine, and no other country for that matter, would exists after a nuclear holocaust,right ?

Lastly, I am against colonization and invasion of any kind.