Sims

I have not tested it, but Zluda are are drop-in cuda replacement for non-nvidia gpus. The speed should be great. You could check if your goto card is supported…

A backup drive ? dotfiles (.config) ? /tmp /var/log /boot ? map it into your home for something temporary - Downloads ? Slow audio files ? If you like to tinker for fun: btrfs / bcachefs could perhaps utilize it for something, like automatically offloading unused/slow files to the slow usb ? Raid ? ;) Create a partition and let zram swap compressed unused memory pages to the usb partition (I think it keeps pages compressed now ?). You can also run ‘usb over ethernet’, so another machine can use it directly - not sure why tho ;)

On a side note. I still use one of my first usb 1.? drives with 32mb in my old router. 32mb is still 12mb more than my first hard drive, and it fits well with the small openwrt packages. It even has a little slide button to select ‘floppy drive’ mode and a physical ‘readonly’ mode. I wonder what the smallest/oldest usb stick people still use, are ?

Looks cool and simple. Here’s for convenience: https://www.youtube.com/watch?v=IkriZGyjoeU

Agree. “Are you perhaps too angry?”, “The HELL I’m not! F*ck off !”, “alrighty then”…

Not judging ? I assume you mean ‘free’ ? It is more difficult now. Groq.com once claimed that they didn’t collect data from the free service, but I just checked their privacy rules, and they have now learned the proper Capitalist virtue of surveilling their users and sell their data. Perhaps one of the other fast npu endpoints, cerberus, or sambanova, that are not directly focused on selling data ?

Duckduckgo ai doesn’t require login, but are very limited for chat.

Alternatively, pick free Chinese AI services ? I guess the data they can accumulate are pretty useless to them unless you buy much over there, but not sure to be honest.

Maybe a vpn and gpt4free ? Someone can see and train on your messages alright, but they don’t know who. This gives different models each time, so maybe not cool.

Something like aihorde maybe okay. Your messages are openly processed elsewhere, but you are not likely to be tracked. There’s a lot of personal chat models available. It afaik still requires a ‘small’ nvidia card to join in. Vpn could help anonymity.

Good thoughts and choices.

A small side tangent: I remember Ubuntu made a small exe file that could install ubuntu on a running windows machine. I cant remember clearly, or fork/develop it, but it could be nice for windows users to have a simple Linux-install.exe app that does all the confusing work for them and perhaps follows your advice here. Just a quick thought…

Compute prices are not going down. If you want a budget friendly compute, then you may need to look for NPU’s. They are slower, and is unfortunately also getting more expensive pr TOPS. Electricity prices in the West are going up as a low output gets bought up by datacenters instead. The newest GPU’s are more efficient in re. to power pr Top, but they are not budget friendly.

I have not seen a recent comparison of GPU vs Npu; TOPS vs price vs power consumption, but @ aliexpress, I saw a small 40 TOPS Npu as a nvme stick with 16gb ram that draws 10 watts, or so (search for ‘ai accelerator nvme’). This little thing can scan files 24/7 and your i5 can help out in peaks. Afaik you can run a distributed model that also runs in, and uses your i5 memory/compute, so if you max out i5 memory, perhaps the combined compute is enough for a larger model ? Maybe a few Npu sticks can work together on the same model ?

Alternatively, you could look for the next version of a Huawei gpu card (first had baby kinks afaik), or one of the other upcoming producers from China. They’ll come faster and faster, but are earmarked for local consumption first.

Another suggestion is to buy one of the old P40/80 (from the ‘pascal’ chipset, I think) (or was it k40/80 ??). They should still support a descent range of modern quantization needs and often have 24gb ram. A refurbished miner card cost around 50-70$+ - cant remember exactly tho.

Lastly, you could try something different. If the i5, with enough memory, can run a large model slowly, you could add a dedicated KV cache, so most of the tokens won’t need to be recalculated. Memory and bandwidth are the most important here, but any old server could be upgraded to be a dedicated KV cache server (might need a network upgrade tho!).

Anyway, Ideas are easy and cheap. If I were you, I would ask an AI for a little python app where you can add a product and it returns a graph where it compares with other products and show optimality over time given prices/power and Tops - Gpu vs Npu. Good hunting…

Can’t see much difference under the ‘crazy hood’.

Biden were a dimwitted demented lunatic/lier also, they all are, and US are doing the exact same things as they did before - just more openly now. The two-party system is merely ‘crowd-control’, and people are fed entertainment, lies and small emotional issues - just like the Romans did with the coliseum etc. All just crowd-control. Nobody in the top of the US government - both ‘parties’ - cares about any people other than themselves and their oligarch donors. Presidents don’t change anything but a few ‘make-up’ issues, and US internal/foreign policy have been the exact same for around 70-100 years. That equals to 100% oppression, political colonialism/control - at home and abroad.

At home in the US, there haven’t been ‘Democracy’ ever. It was and always will be a Plutocracy, by the Plutocracy, and for the Plutocracy - like this Trump oligarch they dug up and placed in power. Free markets = 100% corruption on all ‘markets’, and no oversight allows for rich c*nts to “Manufacture Political Consent” through the media and other channels. The US plutocracy controls the money flow to media, and have hence captured the America/Western people in something called “Colonization of the mind” (type of propaganda control). This are all directly built-in to the US political system through the free markets of politics where peoples interest/well being doesn’t matter - money and Oligarchs does.

On a geopolitical scale US manipulates other nations through their nations oligarchs and newly opened markets, and all the US wars against the nay-sayers around the world where they scoop up other nations/peoples resources in a calculated attempt to preserve their post-WW2 hegemony and power structure. Their campaign against Russia (yes, the war were manufactured by US over 30 years!) went really sour compared to expectations, and now the whole Western economy and power base are contracting, and we are moving towards a so called multi-polar world, where US oligarchs can’t decide global events anymore.

I love the sub and certainly want to boycott US (have done/tried so for 10+ years), but it seems like most here are only reacting to Trump’s open bullying and would stop boycotting US if Trump were removed. I wouldn’t.

However in the end, I guess the underlying reason for what we do isn’t as important, as long as we agree about the boycott and slamming of Psychos in charge of our lives.

Seems incompatible with Capitalism. ‘Surveillance-capitalism’ is a huge problem, and no other system in history have gathered so much private data about its citizens. Everything we do on the internet, or in some closed ecosystem, is indirectly monitored and the market system knows all about us, our activities, our social groupings, opinions etc and this knowledge is not used for our benefit. Further, as long as we might influence the system, or as long as we might have digital data ‘belonging’ to someone else, capitalism will step in and take control of our ‘privacy’. We and privacy are a threat to that system…