Well a local model responding to a prompt on less than 20gb of vram (gaming computer) costs less power than booting up any recent AAA high fps game. The primary cost in power is r&d. Training the next model to be “the best” is an Arms race. 90% of power consumption is trying to train the next model in 100 different ways. China was able to build off of chat cpt tech and built a model that has similar abilities and smartness for only 5 million. I think I won’t update my local model until there is actually more abilities in the next one.
And there is also a cost in the brain atrophy that these are causing in people who use them regularly. LLMs will make a huge segment of the population mentally and emotionally stunted. Who knows what these will do this generation of children (many of whom will virtually raised by LLMs). Television and smartphones have done similar brain damage, particularly to attention spans, but these things will really wreak havoc humanity on another level. Like potentially a Fermi paradox/great filter solution level of harm.
Well a local model responding to a prompt on less than 20gb of vram (gaming computer) costs less power than booting up any recent AAA high fps game. The primary cost in power is r&d. Training the next model to be “the best” is an Arms race. 90% of power consumption is trying to train the next model in 100 different ways. China was able to build off of chat cpt tech and built a model that has similar abilities and smartness for only 5 million. I think I won’t update my local model until there is actually more abilities in the next one.
lol and how much does does 20gb of vram cost right now?
About 600$ before tax and shipping
And there is also a cost in the brain atrophy that these are causing in people who use them regularly. LLMs will make a huge segment of the population mentally and emotionally stunted. Who knows what these will do this generation of children (many of whom will virtually raised by LLMs). Television and smartphones have done similar brain damage, particularly to attention spans, but these things will really wreak havoc humanity on another level. Like potentially a Fermi paradox/great filter solution level of harm.