Code that does not work is just text.
I’ve never thought of it that way. I’m going to add copy writer to my resume.
Maybe fiction writer as well
Write tests and run them, reiterate until all tests pass.
I’ve heard that a Claude 4 model generating code for an infinite amount of time will eventually simulate a monkey typing out Shakespeare
It will have consumed the GigaWattHours capacity of a few suns and all the moisture in our solar system, but by Jeeves, we’ll get there!
…but it won’t be that impressive once we remember concepts like “monkey, typing, Shakespeare” were already embedded in the training data.
Watching the serious people trying to use AI to code gives me the same feeling as the cybertruck people exploring the limits of their car. XD
“It’s terrible and I should hate it, but gosh it it isn’t just so cool”
I wish i could get so excited over disappointing garbage
You definitely could use AI to code, the catch is you need to know how to code first.
I use AI to write code for mundane tasks all the time. I also review and integrate the code myself.
The AI code my “expert in a related but otherwise not helpful field” coworker writes helps me have a lot of extra work to do!
Its like having a junior developer with a world of confidence just change shit and spend hours breaking things and trying to fix them, while we pay big tech for the privilege of watching the chaos.
I asked chat gpt to give me a simple squid proxy config today that blocks everything except https. It confidently gave me one but of course it didnt work. It let through http and despite many attempts to get a working config that did that, it just failed.
So yeah in the end i have to learn squid syntax anyway, which i guess is fine, but I spent hours trying to get a working config because we pay for chat gpt to do exactly that…
It confidently gave me one
IMO, that’s one of the biggest “sins” of the current LLMs, they’re trained to generate words that make them sound confident.
Try to get one of these LLMs to update a package.json.
ones that can run cli tools do great, they just use npm
Define “update”
The image is taken from Zhihu, a Chinese Quora-like site.
The prompt is talking about give a design of a certain app, and the response seems to talk about some suggested pages. So it doesn’t seem to reflect the text.
But this in general aligns with my experience coding with llm. I was trying to upgrade my eslint from 8 to 9, and ask chatgpt to convert my eslint file, and it proceed to spit out complete garbage.
I thought this would be a good task for llm because eslint config is very common and well-documented, and the transformation is very mechanical, but it just cannot do it. So I proceed to read the documents and finished the migration in a couple hour…
I asked ChatGPT with help about bare metal 32-bit ARM (For the Pi Zero W) C/ASM, emulated in QEMU for testing, and after the third iteration of “use printf for output” -> “there’s no printf with bare metal as target” -> “use solution X” -> “doesn’t work” -> “ude printf for output” … I had enough.
Sounds like it’s perfectly replicated the help forums it was trained on.
I used ChatGPT to help me make a package with SUSE’s Open Build Service. It was actually quite good. Was pulling my hair out for a while until I noticed that the project I wanted to build had changes URLs and I was using an outdated one.
In the end I just had to get one last detail right. And then my ChatGPT 4 allowance dried up and they dropped me back down to 3 and it couldn’t do anything. So I had to use my own brain, ugh.
This has beeny experience as well. It keeps emphasizing “beauty” and keeps missing “correctness”
llms are systems that output human-readable natural language answers, not true answers
Honest question: I haven’t used AI much. Are there any AIs or IDEs that can reliably rename a variable across all instances in a medium sized Python project? I don’t mean easy stuff that an editor can do (e.g. rename QQQ in all instances and get lucky that there are no conflicts). I mean be able to differentiate between local and/or library variables so it doesn’t change them, only the correct versions.
I’m going to laugh in Java, where this has always been possible and reliable. Not like ai reliable, but expert reliable. Because of static types.
All programs can be written with on less line of code. All programs have at least one bug.
By the logical consequences of these axioms every program can be reduced to one line of code - that doesn’t work.
One day AI will get there.
On one line of code you say?
*search & replaces all line breaks with spaces*
All programs can be written with on less line of code. All programs have at least one bug.
The humble “Hello world” would like a word.
Just to boast my old timer credentials.
There is an utility program in IBM’s mainframe operating system, z/OS, that has been there since the 60s.
It has just one assembly code instruction: a BR 14, which means basically ‘return’.
The first version was bugged and IBM had to issue a PTF (patch) to fix it.
Okay, you can’t just drop that bombshell without elaborating. What sort of bug could exist in a program which contains a single return instruction?!?
This weekend I successfully used Claude to add three features in a Rust utility I had wanted for a couple years. I had opened issue requests, but no else volunteered. I had tried learning Rust, Wayland and GTK to do it myself, but the docs at the time weren’t great and the learning curve was steep. But Claude figured it all out pretty quick.
Did the generated code get merged? I’d be curious to see the PRs
The lead dev is not available this summer to review, but you can review here: https://github.com/edzdez/sway-easyfocus/pull/22
It’s not great that four changes are rolled into a single PR, but that’s my issue not Claude’s because they were related and I wanted to test them all at once.
This is interesting, I would be quite impressed if this PR got merged without additional changes.
I am genuinely curious and no judgement at all, since you mentioned that you are not a rust/GTK expert, are you able to read and and have a decent understanding of the output code?
For example, in the
sway.rsfile, you uncommented a piece of code about floating nodes inget_all_windowsfunction, do you know why it is uncommented? (again, not trying to judge; it is a genuine question. I also don’t know rust or GTK, just curious.This is interesting, I would be quite impressed if this PR got merged without additional changes.
We’ll see. Whether it gets merged in any form, it’s still a big win for me because I finally was able to get some changes implemented that I had been wanting for a couple years.
are you able to read and and have a decent understanding of the output code?
Yes. I know other coding languages and CSS. Sometimes Claude generated code that was correct but I thought it was awkward or poor, so I had it revise. For example, I wanted to handle a boolean case and it added three booleans and a function for that. I said no, you can use a single boolean for all that. Another time it duplicated a bunch of code for the single and multi-monitor cases and I had it consolidate it.
In one case, It got stuck debugging and I was able to help isolate where the error was through testing. Once I suggested where to look harder, it was able to find a subtle issue that I couldn’t spot myself. The labels were appearing far too small at one point, but I couldn’t see that Claude had changed any code that should affect the label size. It turned out two data structures hadn’t been merged correctly, so that default values weren’t getting overridden correctly. It was the sort of issue I could see a human dev introducing on the first pass.
do you know why it is uncommented?
Yes, that’s the fix for supporting floating windows. The author reported that previously there was a problem with the z-index of the labels on these windows, so that’s apparently why it was implemented but commented out. But it seems due to other changes, that problem no longer exists. I was able to test that labels on floating windows now work correctly.
Through the process, I also became more familiar with Rust tooling and Rust itself.
Holy shit someone on here that know how to use them. Surprised you haven’t been downvoted into oblivion yet.
To be fair, if I wrote 3000 new lines of code in one shot, it probably wouldn’t run either.
LLMs are good for simple bits of logic under around 200 lines of code, or things that are strictly boilerplate. People who are trying to force it to do things beyond that are just being silly.
You managed to get an ai to do 200 lines of code and it actually compiled?
Uh yeah, like all the time. Anyone who says otherwise really hasn’t tried recently. I know it’s a meme that AI can’t code (and still in many cases that’s true, eg. I don’t have the AI do anything with OpenCV or complex math) but it’s very routine these days for common use cases like web development.
I recently tried it for scripting simple things in python for a game. Yaknow, change char’s color if they are targetted. It output a shitton of word salad and code about my specific use case in the specific scripting jargon for the game.
It all based on “Misc.changeHue(player)”. A function that doesn’t exist and never has, because the game is unable to color other mobs / players like that for scripting.
Anything I tried with AI ends up the same way. Broken code in 10 lines of a script, halucinations and bullshit spewed as the absolute truth. Anything out of the ordinary is met with “yes this can totally be done, this is how” and “how” doesn’t work, and after sifting forums / asking devs you find out “sadly that’s impossible” or “we dont actually use cpython so libraries don’t work like that” etc.
It’s possible the library you’re using doesn’t have enough training data attached to it.
I use AI with python for hundreds line data engineering tasks and it nails it frequently.
Well yeah, it’s working from an incomplete knowledge of the code base. If you asked a human to do the same they would struggle.
LLMs work only if they can fit the whole context into their memory, and that means working only in highly limited environments.
No, a human would just find an API that is publically available. And the fact that it knew the static class “Misc” means it knows the api. It just halucinated and responded with bullcrap. The entire concept can be summarized with “I want to color a player’s model in GAME using python and SCRIPTING ENGINE”.
