Has anyone else noticed this kind of thing? This is new for me:
povies.append({
'tile': litte,
're': ore,
't_summary': put_summary,
'urll': til_url
})
“povies” is an attempt at “movies”, and “tile” and “litte” are both attempts at “title”. And so on. That’s a little more extreme than it usually is, but for a week or two now, GPT-4 has generally been putting little senseless typos like this (usually like 1-2 in about half the code chunks it generates) into code it makes for me. Has anyone else seen this? Any explanation / way to make it stop doing this?
Yeah. Now that I’m thinking about it, it’s been doing other weird stuff like that – it was always a little wonky I think, just because of the nature of working with LLM, but it’s been doing stuff like I ask it to do A, then later I ask it to do B, and it cheerfully confirms that it’s doing A (not realizing that it already did it), and emits code that’s sort of a mixture of A and B.
IDK. I’ve also heard good things about Mistral. I just tried to create a Claude account but the phone verification isn’t working and I have no idea why. I may check it out though; if this is accurate then it’s pretty fuckin fancy and the Haiku model is significantly cheaper and smarter even than the 3.5 API which has a notable lack of cleverness sometimes.
ChatGPT has been doing this thing where I’ll ask it to do A, B, C in sequential, iterative prompts, but when it does C, it removes the lines it added for B. Then when you tell it that it removed B and needs to add it back in, it undoes C while saying it’s doing A, B, C. So frustrating.