As AI companies rave about how their products are revolutionizing productivity, Senator Bernie Sanders wants the tech industry to put its money where its automated mouth is.
Does anyone here actually see productivity improvements to their roles from using AI?
I’m a telecoms engineer and I see limited use cases in my role for AI. If I need to process data then I need something that can do math reliably. For document generation I can only reliably get it to build out a structure and even then I’ve more than likely got an existing document the I can use as a structure template.
Network design, system specification and project engineering are all so specific to the use case and have so few examples provided in public data sets that anything AI outputs is usually nonsense.
Am I missing some use cases here?
Also, if you do see productivity improvements from AI, why would you tell your employer? They want a 5 day working week but they know what they expect to be achieved in that week, so that’s what they get.
As a software engineer, definitely. Things that might have taken days writing a boilerplate framework and reading through docs to find out how to configure certain things could take hours now. Now I don’t have to spend hours learning how to use the data visualisation library in order to fix the one donut chart on my company’s site, I can ask AI to do it in minutes and make edits where needed.
It’s certainly not perfect or infallible, it spits out garbage a lot still. But since I have a deep understanding of the stuff I’m working on, I can recognize when it’s spitting out garbage and recalibrate.
A while back when one of the popular models came out it blew all the others away in benchmark tests. That got my coworker and boss super excited, we started coming up with different ways AI can help us. Thankfully as I pointed out, our software is proprietary and super secret, and all it took was a couple Google searches to find out a lot of those companies leak data like crazy and AI will just tell other people your secrets if they ask in the right way. So we needed to run our LLMs locally, but for that we’d need some beefy specs. I did the research, wrote a neovim and sublime plugin to integrate our local LLMs in a ‘copilot’ kind of way. My boss ordered my coworker and I whatever the new macbooks are with 128gb of memory to fit our lovely AI models, bought me a desktop tower and a few GPUs. Then I went on a 3 month paternity leave, came back and have heard nothing about AI since aside that my coworker switched to vs code to try and use continue but got frustrated, switched back to sublime and doesn’t use AI anymore.
So yeah, AI got me 2 new maxed out spec machines and 2 weeks of fuck around time writing plugins that were not nearly as complex as everyone thought, just because AI was involved.
For real though, I’ve tried quite a few times but 9/10 it fails me and I end up spending more time messing around with prompts than it would’ve taken me to do the thing. Occasionally when I have a mile long error message or something super obscure I’ll pass it to ollama and it seems to do well with wittling it down for me, that’s about it.
I’ll say it has been marginally more useful integrated into my code editor, prompt driven for me has been useless output for too much effort, but it ambiently sitting there in code editor can be helpful.
I still can only get it to provide useful suggestions about 15-20% of the time for like two lines, maybe a nice error message, but the failure rate is less obnoxious if you didn’t spend extra effort to ask for it and just ignore and keep going. Getting a feel for whether or not the LLM is likely to have something in the completion worth trying to review is a part of it based on what you’ve typed helps. Notably if you are some keystrokes into a very boilerplate process you might be more optimistic, or if you are about to provide a text string as a human error message, decent chance it wrote that for you well enough.
Still I’m more annoyed and not sure that it’s worth being annoyed, but I could buy that shaving typing out a couple lines 15% of the time could be an objective boost that outweighs the burden of futzing with the high error rate.
For at least some jobs, there’s a ‘work item’ of basically generating a bunch of text for humans that no human will ever read, but management thinks it’s important. AI can generate those walls of text no one actually needs while making management feel good.
It can catch some careless mistakes and guess remediation frequently. For example, if you provide a template string but forget to actually push it through templating, it can see that a string looks like it should be a template and add the templating call and also do a decent job of guessing the variables to pass for the template. However it does have a high false-positive rate, and does hallucinate variables that didn’t exist sometimes, so it’s a bit frustrating and I’m not sure if the false error annoyance is worth it…
On code completion, it can guess the next line or two I was going for about 15% of the time, 20% of the time with some trivial edits to fix it. A bit annoying because along with the suggested line or two it can get right, it will tend to suggest like 6-10 more lines that are completely wrong 99% of the time, so if I accept the completion I have to delete a bunch. The 1% of the time that it manages to land a full, 6 line completion accurately seems magical, but not magical enough to forget being annoyed at usually having to undo most of the work. Further a bit of a challenge as it has a high chance of ‘looking’ correct even as it makes a mistake, and if you are skimming the suggestion you might overlook the mistake because you aren’t forced to process it at the slow speed of typing. One thing it does do pretty well is if I’m about to construct a string intended for a human user, it will auto complete a decent enough error message for the human user, which tends to be a bit more forgiving of little mistakes in the data.
Document that code I wrote 7 years ago, suggest any security or efficiency changes. It’s surprisingly adept at that.
Give me the changes to NixOS 25.05 configuration.nix to add wadroid. Fails with an error, paste the error back into prompt. Oh, you need these kernel modules that are no longer default as of 25.05 make this change. Different error paste it back, Make this one last change and then reboot. It works. I spent a total of 5 minutes on it. If I were just using Google and screwing around that might have been half a morning.
OBS is giving me a pixel resolution warning. AI: it’s one of your cameras or some media you’ve added in an unsupported format. Give me a quick shell script to run through all of my media directories in this tree and convert all the MP4 video that’s yuv720 to a supported format in new tree so I can swap them out in the end with no risk. 30 seconds later it’s there. Yes, I can write that but I’m not going to have it done in 30 seconds. And if one of the files errors I just shove the error right back in the AI. I don’t personally care why one in 50 images failed I just want them to be converted and I’m far enough along and Dunning Kruger scale that I honestly don’t really care about what I don’t know as long as I can learn a little more and still get the job done.
Give me a python script to go through a file full of URLs and verify the SSL key expiration dates. Have a variable for how far the future to alert and then slack me a message at 10:00 a.m. everyday which URLs and IPs are expiring earlier than that variable. Also a bunch of the IPs don’t resolve to external addresses so you’re going to have to fake the calls to check them. Here’s my slack token in the channel name.
3 minute project
It doesn’t do my job for me but it gets rid of a hell of a lot of tech debt that I’ll never get around to. I won’t give it monolithic complicated jobs because it’s not good at it. But I will absolutely tell it to make me a flask app with stubs for half a dozen features. Or give it the source for a shitty old admin web page and ask it to modernize the CSS and add session logins.
Sure, if I’m not watching it it might do something relatively stupid. But honestly it has about the same odds of catching something I did years ago that was relatively stupid and telling me to fix it.
My gut feeling, based on the kind of repetitive nonsense I see them produce and bang on about, is that a lot of management types see AI efficiency because the work they do is repetitive and easily aided by AI input so they assume everything can be improved by it.
Not to say I don’t see the benefits of a good manager, I just don’t think they are that common.
That perspective is consistent with the code completion suggestions I get all the time.
LLM seems to think I really want to just rewrite the same 12 lines of code over and over again instead of calling the function where I wrote those 12 lines of code already.
Claude can spit out powershell scripts up to like, 400 or 500 lines without errors or with minimal, easily debugged errors. Adds things like error correction, colored text, user interaction, comments the code pretty well. Saves me hours every time I fire it up, so that I can in turn save myself dozens of hours with the scripts themselves.
But as far as I tell my boss, there is no AI use, and that’s how we’re keeping that for now/indefinitely
you see, for programming, AI achieved what SQL tried to do with database queries: programming by just telling the computer what you want and the computer figures out the how.
the catch is that human language is imprecise, so if you don’t know how to review what the AI produced, the AI might have written a script to wipe your data in the computer and you don’t even know until you run it and it is too late
The other day it spit out a five line piece of code, except, critically, it had used “archived” where it should have used “received”. Small word difference, huge functionality difference.
It absolutely does help, but we’re gonna have a couple whole new classes of copy/paste errors.
I do. Part of my job involves writing code and I often don’t even know where to start. When I get the first draft I’ll know which documentation to read, and then I make it actually work. Even when the LLM fails completely, writing its prompt serves as a rubber duck.
So do you frame the problem to the LLM, get it to spit out an example piece of code and then run through that initial attempt to get an idea of how to approach the problem? Kind of like prototyping the problem?
I take it you find that more efficient than traditional code planning methods? Or do you then start building flow charts/pseudo code from that prototype and confirm the logic to build more readable or efficient code?
I feel like AI is just going to end up replacing interns or entry level people, it can do easy tasks that would take a while by hand to do. Which based on how bad the job market seems to have been for people like me just trying to enter it somewhat makes sense.
Yeah the whole missing entry level job thing for most industries is going to backfire with exploding wages for experienced people. Without training grads and apprentices there’s an ever decreasing pool of experienced people to pick from.
Yeah we’ve structured the system so companies are mostly only responsible to shareholders, and shareholders really only care about short term gains with their only liability from the company is financial. Companies are always going have a focus on short term gains because thats what the system demands.
I find it useful for correcting my syntax (when it’s correct 😂) for certain networking devices. I touch so many vendors it’s not always one I can remember all the commands for.
It’s kinda become a Google replacement for me.
I have found certain areas it’s weak and I know when to quit when I’m ahead and it just agrees with me and spits out more incorrect info when I call it out.
Also when are we going to hit an AI feedback loop? 😅
I find the AI summary can be helpful when searching, but also not much more helpful than a summary of the first few search results which are mostly only loosely related paid for advertising …
Does anyone here actually see productivity improvements to their roles from using AI?
I’m a telecoms engineer and I see limited use cases in my role for AI. If I need to process data then I need something that can do math reliably. For document generation I can only reliably get it to build out a structure and even then I’ve more than likely got an existing document the I can use as a structure template.
Network design, system specification and project engineering are all so specific to the use case and have so few examples provided in public data sets that anything AI outputs is usually nonsense.
Am I missing some use cases here?
Also, if you do see productivity improvements from AI, why would you tell your employer? They want a 5 day working week but they know what they expect to be achieved in that week, so that’s what they get.
As a software engineer, definitely. Things that might have taken days writing a boilerplate framework and reading through docs to find out how to configure certain things could take hours now. Now I don’t have to spend hours learning how to use the data visualisation library in order to fix the one donut chart on my company’s site, I can ask AI to do it in minutes and make edits where needed.
It’s certainly not perfect or infallible, it spits out garbage a lot still. But since I have a deep understanding of the stuff I’m working on, I can recognize when it’s spitting out garbage and recalibrate.
I think your missing something big here.
A while back when one of the popular models came out it blew all the others away in benchmark tests. That got my coworker and boss super excited, we started coming up with different ways AI can help us. Thankfully as I pointed out, our software is proprietary and super secret, and all it took was a couple Google searches to find out a lot of those companies leak data like crazy and AI will just tell other people your secrets if they ask in the right way. So we needed to run our LLMs locally, but for that we’d need some beefy specs. I did the research, wrote a neovim and sublime plugin to integrate our local LLMs in a ‘copilot’ kind of way. My boss ordered my coworker and I whatever the new macbooks are with 128gb of memory to fit our lovely AI models, bought me a desktop tower and a few GPUs. Then I went on a 3 month paternity leave, came back and have heard nothing about AI since aside that my coworker switched to vs code to try and use continue but got frustrated, switched back to sublime and doesn’t use AI anymore.
So yeah, AI got me 2 new maxed out spec machines and 2 weeks of fuck around time writing plugins that were not nearly as complex as everyone thought, just because AI was involved.
For real though, I’ve tried quite a few times but 9/10 it fails me and I end up spending more time messing around with prompts than it would’ve taken me to do the thing. Occasionally when I have a mile long error message or something super obscure I’ll pass it to ollama and it seems to do well with wittling it down for me, that’s about it.
I’ll say it has been marginally more useful integrated into my code editor, prompt driven for me has been useless output for too much effort, but it ambiently sitting there in code editor can be helpful.
I still can only get it to provide useful suggestions about 15-20% of the time for like two lines, maybe a nice error message, but the failure rate is less obnoxious if you didn’t spend extra effort to ask for it and just ignore and keep going. Getting a feel for whether or not the LLM is likely to have something in the completion worth trying to review is a part of it based on what you’ve typed helps. Notably if you are some keystrokes into a very boilerplate process you might be more optimistic, or if you are about to provide a text string as a human error message, decent chance it wrote that for you well enough.
Still I’m more annoyed and not sure that it’s worth being annoyed, but I could buy that shaving typing out a couple lines 15% of the time could be an objective boost that outweighs the burden of futzing with the high error rate.
So my experience has been:
Yes.
Document that code I wrote 7 years ago, suggest any security or efficiency changes. It’s surprisingly adept at that.
Give me the changes to NixOS 25.05 configuration.nix to add wadroid. Fails with an error, paste the error back into prompt. Oh, you need these kernel modules that are no longer default as of 25.05 make this change. Different error paste it back, Make this one last change and then reboot. It works. I spent a total of 5 minutes on it. If I were just using Google and screwing around that might have been half a morning.
OBS is giving me a pixel resolution warning. AI: it’s one of your cameras or some media you’ve added in an unsupported format. Give me a quick shell script to run through all of my media directories in this tree and convert all the MP4 video that’s yuv720 to a supported format in new tree so I can swap them out in the end with no risk. 30 seconds later it’s there. Yes, I can write that but I’m not going to have it done in 30 seconds. And if one of the files errors I just shove the error right back in the AI. I don’t personally care why one in 50 images failed I just want them to be converted and I’m far enough along and Dunning Kruger scale that I honestly don’t really care about what I don’t know as long as I can learn a little more and still get the job done.
Give me a python script to go through a file full of URLs and verify the SSL key expiration dates. Have a variable for how far the future to alert and then slack me a message at 10:00 a.m. everyday which URLs and IPs are expiring earlier than that variable. Also a bunch of the IPs don’t resolve to external addresses so you’re going to have to fake the calls to check them. Here’s my slack token in the channel name.
3 minute project
It doesn’t do my job for me but it gets rid of a hell of a lot of tech debt that I’ll never get around to. I won’t give it monolithic complicated jobs because it’s not good at it. But I will absolutely tell it to make me a flask app with stubs for half a dozen features. Or give it the source for a shitty old admin web page and ask it to modernize the CSS and add session logins.
Sure, if I’m not watching it it might do something relatively stupid. But honestly it has about the same odds of catching something I did years ago that was relatively stupid and telling me to fix it.
Unless you’re a scammer or a spammer, the answer is legitimately “No”.
My gut feeling, based on the kind of repetitive nonsense I see them produce and bang on about, is that a lot of management types see AI efficiency because the work they do is repetitive and easily aided by AI input so they assume everything can be improved by it.
Not to say I don’t see the benefits of a good manager, I just don’t think they are that common.
That perspective is consistent with the code completion suggestions I get all the time.
LLM seems to think I really want to just rewrite the same 12 lines of code over and over again instead of calling the function where I wrote those 12 lines of code already.
Claude can spit out powershell scripts up to like, 400 or 500 lines without errors or with minimal, easily debugged errors. Adds things like error correction, colored text, user interaction, comments the code pretty well. Saves me hours every time I fire it up, so that I can in turn save myself dozens of hours with the scripts themselves.
But as far as I tell my boss, there is no AI use, and that’s how we’re keeping that for now/indefinitely
you see, for programming, AI achieved what SQL tried to do with database queries: programming by just telling the computer what you want and the computer figures out the how.
the catch is that human language is imprecise, so if you don’t know how to review what the AI produced, the AI might have written a script to wipe your data in the computer and you don’t even know until you run it and it is too late
The other day it spit out a five line piece of code, except, critically, it had used “archived” where it should have used “received”. Small word difference, huge functionality difference.
It absolutely does help, but we’re gonna have a couple whole new classes of copy/paste errors.
Makes me wonder what an LLM trained in lojban would accomplish.
So you’re sharing your data with third parties and relinquishing code copyright without telling your boss?
That’s pretty great, what kind of things do you use the PowerShell scripts for?
I do. Part of my job involves writing code and I often don’t even know where to start. When I get the first draft I’ll know which documentation to read, and then I make it actually work. Even when the LLM fails completely, writing its prompt serves as a rubber duck.
So do you frame the problem to the LLM, get it to spit out an example piece of code and then run through that initial attempt to get an idea of how to approach the problem? Kind of like prototyping the problem?
I take it you find that more efficient than traditional code planning methods? Or do you then start building flow charts/pseudo code from that prototype and confirm the logic to build more readable or efficient code?
Yes. Maybe? I don’t know traditional code planning methods. I guess?
I feel like AI is just going to end up replacing interns or entry level people, it can do easy tasks that would take a while by hand to do. Which based on how bad the job market seems to have been for people like me just trying to enter it somewhat makes sense.
Yeah the whole missing entry level job thing for most industries is going to backfire with exploding wages for experienced people. Without training grads and apprentices there’s an ever decreasing pool of experienced people to pick from.
Yep but of course companies don’t care about it right now as right now it allows them to be more profitable in the immediate term.
Yeah we’ve structured the system so companies are mostly only responsible to shareholders, and shareholders really only care about short term gains with their only liability from the company is financial. Companies are always going have a focus on short term gains because thats what the system demands.
I find it useful for correcting my syntax (when it’s correct 😂) for certain networking devices. I touch so many vendors it’s not always one I can remember all the commands for.
It’s kinda become a Google replacement for me.
I have found certain areas it’s weak and I know when to quit when I’m ahead and it just agrees with me and spits out more incorrect info when I call it out.
Also when are we going to hit an AI feedback loop? 😅
I find the AI summary can be helpful when searching, but also not much more helpful than a summary of the first few search results which are mostly only loosely related paid for advertising …