Bartholomew Gallacher
Well-known member
- Joined
- Sep 26, 2018
- Messages
- 5,769

- SL Rez
- 2002
Answer AI tested an AI coder bot named Devin, which is available since about one year. They gave the AI 20 routine task a software developer is able to solve.
The result is really, really bad: out of 20 tasks Devin only finished 3 successfully.
"Tasks that seemed straightforward often took days rather than hours, with Devin getting stuck in technical dead-ends or producing overly complex, unusable solutions," the researchers explain in their report. "Even more concerning was Devin’s tendency to press forward with tasks that weren’t actually possible."
Oh the joy of AI!
The result is really, really bad: out of 20 tasks Devin only finished 3 successfully.
"Tasks that seemed straightforward often took days rather than hours, with Devin getting stuck in technical dead-ends or producing overly complex, unusable solutions," the researchers explain in their report. "Even more concerning was Devin’s tendency to press forward with tasks that weren’t actually possible."
Oh the joy of AI!