ETH Zurich Study Finds CS Knowledge Trumps AI Habit in Vibe Coding
A new study from ETH Zurich challenges the notion that AI coding tools have made it possible for anyone to build software without needing computer science knowledge. Researchers tested 100 students on their vibe-coding skills, which involve using language models to write code. The results showed that those with strong computer-science achievement performed better than those who relied heavily on language models.
The study found a correlation between computer-science achievement and vibe-coding performance, but not between daily AI habit and success. In fact, students who reported more exposure to large language models in daily life performed worse on the task.
This contradicts the 'no-code' pitch that has been popularized by some AI coding vendors and bootcamp marketers, which claims that one doesn't need to understand loops, state, data structures, or failure modes to ship a product. The study suggests that this is not the case, and that having a solid foundation in computer science is still essential for building software.
The researchers also built a writing test around a technical concept from each participant's coursework, which required them to describe the system precisely enough for someone else or a model to act on it correctly. This highlights the need for precision in coding and how AI tools are not a replacement for human knowledge.