At Diller Digital, we recently went through the exercise of thinking deeply about what it means to produce software for scientific computing in the age of AI. And by “the age of AI”, we mean at a time when tools that use large language models (LLMs) have made it easy to generate code and implement software from natural language prompts. The result is a brand new course at Diller Digital.
Producing software has always had three steps, more or less:
- Specify what you want
- Write the code
- Verify the code is correct
In different contexts, each of these steps may be more or less critical and be more or less time consuming. What’s unmistakable is that modern LLM-driven coding agents have reduced the time and effort of the middle step. Depending on your context, the first and last steps may be a little bit easier, or possibly a lot harder, with an LLM. We spend more time thinking about what code should do and making sure it works correctly and less time actually writing the code than we used to do.
With that in mind, and at the request of the Information Science and Technology Institute at Los Alamos National Laboratory (where the subject matter has sometimes been known to be tricky and the stakes high), we re-wrote our Software Engineering course from the ground up, thinking carefully through what Software Engineering looks like in this new age of cheap code generation.
How can we take advantage of new tools and still maintain accountability, traceability, and responsibility for engineering decisions? How can we demonstrate the correctness of our solution?
The new class poses Software Engineering as the discipline that manages risk and demonstrates correctness in an age when code generation is cheap and human attention and cognition are not. We assume that you know the basics of programming in Python, even if the details are sometimes a mystery. But in any case, you want to write code you can have confidence in that it’s not just a black box, and if something goes wrong or an unanticipated use case arises, someone, probably you, can debug it under pressure.
Software Engineering is the discipline that manages risk and demonstrates correctness in an age when code generation is cheap and human attention and cognition are not.
So while it is a ground-up rewrite, some of the same skills and tools of our former craftsmanship-oriented course are still relevant, but now the emphases are different:
- We present
gitand source control as an accountability record and not just version control and a tool for coordinating multiple threads of activity. In the class, we not only teach the basics ofgitcommits and branch manipulations, but we also collaboratively work on a project together and record prompts, work plans, discussions and reviews in the class record. - Tests are more critical than ever before as executable specifications and demonstrations of correctness. We discuss what kinds of mistakes coding agents likely to make, and how we can implement tests that guard against them. We talk about when it is appropriate to write the tests before the code and when it is appropriate to write them after.
- We spend time talking about the design of code interfaces. If you’re writing code that humans and/or agents will make use of, what is the impact of different choices in the interface? When is it OK to mutate objects? When do you want a functional design? What is a fluent interface, and when might you want to implement one? What features make an API or CLI stable and robust against future changes?
- Large Language Models consume documentation like humans do; just because they can parse code doesn’t mean you don’t need documentation. In this course, we put the tools in your hands that allow you to produce high-quality documentation, including the basics of using Sphinx, which produces polished documentation for most of the major data science oriented projects in Python.
- We show how to use logging, debugging, and profiling as ways of understanding what’s going on in the code. In one exercise, for example, we add logging messages to our code until an agent (or a human) can debug an error given the log file without access to the source code.
- Finally, we talk about what code review means in the age of coding agents, and we use an in-class project to give you practice with peer-review.
By the end of “Software Engineering in the Age of AI”, we expect you’ll agree with one or more of these students who attended a recent session:
“This course was an excellent overview of agentic AI tools and the framework of using well-defined software development practices when working with these tools helped me enhance my own scientific software development.”
“This class greatly helped me understand how the role of a human programmer has shifted in light of agents. It helps me define (for myself) my role in software development. That guidance, coupled with practice using an AI to develop software, document it, use source control, and GitHub-like functions was amazing. After this class, I’m prepared to “hit the ground running” on my own projects using this framework; I’m excited to use it.”
“This course is especially useful for non-software engineers because it explains the core ideas behind how software is planned, built, tested, deployed, and maintained. Even if you are not writing production code every day, understanding topics like requirements, system design, APIs, version control, testing, debugging, security, and collaboration workflows makes it much easier to build small tools yourself and to work effectively with software engineers. It helps you ask better questions, communicate tradeoffs more clearly, understand why engineering work takes time, and make better decisions when turning business or technical needs into working software. I’d highly recommend this course to anyone who doesn’t have formal training as a software developer, but frequently ends up writing software.”
“I had used some of the tools in the course before, but I rarely used (1) Logging (2) Formal debugging tools (3) Profiling. I knew these tools existed, but I never really took the time to get over the “activation energy” to learn them. I will absolutely be using these tools in the software I write going forward.”