Understanding AI Workflows Is Ephemeral Knowledge

Through recent work I've come to understand that the difference between software development and AI workflow development is the difference between persistent and ephemeral knowledge.

When writing a giant Cursor plugin, for example, with thousands of words of markdown, if you have structured input, you cannot go directly to the Cursor plugin, CTRL+F (find), and gain an understanding of how that structured input is working it's way through the workflow. Let's say you have an input field called myCoolField in your structured input. If you search a Cursor plugin markdown for how myCoolField is handled, you're looking at logic, but logic in verbose contractual language. In the same way a contract is a series of assertions meant to coalesce an agreement on an opaque topic, AI markdown, when done well, disambiguates a similarly opaque workflow process so that an AI model is given a rubric of what decisions need to be made, when they need to be made, and how to make them.

Let's say you have an AI workflow where you submit structured input passed down a series of explicit phases A -> B -> C -> D with an overall goal of achieving E. Let's say you intended for myCoolField to be passed through each one of those phases to be used for different purposes at each one in service of achieving E. E in this case is opaque, and its success or failure is more than partially subjective. If myCoolField is only being passed to A and B, because goal E's success is primarily subjective, you might conclude that the results you're seeing are a success to the fullest extent possible. However, this isn't true. In this case, C and D both need myCoolField to perform their duties in service of goal E to the highest competency, but they're not yet receiving it. Therefore, C and D's performance is so far subpar. Goal E is not being achieved to the fullest possible extent because C and D do not have all the input they need to make full contributions to the success of E's workflow.

The difficulty here becomes ascertaining first: are my results for goal E subpar? If so, the next question becomes, why? This is where persistent versus ephemeral knowledge comes into play.

If your AI workflow was instead a codebase, let's say, an OOP Class, then ascertaining this information is actually fairly straightforward. You go to the codebase and spend time searching for myCoolField, and then trace its movement through the code. Because code can be developed to change in highly specific, deterministic, and intentional ways, as you learn the path of myCoolField, you can retain that information expecting that the pattern as you're coming to understand it will hold. If it is changed, you'll see how it's changed in the code diffs, and update your own understanding.

With AI markdown files like a Cursor plugin, you can do no such thing. Even if you do search myCoolField all throughout the plugin, you will find it being referenced, but it will be surrounded in dense, legalistic, verbose contractual language. While this language is in essence as pure logic as language can get, it is not as pure as logic as computer code is. What’s more, it always has the potential to suddenly change drastically, in ways that would be difficult to track. Even if you could read myCoolField and track its movement through the markdown files, which you don’t have the time to do, you couldn’t retain that information because you can’t expect it to be the same through further changes, and when and if it does change the diffs between markdown files are just as verbose and legalistic. Reviewing changes presents the same time and energy resource problem as understanding the markdown files themselves to begin with. Your understanding of the Cursor plugin, both long and short term, will be ephemeral for this reason. Never fully certain, and only in highly condensed summaries that will change between every similar request. 

The only solution then is to use AI to research and summarize the tool. This process works well, but the point is that it’s ephemeral. Every time you need to know how myCoolField works its way through the markdown files, you’ll need to submit another research prompt and then read the returned summary. This is also doable and works, but it’s not persistent in the same way that code is. What’s more, until you think to do the right research, you’ll never know that myCoolField is not working its way fully through all A -> D phases, but only A and B. Another issue is that, as you update and change the structure of the markdown files, you can never be fully sure that the technology isn’t causing unintended side effects, changing things that are affecting other logic you rely on.

It will be interesting to see how the industry copes with this. As time goes on, it’s clear that there are many productive use cases for AI, especially in automating established workflows. What’s not clear is that coding is truly dead. Computer code has its own intrinsic values that can’t be replicated by AI, and the question eventually will start to become, “What are we really getting by abstracting the coding process for everyone entirely?” My example with the AI workflow is specific to automated workflows, but it’s also applicable to AI in general. Asserting that only an ephemeral understanding of codebases is needed to run production software ignores another question entirely, “Why is this superior then just writing the code with established productivity increasing processes?” Speed might be increased, but to go forward, you need to know where you’ve been. It’s a mistake to believe that AI circumvents this idiom.