Researchers from Asari AI, MIT Computer Science and Artificial Intelligence Laboratory, and Caltech have announced a new programming model called Probabilistic Angelic Nondeterminism (PAN) and a corresponding Python implementation named ENCOMPASS. The approach decouples the core workflow logic of an AI agent from the inference strategies that manage uncertainty, aiming to improve scalability and reduce technical debt in production\u2011grade agents.<\/p>\n
Generative large language models (LLMs) are inherently stochastic. A prompt that succeeds once may fail on the next run, prompting developers to embed complex error\u2011handling loops, retries, and branching logic around business rules. This entanglement of business logic and uncertainty handling creates maintenance challenges and limits experimentation with different inference strategies.<\/p>\n
Traditional agent designs combine the sequence of steps required to complete a task with the methods used to navigate uncertainty, such as best\u2011of\u2011N sampling or tree search. Switching from one strategy to another often requires a complete rewrite of the agent\u2019s control flow, discouraging teams from adopting more reliable approaches.<\/p>\n
ENCOMPASS introduces a primitive called branchpoint()<\/em> that marks locations in code where an LLM call may produce divergent outcomes. Developers write the \u201chappy path\u201d of the workflow as if the LLM call will succeed. At runtime, the framework interprets these branch points to build a search tree of possible execution paths.<\/p>\n By treating inference strategies as a search over execution paths, the framework allows developers to apply algorithms such as depth\u2011first search, beam search, or Monte Carlo tree search without modifying the underlying business logic. This separation creates what the authors term \u201cprogram\u2011in\u2011control\u201d agents, where the code defines the overall workflow and the LLM performs only specific subtasks.<\/p>\n The research team applied ENCOMPASS to a Java\u2011to\u2011Python translation agent. The workflow involved translating repository files, generating inputs, and validating outputs through execution. In a conventional Python implementation, adding search logic required defining a state machine, obscuring business logic and complicating code readability. Implementing beam search demanded explicit state management across a dictionary of variables.<\/p>\n Using ENCOMPASS, the team inserted branchpoint()<\/em> statements before LLM calls, keeping the core logic linear and readable. Beam search applied at both the file and method levels outperformed simpler sampling strategies. The study found that performance improved linearly with the logarithm of inference cost, and the most effective fine\u2011grained beam search strategy would have been the most complex to implement with traditional coding methods.<\/p>\n Managing inference cost is a key concern for data officers overseeing AI project budgets. The researchers compared scaling the number of refinement loops in a \u201cReflexion\u201d agent pattern\u2014where an LLM critiques its own output\u2014to using a best\u2011first search algorithm. The search\u2011based approach achieved comparable performance to the standard refinement method while reducing cost per task.<\/p>\n These results suggest that externalizing inference strategy allows teams to balance compute budget and accuracy without rewriting application code. A low\u2011stakes internal tool could employ a cheap, greedy search strategy, whereas a customer\u2011facing application could use a more exhaustive search, all within the same codebase.<\/p>\n Decoupling inference strategy from workflow logic aligns with established software engineering principles of modularity. Hard\u2011coding probabilistic logic into business applications creates technical debt, hampers testing, auditing, and upgrades. Separating concerns enables independent optimization of both logic and inference strategy.<\/p>\n Governance benefits also emerge. If a particular search strategy produces hallucinations or errors, it can be adjusted globally without reviewing each agent\u2019s code. This simplifies versioning of AI behaviors, a requirement in regulated industries where the \u201chow\u201d of a decision is as important as the outcome.<\/p>\n As inference\u2011time compute scales, managing execution paths will become increasingly complex. Enterprise architectures that isolate this complexity are likely to prove more durable than those that allow it to permeate the application layer. The research team plans to further evaluate ENCOMPASS in additional domains, including summarization and creative generation, where defining reliable scoring functions remains a challenge.<\/p>\n Overall, the PAN model and ENCOMPASS framework offer a structured way to separate logic from search in AI agents, potentially improving reliability, reducing maintenance overhead, and enabling more flexible cost\u2011performance trade\u2011offs in production environments.<\/p>\n","protected":false},"excerpt":{"rendered":" Researchers from Asari AI, MIT Computer Science and Artificial Intelligence Laboratory, and Caltech have announced a new programming model called Probabilistic Angelic Nondeterminism (PAN) and a corresponding Python implementation named ENCOMPASS. The approach decouples the core workflow logic of an AI agent from the inference strategies that manage uncertainty, aiming to improve scalability and reduce […]<\/p>\n","protected":false},"author":1,"featured_media":520,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[128],"tags":[493,494,496,454,495],"class_list":["post-519","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-updates","tag-ai_agent","tag-logic_separation","tag-modular_architecture","tag-scalability","tag-search_optimization"],"_links":{"self":[{"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/posts\/519","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/comments?post=519"}],"version-history":[{"count":0,"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/posts\/519\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/media\/520"}],"wp:attachment":[{"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/media?parent=519"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/categories?post=519"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buildconsole.com\/blog\/wp-json\/wp\/v2\/tags?post=519"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Case Study: Legacy Code Migration<\/h2>\n
Cost and Performance Scaling<\/h2>\n
Implications for Enterprise AI<\/h2>\n
Future Directions<\/h2>\n