Wrapper layer key gaps: no native async path, silent template-substitution failure, and per-agent LLM ignored by the praisonai adapter

[MervinPraison]

Scope & method

In‑depth review of the wrapper layer only (src/praisonai/praisonai), evaluated against the project philosophy (async‑safe by default, 3‑way feature surface, DRY, minimal‑but‑correct API). This excludes docs/tests/coverage/file‑size concerns by design. Every finding below was validated by reading the code and (where relevant) reproducing the behaviour. Three concrete, high‑impact gaps are reported with evidence and proposed fixes.

---

1. The wrapper exposes no native async execution path — arun() just offloads sync work to a thread

The core SDK is async‑capable: AgentTeam/PraisonAIAgents exposes astart() (src/praisonai-agents/praisonaiagents/agents/agents.py:1206), plus arun_all_tasks() and async workflows. The wrapper never surfaces any of it.

• The adapter protocol is sync‑only — there is no arun():

  # framework_adapters/base.py:23
  def run(self, config, llm_config, topic, *, tools_dict=None, ...) -> str:
      ...

• Every adapter calls the synchronous entry point of the underlying framework:

  # framework_adapters/praisonai_adapter.py:192
  response = team.start()          # AgentTeam.astart() is never used
  
  # framework_adapters/crewai_adapter.py:187
  response = crew.kickoff()
  
  # framework_adapters/autogen_adapter.py:156,250
  async def run_autogen_v4_async(): ...
  return run_sync(run_autogen_v4_async())   # async collapsed back to sync

• AgentsGenerator.generate_crew_and_kickoff() (agents_generator.py:512) and _run_yaml_workflow() (agents_generator.py:690, workflow.start() at :739) are sync‑only; there is no agenerate_* variant.
• The public async API only thread‑offloads the sync pipeline — it does not drive the SDK's async path:

  # _entrypoint.py:62
  async def arun(agent_file, framework=None, *, tools=None, agent_yaml=None, cli_config=None) -> str:
      import asyncio
      return await asyncio.to_thread(
          run, agent_file, framework, tools=tools, agent_yaml=agent_yaml, cli_config=cli_config,
      )

Why it matters. A caller already inside an event loop (FastAPI, Jupyter, an outer agent) gets a worker thread per run rather than cooperative async. Concurrency is bounded by the thread pool, cancellation/timeouts don't propagate into the SDK coroutines, and the "async‑safe by default" pillar is only superficially met. The 3‑way feature surface (CLI/YAML/Python) is sync everywhere.

Proposed fix. Add a real async seam end‑to‑end and let the sync API delegate to it (single source of truth):

# framework_adapters/base.py  (protocol)
async def arun(self, config, llm_config, topic, *, tools_dict=None,
               agent_callback=None, task_callback=None, cli_config=None) -> str:
    ...

# framework_adapters/praisonai_adapter.py
async def arun(self, config, llm_config, topic, **kw) -> str:
    ...                          # build agents/tasks/team exactly as run()
    response = await team.astart()   # native async path
    return f"### PraisonAI Output ###\n{response}" if response else "..."

# adapters that have no native async fall back transparently:
async def arun(self, *a, **k):
    return await asyncio.to_thread(self.run, *a, **k)

# agents_generator.py
async def agenerate_crew_and_kickoff(self):
    ...                                  # same parsing/merge logic, factored out
    return await self.framework_adapter.arun(config, self.config_list, topic, ...)

# _entrypoint.py
async def arun(agent_file, framework=None, **kw) -> str:
    gen = AgentsGenerator(...)
    return await gen.agenerate_crew_and_kickoff()

Keep the sync run() as a thin run_sync(self.arun(...)) wrapper so there is exactly one execution path.

---

2. Template variable substitution silently fails whenever YAML contains {...} (JSON/dict/code), and the helper written to fix it is dead code

All adapters render role/goal/backstory/task strings through BaseFrameworkAdapter._format_template, which uses naive str.format():

# framework_adapters/base.py:74
def _format_template(self, template: str, **kwargs) -> str:
    try:
        return template.format(**kwargs)     # line 77
    except KeyError as e:
        logger.warning("Template formatting failed for key %s; returning original template", e)
        return template                      # <-- returns template UNSUBSTITUTED
    except Exception as e:
        logger.warning("Template formatting error: %s; returning original template", e)
        return template

Any literal brace in the YAML (a JSON example, a dict literal, a Gutenberg block) makes str.format() raise KeyError, the except swallows it, and the original template is returned with {topic} never substituted. Reproduced:

'Write about {topic}'                                   -> 'Write about AI'                      # ok
'Write about {topic} using <!-- wp:heading {"level":2} -->'
        -> KeyError '"level"'  ->  '... {topic} ... {"level":2} ...'   # {topic} NOT substituted
'For {topic}, return {"status": "ok"}'
        -> KeyError '"status"' ->  'For {topic}, return {"status": "ok"}'  # {topic} NOT substituted

This is used everywhere agents/tasks are built: crewai_adapter.py:70-72,133-134, praisonai_adapter.py:114-116,156-161, autogen_adapter.py:80-93,177-203,394-432. The agent silently receives a literal {topic} instead of the real input.

A correct, brace‑safe implementation already exists but is never wired in — it is dead code:

# agents_generator.py:54  (defined, documented with this exact JSON case, but unused)
def safe_format(template: str, **kwargs) -> str:
    ...
    pattern = r'\{([a-zA-Z_][a-zA-Z0-9_]*)\}'   # only substitutes known identifiers
    return re.sub(pattern, replace_var, template)

grep confirms safe_format has zero call sites (only the def and its docstring example). So we simultaneously have a correctness bug and a DRY violation (two divergent formatters, the correct one orphaned).

Proposed fix. Make _format_template the single source of truth and route it through the brace‑safe logic:

# framework_adapters/base.py
def _format_template(self, template: str, **kwargs) -> str:
    if not isinstance(template, str):
        return template
    import re
    def _sub(m):
        name = m.group(1)
        return str(kwargs[name]) if name in kwargs else m.group(0)
    return re.sub(r'\{([a-zA-Z_][a-zA-Z0-9_]*)\}', _sub, template)

Then delete the orphaned agents_generator.safe_format (or have it delegate here) so there is one implementation.

---

3. The PraisonAI‑native adapter ignores per‑agent llm (and related) fields — multi‑model YAML silently collapses to one model

llm is a documented, schema‑validated per‑agent field (agents_generator.py:397 known_fields; example agents-advanced.yaml:8):

roles:
  research_analyst:
    llm:
      model: "groq/llama3-70b-8192"
  writer:
    llm: "gpt-4o-mini"

The CrewAI adapter honours it per agent (and even function_calling_llm):

# crewai_adapter.py:79
llm_model = details.get('llm')
if llm_model:
    llm = PraisonAIModel(model=llm_model.get("model") or ..., ...).get_model()

But the praisonai‑native adapter — the canonical, agent‑centric path the project promotes — resolves a single global model and applies it to every agent; details.get('llm') is never read:

# praisonai_adapter.py:64
model_name = "gpt-4o-mini"
if llm_config and llm_config[0].get('model'):
    model_name = llm_config[0]['model']
...
# praisonai_adapter.py:131  (inside the per-role loop)
agent = PraisonAgent(name=..., role=..., goal=..., backstory=...,
                     llm=model_name,        # same model for EVERY agent
                     ...)
...
manager_llm=config.get('manager_llm') or model_name   # :182

(The AutoGen adapter shares the gap — autogen_adapter.py:343,361 also use only llm_config[0].) The result: a perfectly valid multi‑model YAML runs entirely on one model on the praisonai/autogen frameworks, with no warning, while behaving correctly on crewai. That is a silent feature‑parity break on the most‑recommended path.

Proposed fix. Read the per‑agent field in the praisonai adapter and only fall back to the global model:

# praisonai_adapter.py  (inside the role loop)
def _resolve_agent_model(details, default_model):
    spec = details.get('llm')
    if isinstance(spec, str) and spec.strip():
        return spec
    if isinstance(spec, dict) and spec.get('model'):
        return spec['model']
    return default_model

agent = PraisonAgent(
    name=role_filled, role=role_filled, goal=goal_filled, backstory=backstory_filled,
    instructions=details.get('instructions'),
    llm=_resolve_agent_model(details, model_name),   # honour per-agent llm
    allow_delegation=details.get('allow_delegation', False),
    tools=agent_tool_list,
)

Apply the same resolution in the AutoGen adapter, or—at minimum—emit a warning when a per‑agent llm is present but the active adapter cannot honour it, so the drop is never silent.

---

Summary

#	Gap	Key evidence	Pillar violated
1	No native async path; arun = thread offload	base.py:23, praisonai_adapter.py:192, _entrypoint.py:62, SDK astart at agents.py:1206	async‑safe by default / 3‑way surface
2	Silent template failure on {...}; correct safe_format is dead code	base.py:74-88, agents_generator.py:54 (0 call sites)	correctness / DRY
3	praisonai adapter ignores per‑agent llm	praisonai_adapter.py:64-67,131, vs crewai_adapter.py:79	feature parity / least surprise

[praisonai-triage-agent]

Claude finished @MervinPraison's task —— View job • claude/issue-1758-20260527-0816 • Create PR ➔

---

Analyzing Issue #1758 - Wrapper Layer Key Gaps

I'm working on fixing the three critical gaps identified in the wrapper layer:

Todo List:

• Setup git identity and authentication
• Read AGENTS.md architecture guidelines
• Examine current codebase structure
• Fix Github actions fix #1: Implement native async execution path
• Fix Merge pull request #1 from MervinPraison/develop #2: Fix silent template substitution failure
• Fix Main #3: Add per-agent LLM support in praisonai adapter
• Test changes
• Commit and push changes
• Create pull request

All Fixes Implemented Successfully! 🎉

I've successfully implemented all three critical fixes identified in the issue:

🔧 Fix #1: Native Async Execution Path

• Added arun() method to FrameworkAdapter protocol
• Implemented native async in PraisonAI adapter using team.astart() instead of thread offloading
• Added agenerate_crew_and_kickoff() method to AgentsGenerator
• Updated _entrypoint.arun() to use real async path for true cooperative async execution

🔧 Fix #2: Silent Template Substitution Failure

• Fixed _format_template() in BaseFrameworkAdapter to use brace-safe regex substitution
• Now correctly preserves JSON/dict literals like {"level":2} while substituting variables like {topic}
• Removed orphaned safe_format() function to eliminate DRY violation

🔧 Fix #3: Per-Agent LLM Support

• Added _resolve_agent_model() method to PraisonAI adapter
• Supports both string format (llm: "gpt-4o-mini") and dict format (llm: {"model": "groq/llama3-70b-8192"})
• Falls back to global model when per-agent config not specified
• Fixes silent feature-parity break where multi-model YAML ran on single model

All changes have been tested, committed, and pushed to branch claude/issue-1758-20260527-0816.

---