The ergonomics of this are good in terms of integration mechanics. I wouldn't worry about performance as long as we are in the tens of milliseconds range on reflection/invoke.
I think the biggest concern is that the # of types & methods is going to be too vast for most practical projects. LLM agents fall apart beyond 10 tools or so. Think about the odds of picking the right method out of 10000+, even with strong bias toward the correct path. A lot of the AI integration pain is carefully conforming to the raw nature of the environment so that we don't overwhelm the token budget of the model (or our personal budgets).
I would consider exposing a set of ~3 generic tools like:
SearchTypes
GetTypeInfo
ExecuteScript
This constrains your baseline token budget to a very reasonable starting point each time.
I would also consider schemes like attributes that explicitly opt-in methods and POCOs for agent inspection/use.
Hi bob,
Thanks for your reply. Yes, reducing tokens and trying to reduce variables to achieve greater determinism in enterprise projects is a fundamental issue.
In my work, we have multi-project solutions, and I understand what you mean. I completely agree. I didn't imagine proposing such a solution to eliminate the architectural creation part, but at this point, that can be done directly in the JSON we send to the model.
With reflection, you don't need wrappers. Let's say we also limit the functions. With this architecture, I don't have to write the wrapper for the three functions I choose. I simply create a JSON file that contains three basic ones.
Yes, I actually discovered that C++ implemented reflection after working on the project, but at the moment I'm too busy to follow up on this with the attention it deserves. However, this is definitely something I need on my TODO.
As for the details, the scheme I started with is “simple” and, as already mentioned, it is valid for all memory-managed languages.
The desktop application starts up and, at the entry point, does the following:
engine.Ingnite(this)
The “engine” library now has the application instance in its belly, and a listener is created for each application, waiting for commands.
On the “chat” side, the prompt to the model starts, and here both the prompt and the classic json that I built with the methods I decided to expose are sent.
When the chat receives the response, it sends the token to the listening listener:
engine.Execute(method.json)
From here, first deserialize the json by creating the objects. At this point, you have both the application instance and its method, as well as the arguments with which it must be called:
method.Invoke(instance, parameters)
I find this interesting, and it breaks with the classic way we are used to communicating with wrapper APIs.
I think that, since it has probably been an established method for decades with the advent of the web, it was natural to move in this direction. However, coming mainly from a desktop environment, I wonder if this is the “easiest” and “smartest” way to handle these cases.
The ergonomics of this are good in terms of integration mechanics. I wouldn't worry about performance as long as we are in the tens of milliseconds range on reflection/invoke.
I think the biggest concern is that the # of types & methods is going to be too vast for most practical projects. LLM agents fall apart beyond 10 tools or so. Think about the odds of picking the right method out of 10000+, even with strong bias toward the correct path. A lot of the AI integration pain is carefully conforming to the raw nature of the environment so that we don't overwhelm the token budget of the model (or our personal budgets).
I would consider exposing a set of ~3 generic tools like:
This constrains your baseline token budget to a very reasonable starting point each time.I would also consider schemes like attributes that explicitly opt-in methods and POCOs for agent inspection/use.
Hi bob, Thanks for your reply. Yes, reducing tokens and trying to reduce variables to achieve greater determinism in enterprise projects is a fundamental issue.
In my work, we have multi-project solutions, and I understand what you mean. I completely agree. I didn't imagine proposing such a solution to eliminate the architectural creation part, but at this point, that can be done directly in the JSON we send to the model. With reflection, you don't need wrappers. Let's say we also limit the functions. With this architecture, I don't have to write the wrapper for the three functions I choose. I simply create a JSON file that contains three basic ones.
At this point, reflection will do its job.
Can you clarify this further?
Hello ddddazed, interesting approach.
With the C++26 release, native reflection was introduced, which is arguably more powerful than RTTR.
So I'm not entirely sure on this point, how do reflection discovery works and how are complex types handled?
nice work anyway
Yes, I actually discovered that C++ implemented reflection after working on the project, but at the moment I'm too busy to follow up on this with the attention it deserves. However, this is definitely something I need on my TODO.
As for the details, the scheme I started with is “simple” and, as already mentioned, it is valid for all memory-managed languages.
The desktop application starts up and, at the entry point, does the following:
engine.Ingnite(this)
The “engine” library now has the application instance in its belly, and a listener is created for each application, waiting for commands.
On the “chat” side, the prompt to the model starts, and here both the prompt and the classic json that I built with the methods I decided to expose are sent.
When the chat receives the response, it sends the token to the listening listener:
engine.Execute(method.json)
From here, first deserialize the json by creating the objects. At this point, you have both the application instance and its method, as well as the arguments with which it must be called:
method.Invoke(instance, parameters)
I find this interesting, and it breaks with the classic way we are used to communicating with wrapper APIs.
I think that, since it has probably been an established method for decades with the advent of the web, it was natural to move in this direction. However, coming mainly from a desktop environment, I wonder if this is the “easiest” and “smartest” way to handle these cases.
I hope this can help.