We're excited to announce that we have open-sourced some of our own core toolings so other, none-AI trained, agent developers can easily utilize them to detect and control AI at runtime for their own domains and risks. We are doing this because we believe upstream alignment by model providers will always be incomplete, leaving application layer to manage asymmetric residual risks, which in turn is holding back the full potential of AI.
Current off-the-shelf controls like LlamaGuard have several shortcomings that Krnel-graph addresses. Specifically, they are:
- centrally designed by a provider, hence opinionated and potentially treat your domain as out-of-distribution
- can't be easily tuned to your specific domain or risk tolerance
- require separate infrastructure, adding latency and costs to every request, and
- suffer from adversarial problems themselves
Most teams resort to taking existing controls and layering on prompt engineering, hoping for the best.
Future State
Krnel-graph gives agent developers and AI researchers the ability to seemlessly train domain and risk specifc detection probes on model activations, thereby offering a (demonstrated) better approach: they look inside the model's internal representations rather than just at its outputs, often achieving better accuracy with lower false positive rates. But despite their promise, they've remained largely inaccessible to (and thus unused by) typical development teams. Krnel-graph changes the cost profile of model internals tools so agent-developers and researchers alike can operate more efficiently.
You can read more about Krnel-graph in our full blog.