Design Choice for Nuclio as A Dependency

[wescef-fahmad]

I recently tried setting up some of the pre-configured AI tools, like Yolo, SiamMask, SAM, etc and while the tools ran fine, trying to change things, for example, trying to use a more up-to-date Python version resulted in an error hell. My takeaway from the exercise was that Nuclio's error reporting is pretty terrible, its Python support is outdated and the documentation is scanty. This made me wonder if there is a compelling reason why AI tools are supported through serverless functions only. An alternative architecture where one can plug in a WebAPI and a spec defining its parameters would be as flexible as will be a bring-your-own-vanilla-docker approach.

[tweigel-dev]

I’d like to reinforce this point from a production / enterprise perspective.

The current dependency on Nuclio for model integration introduces quite a bit of overhead:

Additional platform to operate (deployment, configs, CLI)
Tight coupling to a specific serverless framework
More complex debugging and lifecycle management

This becomes especially problematic in GitOps-based environments, where:

Everything should be declarative and version-controlled
Introducing a separate function registry breaks standard CI/CD flows
Model deployments become harder to manage consistently

Compared to this, approaches like Label Studio are much simpler:

Expose a model via HTTP (URL + port)
Register the endpoint
No extra infrastructure required

A “bring-your-own-endpoint” approach (REST/gRPC) would integrate much better with existing ML stacks and deployment pipelines.

It would be great if CVAT could support calling external model endpoints directly, with a defined input/output contract, instead of relying solely on Nuclio.