ChaM3Leon: from visual workflow to Machine Learning pipeline

In the development of Machine Learning solutions, the recurring problem is not the availability of algorithms or frameworks, but the translation of an experimental idea into an executable, reproducible, and maintainable pipeline.

Building an ML workflow requires the orchestration of interdependent phases (data acquisition, feature engineering, training, validation, and deployment) that demand heterogeneous skills and an in-depth knowledge of the underlying frameworks.

To bridge this gap, ChaM3Leon was born, integrating visual design, template-based code generation, and enrichment through Large Language Models (LLMs) into a single platform.

ChaM3Leon is, in fact, an open-source platform designed as an integrated environment for the automatic generation of Machine Learning workflows. The system combines a drag-and-drop graphical interface based on React Flow, Jinja (a templating engine) for structural code generation, and a generative artificial intelligence layer that enriches outputs with best practices (“MLOps”), error handling, and contextual optimizations.

The platform's role is to break down the technical barriers between the conception of an ML pipeline and its concrete implementation, allowing users to focus on the process logic rather than the syntactic details of the framework.

ChaM3Leon is designed to prioritize clear workflows and interpretable code, with an architecture that allows the complexity of the pipelines to increase progressively as project needs grow.



 

In this perspective, ChaM3Leon aims to be an enabling platform for data science teams with heterogeneous skills, building a bridge between the simplicity of visual representation and the expressive power of Python code. The name recalls the idea of a system capable of adapting quickly to different contexts and requirements, just as a chameleon adapts to its environment.

Model-driven approach and hybrid generation

ChaM3Leon therefore adopts a hybrid model-driven approach to ML pipeline generation. The underlying idea is that the value of a workflow tool lies not in the sophistication of the generated output itself, but in its ability to produce production-ready code: syntactically correct, compliant with the target framework specifications, and enriched with established engineering practices.

During the system design, a constraint typical of real-world contexts emerged: the manual creation of ML pipelines requires in-depth expertise in specific frameworks like Metaflow, an understanding of complex architectural patterns, and the ability to manage infrastructural aspects that often fall outside a data scientist's core competencies. This technical barrier slows down experimental iteration and limits the accessibility of Machine Learning Operations best practices to small teams of specialists.

For these reasons, two operational choices were prioritized:

• Separating the conceptual representation of the workflow from its executable implementation, enabling automatic transformations that preserve the semantics of the original model;
• Combining deterministic generation (template-based) with intelligent enrichment (LLM), maintaining an evolutionary trajectory: models and pipelines can be progressively refined and replaced without compromising the platform's overall structure.

Visual workflows: from nodes to code

ChaM3Leon, as previously mentioned, has the ability to transform a visual composition of nodes and edges into executable Python code, so that the workflow logic can be defined without writing a single line of code. Execution is structured in a guided flow that leads from the design interface to the execution environment. Once the generation is complete, the code is automatically transferred from the Django backend to the ml_runner container via SSH connection, organized into directory structures that prevent conflicts and facilitate resource isolation. At the same time, the user is redirected to a Jupyter interface already configured within the same container, where they can view, analyze, and modify the code before execution. The link generated by the platform incorporates parameters that uniquely identify the workflow and the corresponding file, ensuring a fluid transition between the design environment and the execution environment.

On this basis, two requirements are integrated that make the system truly usable in an operational context. The first concerns structured serialization: the visual workflow is translated into JSON format, capturing the graph topology, the metadata of each node, Metaflow decorators, and configuration parameters, without any loss of semantic information. The second requirement is portability: the user can download the configuration JSON for backup or manual versioning, maintaining a workflow management model that is open and independent of the platform itself.

Intelligent templating with Jinja

A structural feature of ChaM3Leon is the use of the Jinja templating engine to transform the JSON representation of the workflow into executable Python code. This step is essential to ensure structural correctness: deterministic generation ensures that critical aspects (the correct concatenation of steps via self.next(), the definition of @step decorators, the structure of the FlowSpec class) are always correct, eliminating entire classes of syntactic errors.

The Jinja template defines the common structure of the Metaflow code with dynamic placeholders, which are populated starting from the metadata extracted from the JSON configuration. The choice of Jinja as the engine is based on its ability to enrich text templates with iterative loops, conditional constructs, and reusable macros, adapting dynamically to the number of steps and the complexity of the workflow defined by the user.

From an engineering perspective, this template-based generation is set up as a modular and replicable process. The template defines the "how" (the Python structure), while the workflow JSON defines the "what" (the logic and topology), implementing a declarative model driven by metadata.

Enrichment functionality: integration with Large Language Models

To make the platform useful even beyond the generation of structural skeletons, ChaM3Leon integrates a code enrichment functionality built on integration with Large Language Models (ChatGPT, Claude, Gemini). The goal is to produce production-ready implementations without requiring significant manual intervention.

The enrichment process is structured as a multi-phase pipeline with a precise logical thread.

Once the visual definition of the workflow is complete, the user provides a textual description of the desired behavior via a prompt box integrated into the interface. This description is combined with a system prompt that configures the LLM by establishing precise instructions on the expected output format and style. Based on these premises, the skeleton code generated by the templating engine is sent to the language model along with the context provided by the user, implementing a layered prompting pattern that balances structural constraints and contextual flexibility. The model temperature is calibrated to 0.3 (significantly lower than the default value) to favor a deterministic and consistent output, which is a higher priority than creativity when it comes to code generation.

The model's response is post-processed to remove any formatting artifacts, and the enriched code overwrites the initial template, making it immediately ready for deployment in the execution environment.

The value of this choice is twofold: it allows for the production of code with best practices, error handling, and structured logging without requiring advanced expertise; it keeps the process controllable: LLM enrichment is optional, with a graceful fallback mechanism that guarantees the completion of the basic generation even if the service is unavailable.

Operational execution workflow

The generated and enriched code is designed as an output that can be used directly by the user through a dedicated deployment workflow. Execution is structured in a guided flow that leads from the design interface to the execution environment. Once the generation is complete, the code is automatically transferred from the Django backend to the ml_runner container via SSH connection, organized into directory structures that prevent conflicts and facilitate resource isolation. At the same time, the user is redirected to a Jupyter interface already configured within the same container, where they can view, analyze, and modify the code before execution. The link generated by the platform incorporates parameters that uniquely identify the workflow and the corresponding file, ensuring a fluid transition between the design environment and the execution environment. The platform orchestrates the entire process and provides a structured environment, with the code ready to be launched via the Metaflow command, which allows for automatic tracking of parameters, metrics, and artifacts during execution, making it easier to compare runs and identify optimal configurations.

Results and validation

The analysis of the results obtained from the experimentation confirms the effectiveness of the hybrid model-driven approach adopted. The evaluation was conducted across multiple dimensions.

All generated workflows correctly include the mandatory start and end steps, implement valid transition chains, and use decorators in compliance with Metaflow specifications. Semantic validation verified the absence of dead code, unreachable steps, or cycles in the execution graph.

Qualitative analysis of the LLM-enriched code highlighted the incorporation of structured logging, input validation, edge-case handling, and optimizations that would typically be absent in purely mechanically generated code. The LLM integration adds tangible value beyond structural generation, enriching implementations with qualitative patterns derived from pre-training on vast corpora of open-source code.

The system showed competitive generation times and good scalability as workflow complexity increased, with the containerized architecture facilitating distributed deployment and isolation between multiple executions. Native integration with Metaflow also enables the distribution of processing across cloud infrastructures.

Conclusion

The value of ChaM3Leon does not lie in a single feature, but in the continuity of the flow within a single governed platform. The choice to combine template-based generation and LLM enrichment is consistent with the real-world constraints of ML development, where the structural correctness of the code is essential, but the incorporation of contextual best practices represents a competitive differentiator. It is a solid starting point that allows for the generation of functional pipelines without introducing fragility.

The most interesting perspective is its evolution: with the extension of support for conditional and iterative structures (consistent with Metaflow 2.18), the automation of remote execution via SSH, and integration with versioning and collaboration systems, ChaM3Leon can transform from a code generation tool into an enterprise-grade MLOps platform. Smart Shaped has already confirmed its intention to continue development in this direction. In this sense, it is a platform designed to grow incrementally, not an isolated prototype.