Until I can find a solution to the performance and stability issues related to CPU and memory limits on workflow processes in our Kubernetes cluster, expect intermittent service disruptions as I iterate on the deployment configuration. These disruptions may include the API server going offline and jobs being terminated before completion.
If a solution is not discovered within a reasonable amount of time, we may need to create a separate test cluster for the purpose.
As of this post, we are running v1.1.0 at https://ce.musesframework.io, with 7 workers across 7 cluster nodes, each capable of 10 concurrent workflow processes. I have not solved the CPU & memory constraint problem, so there is a chance that heavy use will take nodes down, but in that case, hopefully there will be enough healthy nodes to host the other web services like this forum without interruption. I will continue trying to devising a robust solution.
@mrpelicer Please see the shiny new CHANGELOG.md for a description of the changes I made to support your development work. This allowed me to uninstall the ce-dev instance, bypassing the vexing issue of how to balance the cluster resources between the public and private deployments. What I forgot to include in that changelog explanation is how you include a dev module in your workflow: you add the optional dev boolean scalar to your process spec as demonstrated below, where the default value is false when omitted.
processes:
- name: my_dev_process
module: dev_module
dev: true
config: {...}