announcements

Polyaxon v1.4

Today, we are pleased to announce the v1.4 release, a stable version which brings several new features, enhancements, and fixes. This release does not introduce any breaking changes and is fully compatible with previous releases.

Polyaxon CLI

Polyaxon upload is GA

polyaxon ops upload and polyaxon run -u commands are now available in all Polyaxon distributions. This is one of the features that v0 users have requested after migrating or trying the new Polyaxon v1 series.

The new upload command not only brings all features required for faster experimentation and iteration with Polyaxon, but it is also more optimized than its old version, the new upload does not trigger a rebuild process, instead, it injects the uploaded code or artifacts directly into the user’s main container, which saves a lot of time.

In v0, users would use the upload command to upload code and sometimes data as well, in v1 we made this process easier, you can decide where your uploads need to be located, instead of polyaxon run -u you would use polyaxon run -u-to PATH_TO_USE.

The new upload command has several additional qualities, it is not dependent anymore on the start process, users can use the command to upload artifacts even after an operation is done. For instance, users might need to attach extra artifacts to an experiment, or might want to upload some updated version of their code to resume the same experiment. Users can also upload from and to a different location on the root path of their operations.

Finally, we made the upload logic directly on PolyaxonClient instead of the CLI, which should allow users to upload information programmatically via upload_artifact and upload_artifacts on the run client. In v0, users were copying the logic from the CLI to use it in their code for programmatic upload in Python.

Polyaxon CLI configurations

Although Polyaxon v1 has reduced the number of misconfigurations significantly by introducing sensible defaults for Polyaxon CE and by reducing the complexity of the initial setup, first-time users are still having, from time to time, issues configuring the CLI to talk to Polyaxon Gateway. In v1.3.2 we turned off the compatibility check by default to stop showing the message inciting users to install new versions and made that process optional if users decide to check for new versions via:

> polyaxon version --check

Current cli version: 1.4.0.

Platform:

-------  -----
KEY
VERSION  1.4.0
DIST     ce
-------  -----

Compatibility versions:

--------  -----------------------------------
cli       {'min': '1.2.0', 'latest': '1.4.1'}
platform  {'min': '1.2.0', 'latest': '1.4.0'}
agent     {'min': '1.2.0', 'latest': '1.4.0'}
ui        {'min': '1.2.0', 'latest': '1.4.0'}
--------  -----------------------------------
New version of CLI (1.4.1) is now available. To upgrade run:
    pip install -U polyaxon

We also deprecated polyaxon config -l in favor of a more global sub-command polyaxon config show

> polyaxon config show


Client config:

---------------------------------  -------------------
POLYAXON_HOST                      POLYAXON_HOST
POLYAXON_API_VERSION               v1
POLYAXON_DEBUG                     False
POLYAXON_AUTHENTICATION_TYPE       Token
POLYAXON_IS_MANAGED                False
POLYAXON_IS_OFFLINE                False
POLYAXON_IS_OPS                    False
POLYAXON_K8S_IN_CLUSTER            False
POLYAXON_NO_OP                     False
POLYAXON_TIMEOUT                   20.0
POLYAXON_TRACKING_TIMEOUT          1.0
POLYAXON_TIME_ZONE
POLYAXON_WATCH_INTERVAL            5
POLYAXON_INTERVAL                  5.0
POLYAXON_VERIFY_SSL                True
POLYAXON_ARCHIVE_ROOT              /tmp/plx/archives
POLYAXON_NO_API                    False
POLYAXON_DISABLE_ERRORS_REPORTING  False
---------------------------------  -------------------

CLI config:

Version 1.4.1
-------  -----
KEY
VERSION  1.4.0
DIST     ee
-------  -----

User config:

------------
USERNAME xxx
EMAIL  xxx
NAME  xxx
ORGANIZATION xxx
------------

Changing clusters will automatically reconfigure the CLI to point to the right host and invalidate any stale configuration about auth or current user.

Current owner and project

What organization should be used by default? This was confusing for both CE and Cloud users. For instance, starting a new operation or getting information about a project or an operation, users would try -p PROJECT_NAME and then -p OWNER/PROJECT_NAME. In v1.4, the CLI/Client will automatically use the configured organization for the user, in CE that would be default, in Cloud and EE that would the one configured on the profile, if a user has access to a single organization, that organization will be the default owner.

CLI for Component Hub and Model registry

We added all required commands to the CLI for managing your components and models via:

polyaxon hub --help

and

polyaxon registry --help

Deployment and setup

Polyaxon distributes its Docker containers via Dockerhub, https://hub.docker.com/u/polyaxon/polyaxon. It’s been interesting to see the number of downloads grow since the first release with less than a thousand downloads. Now, most of Polyaxon’s docker images have millions of pulls each.

With the new throttling limitations set by Docker, several users started noticing issues with the sidecar and the init containers.

Although our Helm charts expose configuration to set the pull policy and the image pull secrets for the docker containers, in this new version we made IfNotPresent the default pull policy for all Polyaxon’s related containers, this will make starting jobs faster and should reduce the incidence of the ImagePullError.

UI

UI for Component Hub and Model registry

We opened access to the Component Hub and Model Registry to all organizations following their quota (you can check the feature usage under billing for more information about your quotas, usage, and enabled features).

Component Hub

feature1

The new component hub will allow users to create a knowledge base about all reusable components and templates in their organization. Polyaxon already provides a set of generic and reusable components. However, several users are already customizing those components. At the moment, they copy the manifests and use pathRef: ... to reference their custom versions. This approach works for the CLI, but requires that each user should have access to those manifests, and they should be referenced via a relative path.

The component hub will allow to abstract that behavior by registring those components on the organization’s level, and will allow them to reuse those manifests in all projects by using hubRef: ORG_NAME/COMPONENT_NAME:VERSION.

feature2

They can also run the manifest directly via the UI, Polyaxon will know how to resolve the references and compile the resulting manifest.

Finally, each component can restrict members and teams who can edit and add versions to each component.

Model registry

feature4

Polyaxon’s model registry is now in public beta and accessible to all organizations.

We enabled some of the features that are getting stable such as experiment promotion, locking, and versioning in the registry.

feature4

You can promote an experiment from a Polyaxon project to the model registry, which will trigger the following behavior:

  • Lock that experiment.
  • Collect any relevant lineage data and expose it on the model version.
  • Allow referencing the model version by a semantic name via owner/model:version.

Additionally, users can also signal the stage and also augment a model version with key:value metadata that they can attach to each version.

Finally, organizations with the advanced restriction feature can set restrictions on members’ and teams’ access to each model registry and can list projects that users are allowed to promote experiments from.

The more complex features such as feedback metrics and monitoring will take a bit more time until we finalize the events interface.

Note that the model registry does not provide inferencing or serving technologies and is agnostic to the platform or framework you would use to deploy the models.

Several UI improvements and fixes

In a series of feedback sessions with Cloud users, we gathered feedback about breaking UI elements on some browsers, in this new version we fixed almost all issues reported, the UI is faster, cleaner, smoother, more consistent, and has more utilities for searching, filtering, and sorting all entities.

If you are running an old version of Polyaxon v1, this alone should make you think about upgrading.

Upcoming features

Debugging

One of the major themes of the next release will be about debugging capabilities.

At the moment, polyaxon provides several features for looking and debugging operations:

  • Logs: Users can check the logs of each container running in their operations, including Polyaxon auxiliaries
polyaxon ops logs [--all]

feature5

  • Statuses: Users can have an aggregated view about the state of their operation, with details about each state when they click on a status
polyaxon ops statuses [--watch]

feature6

In some situations having access to the k8s cluster is necessary to debug further. Some companies do not want to distribute k8s config to their users and want to manage access with Polyaxon alone.

In the future, Polyaxon will be providing more features to help with these use cases:

  • Statuses: CLI and UI will have a new feature inspect, this feature will allow users to inspect an operation with warnings or failing status and have a global view about the containers and their states and what failing.
  • Shell: CLI will have a new command polyaxon ops shell which will allow running commands or scripts directly on the users’ main container, together with termination.ttl, this should allow users to investigate their containers without having access to k8s. polyaxon ops shell --attach should be a further iteration after the first version is completed.

Offline mode

The offline mode is something we were trying to make for a very long time, it should essentially allow users to debug their code on a local machine while also executing all Polyaxon related code without triggering any network API calls. The offline mode should allow users to understand how Polyaxon provides tracking and logging features, and how to optimize their code to use this interface. Providing an offline mode should also allow users to build extensions and features around the tracking and lineage APIs more efficiently.

Previous items

We also still have items from the previous monthly update:

  • Control plane connections management.
  • More lineage information.
  • Adding the schedule_at information to the compiler’s context.
  • Add graph visualization for DAG and Matrix runs.

Please see roadmap features announced in v1.3 for more details about these items.

Learn More about Polyaxon

This blog post just goes over a couple of features that we shipped since our last product update, there are several other features and fixes that are worth checking. To learn more about all the features, fixes, and enhancements, please visit the release notes.

Polyaxon continues to grow quickly and keeps improving and providing the simplest machine learning layer on Kubernetes. We hope that these updates will improve your workflows and increase your productivity, and again, thank you for your continued feedback and support.

Subscribe to Polyaxon

Get the latest posts delivered right to your inbox