Updates 2021

Patch Note 2021-12-22

🎶 It's beginning to look a lot like Christmas...

New pipeline features

Pipelines can now be started using "YAML-style" input data.
If using YAML-style data, a new top-level commit value is required for the Create Pipeline endpoint.
Pipelines can now be created directly from a template in a given commit using the Create Pipeline from Template endpoint.
On a similar note, there's a new Run Pipeline trigger action that creates a pipeline from a template in a given commit.

Other enhancements

Like Run Pipeline, there's now also a Run Execution trigger action.
You can now use * and ? wildcards in event log filters.
Object modification times are now available in the API as mtime fields.
YAML files can now be directly previewed.

Bugfixes

Tag filtering in the datum browsers now works once more.
Over-long pipeline titles are correctly validated.
Workers will now abort starting an execution in a stopping/stopped pipeline or task.
A bunch of small papercut-style usability bugs got fixed.

Patch Note 2021-12-15

New features

When enabled for a store, URIs can now be previewed alongside datums.

Performance fixes

For large projects, loading the execution/task/pipeline views is now much quicker.

Bugfixes

Some notification badges were off-color. They've been now taught to be prim and proper.
Scrolling the API docs view's sidebar now works properly on all browsers thanks to our upstream bugfix Redocly/redoc#1823.

Patch Note 2021-12-07

Oh how time flies! It's been nearly a month since the last update.

It's freeze-your-face-off cold in Finland, and electricity costs are nearly record-high, so no better time to hunker down and train your ML models while optimizing resource usage!

Improvements

Default stores for code and logs can now be set on organization level.
Execution events can now be downloaded as CSV at any point during an execution.
You can now opt to allow users to directly download data (e.g. inputs by URI) from a store.
New keys were added to the execution configuration files:
valohai.environment-name
valohai.environment-slug
valohai.execution-image
valohai.execution-tags
valohai.pipeline-tags
valohai.pipeline-title

User Interface Changes

We upgraded from Bootstrap 4 to Bootstrap 5, so things are now decidedly MORE MODERN.
However, things may be broken every here and there, since it was a bit of a change.
The top navigation bar is now full-width. Let your ultrawide screen fly free!
The dashboard view is all-new and decidedly GitHub-inspired.
As always, let us know if there's something you'd like to see there that could be useful to you.
Organization settings got a couple touchups.
The Team Access Grant editor is no longer a weird sidebar, but a modal.
The Team Access Grants view is now more easily found.
Execution tables text-wrap less, for a cleaner look.
Pipeline views also now have tag colors.
Paginators now show first and last page buttons, when available.
Input URLs are now copiable.
Chart views got a tiny bit of polish.

Bugfixes

Parameter-parameter edges from a mimic node now work.
Pipelines can now actually be sorted by status.
Deleted stores (that are invisible) no longer affect signing operations. It'd be silly if they did.
Counter filters in the execution table now work more reliably.
Loading commits for the Create Execution view should now be much faster.
We got rid of a race condition or few, under the hood.
Many spurious heartbeat misses should now be no more.
Confusion matrices are now the right way around, vertically.
Cyclic pipelines no longer return an internal error, but an actual error describing what just happened.
Pipeline notifications work much more reliably.

Patch Note 2021-11-08

Improvements

Searching for datums should be a lot faster this beautiful Monday.

Bugfixes

Deleted executions and data stores could show up as scary, spooky zombie objects. While I'd love to say it was just a Halloween thing, it was, in all honesty, a bug. These zombies have been returned to eternal peace now.
Commits stored in external stores can now be downloaded.

Patch Note 2021-11-04

The leaves are falling, it's cold outside, and it gets dark at 4 PM in Finland. Send help.

New features

Deployments

Deployments are now more easily debuggable thanks to the all new Cluster Status view.
You can now also set the hard memory limit for a deployment.
Deployment details are now actually shown in pipeline deployment nodes.

Other

Organizations may now disallow using Valohai-managed data stores for data management.
Commits and log data can now be stored in user-defined stores.
Being able to set these throughout an organization is planned for the next release.
In pipelines, list-shaped metadata from executions is now automatically "spread" into a multiple variant parameter if the destination node is a task. In plainer English, this means you can use an execution node to parametrize a downstream task node's variations.
You can now invite people to an organization as administrators.
Slack incoming webhooks are now natively supported as notification channels. #nocode
Status detail messages are now exposed for e.g. executions deemed dead by having missed heartbeat messages.

Improvements

The deployment statistics histogram now also uses Plotly under the hood.

Bugfixes

Underscores are now officially disallowed in deployment endpoint names. They were allowed before, but wouldn't work correctly.
Dummy ingress objects are no longer created for disabled deployment versions.
A corner-case bug regarding administering environments got squashed.
URLs that begin with numbers are now allowed for webhook URLs.
The user interface for Team Access Grants now works a whole lot better than before.
The auto-scrolling logic for the execution events view is now more robust, at the expense of it not being quite as automatic.

Patch Note 2021-10-13

SSH Remote Debugging

Where enabled in private environments, you can now enable Remote Debug when starting an execution. This will enable an SSH server in the execution container you can then connect to.
You can also use SSH port forwarding to use debuggers such as pydevd (PyCharm) or debugpy (Visual Studio Code).

See the docs for more details!

Pipeline improvements

You can now configure notifications for pipeline events.
You can now set pipeline nodes to require manual approval before they are run.
Simply add a Node Action to the node: ```yaml actions:
- when: node-starting then: require-approval ```

Git fetch improvements

You can use a .vhignore file in your repository's root directory to exclude files from being ingested by the platform. The syntax is the same as for .gitignore files, powered by gitignorant. This is useful if you have e.g. test data in your repo that you don't need at all in Valohai, but want to keep around.
Git fetches that update multiple refs that share objects between one another should now be faster.

Charting improvements

You can now chart confusion matrices from arrays output as metadata.
The supported formats are:
- An array of [['x', 'y', v], ['x', 'y', v], ...] triples (i.e. x label, y label, value)
- A 2-dimensional array of the shape [[v, v, v], [v, v, v], ...]
- A 2-dimensional array with a header for labels [['a', 'b', 'c'], [v, v, v], [v, v, v], ...]
As always, feedback on the feature is more than welcome!
You can now enable rolling-average smoothing in the metadata charts. Smoooooooth.
The Scatter chart is now backed by Plotly too, for consistency.

Task improvements

The grid view that shows task executions now has icons to make it easier to read for our users with color blindness.
You can now set the maximum number of queued executions per task. This effectively limits the maximum parallelness of the task, so your environment can be used to run other tasks too.

Deployment improvements

Disabled endpoints are no longer visible in the deployment tester.
Versions in disabled deployments are similarly no longer visible in the table view.
Endpoint details are now available in their own view.
Deployment logs are now properly retrieved if there are multiple replicas.

Other improvements

You can now choose to purge all outputs for an execution when marking it as deleted.
This also works for the mass deletion action.

Bugfixes

When already logged in with a 2FA-enabled account, token login no longer fails with an unnecessary error.
The organization environments tab no longer occasionally just ups and fails to render.
Endpoint naming has been emstrictened for better compliance with Kubernetes standards.
Datum aliases are now resolved in the UI, so you can use them as defaults without bumping into issues.
Certain unnecessary remote queries were optimized away from the UI.

Patch Note 2021-09-28

Performance fixes

Creating large tasks is now faster.
A slow component was made quicker.

Small convenience fixes

The data upload UI will let you know if you're trying to upload something big enough to likely fail.
The memory and CPU limits for deployment endpoints are now visible in the frontend.
Pipeline tags are shown in the UI now.

Bugfixes

The memory limit for deployment endpoints is now defaulted to the memory request, unless overridden.
Fetches with errors are shown properly once again and don't just result in "no changes".
Certain API calls could fail to be rendered in the interactive API explorer. No more.
The "Use local time" option for log events works again.

Patch Note 2021-09-13

New features

Refs are now updated separately when fetching commits. This means that repos where multiple refs point to the same commit work better.
The charting component in the execution/task metadata views was replaced with Plotly.js. It's now faster and stabler.
You can now set the number of replicas to enable for deployment endpoints.
A bunch of small and medium accessibility issues were fixed!

Bugfixes

All commits are now loaded when creating deployments.
When fetching commits, the Commit box in the Create Pipeline view is now properly updated.
The Execution List API is now stabler when there are queue connection issues.

Patch Note 2021-08-03

Oh how time flies, it's August already!

New features

The Intercom support chat has been replaced with Happy Fox. 🦊
Bug Reports are now also powered by that same joyful vulpine.
The metadata view got a little cleanup here and there. 🧹 💅
If you know the datum:// URL of a datum, you can now use it cross-project, if the other project has access to the datum's store.
You can now pipe metadata from executions into parameters (in execution and task nodes).

Bugfixes

You can now download the metadata CSV for an arbitrary number of executions.
Under the hood, a lot of frontend components and tables got a little snappier.
The Notebook preview button now works even if you're already on the output tab of an execution.
Input validation in tasks got a lot more user-friendly.
There's no longer a crash when attempting to copy a project with no associated repository.

Patch Note 2021-07-13

New features

Pipeline node actions are now available. For the first iteration, you can configure an execution node to stop the entire pipeline depending on e.g. metadata values.
Environment slugs are now visible in the organization environment dashboard.
Datum aliases can now be deleted.
Many frontend requests were made faster. (For API users: if you don't need the total count of objects in list responses, you, too, can pass ?no_count=true.)
Parameters can now be passed between Task nodes in pipelines.

Bugfixes

The Deployment response viewer became more stable.
Datum aliasing UI operations were fixered-uppered.
The metadata view deals with timezones better than ever before.
Environment selection in the UI based on a step default was made more robust.
Pipeline nodes got some new paint.
The deployment status view now correctly shows unknown statuses and doesn't just, you know, die.
The task/comparison metadata view now correctly counts the executions to be loaded.

Patch Note 2021-06-16

New features

The deployment status UI got some polish to it, making errors easier to see.
The deployment resource configuration UI also got a bit of a new paint coat.
The Execution table keeps getting better:
It got a sticky first column!
The ad-hoc execution icon slid down to a tag badge!
Metadata and parameter columns now have neat groups!
Commit identifiers are now available as a column.
The Metadata chart view's sliders are now entry boxes, for pin-point precise zooming.

Bugfixes

Disabled deployments are no longer shown in the deployment dashboard.
A race condition relating to user preferences for column choices got fixed.
Resolving a lot of s3://... style inputs for a single execution should now be a lot faster. #cachegang #cachegang #cachegang
Numeric strings are now also valid for statistics computation in the metadata views.
Navigating back to the Info tab in the Executions view with the Back button works again.
Certain race conditions involving multiple concurrent repository fetches are no more.
Stores where file adoption can't work are no longer shown in the Adopt tab.
The _time metadata value was renamed a more human-friendly "Timestamp".
Mimic nodes no longer require superhuman vision to see.
On a similar note, the node status icons now work on Firefox, and node borders are colored accordingly.
Pagination worked in a non-deterministic way when lists were sorted by decisively non-unique fields. We poured in a pint of determinism, and it's much better now.
Certain data-related queries were optimized. #poorlittledatabaseserver
The Data Browser view no longer makes queries at a breakneck speed when in a background tab. How's that for a bug...
The Datum Alias creation/editing form got a lot user-friendlier. You no longer need to edit the name of the alias only after selecting the datum...
A bug in the Deployment Tester view was fixed. #typescriptgang

Patch Note 2021-06-07

It's been a while, hasn't it? Nice to see y'all!

New features

Datum Aliases!
You can now configure per-project aliases for given data. You can change the datum the alias points at at will.
Datum aliases are resolved during execution/task creation, which means copying executions will not change the data if the alias had been changed meanwhile.
The change history of datum aliases is tracked.
You can set the value valohai.alias in an output metadata sidecar file to have the platform configure an alias from your output data automatically.
Find the new Aliases tab in your Project Data view.
Datum adoption!
You can now adopt storage URLs (S3 only for the time being – let us know if you need this for other stores) into real Valohai-tracked datums, with all the goodness that implies.
Find the new Adopt tab in your Project Data view.
New Execution View experience!
The iconography for execution statuses got a little facelift courtesy of our facelift department.
The creator and status of an execution is now in a single cell, leaving you with more space for everything actually interesting!
When configuring a deployment endpoint, you can now set a CPU Request value too.
The previously rather arcane "Node Selector" field was elucidated with a help text.
You can now see the effective market price of spot instance environments in the environment management UI.
You can now report bugs directly from the application. Look for "Report a Bug" in the user menu.
The task execution comparison view now allows comparing more than 100 executions. Executions and their metadata is loaded in chunks, making the initial load a little faster.
Very long event log messages now get a nifty "Read More" button!

Bugfixes

Wide View options are now saved for more tables.
We found our crayons and charts now have correct legend coloring when comparing multiple executions
The frontend was made lighter by way of optimizing icon delivery with a synergistic AI solution (or rather, by not including all of Font Awesome).
The Datum browser auto-refreshes once more.
The "you're not using parameters" warning is no longer shown if you use {parameter} or {parameter-value}.
When creating organizations, the messages no longer speak of "usernames". Just "names".
Very rapid logging could previously make the UI wonky. The wonkiness is now decisively reduced.
Metadata statistics computation now works for stringly typed numbers too.
Searching for data was made less intensive bandwidth-and-CPU-wise.

Deprecations

The Hosted Notebook feature is no longer available by default. Let us know if you need the feature.

Patch Note 2021-05-12

New features

Data management improvements

The data browser gained all-new filtering controls. Purged datums are now hidden by default. You can view them via the filter menu.
Datum tags became visible in the input datum select dropdown.
Any metadata associated with datums is now returned to the executor machines too.
Organization members can now get an overview of data used across projects in the Organization view's Data Catalog.

Other things

Organization admins can now tweak environment prices used for price estimation.
Organization admins can now view scaling activity for certain environment types.
You can now pin executions to the top of a project's execution list.
The project settings view now uses the more similar controls as execution creation.
Executions stopped on spot instances by spot instance preemption will now cause the execution's status to correctly become "Stopped".
Enterprise customers can now use git-lfs.
You can sort projects by last_execution_ctime in the API (like in the dashboard).

Bugfixes

Tasks finishing now correctly step pipelines they're members of.
The metadata chart no longer rounds the min/max values for axes.
Deployment logs are again available for all deployment targets (and the health option is not available if it's not, well, available)
The datum browser is now refreshed after tags are edited.
The notification setting list is now refreshed after edits. And routings can now be deleted.
A bug preventing copying projects with a default environment set was fixed.
The execution environment variable editor became more usable.
Pipelines where all nodes were layouted in a single line now layout their edges in a less confusing, non-overlapping manner.

Patch Note 2021-04-21

New features

So many new things you can do and see!

You can now set up webhook notifications for the same events you can get any notifications via email or in the app for.
You can select a custom date range in the deployment health and logs view.
You can now see various statistics of your execution metadata in the metadata chart view.
You can also download all of the metadata in CSV format.
You can filter the execution/task environment box to only show Spot instances (if you have any spot instances set up).
You can now send arbitrary JSON metadata to be stored with your executions. The documentation for this is forthcoming.
When output metadata is sent from your execution, we parse the valohai.tags key from it for, you guessed it, datum tags.
You can now see which deployments a datum is used in in the datum table.

Bugfixes

Some sneaky overlays that remained visible when they should not have have now been shooed away.
The editor for Role ARNs actually works these days.

Patch Note 2021-04-14

New features

You can now interpolate parameter values into input URIs with the syntax {parameter:name}. This is of particular use in tasks, where you can now easily run your execution on multiple variable datasets.
Tasks gained a Detail tab – if you'd been wondering about the parameter configuration for a task, wonder no more. It'll be there, waiting for you.
You can now wire output edges from task nodes too. Outputs from all executions in the task are matched.
On a related note, inputs can also be wired into deployment files these days.
Datums can now be tagged (manually, in the Datum list views).
Being able to tag data as it is output from an execution is a forthcoming feature, don't worry!

Embetterments

Purged datums are now more clearly shown in datum lists.
Axes in metadata line charts are no longer quite as limited as they were.
Large tables render faster thanks to not rendering quite as many invisible things.

Bugfixes

Creating pipelines with task nodes directly from YAML now works.
The "no deployments" text in the deployment dashboard no longer disappears when there are in fact no deployments.
Task nodes no longer require a manual press of the Step button. That's nice.
Even more buttons are now properly disabled when a project is read-only.
Expanding the Input panel for a copied Execution with lots of inputs no longer causes a metric ton of concurrent requests that make your Internet connection and our poor servers cry in agony.
Marketing droids have been politely shooed off the signup page.
Creating a project from a certain template no longer causes an embarrassing self-XSS.
Action dropdown menus in very short tables are no longer cut off, making them actually usable.

Patch Note 2021-03-24

New features

The deployment dashboard now features sparklines describing request rate and success percentages per endpoint.
You can now create task nodes in pipelines.
Initial support for spot instances was added. Refer to your friendly Valohai customer success manager for the low-down.
Initial support for directly mounting persistent data stores supporting NFS (such as Amazon EFS, Google Cloud Filestore, and Azure Files) was added for private worker environments.
Support for AWS STS role delegation was added for S3 stores and EC2 execution environments.

Bugfixes

Read-only projects were further locked down.
Store copying now reliably works.
Linear variant parameters in random search tasks are now correctly quantized when the quantization step >= 1.
The "Latest" metadata view no longer crashes on complex values.
The execution outputs endpoints was given a speed boost. It doesn't ride on walls, though.

Patch Note 2021-03-04

This is going to be one hecking long patch note...

New features

Pipelines can now (finally!) reuse nodes from other pipelines!
A brand new "Reuse Nodes..." button has appeared for this purpose in the Create Pipeline view.
For API users, replace a node definition with {"type": "mimic", "source": "other-node-id"}.
Reused nodes are shown in italic.
It doesn't make much sense to reuse an errored node, but you can do it nevertheless, because who are we to limit your creative expression?
Stop expressions are now applicable to single executions as well as for tasks!
The feature is available starting with agent version 0.36.0; agents are upgraded when execution nodes start.
Step names and queue datetimes are now available as execution configuration data.
Deployment endpoint health information is now available (when the deployment target has been configured accordingly).

User interface embetterments

Notification settings were moved from the dashboard into the project settings view.
The misleading dropdown for metadata chart types is no longer a misleading dropdown when there's only one valid option.
Big long very huge input lists in the Copy Execution view are now hidden by default.
Most messages created by the frontend no longer cause a persistent notification bell. Finally.
You can now sort executions by step name.

Bugfixes

The Organization Restrict Environments dropdown was fixed to actually work. Oops!
Many list API endpoints were made faster. What's this "N+1 queries" thing SQL folks talk about?
The organization deployment dashboard had some sharp corners sanded off:
The endpoint dashboard view now properly shows deployment target names. We heard complaints that Real Human Beings actually can't recall UUIDs that well. How peculiar...
Disabled and deleted deployments no longer show up in the endpoint dashboard.
Endpoint URLs are now properly formatted.
Project stores and notebook instances now also have their ownership transferred when projects are transferred.
Histograms didn't always show up in the deployment monitor. They do now.
Duplicate node names in pipelines now yield a real error message instead of an internal error.
Organization default environments now trickle correctly down to projects, no longer just at creation time.

Patch Note 2021-02-03

New features

You can now set up early stop conditions for tasks based on a metadata field. For example, set the condition to loss < 0.02 and you'll save precious cycles once your model's accurate enough!
You can now view deployment metrics in your local time, not only UTC.
Projects can be set as read-only. Find this in the Non-Danger Zone of your project's Administration view.
You can now purge all outputs for an execution in the execution's Outputs view.
You can also purge all outputs for multiple executions in the execution list view!

Improvements for organizations

You can now see an overview of your organization's deployments in the aptly placed and named Organization > Deployments view.
Organization administrators can now limit available environments to an exact subset of available environments.
Administrators can also now set per-team per-environment queue quotas, to avoid a hoggy team from hogging all of the resources. The quota can also be set to zero to disable an environment for a team.

Bugfixes

Pipelines created by triggers should now start on their own, having been given a degree of autonomy.
The Organization view's environment list now shows environments with the "forbid personal usage" flag.
The environment variable editor gained a helpful error message.
Executions in pipelines are now created in a more granular manner, which should help out with very big, or very multi-inputy pipelines.
Input information can now be fetched only when an execution really starts. If you have long queues and only a few machines in an environment, this'll help out!

Patch Note 2021-01-18

New features

You can now copy projects, including all of their dependent objects such as repository settings, stores, environment variables, etc. Please do note that the new copies will be entirely independent from the original projects and settings; if you'd like to use shared copies of e.g. data, promote the store to organization level first, then simply use it in multiple projects.
The deployment framework now creates a Kubernetes ingress per each endpoint, to allow 100% correct handling of the URL prefix variables.
Parameter-parameter edges are now supported in pipelines.

Bugfixes

We upgraded to a new version of our Stripe integration, which should be smarter regarding postal codes for international credit cards.
Execution titles in the comparison view are now more visible.
The deployment metadata view should blank out a little less.

Patch Note 2021-01-12

Happy new year! May it be less 2020 than last year was!

New features

Hugely embettered deployment monitoring and logging!
On deployment targets where a logging Elasticsearch instance has been installed, you can now view logs from the past too, not just from pods that are currently alive! (We're working on viewing logs from the future, but it'll take some time and a sonic screwdriver.)
Not only that, you can also log metadata lines almost like you would with regular executions and chart them in both time series and histogram modes!
User interface improvements!
Un-filtering tables is now a snap (well, a click, really). When a filter is enabled, a small button to disable them appears!
Execution, task, and pipeline titles are now shown within page titles where applicable.
Attempting to use a Docker image that requires a GPU (i.e. has gpu- in its name) in an environment that doesn't have a GPU now results in a warning.

Bugfixes

The frontend was made a little more frugal regarding certain network requests. This should translate to you, dear user, as a snappier experience.

PreviousUpdates 2022 NextUpdates 2020

Last updated 13 days ago

Was this helpful?