The Arvados team is pleased to announce Arvados 2.5.0. This is a major upgrade, with many new features, changes and bugfixes. We recommend that new and existing installations of 2.4.4 or earlier upgrade to 2.5.0. See Upgrading Arvados for upgrade instructions. For a complete list of changes, visit the release notes.
New Features
Workbench 2
Workbench 2 features a new color scheme, designed for improved contrast and consistency. #19462
The process pages features new Input and Output panels displaying the input and output parameters of the process. #16073 #19700 #19848
The process page also features a new Resources panel showing what resources were requested by the container, and (on cloud installs) what instance type it was actually run on. #19438
The picking dialog now includes the ability to search for projects and to filter the list of collections within a project. #19783
The process panel includes new information about how much a workflow cost to run. #19319
When you view a workflow step or output collection in Workbench 2, there’s now a row of breadcrumbs at the top of the page that shows you the parent workflow and project. This helps users understand how the object was created, and navigate to those parent objects if desired. #19504
The project view now offers the ability to configure a variety of additional columns for display: UUID, Created at/Trash at/Delete at, Properties, Portable Data Hash, Version, File Count, Requesting Container UUID, Container UUID, Output UUID, Log UUID, and Modified By User UUID. #19690
Workbench 2’s left navigation pane is now collapsible. #19434
Workbench 2 fully supports frozen projects: projects can be frozen and unfrozen, and frozen projects can’t be edited in any way. #18692
When you maximize a Workbench 2 panel, there is now an un-maximize button to easily restore the previous panels. #19300
Installer
The Arvados installer includes a new script to manage the process of setting up a fully-featured multi-node cluster. The multi host install guide guide has been rewritten, updated, and expanded. In addition, we now provide a Terraform script to assist in deploying on AWS. #19175 #19215
Client tools
arv-mount now features a memory-mapped disk-backed cache instead of a pure RAM cache. This enables the cache to be much bigger by default and leverage the kernel’s filesystem cache without competing with user programs for RAM. The result is same or better performance reading files without needing to tune cache parameters. This is available both running arv-mount locally and running containers on Arvados. The size of the cache is controlled with --file-cache like before, and the location is controlled with --disk-cache-dir. #18842 #19872
arvados-client shell can now connect to containers running under LSF and SLURM, in addition to the cloud dispatcher. #19166
Running Workflows
Container records now have cost and subrequest_cost fields with information about the container’s cost to run. See the containers reference for details. Container request records now have a cumulative_cost field with information about the request’s total cost to run. See the container requests reference for details. This is calculated using the Price field of the instance type definition in the configuration file. #18205
Workflow steps submitted by arvados-cwl-runner now have properties cwl_input and cwl_output on the container request. See the API properties documentation for details. #19466
SDK
The Python SDK documentation now includes a full overview of how the API client works, with lots of realistic examples for demonstration. #19791
The Python SDK features a disk-backed cache for Keep. This is used by arv-mount as discussed above, but also available to other users of the Python SDK. #18842 #19872
Administrator tools
Arvados includes a new tool sync-users-tool to synchronize Arvados user records described by a CSV file. You can use this to keep Arvados user records up-to-date with other identity sources like Active Directory or SSO providers. The admin guide on synchronizing has more details. #18858
arvados-client diagnostics is now documented in the admin guide. When possible, the diagnostics also performs a cluster “health check” (arvados-server check described below) and includes any problems in the diagnostics report. #19364 #19377
arvados-server check is now documented in the admin guide. The arvados-server check tool now detects and reports several likely error conditions on a cluster: when different services in the cluster are not using the same configuration; different services in the cluster are running from different versions of Arvados; when clock times reported by different services in the cluster are a minute apart or more. #18794
Arvados API
The API server respects a new configuration option Users.CanCreateRoleGroups. If you set this false, non-admin users will not be allowed to create role groups. This is helpful to set when your cluster uses arvados-group-sync as the single source of roles. #19513
The API server now allows sharing objects to be “writable” by the “All Users” group. This entailed some changes to the permissions model for roles: in general, updating a role now requires can_manage permission. Refer to the API permissions model documentation for details. #19269
Group objects returned by the API server now include boolean can_manage and can_write fields indicating the current user’s level of access. The writeable_by field is now deprecated in favor of these new fields. #19146
Keep
If an unauthenticated user visits keep-web they will now be redirected to Workbench 2 to log in first. Once the user is logged in, they will be redirected back to keep-web. #17807
Keepstore now uses a new driver for S3 driver by default. This driver features improved S3 read performance. The old v1 driver will be removed in Arvados 2.6.0. #19582
The keep-web S3 API now publishes project and collection properties through the X-Amz-Meta- headers. Control characters in values are MIME encoded. Refer to the new section in our S3 API documentation for details. #19088 #19249
Thanks,
The Arvados Team