23.10 Release notes

Instabase 23.10 is a major release that introduces new features, enhancements, and bug fixes.

Subsequent patch releases typically contain bug fixes along with testing, optimizations, and other minor internal changes.

Release 23.10.54

November 20, 2024

Release 23.10.53

November 14, 2024

  • Resolved [CVE-2024-5452] | Upgraded pytorch-lightning package to version 2.4.

Release 23.10.52

November 6, 2024

Release 23.10.51

October 31, 2024

  • Resolved [CVE-2024-35255] | Updated the azidentity package to 1.6.0.

  • Resolved [CVE-2024-6345] | Updated pip setuptools.

  • Security | Upgraded prometheus to v2.55.0.

Release 23.10.50

October 30, 2024

Release 23.10.49

October 16, 2024

This patch contains testing, optimizations, security, and other minor internal changes. User functionality is unchanged.

Release 23.10.48

October 9, 2024

This patch contains testing, optimizations, security, and other minor internal changes. User functionality is unchanged.

Release 23.10.47

October 9, 2024

This patch contains testing, optimizations, security, and other minor internal changes. User functionality is unchanged.

Release 23.10.46

October 3, 2024

This patch contains testing, optimizations, security, and other minor internal changes. User functionality is unchanged.

Release 23.10.45

September 27, 2024

This patch contains testing, optimizations, security, and other minor internal changes. User functionality is unchanged.

Release 23.10.44

September 12, 2024

This patch contains testing, optimizations, security, and other minor internal changes. User functionality is unchanged.

Release 23.10.43

September 4, 2024

This patch contains testing, optimizations, security, and other minor internal changes. User functionality is unchanged.

Release 23.10.42

August 29, 2024

This patch contains testing, optimizations, security, and minor internal changes. User functionality is unchanged.

Release 23.10.41

August 5, 2024

This patch contains testing, optimizations, security, and minor internal changes. User functionality is unchanged.

Release 23.10.40

August 5, 2024

This patch contains testing, optimizations, security, and minor internal changes. User functionality is unchanged.

Release 23.10.39

July 29, 2024

This patch contains testing, optimizations, security, and minor internal changes. User functionality is unchanged.

Release 23.10.38

July 20, 2024

This patch contains testing, optimizations, security, and minor internal changes. User functionality is unchanged.

Release 23.10.37

July 19, 2024

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.36

July 13, 2024

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.35

June 27, 2024

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.34

June 20, 2024

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.33

June 12, 2024

  • The job service sometimes erroneously requeued completed tasks.

  • PDF attachments weren’t assigned appropriate names during process files steps.

Release 23.10.32

June 6, 2024

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.31

May 29, 2024

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.30

May 21, 2024

  • When you made a change in the human review screen, the original value incorrectly showed as empty.

Release 23.10.29

May 14, 2024

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.28

May 7, 2024

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.27

May 1, 2024

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.26

  • In Flow V2, if there is an exception in a user-defined function (UDF), the flow is now correctly marked as failed.

Release 23.10.25

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.24

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.23

  • Annotations weren’t carried over when importing an annotation set from the file system into Solution Builder, or when exporting an ML Studio dataset in the same environment.

Release 23.10.22

  • In ML Studio, the field list failed to load in certain circumstances.

Release 23.10.21

  • ImageMagick height and width limit is increased from 16K to 20K. Additionally, new environment variables in celery-app-tasks control ImageMagick height, width, and area: MAGICK_HEIGHT_LIMIT, MAGICK_WIDTH_LIMIT, MAGICK_AREA_LIMIT.

Release 23.10.20

  • The tag search filter in Flow Dashboard, as well as the jobs/list API endpoint, is now case sensitive.

Release 23.10.19

This patch contains testing, optimizations, and other minor internal changes. User functionality is unchanged.

Release 23.10.18

  • All v2 API endpoints now return a 401 status code for authorization errors and a 403 status code for license errors. Previously, a 200 status code was returned, with an error message in the response body, for v2 API errors. See API errors for more information, including v1 API error responses

Release 23.10.17

  • The Python library pyxlsb is now available for use in UDFs. This library enables importing .xlsb files in a pre-flow UDF.

Release 23.10.16

  • You can now view and filter by priority in Flow Dashboard. A new Priority column can be enabled in Flow Dashboard settings, and in the filter sidebar, you can filter by a specific priority. Flow priority can be set when using the API to run a flow.

Release 23.10.15

  • You can now optionally display provenance only for the selected field by enabling Only Show Selected Field Annotations in field list options.

  • If multiple open files have errors in Flow Review, only one appeared in the file list.

Release 23.10.14

  • Table editor cells and validations for the field within the table editor did not update correctly when cells were edited to fix validations.

Release 23.10.13

  • In Flow Review, when you used the pop-out icon to open a document in a separate window, the first file in the file list was displayed, rather than the selected document.

Release 23.10.12

This patch contains internal changes and testing. Functionality is unchanged.

Release 23.10.11

This patch contains internal changes and testing. Functionality is unchanged.

Release 23.10.10

This patch contains internal changes and testing. Functionality is unchanged.

Release 23.10.9

This patch contains internal changes and testing. Functionality is unchanged.

Release 23.10.8

This patch contains internal changes and testing. Functionality is unchanged.

Release 23.10.7

  • During human review, editing a field value sometimes changed the OCR confidence scores for other fields.

Release 23.10.6

This patch contains internal changes and testing. Functionality is unchanged.

Release 23.10.5

This is the first generally available release of Instabase 23.10.

New features

Solution Dashboard

  • New human review metrics for deployed solutions report time spent reviewing by reviewer, class, or field.

Refiner

  • The Value object now includes integrated model confidence scores. You can view confidence scores for individual words or in aggregate, with minimum, maximum, average, and other functions. Additionally, string manipulations, such as concatenation and substring operations, maintain word-based confidence. And in the validations app, minimum model confidence is used on refined phrases. Confidence scores are accessible using the updated interface for the Value object.

Enhancements

Platform

  • You can now authorize access to Azure Blob Storage using service principals, which enables RBAC. RBAC authorization is available on mounted drives, global drives, and Instabase Drive.

Deployment Manager

  • The upgrade process is enhanced with these changes:

    • Validation error messages provide more specific and detailed information for invalid configs.

    • The validation stage now provides a validation error for any configs with no or 0 replicas specified. This error doesn’t prevent the upgrade from proceeding, but provides a checkpoint to ensure resourcing is intentionally set to 0, such as when resourcing is managed through workload autoscaling.

    • Readiness checks that must be completed before the turn down and turn up steps now finish more quickly.

    • A Skip Turn Down toggle lets you skip the turn down step during an upgrade. Skipping the turn down step isn’t recommended during major version upgrades.

    • Logs across multiple steps are now clearer and more descriptive. For example, during the turn up step, you now see detailed progress logs as resources become available, and information about pending resources.

    • After pausing a stage, a Cancel stage button now displays. Clicking Cancel stage lets you force-cancel an in-progress stage, rather than waiting for the stage to complete before the upgrade pauses.

    • If any critical health checks fail, the upgrade fails. Previously, upgrades could proceed despite health check failures.

    • Any patches generated during the upgrade process are now clearly identified in patch comments.

  • The Infra Dashboard deployment alert manager now includes tabs with additional information. The Config tab lets you view your deployment configuration. The Live K8s Config tab lets you view the Kubernetes describe of the object. The Overview tab contains all previously displayed information.

  • When updating base configs from the Base Configs tab, error messages for failed config pushes are more detailed. You no longer need to check the logs or Kubernetes itself to see error details.

  • The login session timeout period for Deployment Manager admins is now 12 hours.

  • Several APIs used in the upgrade process now have an asynchronous variant. Asynchronous variants help avoid gateway timeouts for requests that can take some time to complete when performed synchronously. Tasks that can now be completed by API asynchronously include:

    See the upgrade API documentation, including the asynchronous endpoint conventions section, for more information.

  • An asynchronous API for updating base configs is available, supporting the same behavior available when updating base configs from the Deployment Manager Base Configs tab.

ML Studio

  • When editing a class schema, you can now modify the order of fields by dragging them in the desired order. When annotating or reviewing extraction results, fields are displayed in the specified order.

  • The training metrics page now includes a quick link to open the training results folder in a new tab.

  • You can now delete ML Studio training jobs and they no longer show up in the Training Jobs table.

Flow

  • A setting in the apply checkpoint step supports optional straight-through processing, so individual records that pass validation continue to the next flow step regardless of validation failures in the rest of the batch. Straight-through processing, previously released as a public preview feature, is now generally available.

  • A new retry option surfaced in Flow Dashboard lets you re-execute flows that returned an error at any step. Previously, you could retry only failed checkpoint steps. Additionally, in the retry job endpoint ({URL-BASE}/api/v1/jobs/retry), setting type=step_failure now re-executes steps with any type of failure.

  • The Flow Results API now includes average model confidence and average OCR confidence in responses.

Solution Dashboard

  • When you run a solution from the Solution Dashboard, you can now specify runtime configuration and email notifications for the job. Optional flow run settings are accessible in Advanced options when you run a solution.

Reader

  • Reader, and the Flow process files and fetch files steps, support Microsoft .rtf files.

  • During digitization, Reader—and the process files step in Flow—now automatically sizes table columns to prevent truncation when converting CSV to PDF.

  • Reader output now indicates languages for each page within a document along with confidence for each language.

Marketplace solutions

Bug fixes

Platform

  • Restricted file extensions are now case insensitive.

  • Under certain circumstances, when copying or moving files to a destination in an encrypted drive, the file service sometimes crashed.

  • File retention jobs weren’t correctly purging files.

  • When choosing files, the browser might crash if the file was too large to display a preview.

  • The Ray model is being progressively deployed for SaaS customers. By the week ending on October 23rd, all SaaS customers will have the Ray model deployed in their DEV deployment. By the week ending on October 30th, the Ray model will be deployed for all SaaS customers in both the UAT and PROD deployments.

  • In the Admin app, when you mount an S3 buckets as a global storage drive, an SSE-KMS option is now available. The dialog also now explains the abbreviations SSE-KMS (SSE with Amazon Key Management Service keys) and SSE-S3 (SSE with Amazon S3 managed keys).

  • Digitization of rich text format (.rtf) files is now supported.

  • The model service did not allow the clean up of model artifacts from failed downloads, resulting in wasted storage space.

Deployment Manager

  • During upgrades, you could become stuck on the Confirm Changes step if deprecated configs were added to the list of configs to keep. This behavior was caused by gateway timeouts. This bug fix introduces an asynchronous variant of the API used in this step of the upgrade process, to avoid gateway timeouts.

  • The presence of patches targeting ConfigMap templates no longer causes validation errors during upgrades.

  • A bug causing resource labels to not be recognized during base config validation, leading to validation errors during upgrades, is fixed.

  • If an upgrade is rolled back, ConfigMap template settings are now properly restored. This bug primarily affected rollbacks from release 23.07.x to a previous version.

  • During the Run Actions step of an upgrade, logs continue to populate even after an action failure. Previously, logs stopped populating when any action failed.

  • During installation, database jobs required only for version upgrades no longer run.

  • Network policies have been added for the jobs underlying the Test Runner test post-install action (solution-diagnostic-artifacts) and the base model image upload post-install action (mktplaceupdate). Previously, if your security settings required network policies for all jobs, these actions wouldn’t run. mTLS traffic is not currently supported when running these jobs.

Solution Builder

  • Under certain circumstances, the model training and output set creation portions of the Solution Builder onboarding tutorial would break.

ML Studio

  • Split classifier steps now function correctly when preceded by more than one process files step.

Flow

  • Automation metrics incorrectly assumed that if a validation rule was configured for a document, that the field must exist. This bug fix checks to see that the field also exists in that document.

  • Scheduling a job in the Solution Dashboard appeared successfully, but the scheduled job didn’t show up in the dashboard or the Scheduler app.

  • Checkpoint errors were not always correctly displayed in the Job Status section of the Flow Dashboard.

  • If one row remained in a list or table in the Table Editor view, the values could not be deleted.

Deprecations and removals

See the deprecations and removals page for the latest announcements, as well as previous deprecations and removals.

Deployment guide

  • Workload autoscaling was introduced as a public preview feature in release 23.07 and remains in public preview in release 23.10. As a public preview feature, workload autoscaling is disabled by default. Workload autoscaling has several infrastructure requirements. For instructions on enabling workload autoscaling during your upgrade, see the workload autoscaling feature documentation.