Additionally, Collibra DQ support for Spark 2.x will be limited as of Collibra DQ 2024.01, as Spark 2.x has reached its end of life.

We strongly encourage organizations on Standalone deployments of Collibra DQ to upgrade to the latest Spark package to utilize of the new features and address some of the major vulnerabilities with Spark 3.2 or earlier versions.

We’ve upgraded our out-of-the-box Apache Spark version from 3.2.0 to 3.4.1.

See the Updated UI table below for more details on what's changed. While you can still modify these flags from the Admin Console Configuration Settings Application Config page, we recommend keeping them set to TRUE to allow for full feature functionality and an elevated user experience. Additionally, if a pre-existing install of Collibra DQ had these flags set to FALSE, they are now set to TRUE. New installs of Collibra DQ come with REACT_MUI and UX_REACT_ON admin flags set to TRUE by default.

When all of your data resides in BigQuery, Pushdown reduces the amount of data transfer, eliminates egress latency, and removes the Spark compute requirements of a DQ job. We're excited to announce that Pushdown for BigQuery is now generally available! Pushdown is an alternative compute method for running DQ jobs, where Collibra DQ submits all of the job's processing directly to a SQL data warehouse, such as BigQuery.

Instal EF Commander 2023.10