analytics-catalog-domain/me.ahoo.wow.analytics.catalog.domain.warehouse/WarehouseCatalog

WarehouseCatalog

The warehouse's metadata, the base of the data-source catalog: the tables and views of the configured databases (databases) of one warehouse connection, each a source identified by (connection, database, table), and the wow_catalog rows the Wow Sink appended of them. 10.0 has the ClickHouse implementation (system.tables, system.columns, <database>.wow_catalog); the service binds it.

The catalog is the analytics platform's control plane (②④): only the catalog service calls this port, to discover, adopt and review the sources. The data plane (the Sink's writes, the dataset queries) never calls the catalog on its own path: the Sink only appends its wow_catalog rows, and a query node serves the dataset versions it was given, so the data plane goes on when the catalog is down.

Types

Companion

object Companion

Properties

connection

abstract val connection: String

The name of the warehouse connection, the first part of each source's identity.

databases

abstract val databases: List<String>

The databases whose tables are sources.

engine

abstract val engine: String

The engine type of the warehouse: clickhouse, the one 10.0 implements. Nothing else of the port is an engine's: its types, table kinds and column types are the warehouse's own names, as text.

Functions

catalogChanges

abstract suspend fun catalogChanges(database: String): Map<String, String>

When each table of database last got a wow_catalog row, by its name.

catalogRows

abstract suspend fun catalogRows(database: String, table: String): List<CatalogRow>

Every wow_catalog row of database's table, whoever wrote it; none without wow_catalog.

table

abstract suspend fun table(database: String, table: String): WarehouseTable?

database's table with its columns, or null when the warehouse has none of that name.

tables

abstract suspend fun tables(database: String): List<WarehouseTableInfo>

The tables and views of database that may be sources, without their columns.