analytics-catalog-api/me.ahoo.wow.analytics.catalog.api.source

Package-level declarations

Types

@Summary(value = "Approve a lower sensitivity of a Wow source")

data class ApproveSensitivityChange(val keys: List<String> = emptyList())

Accepts, as a privileged reviewer, the lower protection the Wow source's wow_catalog row now gives the values pending review (keys, each a FieldSensitivity.keyOf; every pending one when empty): lowered or removed since the source adopted it, or gone with the row.

ApproveSource

@Summary(value = "Approve a source for the datasets")

data object ApproveSource

Approves, as an approver, a reviewed or adopted source for the datasets, when the catalog's switch says a new source takes an approver (wow.analytics.catalog.require-source-approval, ③b): until then no dataset reads it.

DraftSource

@CreateAggregate

@Summary(value = "Draft an analyst's source from a warehouse table")

data class DraftSource(val database: String, val table: String)

Drafts the analyst's source of database's table ((b)): the catalog reads the table's columns from the warehouse (system.columns); the source's id is me.ahoo.wow.analytics.catalog.api.AnalyticsCatalogContext.sourceIdOf of the warehouse's connection, database and table. A table that the Wow Sink describes by a trusted wow_catalog row is not an analyst's to draft.

FieldSensitivity

data class FieldSensitivity(val path: String, val level: SensitivityLevel? = null, val keepPrefix: Int = 0, val keepSuffix: Int = 0, val strategy: String? = null, val event: String = "", val revision: Int = 0)

How one value of a source is protected: the value at path (a column, or a JSON path below one such as state.customer.phone), of the event event at revision when it is an event's (empty and 0 otherwise), at level, or not sensitive when level is null, which is a decision too; its mask keeps keepPrefix and keepSuffix characters, or is the custom strategy a Wow schema names.

MarkSourceReviewed

@Summary(value = "Mark an analyst's source reviewed")

data object MarkSourceReviewed

Marks an analyst's source reviewed: its tenant column is declared and each of its string columns is reviewed.

ReviewSourceSensitivity

@Summary(value = "Review the sensitivity of a source's values")

data class ReviewSourceSensitivity(val fields: List<FieldSensitivity>)

Records, value by value, the reviewer's decision on fields: sensitive at a level, or not. An analyst's source is reviewed by column; a Wow source's values may only be tightened above what its wow_catalog row says. Changing an analyst's reviewed source takes it back to its draft.

ReviseSource

@Summary(value = "Revise an analyst's source")

data class ReviseSource(val tenantColumn: String? = null, val ownerColumn: String? = null, val spaceColumn: String? = null, val references: List<SourceReference> = emptyList(), val platform: Boolean = false)

Declares an analyst's source's scope and references: the tenantColumn every row's tenant is in, the optional ownerColumn and spaceColumn, and the columns that refer to an aggregate the catalog knows (references). A table every tenant shares, such as currencies or regions, is instead a platform dimension, without a tenant column (-2): a dataset joins it without a tenant key once an approver marks it reviewed. A reviewed source goes back to its draft.

SensitivityChangeApproved

@Summary(value = "A lower sensitivity was approved")

data class SensitivityChangeApproved(val keys: List<String>, val sensitivity: List<FieldSensitivity>)

A privileged reviewer approved the lower protection of keys: the source's sensitivity is now as given.

SourceApproved

@Summary(value = "A source was approved for the datasets")

data object SourceApproved

An approver approved the source for the datasets.

SourceColumn

data class SourceColumn(val name: String, val type: String)

A column of a table, as system.columns reads it: its name and its warehouse type.

SourceDrafted

@Summary(value = "An analyst's source was drafted")

data class SourceDrafted(val connection: String, val engine: String, val database: String, val table: String, val tableUuid: String, val tableEngine: String, val columns: List<SourceColumn>)

An analyst drafted the source of database's table on connection, a warehouse of the engine type (clickhouse), with the table's uuid, its tableEngine (the kind of table as the warehouse names it) and its columns.

SourceKind

enum class SourceKind : Enum<SourceKind>

Where a source's description comes from.

SourceReference

data class SourceReference(val path: String, val context: String, val aggregate: String)

A value of a source at path that refers to an aggregate: aggregate of context.

SourceReviewed

@Summary(value = "An analyst's source was reviewed")

data object SourceReviewed

The analyst's source is reviewed: a dataset may be built on it.

SourceRevised

@Summary(value = "An analyst's source was revised")

data class SourceRevised(val tenantColumn: String? = null, val ownerColumn: String? = null, val spaceColumn: String? = null, val references: List<SourceReference> = emptyList(), val reopened: Boolean = false, val platform: Boolean = false)

The analyst's source's scope and references were declared, or that it is a platform dimension; reopened when it was reviewed before.

SourceSensitivityReviewed

@Summary(value = "A source's sensitivity was reviewed")

data class SourceSensitivityReviewed(val fields: List<FieldSensitivity>, val reopened: Boolean = false)

A reviewer decided the protection of fields; reopened when an analyst's reviewed source went back to its draft.

SourceStatus

enum class SourceStatus : Enum<SourceStatus>

Where a source is in its review.

SourceTableChanged

@Summary(value = "A source's table changed")

data class SourceTableChanged(val tableUuid: String, val tableEngine: String, val columns: List<SourceColumn>, val reopened: Boolean = false)

The source's table is another table now (tableUuid) or has other columns; reopened when an analyst's reviewed source went back to its draft, because a declared column is gone or a string column is not reviewed.

SourceTableMissing

@Summary(value = "A source's table is missing")

data object SourceTableMissing

The source's table is no longer in the warehouse.

SyncSource

@AllowCreate

@InternalCommand

@Summary(value = "Bring a source up to date with the warehouse")

data class SyncSource(val database: String, val table: String)

Brings a source up to date with the warehouse: its table's uuid and columns, and for a table the Wow Sink describes, the wow_catalog row the trust rules adopt. The catalog sends it when it sees the table or its wow_catalog rows change; it creates the Wow source of a table it does not know yet, and changes nothing for a table that is neither known nor trusted. Only the catalog's watcher sends it: internal.

WowEnrichment

data class WowEnrichment(val service: String, val context: String, val aggregate: String, val role: String, val ownership: String, val fingerprint: String, val writer: String, val writtenAt: String, val eventRevisions: Map<String, Int> = emptyMap(), val access: String = "")

The trusted wow_catalog row a Wow source adopted: the aggregate the table is of (service, context, aggregate, role such as state_flat, ownership), the row's fingerprint, the account that wrote it (writer) and when (writtenAt), and each event's revision. access is whether the aggregate's rows are read by an access policy of its own (POLICY) or by its scope's columns (DEFAULT), as its service says; empty in a row an earlier Sink wrote, which the catalog reads as POLICY.

WowEnrichmentAdopted

@Summary(value = "A trusted wow_catalog row was adopted")

data class WowEnrichmentAdopted(val connection: String, val engine: String, val database: String, val table: String, val tableUuid: String, val tableEngine: String, val columns: List<SourceColumn>, val enrichment: WowEnrichment, val tenantColumn: String, val ownerColumn: String? = null, val spaceColumn: String? = null, val references: List<SourceReference> = emptyList(), val sensitivity: List<FieldSensitivity> = emptyList(), val claimed: List<FieldSensitivity> = emptyList(), val pending: List<String> = emptyList(), val created: Boolean = false, val fromAnalyst: Boolean = false)

The catalog adopted the trusted wow_catalog row enrichment of the table: the table's uuid, table engine and columns, the scope's columns, the references, and the sensitivity the source now gives each value, never lower than before; pending are the keys whose row now says lower or nothing, kept as they were until a privileged reviewer approves the change. created when the table had no source before; fromAnalyst when an analyst's source of the table became a Wow source.

WowEnrichmentLost

@Summary(value = "A Wow source's trusted wow_catalog row is gone")

data class WowEnrichmentLost(val pending: List<String>)

The Wow source's trusted wow_catalog row is gone: each value keeps its protection, and the keys it gave are pending review.

WowEnrichmentRefused

@Summary(value = "A table's wow_catalog rows were refused")

data class WowEnrichmentRefused(val reasons: List<String>)

The table's wow_catalog rows were not adopted, for reasons: an untrusted writer, another table's uuid, other columns than the table has. Recorded for the audit when the reasons change.

Functions

keyOf

fun FieldSensitivity.keyOf(): String

The key a FieldSensitivity is reviewed and approved by: its path, and for an event's value <path>@<event>#<revision>.