When using dialect other than napkin.* Napkin will have limited capabilities (e.g. query optimization will not be available).
Dependency detection and renaming will not affect queries parsed with raw. Napkin will not attempt to parse or modify them.
postgres dialect can be used when some Postgres-specific features are used (e.g. JSON operator such as ->>).
Fix: Napkin would not detect dependencies in some sub-queries.
Napkin is able to perform strict validations (in run, ‘validate’, dump and optimize commands). The supported validations are:
Mustache variable, mentioned in section name is not defined.
Part of nested mustache variable path, mentioned in section name in not an object.
Complex (object, array) mustache value rendered directly.
Suppress reporting 0 rows affected for SQLite
REPL: Fix: helper macros did not work properly due to regression.
REPL: Spec and Hook programs can return values to the REPL environment for debugging purposes.
Internal: LocalFile Polysemy language got split into LocalFile and Template with umbrella LoadQuery language.
Query statistics is not printed as “unknown” if there are no pieces of information available (typical for Sqlite).
Source location (used in error reporting) is now able to print start of inline query for better context.
Post hooks in YAML will have implicit argument table with target table name.
BigQuery: Add support for STRUCTs.
Fix: renamed target table name was passed to programs instead of raw one.
SQLite version has been updated to 3.36.0 and now supports math and JSON functions.
Napkin now prints embedded Sqlite version as part of napkin version CLI command.
‘Simple’ logging format now has table name in the context.
Added --use-spec-names to dump command.
Internal change: query transformers (e.g. table renamers) are not baked into programs – they are applied on runtime.
Fix: regression for assert_expression syntax in YAML specs.
0.5.10 (released 2021-12-29)
<nixpkgs> path in docker now points to the github commit instead of intermediate file.
Is is now possible to specify timeout for google BigQuery in the backend_options.
Napkin will report an error when sql file used with incremental_by_time strategy does not consume cutoff variable.
Fix: SQLite remove extra parents from SQL INSERT INTO SELECT Statement.
Change PostgreSQL parser dialect from ANSI to Postgres.
Fix: Napkin was using incorrect pipeline name for backends (was always “rs”).
Fix: Change CAST operator rendering to be ANSI complainant.
Docker image now uses napkin user by default, it has nicer bash and zsh prompts.
Docker image working directory changed from /project to /home/napkin/project.
Dev-container settings (produced by napkin init) now specify zsh as a default shell.
CI builds is DAG now (overall pipeline time reduced).
Meta arguments handling in programs is more consistent:
Reader MetaArguments is now Input MetaArguments in SpecProgram and HookProgram.
Input SpecMetaArgs is now Input MetaArguments in SpecPreprocessor.
Input MetaArguments helpers are now located in Napkin.Run.Effects.MetaArguments.
incrementalByTime now accepts incremental_reset as string or bool (--arg incremental_reset=true will work again).
napkin auth uses now subcommands to show (napkin auth show) and reset (napkin auth reset).
napkin auth show displays output in human-friendly format.
Fix: namespaceAllTables will no longer rename CTEs and break queries.
Fix: renaming tables will no longer affect aliases.
Fix: table aliases were not rendered correctly in JOIN queries (Postgres, Redshift).
Extensions to table_namespace and table_prefix preprocessors:
added scope parameter that can be either all, managed (default) or unmanaged to control which tables are renamed,
added only and except parameters for fine-grained control on which tables are renamed,
table_namespace has extra on_existing parameter that can be overwrite (default) or keep_original, which allows to keep original namespace if it has been explicitly provided in the spec.
Fix: tables now can be moved between schemas (Postgres, Redshift).
Fix: checkTableExists does not assume that default schema is public (Postgres, Redshift).
0.5.9 (released 2021-12-14)
Napkin now support different log formats though --log-format CLI option.
Added Napkin static binary (does not fully support haskell interpretation) for easier installation.
Renamed live validation option from ‘-l’ (–live) to ‘-i’ (–interactive) to avoid collision with –log-level.
Fix: Interactive validation now doesn’t complain on absent folders.
Added S3 bucket monitoring page.
Consistent log-level setting via command line options: one can use either -v or --log-level (-l). Options can be applied to all commands now.
Error reporting is more consistent.
Improved CLI UI (napkin run -p) look and feel.
Fix: CLI UI did not display query statistics properly when spec execution has been terminated by the user.
Fix: CLI UI did not terminate spec execution.
Napkin will print the information on execution plan (managed tables to be updated, unmanaged tables used as an input, managed tables used as an input, but not scheduled for updated) as well as ETA.
Skipped tables will no longer block dependent tables execution.
Fix: Haskell spec has now access to meta arguments.
Number of concurrent DB operations can be set with backend_options in YAML specs. Use concurrent_queries for Big Query or connection_pool for Postgres and Redshift. The setting defaults to 100.