Commit Graph

34 Commits

Author SHA1 Message Date
Kyle Brandt dd4ffc9918
SQL Expressions: Add setting to limit length of query (#110165)
sql_expression_query_length_limit

Set the maximum length of a SQL query that can be used in a SQL expression. Default is 10000 characters. A setting of 0 means no limit.
2025-08-27 12:08:25 -04:00
Kyle Brandt 4f0cb47d3c
SQL Expressions: Rework backend errors and error instrumentation (#109633)
* Capture error_type label on metrics/traces
* Make error messages more helpful to user
* Use errutil, categorized errors, and tie them to error_type (category in code)
* Misc trace fixes
* Add metric to track SQL input conversion
2025-08-25 11:13:42 -04:00
Kyle Brandt 466aa70179
Sql Expressions: (Chore) Dedicated logger for expr.sql, and pass context (#109549) 2025-08-13 11:20:14 -04:00
Kyle Brandt 1be9e60667
SQL Expressions: Exclude CTEs from required Tables (#106479)
Fixes #105030

---------

Co-authored-by: Sam Jewell <2903904+samjewell@users.noreply.github.com>
2025-06-11 10:36:28 -04:00
Laura Fernández 23ecb9d904
i18n: Use `grafana/i18n` to init the `locale` and manage the `regionalFormat` (#105281) 2025-05-19 13:24:33 +02:00
Mariell Hoversholm 467b88eec5
Test: Make timeout 1ns to minimise flakiness (#105437) 2025-05-15 14:18:09 +02:00
Kyle Brandt 5e056c2a3f
SQL Expressions: Add sql expression specific timeout and output limit (#104834)
Adds settings for SQL expressions:
 sql_expression_cell_output_limit

Set the maximum number of cells that can be returned from a SQL expression. Default is 100000.

sql_expression_timeout

The duration a SQL expression will run before being cancelled. The default is 10s.
2025-05-13 15:22:20 -04:00
Kyle Brandt 6a6ba723a9
SQL Expressions: Add internal GMS traces (#104836)
get trace from inside GMS
2025-05-09 14:48:47 -04:00
Sam Jewell 5c418196f9
SQL Expressions: Add STDDEV(), VARIANCE() and aliases (#103851)
* SQL Expressions: Bump GMS for STDDEV(), VARIANCE()

Bump Go MySQL Server to the latest version, to get support for STDDEV()
and VARIANCE() functions, and their aliases.

See https://github.com/dolthub/go-mysql-server/pull/2928

* Update app-code to comply with latest GMS version

* Run `go mod tidy`

* Update test which had depended on STDDEV being missing

* Run `make update-workspace`
2025-04-15 10:48:19 +01:00
Sam Jewell f8a72214cf
SQL Expressions: Add more functions to the allowlist (#103546)
* SQL Expressions: Allow many more functions and nodes

* Also allow the `REGEXP_SUBSTR` function

* Add window functions

* add more JSON support, remove now and current time (for now)

---------

Co-authored-by: Kyle Brandt <kyle@grafana.com>
2025-04-07 21:23:39 +03:00
Kyle Brandt 856dcfd6fe
SQL Expressions: Permit range conditions (between) in queries (#103238) 2025-04-02 15:13:17 +03:00
Sam Jewell af08a9fae2
SQL Expressions: Add JSON support (#103157)
- Support bi-directional mapping of frame JSON fields and GMS (go-mysql-server) columns 
- Permit GMS json functions

Co-authored-by: Kyle Brandt <kyle@grafana.com>
2025-04-01 07:45:01 -04:00
Kyle Brandt e6480a050c
SQL Expressions: Allow substring_index func (#103021) 2025-03-27 16:49:21 -04:00
Sam Jewell c9e929e2e6
SQL Expressions: Allow more MySQL AST node-types (#102973)
* SQL Expressions: Add Null-literal node

* Retain some order in the code - put NullVal with BoolVal

* Add support for `IN` keyword

* Add GROUP_CONCAT keyword

* Add COLLATE keyword

From Claude:

The test case demonstrates a simple use of COLLATE with a string
literal, but in real MySQL queries, COLLATE is often used in more
complex scenarios like:

1. String comparisons:
   `SELECT * FROM table WHERE name COLLATE utf8mb4_bin = 'John'`
2. Sorting:
   `SELECT * FROM table ORDER BY name COLLATE utf8mb4_unicode_ci`
3. JOINs:
   `SELECT * FROM table1 JOIN table2 ON table1.name COLLATE utf8mb4_bin = table2.name`

The COLLATE clause is particularly useful when you need case-sensitive
comparisons (utf8mb4_bin) or specific language-aware sorting rules.
2025-03-27 11:32:48 +00:00
Sam Jewell 6be7829acf
SQL Expressions: Allow `IS` keyword (#102841) 2025-03-26 09:05:42 +00:00
Kyle Brandt 0f6aff2057
SQL Expressions: allow ParenSelect in queries (#102807)
sql_expressions: allow ParenSelect
2025-03-25 16:19:52 +00:00
Todd Treece 1b1f626edd
SQL Expressions: Fix error handling (#102650) 2025-03-21 18:47:00 -04:00
Sam Jewell d7aeebe5e3
Sql Expressions: State when error is from GMS (#102112) 2025-03-18 22:41:42 +00:00
Sam Jewell 21b9d45ca6
SQL Expressions: Add CASE/WHEN nodes and fixes (and test) for functions just added to allowlist (#102040)
* SQL Expressions: Add CASE/WHEN SQL nodes to allowlist

* Fixed and test for functions added in #102011

* Add remaining functions to the test-case

These are mostly aliases, so the LLM chose to omit them originally.
But adding now for completeness

* Fix ordering of allowed nodes
2025-03-12 15:57:50 +00:00
Sam Jewell 4bd5f29e05
SQL Expressions: Allow more functions (#102011)
I used Cursor and Claude 3.7 thinking to expand the list of functions
we allow. Specifically I needed `abs`.

TODO: Check each of these in the official MySQL docs to ensure they
are all safe:
eg. this doc: https://dev.mysql.com/doc/refman/8.4/en/flow-control-functions.html#operator_case
2025-03-12 11:39:48 +00:00
Kyle Brandt 4d7b9a3c77
SQL Expressions: Improve response to frame conversion handling (#100625)
- use types.Convert to get a more normalized type returned from GMS. After syncing with GMS maintainers, the underlying type of the row could be different things (e.g. int when unit32, so this calls the Convert method from the GMS types library to normalize the interface.
- Clean up fieldValFromRow more
- Use IsText to capture different string types
- Add more types to test, also update test to use same cmp.Diff method as others

---------

Co-authored-by: Sam Jewell <2903904+samjewell@users.noreply.github.com>
2025-02-14 17:21:08 +02:00
Kyle Brandt f7588376df
SQL Expressions: (Chore) Update code for latest plugin-sdk data pkg (#100425)
Use new NilAt and SetRefId methods
2025-02-11 15:15:54 +00:00
Kyle Brandt 6dc98dbbcc
SQL Expressions: Add str_to_date function and unskip test (#100226) 2025-02-07 11:51:55 +02:00
Kyle Brandt d196b789e2
SQL Expressions: Add more SQLNodes and funcs to allow list (#100227)
sql_expr: Add more Nodes and funcs to allow list
2025-02-07 09:44:17 +00:00
Kyle Brandt d64f41afdc
SQL Expressions: Re-implement feature using go-mysql-server (#99521)
* Under feature flag `sqlExpressions` and is experimental
* Excluded from arm32
* Will not work with the Query Service yet
* Does not have limits in place yet
* Does not working with alerting yet
* Currently requires "prepare time series" Transform for time series viz
 
---------

Co-authored-by: Sam Jewell <sam.jewell@grafana.com>
2025-02-06 07:27:28 -05:00
Sam Jewell 1915efbc95
SQL Expressions: Simplify where code is being stubbed out (#96227)
* Rename from DuckDB

* Remove unused function
2024-11-18 12:05:08 +00:00
Sam Jewell ea71201ddc
ServerSideExpressions: Disable SQL Expressions to prevent RCE and LFI vulnerability (#94942)
* disable sql expressions

remove duckdb ref

* Run `make update-workspace`

---------

Co-authored-by: Scott Lepper <scott.lepper@gmail.com>
2024-10-18 11:28:26 +01:00
Scott Lepper cac40795a5
sql expressions - logging (#88193)
* sql expressions - logging
2024-05-22 17:18:46 -04:00
Scott Lepper 93acc0c932
sqlexpressions - fix escape quote (#87921) 2024-05-15 10:55:58 -04:00
Scott Lepper 14a814a280
[sql expressions] fix: use ast to read tables (#87867)
* [sql expressions] fix: use ast to read tables

* can't run tests during ci yet.  need to install duckdb

* skip for now.  need duckdb cli
2024-05-14 17:05:29 -04:00
Scott Lepper 1a2bbd61fd
sql expressions: improve parser (#87277)
sql expressions: improve parser
2024-05-03 08:08:07 -04:00
Scott Lepper cbe3e3a5e0
fix: sql expressions parser with join (#87263) 2024-05-02 14:48:05 -04:00
Scott Lepper 4fd2cb6014
fix: sql expressions - sql parser table name case (#87196)
* fix: sql parser table name case
2024-05-02 08:43:20 -04:00
Scott Lepper 70009201d4
Expressions: Sql expressions with Duckdb (#81666)
duckdb temp storage of dataframes using parquet and querying from sql expressions
---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2024-02-27 16:16:00 -05:00