Commit Graph

8326 Commits

Author SHA1 Message Date
Nick Thomas 69645389e9
Prevent a path traversal attack on global file templates
The API permits path traversal characters like '../' to be passed down
to the template finder. Detect these requests and cause them to fail
with a 500 response code.
2018-12-05 14:12:35 +00:00
Fabio Busatto 568cc2dd6a Revert "Merge branch 'auto-devops-support-for-group-security-dashboard' into 'master'"
This reverts merge request !23165
2018-12-04 07:17:27 +00:00
Andreas Brandl 9c059a026d
Lower the tablesample thresholds/targets.
The numbers in
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22650#note_117645395
are quite promising with that target.
2018-12-03 21:26:54 +01:00
Andreas Brandl 474fd9138c
Move strategies in their own files
This improves readability quite a bit.
2018-12-03 21:26:53 +01:00
Andreas Brandl ff35cb45e9
Fix comment. 2018-12-03 21:26:53 +01:00
Andreas Brandl df6166f4b1
Refactor estimate query 2018-12-03 21:26:52 +01:00
Andreas Brandl 9bb7c690c4
Fix rubocop offenses. 2018-12-03 21:26:51 +01:00
Andreas Brandl 3266642739
Add feature flag for tablesample counting. 2018-12-03 21:26:51 +01:00
Andreas Brandl 3a7d9b4b02
Implement TablesampleCountStrategy.
A tablesample count executes in two phases:
* Estimate table sizes based on reltuples.
* Based on the estimate:
  * If the table is considered 'small', execute an exact relation count.
  * Otherwise, count on a sample of the table using TABLESAMPLE.

The size of the sample is chosen in a way that we always roughly scan
the same amount of rows (see TABLESAMPLE_ROW_TARGET).
2018-12-03 21:26:50 +01:00
Andreas Brandl 72347448db
More specs for fallback testing. 2018-12-03 21:26:49 +01:00
Andreas Brandl 01c7cb90da
Refactor specs to separate concerns. 2018-12-03 21:26:49 +01:00
Andreas Brandl c5fb468255
Flexible approximate counts with fallback strategies. 2018-12-03 21:26:48 +01:00
Andreas Brandl b6a530c9b1
Simplify to use models instead of tables. 2018-12-03 21:26:48 +01:00
Andreas Brandl f2ec9d2d34
Add ExactCountStrategy. 2018-12-03 21:26:48 +01:00
Andreas Brandl f3a9315177
Extract ReltuplesCountStrategy. 2018-12-03 21:26:47 +01:00
Dmitriy Zaporozhets 8cd5004b35 Merge branch 'improve_auto_devops_migration_debug' into 'master'
Add echo so that we know which branch was taken

See merge request gitlab-org/gitlab-ce!23499
2018-12-03 11:26:49 +00:00
Thong Kuah e194d7b4aa Add echo so that we know which branch was taken
This should help debug issues like
https://gitlab.com/gitlab-org/gitlab-ce/issues/54760
2018-12-03 09:22:27 +13:00
Kamil Trzciński 64b1044e7a ci/config: generalize Config validation into Gitlab::Config:: module
This decouples Ci::Config to provide a common interface for handling
user configuration files.
2018-11-29 16:09:18 +01:00
Rémy Coutable 7c0718cd79 Merge branch '51083-fix-move-api' into 'master'
Commits API: Preserve file content in move operations if unspecified

Closes #52974 et #51083

See merge request gitlab-org/gitlab-ce!23387
2018-11-29 11:36:16 +00:00
Imre Farkas bd3a484032
Add config to disable impersonation
Adds gitlab.impersonation_enabled config option defaulting to true to
keep the current default behaviour.

Only the act of impersonation is modified, impersonation token
management is not affected.
2018-11-29 09:37:16 +01:00
Steve Azzopardi a9f5b22394
Merge branch 'security-11-5-fix-webhook-ssrf-ipv6' into 'security-11-5'
[11.5] Fix SSRF in project integrations

See merge request gitlab/gitlabhq!2611
2018-11-28 19:14:36 -05:00
Cindy Pallares c0e5d9afee
Merge branch 'security-fj-crlf-injection' into 'master'
[master] Fix CRLF issue in UrlValidator

See merge request gitlab/gitlabhq!2627
2018-11-28 19:14:06 -05:00
Cindy Pallares fe5f75930e
Merge branch 'security-fix-pat-web-access' into 'master'
[master] Resolve "Personal access token with only `read_user` scope can be used to authenticate any web request"

See merge request gitlab/gitlabhq!2583
2018-11-28 19:13:59 -05:00
Cindy Pallares 4bc6f2e3ac
Merge branch 'security-stored-xss-for-environments' into 'master'
[master] Stored XSS for Environments

Closes #2727

See merge request gitlab/gitlabhq!2594
2018-11-28 19:07:29 -05:00
Nick Thomas 14076062df
Commits API: Preserve file content in move operations if unspecified 2018-11-28 11:50:05 +00:00
Douwe Maan 5e3076f2d7 Merge branch '50341-cleanup-useless-project-import-attributes' into 'master'
Removes all the irrelevant code and columns that were migrated from the Project…

See merge request gitlab-org/gitlab-ce!21497
2018-11-28 11:19:55 +00:00
Toon Claes f35ff1ea48 Ensure all Routables have a parent
Or otherwise do not try to write repo config.
2018-11-27 22:48:55 +01:00
Toon Claes de0cc8e46a Disable the `type` STI column for the migration
There is no need to make a difference between a Group or User
namespace, so also no need to define the classes of those.
2018-11-27 22:48:55 +01:00
Toon Claes 4711100164 Always run CleanUp before writing the git config 2018-11-27 22:48:55 +01:00
Toon Claes 4908e4b3a2 Run repository cleanup on failure 2018-11-27 22:48:55 +01:00
Toon Claes 220208c051 Retry the failing projects 2018-11-27 22:48:55 +01:00
Toon Claes cc70bd8440 Move code to a BackgroundMigration
And run in intervals.
2018-11-27 22:48:55 +01:00
Stan Hu 6c83c2d8b9 Merge branch 'lock-trace-writes' into 'master'
Lock writes to trace stream

Closes #51502

See merge request gitlab-org/gitlab-ce!23332
2018-11-27 20:56:40 +00:00
Stan Hu 1524a19302 Merge branch '53763-fix-encrypt-columns-data-loss' into 'master'
Correctly handle data-loss scenarios when encrypting columns

Closes #53763

See merge request gitlab-org/gitlab-ce!23306
2018-11-27 19:08:53 +00:00
Nick Thomas 6ddefe7cad
Correctly handle data-loss scenarios when encrypting columns
If the EncryptColumns background migration runs in a sidekiq with a
stale view of the database schema, or when the purported destination
columns don't actually exist, data loss can result. Attempt to work
around these issues by reloading schema information before running
the migration, and raising errors if the model reports that any of its
source or destination columns are missing.
2018-11-27 18:24:18 +00:00
Filipa Lacerda a99f342b42 Merge branch 'issuable-suggestions' into 'master'
Suggest issues when typing title

Closes #22071

See merge request gitlab-org/gitlab-ce!22866
2018-11-27 17:58:27 +00:00
Kamil Trzciński 31a1ce2132 Lock writes to trace stream 2018-11-27 17:55:20 +01:00
Stan Hu 681d7139a9 Merge branch 'json-logging-for-k8s' into 'master'
Json logging for k8s Integration

See merge request gitlab-org/gitlab-ce!23328
2018-11-27 16:51:34 +00:00
Phil Hughes 50e21a89a0
Suggests issues when typing title
This suggests possibly related issues when the user types a title.

This uses GraphQL to allow the frontend to request the exact
data that is requires. We also get free caching through the Vue Apollo
plugin.

With this we can include the ability to import .graphql files in JS
and Vue files.
Also we now have the Vue test utils library to make testing
Vue components easier.

Closes #22071
2018-11-27 15:10:40 +00:00
Dylan Griffith 7e7fb6deba Use JSON logging for helm install services 2018-11-27 15:23:02 +01:00
Dmitriy Zaporozhets fcca3f6452 Merge branch '51792-dont-delete-failed-install-pods' into 'master'
Don't remove failed install pods

Closes #51792

See merge request gitlab-org/gitlab-ce!23350
2018-11-27 13:14:51 +00:00
Tiago Botelho 4bd8a427d4
Removes all the irrelevant import related code and columns
Clears the import related columns and code from the Project
model over to the ProjectImportState model
2018-11-27 12:58:13 +00:00
Dylan Griffith 6a66e4a1f1 Don't remove failed install pods
We want to keep failed install pods around so that it is easier to debug
why a failure occured. With this change we also need to ensure that we
remove a previous pod with the same name before installing so that
re-install does not fail.

Another change here is that we no longer need to catch errors from
delete_pod! in CheckInstallationProgressService as we now catch the
ResourceNotFoundError in Helm::Api. The catch statement in
CheckInstallationProgressService was also probably too broad before and
should have been narrowed down simply to ResourceNotFoundError.
2018-11-27 12:54:32 +01:00
Douwe Maan d3f033d69c Merge branch 'bvl-use-shell-writeref' into 'master'
Don't use rugged write-ref anymore

See merge request gitlab-org/gitlab-ce!23286
2018-11-27 08:56:26 +00:00
Douwe Maan 124a0cab33 Merge branch 'sh-handle-string-null-bytes' into 'master'
Gracefully handle references with null bytes

Closes #54466

See merge request gitlab-org/gitlab-ce!23365
2018-11-27 08:54:59 +00:00
Stan Hu a0548a2277 Merge branch '8584-move-ee-specific-code-from-gitbab-database-ee-gitlab-database' into 'master'
CE port of 'Move EE specific code from Gitbab::Database into ee'

See merge request gitlab-org/gitlab-ce!23361
2018-11-26 23:25:27 +00:00
Stan Hu e36c347ff9 Gracefully handle references with null bytes
`Rugged::Reference.valid_name?` used in
`Gitlab::GitRefValidator.validate` fails on strings containing null
bytes because it uses `StringValueCStr()`. Per
https://silverhammermba.github.io/emberb/c/:

Ruby’s String kinda corresponds to C’s char*. The simplest macro is
StringValueCStr() which returns a null-terminated char* for a
String. The problem here is that a Ruby String might contain nulls - in
which case StringValueCStr() will raise an ArgumentError!

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/54466
2018-11-26 14:14:16 -08:00
Stan Hu deaf3af7e5 Merge branch 'fix-deadlock-chunked-io' into 'master'
Fix deadlock on ChunkedIO

See merge request gitlab-org/gitlab-ce!23329
2018-11-26 21:16:40 +00:00
Douglas Barbosa Alexandre 5f8423dd85
CE port of 'Move EE specific code from Gitbab::Database into ee' 2018-11-26 17:14:05 -02:00
Stan Hu 95ebdf3053 Merge branch '54327-profiler-doesn-t-work-with-auth-now' into 'master'
Allow profiler to authenticate by stubbing users directly

Closes #54327

See merge request gitlab-org/gitlab-ce!23320
2018-11-26 18:46:32 +00:00