Commit Graph

5882 Commits

Author SHA1 Message Date
Bob Van Landuyt 589b2db06c Setup Phabricator import
This sets up all the basics for importing Phabricator tasks into
GitLab issues.

To import all tasks from a Phabricator instance into GitLab, we'll
import all of them into a new project that will have its repository
disabled.

The import is hooked into a regular ProjectImport setup, but similar
to the GitHub parallel importer takes care of all the imports itself.

In this iteration, we're importing each page of tasks in a separate
sidekiq job.

The first thing we do when requesting a new page of tasks is schedule
the next page to be imported. But to avoid deadlocks, we only allow a
single job per worker type to run at the same time.

For now we're only importing basic Issue information, this should be
extended to richer information.
2019-05-31 09:40:54 +02:00
Vladimir Shushlin 39e21fb266 Generate lets_encrypt_private_key on the fly
Remove migration generating lets encrypt key
Don't generate private_key if database is readonly

For reference:
This reverts commit 988a7f70489b99383b95e9f271a2caf6bb5b3a44.
This reverts commit 21acbe531592d55caf0e5b8716a3b551dafd6233.
2019-05-31 05:22:55 +00:00
Lin Jen-Shin c90ba127bf Extract roulette to its own module
So it's more modular and extensible
2019-05-30 19:24:28 +08:00
Stan Hu 82f39315b1 Merge branch 'add-wiki-size-to-statistics' into 'master'
Add wiki size to project statistics

Closes #57822

See merge request gitlab-org/gitlab-ce!25321
2019-05-29 22:01:50 +00:00
Jan Provaznik 6d6bae66df Added rack-timeout for Puma
It assures that requests are aborted after 60 seconds, otherwise
an exception is raised. This exception is logged by Sentry, also
there is a Prometheus counter for measuring number of requests in each
state.
2019-05-29 18:53:44 +00:00
Peter Marko 40490cc492 Add wiki size to project statistics 2019-05-29 16:08:25 +02:00
Jan Provaznik a5adc6a024 Add Puma sampler
This sampler gathers Puma-specific metrics which can be used by
Prometheus then.
2019-05-29 12:27:44 +00:00
Liam McAndrew 5ddb32f382 Fix order dependency with user params during imports 2019-05-28 10:31:51 +00:00
Ash McKenzie 2714f85c12 Merge branch 'use-source-ref-name-in-webhook' into 'master'
Use source ref in pipeline webhook

Closes #61553

See merge request gitlab-org/gitlab-ce!28772
2019-05-28 08:16:26 +00:00
Vladimir Shushlin 4687ff7c9b Store Let's Encrypt private key in settings
Storing this key in secrets.yml was a bad idea,
it would require users using HA setups to manually
replicate secrets across nodes during update,
it also needed support from omnibus package

* Revert "Generate Let's Encrypt private key"
  This reverts commit 444959bfa0.

* Add Let's Encrypt private key to settings
  as encrypted attribute

* Generate Let's Encrypt private key
  in database migration
2019-05-28 04:47:34 +00:00
Shinya Maeda 7e05f3b78b Use source ref for pipeline webhook
When user uses Pipelines for merge requests, the pipeline is a run on
a merge request ref instead of branch ref. However, we should send
source ref as a webhook in order to respect the original behavior.
2019-05-28 09:29:59 +07:00
Thong Kuah f0efa2f91f Stop emitting AUTO_DEVOPS_DOMAIN Ci variable
Update documentation to reflect removal
2019-05-24 14:40:12 +12:00
Stan Hu bf8f4c135a Fix OmniAuth OAuth2Generic strategy not loading
In https://github.com/rails/rails/commit/83b767ce, Rails 5.1 removed
support for using a String to specify a middleware. When the
strategy_class argument is passed from the GitLab YAML config to Devise,
Devise passes the string value straight through to Rails, and GitLab
would crash with a NoMethodError inside ActionDispatch::MiddlewareStack.

To make this OmniAuth strategy work again, we normalize the arguments by
converting the strategy_class value into an actual Class.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/62216
2019-05-23 12:17:56 -07:00
James Lopez 30d915110f Fix issue importing members with owner access 2019-05-23 13:10:38 +00:00
Stan Hu 5c8cd42bbd Fix invalid visibility string comparison in project import
This resolves an "ArgumentError: comparison of String with 0 failed"
issue where the visibility_level is stored as a string in the project
import data because the value comes directly from the Web form. This
problem happened upon creating a project from a template or restoring a
project.

We now cast the value to an integer to guard against these kinds of
failures.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/61692
2019-05-22 11:32:23 -07:00
Nick Thomas 9f54fdcc89 Merge branch '61935-remove-code-left-over-from-when-clusters-were-always-project-specific' into 'master'
remove `Clusters::Platforms::Kubernetes#actual_namespace`

Closes #61935

See merge request gitlab-org/gitlab-ce!28391
2019-05-22 10:53:30 +00:00
Valery Sizov 07a3532594
Geo: Remove Gitlab::LfsToken::LegacyRedisDeviseToken implementation
We kept it for smooth update only
2019-05-22 16:11:08 +10:00
Tiger 101c4480b3 Remove legacy Kubernetes #actual_namespace
When Kubernetes clusters were originally built they could only
exist at the project level, and so there was logic included
that assumed there would only ever be a single Kubernetes
namespace per cluster. We now support clusters at the group
and instance level, which allows multiple namespaces.

This change consolidates various project-specific fallbacks to
generate namespaces, and hands all responsibility to the
Clusters::KubernetesNamespace model. There is now no concept of
a single namespace for a Clusters::Platforms::Kubernetes; to
retrieve a namespace a project must now be supplied in all cases.

This simplifies upcoming work to use a separate Kubernetes
namespace per project environment (instead of a namespace
per project).
2019-05-21 11:38:11 -05:00
Douglas Barbosa Alexandre d951f04719 Merge branch 'sh-fix-rugged-get-tree-entries-recursive' into 'master'
API: Fix recursive flag not working with Rugged get_tree_entries flag

Closes #61979

See merge request gitlab-org/gitlab-ce!28494
2019-05-20 20:19:29 +00:00
Stan Hu c1827f1c50 API: Fix recursive flag not working with Rugged get_tree_entries flag
Attempting to use the API endpoint
/projects/:id/repository/tree?recursive=true would only return a subset
of the results since the full recursive list wasn't actually being
returned.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/61979
2019-05-20 12:19:15 -07:00
Ryan Cobb 2334b07726 Revert "Merge branch '56850-add-new-unicorn-metrics' into 'master'"
This reverts merge request !27474
2019-05-20 14:34:00 +00:00
Mayra Cabrera c513434c9d Merge branch '49517-fix-notes-import-export' into 'master'
Include type to notes import / export

Closes #49517

See merge request gitlab-org/gitlab-ce!28401
2019-05-20 13:48:53 +00:00
Robert Speicher a3795a0cff Merge branch 'fix-too-many-loops-cron-error' into 'master'
Rescue too many loops cron error

Closes #58241

See merge request gitlab-org/gitlab-ce!28002
2019-05-17 17:12:50 +00:00
Zeger-Jan van de Weg a3f80f9f2e Add PoolRepository to the usage ping
PoolRepository is a relatively new model of which the counts could help
to further determine the priority of new features. Also gives some
insight into the number of forks customers have.
2019-05-17 09:10:29 +00:00
Heinrich Lee Yu e938d241ea Include type to notes import / export 2019-05-17 16:45:40 +08:00
Vladimir Shushlin 3c33724e2e Add Let's Encrypt client
Part of adding Let's Encrypt certificates for pages domains

Add acme-client gem

Client is being initialized by private key stored in secrets.yml
Let's Encrypt account is being created lazily.
If it's already created, Acme::Client just gets account_kid by
calling new_account method

Make Let's Encrypt client an instance
Wrap order and challenge classes
2019-05-16 09:32:25 +00:00
Douglas Barbosa Alexandre 36e15a866d Merge branch 'bvl-add-corrilation-id-to-all-exception' into 'master'
Add correlation id to all sentry errors

Closes #61699

See merge request gitlab-org/gitlab-ce!28253
2019-05-14 21:36:03 +00:00
Bob Van Landuyt 4c2f681490 Add correlation id to all sentry errors
Before this, we were only adding the correlation id to the "acceptable
exceptions" which we handle in code.

But we need to add it to the default raven context so the information
would be available for uncaught exceptions.
2019-05-13 13:27:41 -05:00
lmcandrew e48305c02c EE backport to CE of favicon_spec.rb 2019-05-13 13:23:04 +01:00
James Lopez 131f66c91a Update usage_data_spec to match EE 2019-05-09 16:32:18 +00:00
Patrick Derichs b145d9b873 Refactor spec to reload existing_label right after creation
This will avoid timestamp comparison issues later within equality check
with attributes
2019-05-08 12:39:54 +00:00
Ash McKenzie c5a9bc17a1 Merge branch '56850-add-new-unicorn-metrics' into 'master'
Resolve "Add new Unicorn metrics"

Closes #56850

See merge request gitlab-org/gitlab-ce!27474
2019-05-07 16:21:40 +00:00
Fabio Pitino c75b8ad4da Remove workaround by upgrading Fugit gem
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/58241
* Upgrade Fugit gem to 1.2.1 which recognizes cron expressions
for invalid days
* Reverted previously implemented workaround
* Leave test case which previously exposed the bug
2019-05-07 08:07:16 +01:00
Fabio Pitino 673ea5d2ac Rescue RuntimeError when "too many loops" occur
With this workaround we temporarily prevent an
exception from Fugit gem to be raised in
Gitlab::Ci::CronParser
2019-05-07 08:06:49 +01:00
Fabio Pitino 0edd1e6714 Write test that exposes bug with Fugit gem 2019-05-07 07:46:07 +01:00
Felipe Artur c40bad741f Fix issuables state_id nil when importing projects from GitHub
Issues and merge requests imported from GitHub are having state_id
set to null. This fixes the GitHub project importer and schedule
migrations to fix state_id.
2019-05-06 19:45:17 +00:00
Douwe Maan 908860087f Merge branch '30093-apply-bfg-object-map-to-database' into 'master'
Remove cleaned up OIDs from database and cache

Closes #30093

See merge request gitlab-org/gitlab-ce!26555
2019-05-06 11:33:11 +00:00
Stan Hu 02ca1577ea Merge branch 'jc-disconnect-git-alternates-client' into 'master'
Add client methods for DisconnectGitAlternates rpc

See merge request gitlab-org/gitlab-ce!27938
2019-05-06 10:54:16 +00:00
Bob Van Landuyt d7adc4cf3c Merge branch 'fix-merge-request-pipeline-exist-method' into 'master'
Fix duplicate merge request pipelines created by Sidekiq worker retry

See merge request gitlab-org/gitlab-ce!26643
2019-05-06 10:45:21 +00:00
Nick Thomas 8973f32d42
Remove cleaned up OIDs from database and cache 2019-05-06 11:35:03 +01:00
Dmitriy Zaporozhets d7eb886b9f Merge branch 'jej/session-stored-globaly' into 'master'
Session stored globally per request

See merge request gitlab-org/gitlab-ce!27658
2019-05-06 10:33:52 +00:00
Shinya Maeda d4d2cf7327 Fix merge request pipeline exist method
Refactor
2019-05-06 13:20:47 +07:00
Stan Hu f93b2e02a5 Run rubocop -a on CE files 2019-05-05 03:24:28 -07:00
John Cai 0e033b3124 Add client methods for DisconnectGitAlternates rpc
DisconnectGitAlternates is a Gitaly RPC that will delete a repository's
objects/info/alternates file. This commit adds client support for that
RPC.
2019-05-03 15:59:28 -07:00
Mayra Cabrera 89132bbdd6 Add gitlab-managed option to clusters form
When this option is enabled, GitLab will create namespaces and service
accounts as usual. When disabled, GitLab wont create any project
specific kubernetes resources

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/56557
2019-05-03 01:05:53 +00:00
Jan Provaznik d25239ee0b Use git_garbage_collect_worker to run pack_refs
PackRefs is not an expensive gitaly call - we want to
call it more often (than as part of full `gc`) because
it helps to keep number of refs files small - too many
refs file may be a problem for deployments with
slow storage.
2019-05-02 21:41:05 +00:00
Mayra Cabrera 5432f5480f Adds a way to start multiple manual jobs in stage
- Adds an endpoint on PipelinesController
- Adds a service that iterates over every build in a stage and
plays it.
- Includes 'play_manual' details on EntitySerializer
- Builds a new Stage state: PlayManual. An stage can take this status if
it has manual builds or an skipped, scheduled or manual status
- Includes FE modifications and specs
2019-05-02 18:27:35 +00:00
Jason Goodman 1b8646804d Update deployment chat message notification
Include link to user and commit title.
Rearrange text
2019-05-02 16:07:26 +00:00
Douwe Maan 86900f0000 Merge branch '53064-bypassing-pipeline-jobs-by-canceling-the-pipeline-and-manually-running-later-jobs' into 'master'
Disable retrying cancelled jobs

Closes #53064

See merge request gitlab-org/gitlab-ce!27503
2019-05-02 10:28:39 +00:00
Jan Provaznik 96750fac4c Add opentracing integration for graphql
Extends existing graphql's tracer with opentracing measurements. Because
it also adds Tracing::Graphql class (for opentracing), it also renames
Graphql::Tracing class to Graphql::GenericTracing to minimize confusion
with similar class names.
2019-05-02 07:01:14 +00:00