Commit Graph

131 Commits

Author SHA1 Message Date
GitLab Bot d5b4360051 Add latest changes from gitlab-org/gitlab@master 2019-11-26 03:06:24 +00:00
GitLab Bot 0a6ffb540e Add latest changes from gitlab-org/gitlab@master 2019-11-21 12:06:40 +00:00
GitLab Bot 29c01c6c91 Add latest changes from gitlab-org/gitlab@master 2019-11-14 03:06:25 +00:00
GitLab Bot 56d96ad7fa Add latest changes from gitlab-org/gitlab@master 2019-11-01 03:06:26 +00:00
GitLab Bot 8078bd185f Add latest changes from gitlab-org/gitlab@master 2019-11-01 00:06:02 +00:00
GitLab Bot 6653aab95d Add latest changes from gitlab-org/gitlab@master 2019-10-22 15:06:06 +00:00
GitLab Bot 94be244a9f Add latest changes from gitlab-org/gitlab@master 2019-10-15 03:06:19 +00:00
GitLab Bot b4e072cbaf Add latest changes from gitlab-org/gitlab@master 2019-10-11 06:06:27 +00:00
GitLab Bot 012e59c2af Add latest changes from gitlab-org/gitlab@master 2019-10-06 12:05:58 +00:00
GitLab Bot 83ad9ec8cc Add latest changes from gitlab-org/gitlab@master 2019-09-24 00:06:02 +00:00
Grzegorz Bizon 1068ec2e92 Merge branch '66741-remove-spec-rails_helper-rb' into 'master'
Remove spec/rails_helper.rb

Closes #66741

See merge request gitlab-org/gitlab-ce!32380
2019-09-03 13:48:17 +00:00
Ash McKenzie 19ff9d9899
Replace rails_helper.rb with spec_helper.rb
rails_helper.rb's only logic was to require
spec_helper.rb.
2019-08-30 12:26:18 +10:00
Thong Kuah e0826b0cb5 Override hostname when connecting via Kubeclient
Kubeclient uses rest-client. We hack into to access the net/http object
so that we can patch to connect to the resolved IP + set
hostname_override.

Add specs for discord. The discord integration also uses rest-client, so
since we patched rest-client, spec that the DNS rebinding protection
works
2019-08-21 10:49:22 +12:00
João Cunha f8821f828e Make use of Gitlab::Kubernetes
- refactor Knative and Prometheus
2019-08-14 20:02:37 +00:00
Thong Kuah b577825e54 Bump Helm to 2.14.3 and kubectl to 1.11.10 2019-08-13 00:07:27 +12:00
Tiger Watson 36a01a88ce Use separate Kubernetes namespaces per environment
Kubernetes deployments on new clusters will now have
a separate namespace per project environment, instead
of sharing a single namespace for the project.

Behaviour of existing clusters is unchanged.

All new functionality is controlled by the
:kubernetes_namespace_per_environment feature flag,
which is safe to enable/disable at any time.
2019-08-07 04:40:29 +00:00
Rémy Coutable f74387d298 Merge branch 'georgekoltsov/55474-outbound-setting-system-hooks' into 'master'
Add outbound setting for system hooks

See merge request gitlab-org/gitlab-ce!31177
2019-08-05 08:55:16 +00:00
Dylan Griffith 5027979b9b Implement Helm ResetCommand for removing Tiller
Also creates specs

Only allow Helm to be uninstalled if it's the only app

- Remove Tiller leftovers after reser command
- Fixes specs and offenses

Adds changelog file

Fix reset_command specs
2019-08-02 19:02:57 +00:00
George Koltsov e5e1c907c0 Add outbound requests setting for system hooks
This MR adds new application setting to network section
`allow_local_requests_from_system_hooks`. Prior to this change
system hooks were allowed to do local network requests by default
and we are adding an ability for admins to control it.
2019-08-02 15:39:18 +01:00
Dylan Griffith c9b0ac5fc4 GitLab Managed App ensure helm version uses tls opts
The TLS opts were missing from helm version command which meant that it
was just perpetually failing and hence wasting 30s of time waiting for a
command to be successful that was never going to be successful. This
never actually caused any errors because this loop will happily just
fail 30 times without breaking the overall script but it was just a
waste of installation time so now installing apps should be ~30s faster.
2019-07-15 11:38:40 +10:00
Hordur Freyr Yngvason 6971fd261d Give Knative serving permissions to service account
GitLab uses a kubernetes service account to perform deployments. For
serverless deployments to work as expected with externally created
clusters with their own knative installations (e.g. via Cloud Run), this
account requires additional permissions in the serving.knative.dev API
group.
2019-07-11 11:26:15 +00:00
Thong Kuah c7d50ddf55 Fix connection to Tiller error while uninstalling
Both the `install-<app>` and `uninstall-<app>` pods loads the
`values-content-configuration-<app>` configmap into the pod
(see `#volume_specification`). This configmap contains the cert
necessary to connect to Tiller. The cert though is only valid for 30
minutes.

So this fixes the bug where the configmap when uninstalling should be
updated as well.
2019-06-05 00:07:47 +12:00
Thong Kuah 33a765c17a Teach Helm::Api about #uninstall 2019-04-29 22:55:11 -07:00
Thong Kuah f8326af565 Implement commands to uninstall cluster applications
This is the backend part which just allows uninstalling Prometheus for
now.
2019-04-16 09:10:11 +10:00
Dylan Griffith c239bfcb17 Add more info logging to cluster apps
Log events so that it's easy to see
when different requests are starting.
2019-04-10 06:38:27 +00:00
Mayra Cabrera 135b78bb25 Revert "Merge branch 'bump_kubernetes_1_11_9' into 'master'"
This reverts merge request !26991
2019-04-05 20:22:50 +00:00
Douwe Maan a827092b65 Merge branch 'bump_kubernetes_1_11_9' into 'master'
Bump Helm to 2.13.1 and kubectl to 1.11.9

See merge request gitlab-org/gitlab-ce!26991
2019-04-05 09:43:46 +00:00
Thong Kuah 4ec16912b8 Autocorrect with RSpec/ExampleWording cop
- rewords examples starting with 'should'
- rewords examples starting with 'it'

Note: I had to manually fixup "onlies" to "only"
2019-04-05 08:43:27 +00:00
Thong Kuah 38b40f9164 Bump Helm to 2.13.1 and kubectl to 1.11.9 2019-04-05 00:08:11 +13:00
Yorick Peterse 03340f0987 Merge branch 'security-kubernetes-local-ssrf' into 'master'
Block local URLs for Kubernetes integration

See merge request gitlab/gitlabhq!2901
2019-03-04 18:36:50 +00:00
Thong Kuah af16fd687e Do not allow local urls in Kubernetes form
Use existing `public_url` validation to block various local urls. Note
that this validation will allow local urls if the "Allow requests to the
local network from hooks and services" admin setting is enabled.

Block KubeClient from using local addresses

It will also respect `allow_local_requests_from_hooks_and_services` so
if that is enabled KubeClinet will allow local addresses
2019-02-21 23:16:11 +13:00
Thong Kuah 8004508943 Bump helm and kubectl versions
Bump the helm and kubectl used in our Kubernetes integration, used e.g.
to install apps.

Note I have only bumped to the latest patch of the v1.11 series for
kubectl as GKE clusters are still on 1.10/1.11
2019-02-20 09:54:07 +13:00
Thong Kuah f67fc23727 Upgrade cluster applications, starting with runner 2019-02-07 21:40:55 +00:00
Thong Kuah f234aef994 Use http_max_redirects opt to replace monkeypatch
http_max_redirects was introduced in 4.2.2, so upgrade kubeclient.

The monkey-patch was global so we will have to check that all instances
of Kubeclient::Client are handled.

Spec all methods of KubeClient

This should provide better confidence that we are indeed disallowing
redirection in all cases
2019-01-25 16:48:37 +13:00
Rémy Coutable 3a2abc1d50
Enable the Layout/ExtraSpacing cop
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-01-24 13:05:45 +01:00
Takuya Noguchi db237d8431 Update Helm to 2.12.2 to address Helm client vulnerability
Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com>
2019-01-18 09:21:47 +09:00
Thong Kuah dc18272091 Modify service so that it can be re-run
If the service fails mid-point, then we should be able to re-run this
service. So, detect presence of any previously created Kubernetes
resource and update or create accordingly.

Fix specs accordingly. In the case of finalize_creation_service_spec.rb,
I decided to stub out the async worker rather than maintaining
individual stubs for various kubeclient calls for that worker.

Also add test cases for group clusters
2018-12-04 15:46:46 +13:00
Dylan Griffith 6a66e4a1f1 Don't remove failed install pods
We want to keep failed install pods around so that it is easier to debug
why a failure occured. With this change we also need to ensure that we
remove a previous pod with the same name before installing so that
re-install does not fail.

Another change here is that we no longer need to catch errors from
delete_pod! in CheckInstallationProgressService as we now catch the
ResourceNotFoundError in Helm::Api. The catch statement in
CheckInstallationProgressService was also probably too broad before and
should have been narrowed down simply to ResourceNotFoundError.
2018-11-27 12:54:32 +01:00
Amit Rathi 2945ab47d6 Fix usage data and install command spec 2018-11-20 16:48:23 +05:30
Amit Rathi 5e59c40baa Fixing more tests 2018-11-20 12:33:45 +05:30
Kamil Trzciński 5b75fcf6f2 Merge branch '49726-upgrade-helm-to-2-11' into 'master'
Resolve "Upgrade Helm Tiller Version Used By GitLab Managed Apps"

Closes #49726

See merge request gitlab-org/gitlab-ce!22693
2018-11-16 12:09:26 +00:00
Dylan Griffith 1244533b77 Remove unnecessary --tiller-namespace 2018-11-16 10:57:48 +00:00
Dylan Griffith 9b1a99b03d Use helm version loop instead of sleep in ClientCommand 2018-11-16 10:57:27 +00:00
Dylan Griffith fe1469e12f Upgrade helm to 2.11.0 and upgrade on every install 2018-11-16 10:22:40 +00:00
Kamil Trzciński 5059c15d06 Merge branch '53626-update-config-map-on-install-retry' into 'master'
Update config map if already present on install

Closes #53626

See merge request gitlab-org/gitlab-ce!22969
2018-11-15 12:33:40 +00:00
Chris Baumbauer 327892b23e Add knative client to kubeclient library 2018-11-15 11:39:43 +00:00
Thong Kuah 429d3e4952 Update config map if already present on install
When an application install fails, and the user retries install, the
configmap for the application will already exists. If so, we simply
update instead of create.
2018-11-15 22:39:42 +13:00
Thong Kuah 14b3033a0d Show HTTP response code for Kubernetes errors 2018-11-13 12:46:01 +00:00
Dylan Griffith da7836956e Improve Debugging Capabilities For Helm Application Installs 2018-11-09 15:03:24 +00:00
Chris Baumbauer 97441486ce Fix issue with missing knative cluster role binding, and cleanup tests 2018-11-05 23:55:18 -08:00
Chris Baumbauer aa44393e8f Merge branch 'master' into triggermesh-phase1-knative 2018-11-03 10:13:35 -07:00
Dylan Griffith d86f76320c Use our own docker image for helm install pods
This will reduce dependencies and failure points during installation. It
will also reduce security risks from untrusted dependencies being able
to effect all our users
2018-11-02 17:46:33 +00:00
Thong Kuah 5ede567d71 Incorporates Kubernetes Namespace into Cluster's flow 2018-11-02 15:46:15 +00:00
Chris Baumbauer dc078c2417 Merge branch 'master' into triggermesh-phase1-knative 2018-11-02 08:39:25 -07:00
Kamil Trzciński 9e6098a6db Merge branch 'refactor_gitlab_kube_client' into 'master'
Refactor Gitlab::KubeClient

Closes #52131

See merge request gitlab-org/gitlab-ce!22073
2018-10-25 12:02:11 +00:00
Chris Baumbauer 661fd213de Update helm version 1.7.2 -> 2.11.0 2018-10-24 15:13:33 -07:00
Thong Kuah a5419138fd Store version within SUPPORTED_API_GROUPS hash
This removes the ability to pass in a different version. We can instead
create a new entry in the SUPPORTED_API_GROUPS hash for a different
version if need be.
2018-10-23 23:52:34 +13:00
Thong Kuah 983c4a50d0 Remove api_groups from KubeClient constructor
We should have access to #core_client, #rbac_client,
and #extensions_client without having to pass in an awkward array.

Also change api_version to default_api_version, which allows us to use a
different version for an individual client. Special case for
apis/extensions which only go up to v1beta1

Makes #hashed_client private

Removes the #clients and #discover! methods which are un-used
2018-10-23 23:36:43 +13:00
Mayra Cabrera 1e4d615000 Add RoleBinding methods
Includes RoleBinding methods to Kubeclient and introduce a new lib class
to generate RoleBinding resources.

This MR is part of
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22011
2018-10-22 17:16:14 -05:00
Thong Kuah f6ff32d9bd Port Helm::Api EE extensions to CE
We will need these utility level code in the future to help upgrade all
helm applications.
2018-10-01 11:16:07 +13:00
Thong Kuah e6fd3f1986 Port UpgradeCommand to CE
This is a utility class that we will need in the future to update and
upgrade our managed helm applications, which we do plan to do in CE.
2018-09-27 18:15:04 +12:00
Thong Kuah a02e35308b Always create `gitlab` service account and service account token regardless of ABAC/RBAC
This also solves the async nature of the automatic creation of default
service tokens for service accounts. It also makes explicit which
service account token we always use.

create cluster role binding only if the provider has legacy_abac
disabled.
2018-09-14 16:26:51 +12:00
Thong Kuah 8c8ccd3167 Teach GitLab how to create Secret of type ServiceAccountToken
Add create_secret to KubeClient
2018-09-14 16:26:51 +12:00
Thong Kuah 6f2ad2b604 Enable Kubernetes RBAC for GitLab Managed Apps for existing clusters 2018-09-06 10:03:38 +00:00
Mayra Cabrera fc13409637 Resolve "Mutual SSL Auth For Helm TIller" 2018-08-07 12:39:38 +00:00
Thong Kuah 0cd76190de Lock helm charts to the VERSION already specified for each application.
Fix up VERSION for each of the applications
* There is no 0.0.1 helm version for jupyterhub. Use the latest version instead
* `:nginx` is not a valid chart version. Lock the ingress application GitLab installs to the latest chart version.
* Use the latest gitlab-runner chart to prevent GitLab installing older versions when users have been installing the lastest version

Always install from the VERSION and not the database `version` column.
This should fix cases like https://gitlab.com/gitlab-org/gitlab-ee/issues/6795 in
the instances where an install command failed previously, which locked the version
in the database to an older version.

Also, ensure that the version column is updated to the version we are
installing.

Add specs to show how previously failed appplications will be handled when the helm installation is run again

Add changelog entry
2018-08-02 10:22:38 +12:00
Mayra Cabrera b21730116e Revert "Merge branch '48098-mutual-auth-cluster-applications' into 'master'"
This reverts merge request !20801
2018-07-31 13:15:18 +00:00
Dylan Griffith 628ff2cf17 Use anonymous class in base_command_spec 2018-07-30 13:08:32 +00:00
Dylan Griffith cb21560b91 Ensure CA + Tiller cert never expire and Helm client cert expires quickly 2018-07-30 13:08:31 +00:00
Dylan Griffith 11edbcccef Get mutual SSL working with helm tiller 2018-07-30 13:08:30 +00:00
Dylan Griffith ce897f11a0 Refactor Cluster Application classes to pass through a has of config files
This is refactoring in the lead up to passing mutual TLS certs for helm applications. As such we expect all applications to need config files so we can remove the logic about which applications need and do not need this (ie `#config_map?`).
2018-07-28 09:50:31 +00:00
Tiago Botelho eb7c08c7a6 Backports relevant changes made in https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/5163 to CE 2018-07-25 09:39:39 +01:00
Mayra Cabrera 5841e92390 Resolve "Unable to install Prometheus on Clusters: 'Error: Chart incompatible with Tiller v2.7.0'" 2018-06-20 12:10:14 +00:00
Alessio Caiazza ce5723c849
Add alpine mirrors while operating on k8s cluster 2018-04-30 15:12:26 +02:00
Pirate Praveen 53e2987ba6 Replace KubeException with Kubeclient::HttpError (kubeclient 3.0) 2018-03-15 08:14:13 +01:00
Mayra Cabrera c607008ee5 Extend Cluster Applications to install GitLab Runner to Kubernetes cluster 2018-03-01 23:46:02 +00:00
Joshua Lambert 1f912880db Enable Prometheus metrics for deployed Ingresses 2018-02-05 12:46:31 +00:00
Mayra Cabrera 1827f398ad Resolve "Managed Prometheus deployment installs a lot of extra stuff" 2018-01-15 16:27:58 +00:00
Mayra Cabrera 0d4548026f Extend Cluster Applications to allow installation of Prometheus 2017-12-22 17:23:43 +00:00
Alessio Caiazza 8ec618a6ed Add Helm InstallCommand 2017-11-07 17:12:41 +01:00
Alessio Caiazza 3f261a499e Add k8s namespace implementation and tests 2017-11-07 14:05:58 +01:00