Commit Graph

1383 Commits

Author SHA1 Message Date
Z.J. van de Weg f749fb7fe0 Improve style, add more tests 2016-11-18 11:38:54 +01:00
James Lopez c76ef84781 run pipeline worker in cycle analytics dev setup 2016-11-18 10:50:47 +01:00
James Lopez cd222357ad refactor events 2016-11-18 10:13:42 +01:00
Z.J. van de Weg 0d04724fa1 More coverage on service level 2016-11-18 10:00:40 +01:00
James Lopez e56c305be0 fix spec failures 2016-11-18 08:13:34 +01:00
Sean McGivern dd9cd24e36 Merge branch 'feature/24072-improve-importing-of-github-pull-requests' into 'master'
Improve importing of github pull requests

Closes #24072

See merge request !7241
2016-11-17 23:40:47 +00:00
Andrew Smith 3228798b37
Refactor github import to reduce number of API calls
Ref #24073
2016-11-18 07:47:00 +10:00
Z.J. van de Weg 778b5a5a04 Merge remote-tracking branch 'origin/master' into zj-slash-commands-mattermost 2016-11-17 21:56:38 +01:00
Z.J. van de Weg 166ee0965b More refactoring, push present to base command 2016-11-17 21:44:26 +01:00
Z.J. van de Weg 1607efa400 Add tests for increased converage 2016-11-17 21:34:24 +01:00
Z.J. van de Weg 6737ada0c8 Remove some commands for now 2016-11-17 21:34:24 +01:00
Z.J. van de Weg d4def9cbcd Incorporate feedback, improve presenter class
[ci skip]
2016-11-17 21:34:24 +01:00
Z.J. van de Weg 1b4fdb9893 Rename from service, and move to lib/gitlab 2016-11-17 21:34:24 +01:00
Kamil Trzcinski bb988fd214 Add most of specs for chat names 2016-11-17 21:34:23 +01:00
Rémy Coutable 726a414169 Merge branch 'chat-name-authorize' into 'master'
Allows to authorize chat user against GitLab.

This is needed for: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7438

See merge request !7450
2016-11-17 18:08:24 +00:00
James Lopez d5b673da79 more refactoring and added some auth checks 2016-11-17 18:00:37 +01:00
James Lopez af80fcb477 fix specs 2016-11-17 14:14:23 +01:00
James Lopez ed39d61d74 refactor events facade so it uses separate classes and refactor query stuff 2016-11-17 13:22:27 +01:00
James Lopez cbd7d00039 added custom date helper and spec and fixed some unrelated spec failures 2016-11-17 08:22:59 +01:00
James Lopez cf2dcf043c Refactor all query config stuff into separate classes and added specs 2016-11-17 08:22:59 +01:00
James Lopez f93607a305 get rid of light url builder and fix wrong spec 2016-11-17 08:22:58 +01:00
James Lopez 73e9ec6319 serialize all the things! 2016-11-17 08:22:58 +01:00
James Lopez 8743e59f78 get all stages to use serlalizers - apart from plan - WIP 2016-11-17 08:22:58 +01:00
James Lopez f560099751 fix issue events and related spec - now using generic serializer for a hash 2016-11-17 08:22:58 +01:00
James Lopez 3b179bc37b WIP - refactored events to use build serializer, related spec passing 2016-11-17 08:22:58 +01:00
James Lopez 7ac7cfeb75 refactored and added missing spec to light URL builder 2016-11-17 08:22:57 +01:00
James Lopez 83130ae877 Updated production events with new fields 2016-11-17 08:22:57 +01:00
James Lopez beeb64610e Refactored staging events, added missing fields and fixed specs 2016-11-17 08:22:57 +01:00
James Lopez bd31f24c54 Added branch option to test events
Also fixed test events ordering issue
2016-11-17 08:22:57 +01:00
James Lopez eccb6a5e92 Refactored test events
Now test events return the builds instead a list of pipelines to avoid calling pipeline.builds per each and get the info. Also, added missing fields/data, URLs, and fixed specs in events spec.
2016-11-17 08:22:57 +01:00
James Lopez 11bad33a42 added missing fields to code events and updated spec 2016-11-17 08:22:57 +01:00
James Lopez 1b5b2eac22 added missing fields to plan events and updated spec 2016-11-17 08:22:56 +01:00
James Lopez 8f7266cd4b added missing fields to issue. Also, added a light url builder to add URLs easily from arel. Updated specs. 2016-11-17 08:22:56 +01:00
James Lopez ca9ae8bf63 add email to user related queries so it can be used for displaying avatar in the UI 2016-11-17 08:22:56 +01:00
James Lopez 3cdc9af78e added production events and related spec 2016-11-17 08:22:55 +01:00
James Lopez 1f701cb5e2 added staging events and spec 2016-11-17 08:22:55 +01:00
James Lopez f8acc7ea77 fixing spec failures 2016-11-17 08:22:55 +01:00
James Lopez d99cec7f55 review events - spec and logic 2016-11-17 08:22:55 +01:00
James Lopez ebd5ced7eb Added test events specs and logic. Also fixed some SQL and refactored the pipeline worker spec. 2016-11-17 08:22:55 +01:00
James Lopez 1a4ff5d720 Added code events spec and logic. Also fixed SQL issues and refactored the code a bit. 2016-11-17 08:22:55 +01:00
James Lopez 72660d58af plan events and spec working. Also added some TODOs to the code. 2016-11-17 08:22:54 +01:00
James Lopez 3b5d947730 commit events and spec 2016-11-17 08:22:54 +01:00
James Lopez 1d6068a1c3 fix specs - issue events working 2016-11-17 08:22:54 +01:00
James Lopez c545968ece fix timestamp diff and spec 2016-11-17 08:22:54 +01:00
James Lopez 847d2796cf fixed spec and SQL query 2016-11-17 08:22:54 +01:00
James Lopez 470e39d64f WIP - refactored some arel queries 2016-11-17 08:22:54 +01:00
James Lopez ec7db295d2 Started refactoring stuff to add events to cycle analytics
- Refactored cycle analytics class to extract DB logic
- Reuse logic in new events fetcher
- Started adding cycle analytics events class and spec (still not functional)
2016-11-17 08:22:54 +01:00
http://jneen.net/ 1734124433 fix rubocop failures 2016-11-17 11:59:44 +09:00
http://jneen.net/ f7b0692912 add parsing support for incoming html email 2016-11-17 11:59:44 +09:00
Kamil Trzcinski 6f714dfb4a Improve code design after code review 2016-11-16 23:10:27 +01:00
Kamil Trzcinski d2d64393ed Merge remote-tracking branch 'origin/master' into chat-name-authorize 2016-11-16 16:05:03 +01:00
Kamil Trzcinski d64183e1fa Add most of specs for chat names 2016-11-16 14:56:30 +01:00
Valery Sizov a5632e802b Search for a filename in a project 2016-11-16 14:25:54 +02:00
Douwe Maan fdd7e3f6e9 Merge branch 'stanhu/gitlab-ce-fix-error-500-with-mr-images' into 'master'
Fix Error 500 when creating a merge request that contains an image that was deleted and added

_Originally opened at !4816 by @stanhu._

- - -

## What does this MR do?

This MR fixes an Error 500 when creating a merge request that contains an image that was deleted and added. Before, when displaying the before and after image, the code would always retrieve the image from the parent commit. However, in a diff, this could cause two different problems:
The "before" image may not actually be the image you want to compare against (regression of #14327)
It may appear as though a file was modified when it was really just added during the diff

## Are there points in the code the reviewer needs to double check?

There may be a more elegant to fix this bug.

## What are the relevant issue numbers?

Closes #3893, gitlab-org/gitlab-ee#678

See merge request !7457
2016-11-15 10:22:36 +00:00
Douwe Maan 3a3b06b4fa Merge branch 'fix_saml_ldap_link' into 'master'
Omniauth auto link LDAP user falls back to find by DN when user cannot be found by uid

Unfortunately, SAML IDs can be an LDAP UID, DN, or something else entirely. UID and DN are most common, though. This adds a fallback scenario so we first try to find a matching LDAP user by UID, then by DN. This will fix a problem for the customer in https://gitlab.zendesk.com/agent/tickets/43298

See merge request !7002
2016-11-15 10:20:11 +00:00
Stan Hu 0f61bb2dc5
Fix Error 500 when creating a merge request that contains an image that was deleted and added
Steps to reproduce:

1. Start with a repo with an image
2. Add a commit to delete the image
3. Add another commit to replace the image with another image

In a diff comparison, we really just compare about what the image was before the diff, not
the direct parent of the last commit. This MR fixes that.

Closes #3893, gitlab-org/gitlab-ee#678

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-11-14 15:57:01 +01:00
Kamil Trzciński 39ec62ea98 Merge branch 'fix/improve-naming-convention-in-ci-config' into 'master'
Improve naming convention in ci configuration module

## What does this MR do?

This MR improves the naming convention in CI configuration module to reflect the domain design better.

## What are the relevant issue numbers?

Related to #15060

See merge request !7448
2016-11-14 11:59:11 +00:00
Grzegorz Bizon a761d29366 Change name of method for setting CI config entries 2016-11-14 10:51:56 +01:00
Grzegorz Bizon c701468390 Improve naming convention in ci configuration module 2016-11-14 10:31:45 +01:00
Drew Blessing c50b98da72 Centralize LDAP config/filter logic
Centralize all LDAP config logic in `GitLab::LDAP::Config`. Previously,
some logic was in the Devise initializer and it was not honoring the
`user_filter`. If a user outside the configured `user_filter` signed
in, an account would be created but they would then be denied access.
Now that logic is centralized, the filter is honored and users outside
the filter are never created.
2016-11-11 15:58:33 -06:00
Patricio Cano e840749b84 Refactored Sidekiq Throttler and updated documentation 2016-11-11 11:14:58 -06:00
Drew Blessing 3cff3a2e5b Omniauth auto link LDAP user falls back to find by DN when user cannot be found by uid 2016-11-10 15:47:40 -06:00
Patricio Cano 208530494e Refactored initializer code to its own class and added tests 2016-11-10 11:38:11 -06:00
Sean McGivern de8c2b79c0 Merge branch 'ldap_check_bind' into 'master'
Improve ldap:check errors

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/21621. 

See merge request !6601
2016-11-09 15:37:23 +00:00
Robert Speicher b0088b527e
Merge branch '23403-fix-events-for-private-project-features' into 'security'
Respect project visibility settings in the contributions calendar

This MR fixes a number of bugs relating to access controls and date selection of events for the contributions calendar

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/23403

See merge request !2019

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-11-09 12:27:41 +01:00
Douwe Maan b0bf92140f
Merge branch 'fix-unathorized-cloning' into 'security'
Ensure external users are not able to clone disabled repositories.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/23788

See merge request !2017

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-11-09 12:27:17 +01:00
Douwe Maan bf061d0aff
Merge branch 'issue_23548_dev' into 'master'
disable markdown in comments when referencing disabled features

fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/23548

This MR prevents the following references when tool is disabled:

- issues
- snippets
- commits - when repo is disabled
- commit range - when repo is disabled
- milestones

This MR does not prevent references to repository files, since they are just markdown links and don't leak
information.

See merge request !2011

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-11-09 12:25:17 +01:00
Drew Blessing dc30783057 Introduce better credential and error checking to `rake gitlab:ldap:check`
It was previously possible for invalid credential errors to go unnoticed
in this task. Users would believe everything was configured correctly and
then sign in would fail with 'invalid credentials'. This adds a specific
bind check, plus catches errors connecting to the server. Also, specs :)
2016-11-08 15:46:10 -06:00
Douwe Maan 9c3f3e9e35 Merge branch 'use-separate-token-for-incoming-email' into 'master'
Use separate email-friendly token for incoming email

See merge request !5914
2016-11-08 10:47:45 +00:00
tiagonbotelho 011e561bfa implements reset incoming email token on issues modal and account page,
reactivates all tests and writes more tests for it
2016-11-07 15:56:18 +00:00
Douwe Maan 9d51421346 Use separate email-friendly token for incoming email and let incoming
email token be reset
2016-11-07 15:55:42 +00:00
Elan Ruusamäe b9176afbc8 update existing test 2016-11-06 20:27:58 +02:00
Jacob Vosmaer 6bcc52a536 Refine Git garbage collection 2016-11-04 14:30:11 +01:00
Douwe Maan dc9b3db8b0 Merge branch 'fix/import-export-symlink-vulnerability' into 'security'
Fix symlink vulnerability in Import/Export

Replaces https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2018 made by @james

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/23822

See merge request !2022

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-11-03 16:04:18 +01:00
Douwe Maan cfb511ea69 Merge branch 'fix/import-projectmember-security' into 'security'
Fix Import/Export foreign key issue to do with project members

Cleans-up any foreign keys in `ProjectMember` - same as we do with the rest of the models when importing.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/23837 and https://gitlab.com/gitlab-org/gitlab-ce/issues/23739

See merge request !2020

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-11-03 16:04:18 +01:00
Rémy Coutable d733a966e4 Merge branch 'allow-owner-to-run-ci-builds' into 'master'
Allow owners to fetch source code in CI builds

Due to different way of handling owners of a project, they were not allowed to fetch CI sources for project.

This adds a separate code path for handling owners, that are not admins.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/23437

See merge request !6943
2016-11-02 09:48:22 +00:00
Felipe Artur f4e31b820e Fix project features default values 2016-11-01 11:03:56 -02:00
Kamil Trzcinski 517dd4a3f3 Allow owners to fetch source code in CI builds
Due to different way of handling owners of a project, they were not allowed to fetch CI sources for project.
2016-11-01 09:37:20 +01:00
Stan Hu d306b0d7c2 Merge branch 'use-optimistic-locking' into 'master'
Use optimistic locking

## What does this MR do?
Removes the usage of pessimistic locking in favor of optimistic which is way cheaper and doesn't block database operation.

Since this is very simple change it should be safe. If we receive `StaleObjectError` message we will reload object a retry operations in lock.

However, I still believe that we need this one: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7005 as this will reduce a load on Database and FS.
This changes a behavior from:

### Pesimistic locking (previous behavior)

#### For updating
1. SELECT * FOR UPDATE (other updates wait on this)
2. we update ci_pipeline
3. latest_build_status
4. enqueue: (use: transition :created -> :pending)
5. [state_machine] we are in  state created, we can go to pending
6. [state_machine] ci_pipeline.status = created
7. [state_machine] ci_pipeline.save
8. [state_machine] after_transition: (if for success): PipelineSuccessWorker on Sidekiq
9. release DB lock

#### If no update is required
1. SELECT * FOR UPDATE (other updates wait on this)
2. we update ci_pipeline
3. latest_build_status
4. we are in pending, we can't transition to pending, because it's forbidden
5. release DB lock

### Optimistic locking (implemented by this MR)

#### For updating
1. latest_build_status
2. enqueue: (use `transition :created -> :pending`)
3. [state_machine] we are in state created, we can go to pending
4. [state_machine] ci_pipeline.status = created
5. [state_machine] ci_pipeline.save
6. [state_machine] [save] where(lock_version: ci_pipeline.lock_version).update_all(status: :created, updated_at: Time.now)
7. [state_machine] [save] unless we_updated_row then raise ObjectInconsistentError

#### If no update is required
1. we update ci_pipeline
2. latest_build_status
3. we are in pending, we can't transition to pending, because it's forbidden

## Why was this MR needed?
We have been seeing a number of problems when we migrated Pipeline/Build processing to Sidekiq. Especially we started seeing a lot of blocking queries.

We used a pessimistic locking which doesn't seem to be required. This effectively allows us to fix our issues with blocked queries by using more efficient method of operation.

## What are the relevant issue numbers?
Issues: https://gitlab.com/gitlab-com/infrastructure/issues/623 and https://gitlab.com/gitlab-com/infrastructure/issues/584, but also there's a bunch of Merge Requests that try to improve behavior of scheduled jobs.

cc @pcarranza @yorickpeterse @stanhu

See merge request !7040
2016-10-28 14:41:24 +00:00
Ahmad Sherif 14fbd25d06 Modify GitHub importer to be retryable 2016-10-28 11:30:20 +02:00
Sean McGivern 66870960af Merge branch 'fix/gh-import-bugs' into 'master'
Fix couple of GitHub importing bugs

Fix a bug in GH comment importing and label applying for imported MRs.

See merge request !7139
2016-10-28 08:48:08 +00:00
Douwe Maan e5950e687c Merge branch '21645-mail_room_sentinel' 2016-10-27 15:51:34 +01:00
Ahmad Sherif 4259334fb6 Fix applying labels for GitHub-imported MRs 2016-10-27 15:42:28 +02:00
Kamil Trzcinski 2822526e7b Make retry_lock to not be infinite 2016-10-27 12:34:35 +01:00
Kamil Trzcinski d8aed6a27b Fix optimistic locking 2016-10-26 11:37:23 +02:00
Kamil Trzcinski 47b2add4f6 Add tests for optimistic locking 2016-10-26 11:37:23 +02:00
Gabriel Mazetto f9126fbe0a Updated mail_room and added sentinel support to Reply by Email 2016-10-26 05:02:47 +02:00
Yorick Peterse 3b4af59a5f
Don't schedule ProjectCacheWorker unless needed
This changes ProjectCacheWorker.perform_async so it only schedules a job
when no lease for the given project is present. This ensures we don't
end up scheduling hundreds of jobs when they won't be executed anyway.
2016-10-25 16:02:36 +02:00
Lin Jen-Shin a4ef4244d4 Preserve note_type and position for notes from emails
Closes #23208
2016-10-20 16:18:53 +08:00
James Lopez 891e5f4851 Update specs to cope with new label types and priorities
Fixed all related specs and also changed the logic to handle edge cases. This includes exporting and exporting of group labels, which will get associated with the new group (if any) or they will become normal project labels otherwise.

Found other issues to do with not being able to import all labels at once in the beginning of the JSON - code was much simpler when we import all labels and milestones associated to a project first, then the associations will find the already created labels instead of creating them from the associations themselves.
2016-10-19 14:58:28 -02:00
Douglas Barbosa Alexandre f0ad0ceff5 Fix GitHub importer spec 2016-10-19 14:58:28 -02:00
Douglas Barbosa Alexandre 7e11ca86fd Reuse LabelsFinder on Issueable#add_labels_by_names 2016-10-19 14:58:25 -02:00
James Lopez 723e576782 fix rubocop warning 2016-10-19 14:58:25 -02:00
James Lopez 77b7bfd463 Fix import/export labels to cope with project and group labels. Added relevant specs. 2016-10-19 14:58:25 -02:00
Douglas Barbosa Alexandre 07709c5576 Unfold references for group labels when moving issue to another project 2016-10-19 14:58:25 -02:00
Douglas Barbosa Alexandre e00c739f97 Add Label attributes: type, and group_id to safe model attributes 2016-10-19 14:58:25 -02:00
Douwe Maan 7a650d1578 Merge branch 'backport-git-access-spec-changes' into 'master'
Backport git access spec changes from EE

https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/645#note_16391185

See merge request !6961
2016-10-19 09:37:58 +00:00
Rémy Coutable f0c7e6713f Merge branch '22191-delete-dynamic-envs-mr' into 'master'
Delete dynamic environments

- Adds "close environment" action to a merge request
- Adds tabs to environments list
- Adds close button to each environment row in environments list
- Replaces Destroy button with Close button inside an environment
- Adds close button to builds list inside an environment

#### Configuration

In order to enable stopping environments a valid `.gitlab-ci.yml` syntax has to be used:

```
review:
  environment:
    name: review/$app
    on_stop: stop_review

stop_review:
  script: echo Delete My App
  when: manual
  environment:
    name: review/$app
    action: stop
```

This MR requires that `stop_review` has to have: `when`, `environment:name` and `environment:action` defined.
The next MR after this one will verify that and enforce that these settings are configured.

It will also implicitly configure these settings, making it possible to define it like this:

```
review:
  environment:
    name: review/$app
    on_stop: stop_review

stop_review:
  script: echo Delete My App
```

Closes #22191 

See merge request !6669
2016-10-19 07:53:05 +00:00
Kamil Trzciński 2f7e1c0ead Merge branch 'pipeline-emails' into 'master'
Add a new pipeline email service

## What does this MR do?

Add a new pipeline email service

## What are the relevant issue numbers?

Closes #3976 

## Remaining tasks

* [x] Preserve `&middot;` and `&nbsp;`
* [x] Use XHTML 1.0
* [ ] Use the same layout (`app/views/layouts/notify.html.haml`)
* [ ] Digest or not (assets or public)
* [x] A similar email for succeeded pipeline
* [x] Plain text versions for both emails

## Screenshots (if relevant)

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6019#note_16594345

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
  - [x] `PipelinesEmailService`
  - [x] `SendPipelineNotificationService`

See merge request !6019
2016-10-18 16:56:13 +00:00
Sean McGivern cc6d42861b Backport git access spec changes from EE
These were introduced in:
<https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/645>
2016-10-18 17:28:57 +01:00