3228798b37
Refactor github import to reduce number of API calls
...
Ref #24073
2016-11-18 07:47:00 +10:00
eaa0303b3d
Cache imported GH labels no matter what
2016-11-04 15:59:53 +02:00
6a9fb20fe1
Fix applying GitHub-imported labels when importing job is interrupted
...
Closes #24075
2016-11-04 15:59:53 +02:00
44cbfeaba8
Merge branch 'adam-fix-labels-find-or-create' into 'master'
...
Pass user instance to Labels::FindOrCreateService or skip_authorization: true
## What does this MR do?
It fixes a bug described in #23694 when `project.owner` was passed to `Labels::FindOrCreateService`. `Labels::FindOrCreateService` expected a user instance and `project.owner` may return a group as well. This MR makes sure that we either pass a user instance or `skip_authorization: true`.
## Are there points in the code the reviewer needs to double check?
- places where we pass `skip_authorization: true`
## Does this MR meet the acceptance criteria?
- Tests
- [x] Added for this feature/bug
- [ ] All builds are passing
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html )
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides )
- [x] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits )
## What are the relevant issue numbers?
Fixes #23694
See merge request !7093
2016-10-28 15:01:59 +00:00
e2c603696a
Pass user instance to Labels::FindOrCreateService or skip_authorization: true
...
Do not pass project.owner because it may return a group and Labels::FindOrCreateService
throws an error in this case.
Fixes #23694 .
2016-10-28 11:31:45 +02:00
f68d9261b0
Fix typos
2016-10-28 11:30:20 +02:00
07275dd79c
Abstract the use of imported[!?] and {current,increment}_page in GitHub importer
2016-10-28 11:30:20 +02:00
bc6302b942
Use public_send instead of send
2016-10-28 11:30:20 +02:00
14fbd25d06
Modify GitHub importer to be retryable
2016-10-28 11:30:20 +02:00
ce38ae8ca1
Fix importing MR comments from GitHub
2016-10-27 15:42:29 +02:00
4259334fb6
Fix applying labels for GitHub-imported MRs
2016-10-27 15:42:28 +02:00
4f6d1c1d70
Rename Labels::CreateService to Labels::FindOrCreateService
2016-10-19 14:58:28 -02:00
f0ad0ceff5
Fix GitHub importer spec
2016-10-19 14:58:28 -02:00
d009d38ed6
User Labes::CreateService to create labels
2016-10-19 14:58:27 -02:00
333c02a8c8
Fix broken handling of certain calls in GitHub importer client
...
Closes #22998
2016-10-06 11:14:25 +02:00
b5f9541778
Skip wiki creation when GitHub project has wiki enabled
...
If the GitHub project repository has wiki, we should not create
the default wiki. Otherwise the GitHub importer will fail because
the wiki repository already exist.
This bug was introduced here
https://gitlab.com/gitlab-org/gitlab-ce/commit/892dea67717c0efbd6a28f763
9f34535ec0a8747
2016-10-04 13:06:45 -03:00
dca1acd6a6
Call after_remove_branch only once after importing all GitHub PRs
2016-09-27 20:45:07 +02:00
e30bfb809a
Import all GitHub comments after importing issues and PRs
2016-09-27 20:45:07 +02:00
dbcbbf262b
Speed up label-applying process for GitHub importing
...
* No need to re-fetch issues from GH to read their labels, the labels
are already there from the index request.
* No need to look up labels on the database for every application, so we
cache them.
2016-09-27 20:45:07 +02:00
395a9301b2
Process each page of GitHub resources instead of concating them then processing
...
This should avoid having large memory growth when importing GitHub repos
with lots of resources.
2016-09-27 20:45:07 +02:00
0c65112da7
modify github import JS and controller so we can now specify a namespace and/or name for a project.
...
- Fixed and added specs.
- Added different namespace options depending on user privilages
- Updated docs.
2016-09-20 10:14:39 +02:00
05b52e0f5e
Import GitHub release notes
...
# Conflicts:
# lib/gitlab/github_import/importer.rb
2016-09-13 14:01:22 +10:00
c87235a24d
Merge branch '21569-dont-add-created-by-for-matched-users' into 'master'
...
Only add original author tag line when importing from GitHub if there isn't a linked GitLab account
## What does this MR do?
If there we've found a linked GitLab user for a creator of an issue or comment don't add the 'Created By:' line.
## What are the relevant issue numbers?
Closes #21569
See merge request !6081
2016-09-12 20:08:33 +00:00
02761638f1
Avoid conflict with admin labels when importing GitHub labels
2016-09-09 11:22:38 -03:00
4a788f8a0c
Only add the original author if there isn't a linked GitLab account
2016-09-08 20:48:35 +10:00
9a9e22a88c
Rename `gl_user_id` to `gitlab_user_id` in importer classes
2016-09-08 11:57:22 +10:00
d308a3f433
Merge branch 'issue_19734' into 'master'
...
Project tools visibility level
## part of #19734
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG ) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md )
- [x] API support added
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides )
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits )
See merge request !5606
2016-09-01 15:28:14 +00:00
892dea6771
Project tools visibility level
2016-09-01 11:47:59 -03:00
9b376e772e
GitHub importer use default project visibility for non-private projects
2016-08-31 18:27:42 -03:00
80c4f10937
Don't touch Issue/Merge Request when importing GitHub comments
2016-08-31 13:20:28 -03:00
52daddc0b7
Use updated_at as the last updated date when importing from GitHub
2016-08-31 13:20:28 -03:00
2986de7c8c
Add readable error message when remote data could not be fully imported
2016-08-24 12:17:13 -03:00
3c09000e18
Does not halt the GitHub import process when an error occurs
2016-08-24 12:16:42 -03:00
0a1535a9f4
Prefixes removed branches name with PR number when importing PR from GH
2016-08-08 20:24:40 -03:00
167a0c7f0b
Remove SHA suffix for removed branches name when importing PR from GH
2016-08-08 19:39:59 -03:00
1e736fb527
Allow users to import cross-repository pull requests from GitHub
2016-08-04 17:16:50 -03:00
51cf14b37c
Does not need to disable GitHub webhooks since PRs are check out locally
2016-08-04 17:16:50 -03:00
ceff0c9170
Check out locally PRs where the source/target branch were removed
2016-08-04 17:16:50 -03:00
78a5de99e9
Memoize response from `has_rate_limit?` to avoid extra API call
2016-07-12 12:14:17 -03:00
be9262dd95
Checks if rate limit is enabled instead of stubbing response
2016-07-12 12:14:17 -03:00
38a58978aa
Fix GItHub client requests when rate limit is disabled
2016-07-12 12:14:17 -03:00
6ce25e7b4c
Rename MergeRequest methods that return commits or shas to be more clear and consistent
2016-07-06 18:50:58 -04:00
5f86a084f2
Consider that a GH exists only if both `ref`, and `sha` exist
2016-07-06 13:08:12 -03:00
4e5de3e6ee
Doesn't trigger Git hooks when cleaning up restored branches from GitHub
2016-07-06 13:08:12 -03:00
12aa1f898d
Import from Github using Personal Access Tokens.
...
This stands as an alternative to using OAuth to access a user's Github
repositories. This is setup in such a way that it can be used without OAuth
configuration.
From a UI perspective, the how to import modal has been replaced by a full
page, which includes a form for posting a personal access token back to the
Import::GithubController.
If the user has logged in via GitHub, skip the Personal Access Token and go
directly to Github for an access token via OAuth.
2016-06-30 18:48:17 +02:00
86359ec854
Refactor repository paths handling to allow multiple git mount points
2016-06-29 22:30:31 -04:00
7a34c7997b
Listing GH Webhooks doesn't stop import process for non GH admin users
2016-06-17 18:17:34 -03:00
cef3fd2899
Wrap all rate limiting logic inside GitHub API client
2016-06-09 12:43:06 -03:00
59e5f4705b
Check if GitHub rate limite API was reached before update Webhooks
2016-06-07 10:57:58 -03:00
8c5712a422
Merge branch 'gh-disable-webhooks'
2016-06-07 13:06:43 +02:00
acef028d85
Merge branch 'gh-fix-comments-on-diff'
2016-06-07 13:04:32 +02:00
67c02edccf
Move create! method from formatters to the BaseFormatter
2016-06-06 14:26:28 -03:00
7c072c76fc
Fix importer for GitHub comments on diff
...
Fix comments on diff after LegacyDiffNote was extracted from Note
2016-06-06 12:19:03 -03:00
9437b8a2e4
Import GitHub repositories respecting the API rate limit
...
While Octokit auto pagination set the page size to the maximum 100, and
seek to not overstep the rate limit. When the rate limit is reached its
raises an exception, and stop doing new requests.
Here we use a custom pattern for traversing large lists, so we can
check if we’ll reach the rate limit and wait the API to reset the rate
limit before making new requests.
2016-06-06 12:12:18 -03:00
659d5d4830
Disable Webhooks before proceeding with the GitHub import
2016-06-06 12:07:45 -03:00
3b4f03de8f
Ensure branch cleanup regardless of whether the import process succeeds
2016-06-02 19:48:11 -03:00
86cf9dd253
Enable Lint/LiteralInCondition rubocop cop
...
Checks of literals used in conditions.
See #17478
2016-05-30 12:34:25 +02:00
290aa75e6f
Remove branches that does exist on GitHub after PRs were imported
2016-05-13 17:55:43 -05:00
ebaa19c162
Fix validation method for Gitlab::GithubImport::PullRequestFormatter
2016-05-13 17:55:43 -05:00
795a7ca8f1
Extract GitHub branch formatter
2016-05-13 17:55:42 -05:00
e001bd5e3d
Import PRs where branch names were reused across PRs
2016-05-13 17:55:42 -05:00
7e1f14e215
Preserve commits/diff/comments for PRs that were merged on GitHub
2016-05-13 17:55:42 -05:00
3b50867550
Fix spec for Gitlab::GithubImport::PullRequestFormatter
2016-05-13 16:49:27 -05:00
9b4dc552cb
Import pull requests from GitHub where the target branch was removed
2016-05-13 16:49:27 -05:00
8532dc0d77
Import pull requests from GitHub where the source branch was removed
2016-05-13 16:49:27 -05:00
6fbf6b2936
Fix the line code when importing PR review comments from GitHub
...
Pull Request Review Comments are comments on a portion of the unified
diff.
2016-05-05 10:45:14 -03:00
eede032345
Backport GitHub Enterprise import support from EE
...
These changes were pulled from GitLab EE to support configuring
an alternative API URL than the default https://api.github.com .
In addition, the `verify_ssl` flag allows users to disable SSL cert
checking.
One modification: add a default `args` option if it does not exist
to avoid breaking existing configurations.
2016-04-26 21:00:55 -07:00
f2fe4af19d
Set GitHub milestones to Issue/Merge Request that were imported
2016-04-18 20:07:05 -03:00
06ec511164
Import milestones from GitHub
2016-04-18 20:07:04 -03:00
7462a8f90a
Apply GitHub labels to Issue/Merge Request that were imported
2016-04-18 17:14:59 -03:00
05a4f444c3
Import labels from GitHub
2016-04-18 17:14:59 -03:00
d13bba44f0
Use GitHub Issue/PR number as iid to keep references
...
With these changes we don’t lost the issue/pr references when importing
them to GitLab.
2016-04-18 12:15:50 -03:00
6aeb753ba8
fix github import issues
2016-04-01 15:51:54 +02:00
23146fca18
update bitbucket importer
2016-03-22 12:25:28 +01:00
030b139445
more refactoring
2016-03-21 15:11:05 +01:00
383cc84047
some refactoring based on feedback
2016-03-21 13:15:51 +01:00
5f86912ef0
Merge branches 'fix/project-import_url' and 'master' of gitlab.com:gitlab-org/gitlab-ce into fix/project-import_url
...
# Conflicts:
# db/schema.rb
2016-03-21 09:20:55 +01:00
37b00b16a5
Fix importing PR's from GitHub when the source repo was removed
2016-03-10 19:29:54 -03:00
735563329d
refactored a bunch of stuff based on MR feedback
2016-03-07 12:50:35 +01:00
c2b33d3b71
added import url exposer to construct URL withunencrypted credentials
2016-03-04 16:23:19 +01:00
06b36c00d5
some refactoring in the migration. Also fixed github import issue and updated spec
2016-03-04 12:21:53 +01:00
b58a2e30b2
Wrap errors on GitHub importer to raise Projects::ImportService::Error
2016-01-26 10:56:22 -02:00
4861b1bbfb
Fix import of GitHub's wiki when the repository has not been created
2016-01-21 13:11:38 -02:00
f4bdbecfd5
Fix cross projects detection when importing GitHub pull requests
2016-01-20 19:36:35 -02:00
89e8b82b63
Make sure the .git is at the end on Gitlab::GithubImport::WikiFormatter
2016-01-14 12:09:31 -02:00
60f8434a9d
Refactoring Gitlab::GithubImport::Importer
2016-01-14 12:09:31 -02:00
78f5eb94fb
Import GitHub wiki into GitLab
2016-01-14 12:09:31 -02:00
837a9065f0
Ensure that we're only importing local pull requests
2016-01-05 15:24:55 -02:00
98909dd12c
Generate separate comments when importing GitHub Issues into GitLab
2016-01-05 15:24:55 -02:00
dc72a8b305
Refactoring GithubImport::Importer
2016-01-05 15:24:55 -02:00
6c846ef83d
Extract methods to import comments on a GitHub Pull Request
2016-01-05 15:24:55 -02:00
d72b25811e
Doesn't import GitHub PR where branches were no longer available
2016-01-05 15:24:55 -02:00
f19bf0eaa7
Import comments on the diff of a GitHub Pull Request into GitLab
2016-01-05 15:24:55 -02:00
223f7f5345
Import comments on GitHub Pull Request into GitLab
2016-01-05 15:24:55 -02:00
95f1fe724a
Import GitHub Pull Requests into GitLab
2016-01-05 15:24:55 -02:00
e92ceb7b57
fix specs
2015-11-30 16:12:31 +02:00
e809669383
Fix github importer to handle empty issues
2015-11-18 11:02:25 +00:00
2e9c922dd1
Fix bug where only the first 15 Bitbucket issues would be imported.
...
Also fix a number of issues where author/reporter names were not available.
2015-09-01 08:50:20 -07:00
ed1d4fa477
Remove user OAuth tokens stored in database for Bitbucket, GitHub, and GitLab
...
and request them each session. Pass these tokens to the project import data.
This prevents the need to encrypt these tokens and clear them in case they
expire or get revoked.
For example, if you deleted and re-created OAuth2 keys for Bitbucket, you would get
an Error 500 with no way to recover:
```
Started GET "/import/bitbucket/status" for x.x.x.x at 2015-08-07 05:24:10 +0000
Processing by Import::BitbucketController#status as HTML
Completed 500 Internal Server Error in 607ms (ActiveRecord: 2.3ms)
NameError (uninitialized constant Import::BitbucketController::Unauthorized):
app/controllers/import/bitbucket_controller.rb:77:in `rescue in go_to_bitbucket_for_permissions'
app/controllers/import/bitbucket_controller.rb:74:in `go_to_bitbucket_for_permissions'
app/controllers/import/bitbucket_controller.rb:86:in `bitbucket_unauthorized'
```
Closes #1871
2015-08-23 09:23:44 -07:00
4a203aab57
Fix wrong order of issues when importing from github
...
See API details at https://developer.github.com/v3/issues/ .
Refs #1791
2015-06-28 04:02:21 +09:00
Andrew Smith
Ahmad Sherif
Douwe Maan
Adam Niedzielski
Douglas Barbosa Alexandre
James Lopez
Robert Speicher
Felipe Artur
Douwe Maan
Eric K Idema
Alejandro Rodríguez
Grzegorz Bizon
Stan Hu
Valery Sizov
Arseny Zarechnev
Hiroyuki Sato