Added helper methods for database migrations
These helpers can ultimately be used to write migrations that don't
require downtime.
See #15464 for more information.
See merge request !3860
Add container registry support
Tasks:
- [x] Merge docker/distribution authentication service: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3787
- [x] Implement Docker Registry API
- [x] Show a list of docker images in GitLab
- [x] Remove registry repository on project deletion
- [x] Support project rename, move and namespace rename
- [x] Use token when connecting the registry
- [x] Allow to delete images from GitLab
- [x] Support pushing from GitLab CI (gitlab-ci-token / $CI_BUILD_TOKEN)
- [x] Support GitLab Runner pulling for public repositories
- [ ] Support GitLab Runner pulling for private repositories
- [x] Add tests for Docker Registry API
- [x] Add tests for a views
- [x] Make texts nicer
- [x] Implement a backup support
- [ ] Create administration documentation https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4141
- [ ] Create user documentation
See merge request !4040
So we extend Gitlab::Email::Receiver for this new behaviour,
however we might want to split it into another class for better
testing it.
Another issue is that, currently it's using this to parse project
identifier:
Gitlab::IncomingEmail.key_from_address
Which is using:
Gitlab.config.incoming_email.address
for the receiver name. This is probably `reply` because it's used
for replying to a specific issue. We might want to introduce another
config for this, or just use `reply` instead of `incoming`.
I'll prefer to introduce a new config for this, or just change
`reply` to `incoming` because it would make sense for replying to
there, too.
The email template used in tests were copied and modified from:
`emails/valid_reply.eml` which I hope is ok.
Use the relative url prefix for links in Wiki
Retry of gitlab-org/gitlab-ce!4026
@rymai !4050 solved all other problems how it looks like. I [tested](ff01eca7b5/builds) this with ruby22
Fixes#17071
See merge request !4131
Enable the Rubocop DeprecatedClassMethods cop
This reports uses of `File.exists?` and `Dir.exists?`, which were both
deprecated in Ruby and will eventually be removed in favor of `.exist?`.
Also fixes all existing uses of the deprecated methods.
See merge request !4087
Because method call timings are inclusive (that is, they include the
time of any sub method calls) this would lead to the total method
execution time often being far greater than the total transaction time.
Because this is incredibly confusing it's best to simply _not_ track the
total method execution time, after all it's not that useful to begin
with.
Fixesgitlab-org/gitlab-ce#17239
These helpers can be used to perform migrations without taking down the
entire application.
For example, the method "add_column_with_default" can be used to add a
new column with a default value without locking the entire table.
Only generate repository push email once
The repository push email can be very expensive to generate, especially
with syntax-highlighted diffs. Instead of generating the email for each
recipient, generate one email object and reset the Message-Id and To
headers for each recipient. (Cloning would also be expensive in the case
of large emails, although probably not as bad as generating from
scratch.)
Closes#17270.
See merge request !4070
Enable the StringReplacement cop.
Also fix one use of `gsub` that would be faster as `delete`.
Use `tr` instead of `gsub` when you are replacing the same number of characters. Use `delete` instead of `gsub` when you are deleting characters.
See merge request !3960
The repository push email can be very expensive to generate, especially
with syntax-highlighted diffs. Instead of generating the email for each
recipient, generate one email object and reset the Message-Id and To
headers for each recipient. (Cloning would also be expensive in the case
of large emails, although probably not as bad as generating from
scratch.)
Also fix one use of `gsub` that would be faster as `delete`.
Use `tr` instead of `gsub` when you are replacing the same number of
characters. Use `delete` instead of `gsub` when you are deleting
characters.
If you attempt to push thousands of branches at once, the 60-second timeout
will occur because GitAccess checking does a lot of work to check if the
user has permission to push to a branch. This changes does two things:
1. Instead of making 1 DB query per branch push, use a memoized list of protected branches to check
2. Memoize what permissions the user has to perform on this project
On a test of 10,000 branch pushes, this prevents gitlab-shell from hitting the 60-second
timeout.
Closes#17225
This reports uses of `File.exists?` and `Dir.exists?`, which were both
deprecated in Ruby and will eventually be removed in favor of
`.exist?`. Also fixes all existing uses of the deprecated methods.
Use a custom Loofah scrubber since sanitize 2.x transformers are inadequate
to handle case-sensitive SVG attributes. sanitize parses documents as HTML
instead of XML, which causes all SVG attribute names (e.g. viewBox) to be downcased.
* SVG element list: https://www.w3.org/TR/SVG/eltindex.html
* SVG attribute list: https://www.w3.org/TR/SVG/attindex.htmlCloses#14555
Previously this method would directly receive the output of tag_name().
This method could either return a String or nil. In the previous setup
this would somehow magically work but because Rugged::TagCollection#[]
doesn't accept nil values it started to fail.
To work around this the elsif in change_access_check() assigns the
result of tag_name() to a local and then _only_ calls protected_tag?()
if the tag name is not nil. The extra parenthesis are put in place to
ensure that things are parsed correctly, without these the code would be
parsed as follows:
elsif tag_ref = (tag_name(ref) && protected_tag(tag_ref))
During runtime this would basically resolve to:
elsif tag_ref = (tag_name(ref) && protected_tag(nil))
This is because when you refer to the variable you're assigning _in_ the
assignment Ruby returns nil instead of raising an error.
Use Rugged's TagCollection#create instead of gitlab-shell's Repository#add_tag for better performance
This was originally opened at !1757 by @pcarranza but I changed it to use Rugged instead of gitlab_git, following @DouweM's request.
Once this is merged, https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/26 will be mergeable too.
See merge request !3745
Use ActionDispatch Remote IP for Akismet checking
Previously all remote IPs appeared at 127.0.0.1, which made Akismet
not very useful. Using the ActionDispatch Remote IP (http://api.rubyonrails.org/classes/ActionDispatch/RemoteIp.html)
should provide more reliable results.
Closes#16629
See merge request !3961
These changes were pulled from GitLab EE to support configuring
an alternative API URL than the default https://api.github.com.
In addition, the `verify_ssl` flag allows users to disable SSL cert
checking.
One modification: add a default `args` option if it does not exist
to avoid breaking existing configurations.
Fix setting of "action" for Grape transactions
After wondering why we're not getting all the data in Grafana that I
wanted I realized I wasn't setting the "action" field properly here.
See merge request !3842
Merely setting the "action" tag will only result in the transaction
itself containing a value for this tag. To ensure other metrics also
contain this tag we must set the action using Transaction#action=
instead.
This allows users to configure the number of points stored in a single
UDP packet. This in turn can be used to reduce the number of UDP packets
being sent at the cost of these packets being somewhat larger.
The default setting is 1 point per packet so nothing changes for
existing users.
By using Module#prepend we can define a Module containing all proxy
methods. This removes the need for setting up crazy method alias chains
and in turn prevents us from having to deal with all that madness (e.g.
methods calling each other recursively).
Fixesgitlab-org/gitlab-ce#15281
Fix revoking of authorized OAuth applications
Users were not able to revoke access to authorized OAuth applications. Clicking the "Revoke" button would result in a 404 page, and the application would still be authorized.
Added a spec and also found that the `gon` variables were not being set for this view.
Closes#14370
See merge request !3690
Allow Omniauth providers to be marked as external
Related to #4009
With this MR we will be able to allow the user to set which Omniauth Providers they would like to have as external. All users login in via these providers will be marked as external, even if they already had an account before. If the provider is removed form the list of external providers, the users will be marked as internal at their next login.
MR for Omnibus: gitlab-org/omnibus-gitlab!727
/cc @dblessing @DouweM
See merge request !3657
One use case for this is manually setting the "action" tag for Grape API
calls. Due to Grape running blocks there are no human readable method
names that can be used for the "action" tag, thus we have to set these
manually on a case by case basis.
Add proper nil and error handling to SAML login process
While writing the feature that would allow certain Omniauth providers to be marked as external I noticed that there is a scenario where the `gl_user` method can return `nil` and if this is not properly checked, it will lead to exceptions that will cause 500 errors. It is quite easy to land in this scenario, so I added `nil` checks.
I also noticed that the `saml` method in the `omniauth_callbacks_controller.rb` file lacked a `rescue` for `Gitlab::OAuth::SignupDisabledError`, which can happen if the default configuration from `1_settings.rb` is applied. So I also added this check.
See merge request !3609
If the measure method uses Transaction.current directly the SQL
subscriber (Subscribers::ActiveRecord) will add timings of queries
triggered by DB cleaner.
This makes it easier to query, simplifies the code, and makes it
possible to figure out what transaction the data belongs to (simply
because it's now stored _in_ the transaction).
This new setup keeps track of both the real/wall time _and_ CPU time
spent in a block, both measured using milliseconds (to keep all units
the same).
Redis connection pool
Split from https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3232
Having an easily accessible Redis connection pool allows us to do more
cool stuff with Redis in GitLab (instead of having to go through e.g.
the Rails cache).
See merge request !3521
This changes the timestamp of metrics to be more accurate/unique by
using Time#to_f combined with a small random jitter value. This
combination hopefully reduces the amount of collisions, though there's
no way to fully prevent any from occurring.
Fixesgitlab-com/operations#175
Allow SAML to identify external users and set them as such
Related to #4009Fixes#14577
This allows SAML to retrieve group information form the `SAML Response`
and match that to a setting that will flag all matching users as external.
See merge request !3530
Unblocks user when active_directory is disabled and it can be found
We implemented a specific block state to handle user blocking that originates from LDAP filtering rules / directory state in !2242.
That introduced a regression in LDAP authentication when Active Directory support was disabled. You could have a scenario where the user would not be temporarily found (like a filtering rule), that would mark the user as `ldap_blocked`, but will never unblock it automatically when that state changed.
Fixes#14253, #13179, #13259, #13959
See merge request !3550
This allows measuring of timings of arbitrary Ruby blocks, this allows
for more fine grained performance monitoring. Custom values and tags can
also be attached to a block.
Rails' "url_helpers" method creates an anonymous Module (which a bunch
of methods) on every call. By caching the output of this method in a
dedicated method we can shave off about 10 seconds of loading time for
an issue with around 200 comments.
* master: (27 commits)
Fix commit comment alignment
minor cleanup in system_hook_spec
Pre-calculate Emoji digests
Clear .todo listener
Change window.location to use turbolinks
Make entire todo row clickable
Add 8.6.2 CHANGELOG items
Ensure uploads dir exists when running backup specs
Move CarrierWave test env config to separate file
Remove console logs
Off the event initially
Collapsed sidebar opens over instead of pushing content.
Sidebar collapse update issue
User selection from collapsed sidebar
Add json response for user avatar in merge request
Make changed values visible in minimized sidebar.
Fixed MergeRequestController spec
We need `sha` reference from `diff_base_commit` to generate the diff
Use `diff_base_commit` instead of `target_branch` to generate diffs
Isolate CarrierWave uploads in test enviroment
...
James Lopez
Grzegorz Bizon
Lin Jen-Shin
Douwe Maan
Zeger-Jan van de Weg
Douwe Maan
Yorick Peterse
Robert Speicher
Stan Hu
Rubén Dávila
Rémy Coutable
Fatih Acet
Kamil Trzciński
Adam Butler
Sean McGivern
Phil Hughes
Pablo Carranza
Douglas Barbosa Alexandre
Artem Sidorenko
Gabriel Mazetto
Connor Shea
Jacob Vosmaer
Paco Guzman
Alfredo Sumaran
Jacob Schatz
Sebastian Klier
Arinde Eniola
Robert Schilling
Valery Sizov
Marin Jankovski
Patricio Cano