Limiting the size of issuable description and comments to 1_000_000,
which is close to ~1MB of ASCII characters, which represents 99.9% of
all descriptions and comments we have in DB at the moment. This should
help prevent DoS attacks when comments contain refference strings.
Also this change updates regexp matching the namespaces paths by
limiting the namespaces paths to Namespace::NUMBER_OF_ANCESTORS_ALLOWED,
as we allow 20 levels deep groups.
see https://gitlab.com/gitlab-org/gitlab-ce/issues/61974#note_191274234
Currently we don't have good insight into the affect of Rails
transaction blocks on the application. If these blocks are held open for
extended periods, they can have detrimental effects on the application.
This change will allow us to track these transactions, with the aim
of reducing their duration.
This is a small change to move AfterCommitQueue.inside_transaction?
to Gitlab::Database.inside_transaction? Since this change is required
by different changes which may not arrive in sequence, it's easier
to extract this change out on it's own.
If doing a schema load, the post_migrations should also be marked as up,
even if SKIP_POST_DEPLOYMENT_MIGRATIONS was set, otherwise future
migration runs will be broken.
This adds the keyword argument "return_ids" to
Gitlab::Database.bulk_insert. When set to `true` (and PostgreSQL is
used) this method will return an Array of the IDs of the inserted rows,
otherwise it will return an empty Array.
In GitLab EE, a GitLab instance can be read-only (e.g. when it's a Geo
secondary node). But in GitLab CE it also might be useful to have the
"read-only" idea around. So port it back to GitLab CE.
Also having the principle of read-only in GitLab CE would hopefully
lead to less errors introduced, doing write operations when there
aren't allowed for read-only calls.
Closesgitlab-org/gitlab-ce#37534.
This adds a bunch of checks to migrations that may create or drop
triggers. Dropping triggers/functions is done using "IF EXISTS" so we
don't throw an error if the object in question has already been dropped.
We now also raise a custom error (message) when the user does not have
TRIGGER privileges. This should prevent the schema from entering an
inconsistent state while also providing the user with enough information
on how to solve the problem.
The recommendation of using SUPERUSER permissions is a bit extreme but
we require this anyway (Omnibus also configures users with this
permission).
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/36633
This changes various controllers to use the new EventCollection class
for retrieving events. This class uses a JOIN LATERAL query on
PostgreSQL to retrieve queries in a more efficient way, while falling
back to a simpler / less efficient query for MySQL.
The EventCollection class also includes a limit on the number of events
to display to prevent malicious users from cycling through all events,
as doing so could put a lot of pressure on the database.
JOIN LATERAL is only supported on PostgreSQL starting with version 9.3.0
and as such this optimisation is only used when using PostgreSQL 9.3 or
newer.
This adds an ID-less table containing one row per file, per merge request
diff. It has a column for each attribute on Gitlab::Git::Diff that is serialised
currently, with the advantage that we can easily query the attributes of this
new table.
It does not migrate existing data, so we have fallback code when the legacy
st_diffs column is present instead. For a merge request diff to be valid, it
should have at most one of:
* Rows in this new table, with the correct merge_request_diff_id.
* A non-NULL st_diffs column.
It may have neither, if the diff is empty.
This returns the ActiveRecord configuration for the current environment.
While CE doesn't use this very often, EE will use it in a few places for
the database load balancing code. I'm adding this to CE so we don't end
up with merge conflicts in this file.
This allows you to set a custom host when calling
Gitlab::Database.create_connection_pool. This is necessary for load
balancing as in this case we want to inherit all settings except for the
hostname.
We don't need to connect when requesting the name of the database
adapter. This in turn should prevent us from requesting/leaking
connections just by asking whether we're using PostgreSQL or MySQL.
When I proposed using serializable transactions I was hoping we would be
able to refresh data of individual users concurrently. Unfortunately
upon closer inspection it was revealed this was not the case. This could
result in a lot of queries failing due to serialization errors,
overloading the database in the process (given enough workers trying to
update the target table).
To work around this we're now using a Redis lease that is cancelled upon
completion. This ensures we can update the data of different users
concurrently without overloading the database.
The code will try to obtain the lease until it succeeds, waiting at
least 1 second between retries. This is necessary as we may otherwise
end up _not_ updating the data which is not an option.
GitLab Bot
Robert Speicher
Tiger
Alexandru Croitor
Andreas Brandl
Nick Thomas
Heinrich Lee Yu
Mayra Cabrera
DJ Mountney
Andrew Newdigate
DJ Mountney
Douglas Barbosa Alexandre
Ash McKenzie
Jasper Maes
gfyoung
Yorick Peterse
Brett Walker
Jarka Kadlecová
Stan Hu
Brett Walker
Michael Kozono
Alejandro Rodríguez
Toon Claes
Sean McGivern
Tiago Botelho
Lin Jen-Shin
Douwe Maan
Ahmad Sherif