Previously, we stored these as serialised fields - `st_{commits,diffs}` - on the
`merge_request_diffs` table. These now have their own tables -
`merge_request_diff_{commits,diffs}` - with a column for each attribute of the
serialised data.
Add a background migration to go through the existing MR diffs and migrate them
to the new format. Ignore any contents that cannot be displayed. Assuming that
we have 5 million rows to migrate, and each batch of 2,500 rows can be
completed in 5 minutes, this will take about 7 days to migrate everything.
Previously, we used Psych, which would:
1. Check if a string was encoded as binary, and not ASCII-compatible.
2. Add the !binary tag in that case.
3. Convert to base64.
We need to do the same thing, using a new column in place of the tag.
This is an ID-less table with just three columns: an association to the merge
request diff the commit belongs to, the relative order of the commit within the
merge request diff, and the commit SHA itself.
Previously we stored much more information about the commits, so that we could
display them even when they were deleted from the repo. Since 8.0, we ensure
that those commits are kept around for as long as the target repo itself is, so
we don't need to duplicate that data in the database.
This adds an ID-less table containing one row per file, per merge request
diff. It has a column for each attribute on Gitlab::Git::Diff that is serialised
currently, with the advantage that we can easily query the attributes of this
new table.
It does not migrate existing data, so we have fallback code when the legacy
st_diffs column is present instead. For a merge request diff to be valid, it
should have at most one of:
* Rows in this new table, with the correct merge_request_diff_id.
* A non-NULL st_diffs column.
It may have neither, if the diff is empty.
The problem is that we often go via a diff object constructed from the diffs
stored in the DB. Those diffs, by definition, don't overflow, so we don't have
access to the 'correct' `real_size` - that is stored on the MR diff object
iself.
* upstream/master: (1122 commits)
Update CHANGELOG.md for 8.16.2
Display project ID in project settings (!8572)
fixed points from comments to improve code quality
Update CHANGELOG.md for 8.14.8
Statisfy eslint
Add CHANGELOG entry
Fix access to the wiki code via HTTP when repository feature disabled
Display fullscreen button on small screens (!5302)
Prevent removing fields from dropdowns on input elements
fix for all themes
Return struct instead of multiple values
Fix race conditions for AuthorizedProjectsWorker
Add User#nested_groups and User#nested_projects methods
Fix spec failure due to timestamp ordering issue in mySQL
Fixed error with filter keyboard tests
`can?` already includes the `feature_available?` check
Test there is no Merge Request button when MRs are disabled
Ensure the correct Merge Request button is found
Add 409 conflict tests
Add CHANGELOG
...
Customers running old versions of GitLab may have MergeRequestDiffs with
the text ["--broken diff"] due to text generated by gitlab_git 1.0.3.
To avoid the Error 500, verify that each element is a type that gitlab_git
will accept before attempting to create a DiffCollection.
Closes#20776
* remove ref fetch (we do it during creation anyway)
* remove safe_head_commit_sha for diff compare (do not depend on the source branch)
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
Alejandro Rodríguez
Sean McGivern
Yorick Peterse
Bob Van Landuyt
Robert Speicher
Douwe Maan
James Lopez
Lin Jen-Shin
Sean McGivern
Felipe Artur
Robert Speicher
Adam Niedzielski
Dmitriy Zaporozhets
Stan Hu