64bit mtime

[individual-it]

Description

allow to store mtime as 64bit int
build on top of #28066

Related Issue

#23228

Motivation and Context

the database should hold 64bit int to be able to store dates after 2038

How Has This Been Tested?

run new tests against pgsql, mysql & sqlite on PHP 7.1

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.

[PVince81]

we need to use the new DB migrations approach to make DB changes, don't reuse the old db_structure.xml.

See https://doc.owncloud.org/server/10.0/developer_manual/app/schema.html

[PVince81]

If all databases can deal with it then why not.

[individual-it]

@PVince81 is that the correct way of changing the DB Structure?

according to the documentation bigint will be converted to PHP's string type

I can imagine 2 problems with that:

1. on 32bit systems we could theoretically have a wrong mtime if a long string is casted to int. But that should not happen because there is no way to enter a value bigger than PHP_INT_MAX. Except of course somebody writes the value directly to the DB or moves a DB with files that are far in the future or past from a 64bit system to a 32bit system. So I think this is more of a theoretical problem.
2. in places where mtime is used without being casted to int but compared with === the code will fail

[individual-it]

rebased on current master

[phil-davis]

Looks good, assuming the Jenkins runs some time and passes. As long as the actual storage filesystems also support setting 64-bit then that gets us well past the end of the sun and practical heat death of the universe.
http://ximalas.info/2015/03/10/when-does-the-64-bit-unix-time_t-really-end/

[PVince81]

Would be good to do a quick test on a 32 bit system (VM or rasperry pi) to confirm that it doesn't break anything

[PVince81]

uh oh owncloud/files_texteditor#172 ?! do we need to support float mtimes ?

[phil-davis]

We thought about this. Modern file systems happily keep a record of mtime and the like to some fractions of a second. e.g. I think something like:
ext4 nanoseconds
UFS - ?
FAT - 2 seconds
exFAT - 10 milliseconds
NTFS - 100 nanoseconds

Rather than using float, the database could store the int nanoseconds (traditional unix timestamp seconds x 10^9). According to my calculations, (2^63)/(60 * 60 * 24 * 365 * 10^9) is about 292. That would allow storing times +-292 years from 1970. Enough for normal files, and dates that current people might have been born etc,

Or it can store the current seconds (in 64 bit to include the beginning and end of the universe) and add another 64-bit field for the nanoseconds offset.

That allows actual timestamps from ext4 and similar to be remembered in the database.

Some issues are:

1. What is the actual resolution of the backend storage that the server happens to have? What to do about having a potentially more accurate timestamp in the database than in the file system?
2. Clients will have local file systems with different resolutions. So if they are provided nanosecond resolution when they download a file, actually they cannot store that in their local filesystem. So what will they do in the future when comparing local and server time stamps? (if the client also has its own database, then it can remember the nanosecond timestamps itself)

[individual-it]

I've run the unit tests on a 32bit system with PHP 5.6 and they are passing on sqlite & mysql
on a 32bit system it would not be possible to store dates > 2038 because the mtime is always cast to int #28066

[PVince81]

Fine 👍

Please rebase once to refresh CI

[individual-it]

rebased, @DeepDiver1975 are you happy with the changes?

[DeepDiver1975]

We might want to rebase this one and see for conflicts

[individual-it]

@DeepDiver1975 rebased the branch.
UI tests are failing because of #28748

[DeepDiver1975]

@PVince81 shall we backport this?

[PVince81]

Later maybe, I don't feel safe about this

[PVince81]

@individual-it please backport for 10.0.4 so we can have it in an early beta

[phil-davis]

Backport stable10 #28961

[phil-davis]

Note: the migration code is being moved from dav to core in #29120

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.