Support for a 64bit time_t type

[th-otto]

As you are surely aware, there will be a problem with our current 32bit type time_t, since values after 19 Jan 2038 will become negative and thus cannot be represented. Although that will give us about 13 years before it really will become a problem, an increasing number of packages already may misbehave when working with 32bit time_t values. Eg. the some backends in gcc are starting to deprecate such a type in their runtime library, and as a result expect the underlying c-library to support a 64bit time_t type.

Actually, i think the situation is not that bad. For example, our struct stat already has members for the high values of time values:

mintlib/include/bits/stat.h

Line 45 in fe785b5

long __st_high_mtime;

So using a 64bit time_t for those will not change the layout.

Introducing such a type (from the application point of view) should be implement following glibc. The application can define _TIME_BITS to be either 32 or 64, and will get a default value for it if that is undefined (much like is done for _FILE_OFFSET_BITS for file related functions). For a start, we could even set the default to 32 so existing applications don't break.

Using a 64bit time_t has (after a first look) direct consequences to the following structures

• struct timeval
• struct timespec
• struct itimerspec
• struct itimerval
• struct utimbuf
• struct timeb
• struct msqid_ds
• struct shmid_ds
• struct semid_ds

So we will need alternate names for all those. As a result, following functions in mintlib will be affected:

• gettimeofday()

  mintlib/unix/gettimeofday.c

  Line 18 in fe785b5

  __gettimeofday (struct timeval *tp, struct timezone *tzp)

• settimeofday)

  mintlib/unix/settimeofday.c

  Line 17 in fe785b5

  __settimeofday (const struct timeval *tp, const struct timezone *tzp)

• time()

  mintlib/unix/time.c

  Line 17 in fe785b5

  __time (time_t *buf)

• mktime()

  mintlib/tz/localtime.c

  Line 2367 in fe785b5

  time_t mktime(struct tm *tmp)

• difftime()

  mintlib/tz/difftime.c

  Line 16 in fe785b5

  double difftime(time_t time1, time_t time0)

• gmtime()

  mintlib/tz/localtime.c

  Line 1762 in fe785b5

  struct tm *gmtime(const time_t * timep)

• gmtime_r()

  mintlib/tz/localtime.c

  Line 1756 in fe785b5

  struct tm *gmtime_r(const time_t *__restrict timep, struct tm *__restrict tmp)

• localtime()

  mintlib/tz/localtime.c

  Line 1711 in fe785b5

  struct tm *localtime(const time_t *timep)

• localtime_r()

  mintlib/tz/localtime.c

  Line 1722 in fe785b5

  struct tm *localtime_r(const time_t *__restrict timep, struct tm *__restrict tmp)

• ctime()

  mintlib/tz/ctime.c

  Line 9 in fe785b5

  char *ctime(const time_t *timep)

• ctime_r()

  mintlib/tz/ctime.c

  Line 18 in fe785b5

  char *ctime_r(const time_t *timep, char *buf)

• timegm()

  mintlib/tz/localtime.c

  Line 2393 in fe785b5

  time_t timegm(struct tm *tmp)

• timelocal()

  mintlib/tz/localtime.c

  Line 2386 in fe785b5

  time_t timelocal(struct tm *tmp)

• clock_gettime()

  mintlib/posix/clock_gettime.c

  Line 22 in fe785b5

  __clock_gettime(clockid_t clock_id, struct timespec *tp)

• clock_settime() NYI
• utime()

  mintlib/unix/utime.c

  Line 28 in fe785b5

  __utime (const char *_filename, const struct utimbuf *_tset)

• utimes()

  mintlib/unix/utimes.c

  Line 24 in fe785b5

  __utimes (const char *filename, const struct timeval tvp[2])

• futimens() NYI
• semctl()

  mintlib/sysvipc/semctl.c

  Line 38 in fe785b5

  __semctl (int semid, int semnum, int cmd, ...)

• semtimedop() NYI
• sendmsg()

  mintlib/socket/sendmsg.c

  Line 22 in fe785b5

  __sendmsg (int fd, const struct msghdr *msg, int flags)

• recvmsg()

  mintlib/socket/recvmsg.c

  Line 22 in fe785b5

  __recvmsg (int fd, struct msghdr *msg, int flags)

• adjtime

  mintlib/unix/adjtime.c

  Line 16 in fe785b5

  int __adjtime (__const struct timeval *__delta,

• getitimer()

  mintlib/unix/getitimer.c

  Line 19 in fe785b5

  __getitimer (enum __itimer_which which, struct itimerval* old)

• setitimer()

  mintlib/unix/setitimer.c

  Line 19 in fe785b5

  __setitimer (enum __itimer_which which, const struct itimerval* new,

• ftime()

  mintlib/time/ftime.c

  Line 23 in fe785b5

  ftime (struct timeb *timebuf)

• futimes()

  mintlib/unix/futimes.c

  Line 26 in fe785b5

  int __futimes (int fd, const struct timeval tvp[2])

• stime()

  mintlib/unix/stime.c

  Line 18 in fe785b5

  __stime (const time_t *now)

Note that some of the IPC functions are not yet implement in mintlib and/or the kernel so are currently not that important.
There would also be some more posix functions that are affected, but which are not implemented in mintlib yet.

There are also a bunch of other functions that take a timeval/timespec as argument, but use it as a relative timeout value rather than an absolute value, so maybe don't need to be changed for now (you don't want to do a sleep() for more than 140 years). This also includes pthread_* functions, should they be implemented.

• select()
• pselect()
• poll()
• ppoll()
• fcntl()
• sem_timedwait()
• sem_clockwait()
• nanosleep()
• clock_getres()
• clock_nanosleep()
• getrusage()
• wait3()
• wait4()

As a result of the above, it seems we would need only 2 new functions in the mint kernel for now:

• Tgettimeofday64()
• Tsettimeofday64()

Tgetdate()/Tsetdate are not affected because

1. they cannot be changed for TOS-compatibility
2. they use a base year of 1980 and therefor will only overflow at 2108

Some other functions should be checked whether they behave correctly. Eg. all the functions that deal with struct stat should check the time values from filesystems to fit into 32bits if needed.

Obviously, we need a kernel first which implements the new functions, otherwise those new interfaces can't be used.

Any comments on these are welcome, especially should i have missed any types/functions that are affected.

PS.: Note that the timezone database is not affected by this. zic (the timezone info compiler) uses a 64bit type internally for quite some time, and produces tables that contain both 32bit and 64bit values.

[mikrosk]

Great analysis. I have nothing smart to add but I'll just menion that from Atari perspective, we are even luckier than the rest of the world: basically we don't have Unix/Linux/MiNT-only applications without source code. So basically we can recompile everything after the update.

And for the very few people who actively develop stuff on/for FreeMiNT, we can always reach out with advice and best practice.

[th-otto]

So basically we can recompile everything after the update.

Yes, luckily. You only have to be careful if libraries use time_t in public headers. For those we maybe have to have add redirections similar to those for the system functions. But from what if seen so far, this is only the case for png.h:

PNG_EXPORT(25, void, png_convert_from_time_t, (png_timep ptime, time_t ttime));

[th-otto]

There will be one thing in the kernel i have to think about:
https://github.com/freemint/freemint/blob/108f4cbdd4ee59bccbef1fa6f6796f5337a3056e/sys/dosfile.c#L440-L442

Passes a 32bit unix time to the filesystem layer. That will overflow the range of a signed 32bit value, but maybe not the range of an unsigned 32bit value (assuming that you never will try to set filetimes to some date before 1970, or after 2106).

[th-otto]

Hm, functions recvmsg() and sendmsg() imho do not need to be redirected. I added them, because i looked at linux glibc headers, and there was a redirection depending on __USE_TIME_BITS64. However i don't see any dependency on time_t in the parameters, neither in struct msghdr, nor in struct iovec that is used by it. Did i miss anything?

[th-otto]

While looking at was needs to be fixed in the kernel, i also wonder whether code like

https://github.com/freemint/freemint/blob/108f4cbdd4ee59bccbef1fa6f6796f5337a3056e/sys/xfs/ext2fs/main.c#L87

make sense. That xtime member will never be null with recent kernels, and older kernels are already ruled out by checking the version. XFS/XDD drivers must always use the same interface as the kernel, or they fail.

[th-otto]

What i also noticed: it looks like ext2fs is the only filesystem where a 64bit time_t could actually be written. Is that possible? I thought that atleast nfs should be able to handle that, but that does not seem to be the case: https://github.com/freemint/freemint/blob/108f4cbdd4ee59bccbef1fa6f6796f5337a3056e/sys/xfs/nfs/nfs_xdr.h#L136-L140

Or is that only because our client only support NFSv2?

For minix, there is not much hope to be able to support that, but i don't think that there is much use of it. For FAT32, it is basically the same as for TOS, there is no 64bit time in the filesystem itself, but timestamps are based at 1980, and could represent dates up to 2108. We just have to make sure that the values don't overflow somewhere in the conversion.

[mikrosk]

Or is that only because our client only support NFSv2?

From Wikipedia:

Network File System version 4 has defined its time fields as struct nfstime4 {int64_t seconds; uint32_t nseconds;} since December 2000.[25] Version 3 supports unsigned 32-bit values as struct nfstime3 {uint32 seconds; uint32 nseconds;};.[26] Values greater than zero for the seconds field denote dates after the 0-hour, January 1, 1970. Values less than zero for the seconds field denote dates before the 0-hour, January 1, 1970. In both cases, the nseconds (nanoseconds) field is to be added to the seconds field for the final time representation.

[th-otto]

Hm, ok. But NFSv4 also mandates "strong security", i don't think anybody wants to use that on a 68k machine ;)

Well, still nearly 13 years to think about it, so no hurry...

BTW, i'm not even sure whether our ext2fs port supports that. The extra data is contained in an "ext2_inode_large", and it seems that our tools cannot handle that yet.

[th-otto]

BTW, there is also a (potentially) Y2K problem lurking in teradesk. I've touched a file with a timestamp of the year 2087, and in teradesk it is displayed as

The year is displayed with 2 digits only, which is ambiguous (could be either 1987, or 2087 etc).

Same file is currently displayed with a broken date by ls (without any kernel and/or mintlib patches):

[th-otto]

I've also started pushing changes to the kernel, to the time64 branch: https://github.com/freemint/freemint/tree/time64

Maybe someone wants to have a look already.

Same for mintlib: https://github.com/freemint/mintlib/tree/time64 but those are incomplete yet.

[th-otto]

BTW, interesting fact: while looking after the ext2 filesystem driver, i realized that they only use 2 bits to extend the timestamps beyond 2038, the other 30 bits are used for nanoseconds. So they just changed the Y2038 problem into a Y2514 problem ;)

[th-otto]

Regarding msqid_ds etc: the underlying kernel function are not implemented yet. And mintlib passes the just a pointer to such struct along. I think we could take the opportunity and just change those structure to something like:

struct msqid_ds
{
  struct ipc_perm msg_perm;	/* structure describing operation permission */
#ifdef __USE_TIME_BITS64
  __time64_t msg_stime;
  __time64_t msg_rtime;
  __time64_t msg_ctime;
#else
  long __msg_high_stime;
  __time_t msg_stime;		/* time of last msgsnd command */
  long __msg_high_rtime;
  __time_t msg_rtime;		/* time of last msgrcv command */
  long __msg_high_ctime;
  __time_t msg_ctime;		/* time of last change */
#endif
  msgqnum_t msg_qnum;		/* number of messages currently on queue */
  msglen_t msg_qbytes;		/* max number of bytes allowed on queue */
  __pid_t msg_lspid;		/* pid of last msgsnd() */
  __pid_t msg_lrpid;		/* pid of last msgrcv() */
};

Similar to what was done for struct stat. That would simplify later implementation in the kernel, and also mintlib. What do you think?

[mikrosk]

If the handlers are not implement then sure, this looks like a good idea.

[th-otto]

https://github.com/freemint/freemint/blob/54e7078cdafddf5c4073185ab72ab086be33ee56/sys/sysv_shm.c#L48
https://github.com/freemint/freemint/blob/54e7078cdafddf5c4073185ab72ab086be33ee56/sys/sysv_sem.c#L36
https://github.com/freemint/freemint/blob/54e7078cdafddf5c4073185ab72ab086be33ee56/sys/sysv_msg.c#L36