Posix compliant cpu clocks: V7 [0/3] Rationale and test program Signed-off-by: Christoph Lameter Changes from V5: * add a simple means of accessing other processes cputimers. * Simplified glibc patch that makes glibc return errors when an applications accesses cputimers and is run with glibc under a kernel not providing cputimers. This may allow the detection of applications that need to be updated since they depend on the earlier non posix compliant version of glibc. Also should be more readable since the sections are no longer moved betweeen directory hierachies. * remove ability to set process and thread clocks. POSIX clocks are to be implemented in the following way according to V3 of the Single Unix Specification: 1. CLOCK_PROCESS_CPUTIME_ID Implementations shall also support the special clockid_t value CLOCK_PROCESS_CPUTIME_ID, which represents the CPU-time clock of the calling process when invoking one of the clock_*() or timer_*() functions. For these clock IDs, the values returned by clock_gettime() and specified by clock_settime() represent the amount of execution time of the process associated with the clock. 2. CLOCK_THREAD_CPUTIME_ID Implementations shall also support the special clockid_t value CLOCK_THREAD_CPUTIME_ID, which represents the CPU-time clock of the calling thread when invoking one of the clock_*() or timer_*() functions. For these clock IDs, the values returned by clock_gettime() and specified by clock_settime() shall represent the amount of execution time of the thread associated with the clock. These times mentioned are CPU processing times and not the time that has passed since the startup of a process. Glibc currently provides its own implementation of these two clocks which is designed to return the time that passed since the startup of a process or a thread. Moreover Glibc's clocks are bound to CPU timers which is problematic when the frequency of the clock changes or the process is moved to a different processor whose cpu timer may not be fully synchronized to the cpu timer of the current CPU. This patchset results in a both clocks working reliably. The patch also implements the access to other the thread and process clocks of linux processes by using negative clockid's: 1. For CLOCK_PROCESS_CPUTIME_ID: -pid 2. For CLOCK_THREAD_CPUTIME_ID: -(pid + PID_MAX_LIMIT) This allows clock_getcpuclockid(pid) to return -pid and pthread_getcpuiclock(pid) to return -(pid + PID_MAX_LIMIT) to allow access to the corresponding clocks. Todo: - The timer API to generate events by a non tick based timer is not usable in its current state. The posix timer API seems to be only useful at this point to define clock_get/set. Need to revise this. - Implement timed interrupts through mmtimer The patchset consists of the following components: [0/2] Contains an explanation as to why these patches are necessary as well as a test program and the output of a sample run. [1/2] Linux Kernel Patch: Implements the two clocks and enhances some pieces of the posix-timers implementation in the kernel for these clocks and also makes it possible for device drivers to define additional clocks. [2/2] Glibc patch: Use kernel clocks and also makes glibc able to use any posix clock provided by the kernel so that posix clocks by driver may be accessed. Break apps that use the old cputimer based CLOCK_*_CPUTIME_IDs on kernels that do not support CLOCK_*_CPUTIME_ID. The mmtimer patch is unchanged from V6 and stays as is in 2.6.9-rc3-mm2. But I expect to update the driver as soon as the interface to setup hardware timer interrupts is usable. Rolands patches: ---------------- This patch is not as comprehensive as Rolands but also not as invasive. Roland's high resolution time through tsc is a hack but would significantly improve the statistics we get for a process. However, we may get the same result in a cleaner way after John Stultz and my plan for revising the time subsystem is through. This would replace jiffies with time access with nanosecond accuracy throughout the kernel. Nanosecond accuracy is already available through getnstimeofday() on some platforms. Some of the functionality in Roland's patch may be implemented in a TSC independent way using this function. The use of negative pid values in certain numeric ranges for selecting the type of clock is something that is also in a smaller way present in my patch. I do not like this and would rather see this incorporated in glibc if the glibc people can come up with a sane implementation. Single Thread Testing CLOCK_THREAD_CPUTIME_ID= 0.494140878 resolution= 0.000976563 CLOCK_PROCESS_CPUTIME_ID= 0.494140878 resolution= 0.000976563 Multi Thread Testing Starting Thread: 0 1 2 3 4 5 6 7 8 9 Joining Thread: 0 1 2 3 4 5 6 7 8 9 0 Cycles= 0 Thread= 0.000000000ns Process= 0.495117441ns 1 Cycles=1000000 Thread= 0.140625072ns Process= 2.523438792ns 2 Cycles=2000000 Thread= 0.966797370ns Process= 8.512699671ns 3 Cycles=3000000 Thread= 0.806641038ns Process= 7.561527309ns 4 Cycles=4000000 Thread= 1.865235330ns Process= 12.891608163ns 5 Cycles=5000000 Thread= 1.604493009ns Process= 11.528326215ns 6 Cycles=6000000 Thread= 2.086915131ns Process= 13.500983475ns 7 Cycles=7000000 Thread= 2.245118337ns Process= 13.947272766ns 8 Cycles=8000000 Thread= 1.604493009ns Process= 12.252935961ns 9 Cycles=9000000 Thread= 2.160157356ns Process= 13.977546219ns Clock status at the end of the timer tests: Gettimeofday() = 1097084999.489938000 CLOCK_REALTIME= 1097084999.490116229 resolution= 0.000000040 CLOCK_MONOTONIC= 177.071675109 resolution= 0.000000040 CLOCK_PROCESS_CPUTIME_ID= 13.978522782 resolution= 0.000976563 CLOCK_THREAD_CPUTIME_ID= 0.497070567 resolution= 0.000976563 CLOCK_SGI_CYCLE= 229.967982280 resolution= 0.000000040 PROCESS clock of 1 (init)= 4.833986850 resolution= 0.000976563 THREAD clock of 1 (init)= 0.009765630 resolution= 0.000976563 #include #include #include #include #include #include #define clock_getres(x,y) syscall(__NR_clock_getres, x,y) #define clock_gettime(x,y) syscall(__NR_clock_gettime, x, y) #define clock_settime(x,y) syscall(__NR_clock_settime, x, y) void pr(int clock,const char *n) { struct timespec tv = {1,2}; struct timespec res = {3,4}; int rc; rc=clock_getres(clock,&res); if (rc) { printf("getres return code on %s=%d errno=%d\n",n,rc,errno); } rc=clock_gettime(clock,&tv); if (rc) { printf("gettime return code on %s=%d errno=%d\n",n,rc, errno); } else printf("%25s=% 11d.%09d resolution=% 2d.%09d\n",n,tv.tv_sec,tv.tv_nsec,res.tv_sec,res.tv_nsec); } int y; void kx(long long x) { y=x; }; struct timespec zero; pthread_t thread[10]; struct tinfo { int i; struct timespec ttime,ptime; } tinf[10]; void *thread_function(void *x) { struct tinfo *t=x; int i; for(i=1;i< t->i;i++) kx(1000000000000LL/i); clock_gettime(CLOCK_THREAD_CPUTIME_ID,&t->ttime); clock_gettime(CLOCK_PROCESS_CPUTIME_ID,&t->ptime); } int main(char argc, char *argv[]) { struct timespec tv; int i; /* Waste some time */ printf("Single Thread Testing\n"); for(i=1;i<10000000;i++) kx(1000000000000LL/i); pr(CLOCK_THREAD_CPUTIME_ID,"CLOCK_THREAD_CPUTIME_ID"); pr(CLOCK_PROCESS_CPUTIME_ID,"CLOCK_PROCESS_CPUTIME_ID"); /* Waste some more time in threads */ printf("Multi Thread Testing\nStarting Thread:"); clock_settime(CLOCK_PROCESS_CPUTIME_ID,&zero); for(i=0;i<10;i++) { tinf[i].i=i*1000000; if (pthread_create(&thread[i], NULL, thread_function, tinf+i)) perror("thread"); else printf(" %d",i); } printf("\n Joining Thread:"); for(i=0;i<10;i++) if (pthread_join( thread[i], NULL)) perror("join"); else printf(" %d",i); printf("\n"); for(i=0;i<10;i++) { printf("%d Cycles=%7d Thread=% 3d.%09dns Process=% 3d.%09dns\n",i,tinf[i].i,tinf[i].ttime.tv_sec,tinf[i].ttime.tv_nsec,tinf[i].ptime.tv_sec,tinf[i].ptime.tv_nsec); } gettimeofday((struct timeval *)&tv); tv.tv_nsec = tv.tv_nsec*1000; printf("\nClock status at the end of the timer tests:\n"); printf(" Gettimeofday() =% 11d.%09d\n",tv.tv_sec,tv.tv_nsec); pr(CLOCK_REALTIME,"CLOCK_REALTIME"); pr(CLOCK_MONOTONIC,"CLOCK_MONOTONIC"); pr(CLOCK_PROCESS_CPUTIME_ID,"CLOCK_PROCESS_CPUTIME_ID"); pr(CLOCK_THREAD_CPUTIME_ID,"CLOCK_THREAD_CPUTIME_ID"); pr(10,"CLOCK_SGI_CYCLE"); pr(-1,"PROCESS clock of 1 (init)"); pr(-1 - 4*1024*1024,"THREAD clock of 1 (init)"); printf("\n"); }