Hiro Yoshioka c22ce143d1 [PATCH] x86: cache pollution aware __copy_from_user_ll()
Use the x86 cache-bypassing copy instructions for copy_from_user().

Some performance data are

Total of GLOBAL_POWER_EVENTS (CPU cycle samples)

2.6.12.4.orig    1921587
2.6.12.4.nt      1599424
1599424/1921587=83.23% (16.77% reduction)

BSQ_CACHE_REFERENCE (L3 cache miss)
2.6.12.4.orig      57427
2.6.12.4.nt        20858
20858/57427=36.32% (63.7% reduction)

L3 cache miss reduction of __copy_from_user_ll
samples  %
37408    65.1412  vmlinux                  __copy_from_user_ll
23        0.1103  vmlinux                  __copy_user_zeroing_intel_nocache
23/37408=0.061% (99.94% reduction)

Top 5 of 2.6.12.4.nt
Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (mandatory) count 100000
samples  %        app name                 symbol name
128392    8.0274  vmlinux                  __copy_user_zeroing_intel_nocache
64206     4.0143  vmlinux                  journal_add_journal_head
59746     3.7355  vmlinux                  do_get_write_access
47674     2.9807  vmlinux                  journal_put_journal_head
46021     2.8774  vmlinux                  journal_dirty_metadata
pattern9-0-cpu4-0-09011728/summary.out

Counted BSQ_CACHE_REFERENCE events (cache references seen by the bus unit) with a unit mask of 0x3f (multiple flags) count 3000
samples  %        app name                 symbol name
69755     4.2861  vmlinux                  __copy_user_zeroing_intel_nocache
55685     3.4215  vmlinux                  journal_add_journal_head
52371     3.2179  vmlinux                  __find_get_block
45504     2.7960  vmlinux                  journal_put_journal_head
36005     2.2123  vmlinux                  journal_stop
pattern9-0-cpu4-0-09011744/summary.out

Counted BSQ_CACHE_REFERENCE events (cache references seen by the bus unit) with a unit mask of 0x200 (read 3rd level cache miss) count 3000
samples  %        app name                 symbol name
1147      5.4994  vmlinux                  journal_add_journal_head
881       4.2240  vmlinux                  journal_dirty_data
872       4.1809  vmlinux                  blk_rq_map_sg
734       3.5192  vmlinux                  journal_commit_transaction
617       2.9582  vmlinux                  radix_tree_delete
pattern9-0-cpu4-0-09011731/summary.out

iozone results are

original 2.6.12.4 CPU time = 207.768 sec
cache aware       CPU time = 184.783 sec
(three times run)
184.783/207.768=88.94% (11.06% reduction)

original:
pattern9-0-cpu4-0-08191720/iozone.out:  CPU Utilization: Wall time   45.997    CPU time   64.527    CPU utilization 140.28 %
pattern9-0-cpu4-0-08191741/iozone.out:  CPU Utilization: Wall time   46.878    CPU time   71.933    CPU utilization 153.45 %
pattern9-0-cpu4-0-08191743/iozone.out:  CPU Utilization: Wall time   45.152    CPU time   71.308    CPU utilization 157.93 %

cache awre:
pattern9-0-cpu4-0-09011728/iozone.out:  CPU Utilization: Wall time   44.842    CPU time   62.465    CPU utilization 139.30 %
pattern9-0-cpu4-0-09011731/iozone.out:  CPU Utilization: Wall time   44.718    CPU time   59.273    CPU utilization 132.55 %
pattern9-0-cpu4-0-09011744/iozone.out:  CPU Utilization: Wall time   44.367    CPU time   63.045    CPU utilization 142.10 %

Signed-off-by: Hiro Yoshioka <hyoshiok@miraclelinux.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-06-23 07:42:56 -07:00
..
2006-01-08 20:14:02 -08:00
2006-03-21 08:52:18 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-01-13 09:12:21 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-10-04 13:22:01 -07:00
2005-06-23 09:45:26 -07:00
2005-09-10 10:06:21 -07:00
2006-03-24 07:33:20 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-28 09:16:05 -08:00
2005-04-16 15:20:36 -07:00
2006-03-23 07:38:09 -08:00
2005-09-10 10:16:27 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-27 08:44:59 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-05-15 18:31:07 -04:00
2006-03-27 08:44:59 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-08 15:14:23 -07:00
2005-04-16 15:20:36 -07:00
2005-05-19 22:43:37 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-31 12:18:54 -08:00
2005-04-16 15:20:36 -07:00
2006-01-18 19:20:29 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-07-12 12:08:43 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-11-10 02:26:41 +01:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-26 08:56:56 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-27 08:44:51 -08:00
2005-06-15 12:38:14 +01:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-11-08 17:14:08 +01:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-05-05 14:25:59 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-11-08 09:40:47 -08:00
2005-07-05 15:03:46 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-09-19 15:41:28 -07:00
2005-04-16 15:20:36 -07:00
2006-03-20 17:09:11 -08:00
2005-04-16 15:20:36 -07:00
2006-03-20 22:41:23 -08:00
2006-01-06 13:24:54 -08:00
2005-04-16 15:20:36 -07:00
2006-06-20 20:24:58 -07:00
2006-02-08 01:03:52 -05:00
2005-06-21 18:46:32 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-01-06 13:24:29 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-20 22:21:10 -08:00
2006-06-21 12:40:49 -07:00
2005-04-16 15:20:36 -07:00
2005-04-25 18:32:12 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-23 07:42:49 -07:00
2005-04-16 15:20:36 -07:00
2005-10-08 15:00:57 -07:00
2005-04-16 15:20:36 -07:00
2005-10-30 17:37:17 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-04-29 18:33:15 -07:00
2005-04-16 15:20:36 -07:00
2006-03-23 07:38:12 -08:00
2005-04-16 15:20:36 -07:00
2006-06-05 12:29:17 -07:00
2005-04-16 15:20:36 -07:00
2005-10-28 08:16:47 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-22 07:53:57 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-23 07:38:14 -08:00
2006-03-23 07:38:14 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-20 13:44:40 -05:00
2006-03-20 13:44:27 -05:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-23 07:42:46 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-22 15:05:56 -07:00
2005-04-16 15:20:36 -07:00
2006-03-27 08:44:48 -08:00
2005-04-16 15:20:36 -07:00
2006-05-04 06:55:12 +02:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-23 07:42:49 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-11-07 07:53:46 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-27 08:44:51 -08:00
2005-04-16 15:20:36 -07:00
2005-10-30 11:14:39 +11:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-17 21:29:55 -07:00
2006-03-23 07:38:12 -08:00
2005-04-16 15:20:36 -07:00
2005-10-30 17:37:32 -08:00
2006-04-02 00:08:05 -05:00
2005-11-07 07:53:37 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-28 09:16:05 -08:00
2005-09-10 10:06:21 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-06-23 07:42:47 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-10-30 17:37:32 -08:00
2005-09-18 00:18:32 -07:00
2005-04-16 15:20:36 -07:00
2006-03-26 08:57:03 -08:00
2005-04-16 15:20:36 -07:00
2006-03-31 12:18:56 -08:00
2006-01-12 14:06:38 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-01-10 08:01:59 -08:00
2006-03-23 07:38:14 -08:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2006-03-23 07:12:57 -05:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00
2005-04-16 15:20:36 -07:00