1 2 --- GEOM BASED DISK SCHEDULERS FOR FREEBSD --- 3 4This code contains a framework for GEOM-based disk schedulers and a 5couple of sample scheduling algorithms that use the framework and 6implement two forms of "anticipatory scheduling" (see below for more 7details). 8 9As a quick example of what this code can give you, try to run "dd", 10"tar", or some other program with highly SEQUENTIAL access patterns, 11together with "cvs", "cvsup", "svn" or other highly RANDOM access patterns 12(this is not a made-up example: it is pretty common for developers 13to have one or more apps doing random accesses, and others that do 14sequential accesses e.g., loading large binaries from disk, checking 15the integrity of tarballs, watching media streams and so on). 16 17These are the results we get on a local machine (AMD BE2400 dual 18core CPU, SATA 250GB disk): 19 20 /mnt is a partition mounted on /dev/ad0s1f 21 22 cvs: cvs -d /mnt/home/ncvs-local update -Pd /mnt/ports 23 dd-read: dd bs=128k of=/dev/null if=/dev/ad0 (or ad0-sched-) 24 dd-writew dd bs=128k if=/dev/zero of=/mnt/largefile 25 26 NO SCHEDULER RR SCHEDULER 27 dd cvs dd cvs 28 29 dd-read only 72 MB/s ---- 72 MB/s --- 30 dd-write only 55 MB/s --- 55 MB/s --- 31 dd-read+cvs 6 MB/s ok 30 MB/s ok 32 dd-write+cvs 55 MB/s slooow 14 MB/s ok 33 34As you can see, when a cvs is running concurrently with dd, the 35performance drops dramatically, and depending on read or write mode, 36one of the two is severely penalized. The use of the RR scheduler 37in this example makes the dd-reader go much faster when competing 38with cvs, and lets cvs progress when competing with a writer. 39 40To try it out: 41 421. PLEASE MAKE SURE THAT THE DISK THAT YOU WILL BE USING FOR TESTS 43 DOES NOT CONTAIN PRECIOUS DATA. 44 This is experimental code, so we make no guarantees, though 45 I am routinely using it on my desktop and laptop. 46 472. EXTRACT AND BUILD THE PROGRAMS 48 A 'make install' in the directory should work (with root privs), 49 or you can even try the binary modules. 50 If you want to build the modules yourself, look at the Makefile. 51 523. LOAD THE MODULE, CREATE A GEOM NODE, RUN TESTS 53 54 The scheduler's module must be loaded first: 55 56 # kldload gsched_rr 57 58 substitute with gsched_as to test AS. Then, supposing that you are 59 using /dev/ad0 for testing, a scheduler can be attached to it with: 60 61 # geom sched insert ad0 62 63 The scheduler is inserted transparently in the geom chain, so 64 mounted partitions and filesystems will keep working, but 65 now requests will go through the scheduler. 66 67 To change scheduler on-the-fly, you can reconfigure the geom: 68 69 # geom sched configure -a as ad0.sched. 70 71 assuming that gsched_as was loaded previously. 72 735. SCHEDULER REMOVAL 74 75 In principle it is possible to remove the scheduler module 76 even on an active chain by doing 77 78 # geom sched destroy ad0.sched. 79 80 However, there is some race in the geom subsystem which makes 81 the removal unsafe if there are active requests on a chain. 82 So, in order to reduce the risk of data losses, make sure 83 you don't remove a scheduler from a chain with ongoing transactions. 84 85--- NOTES ON THE SCHEDULERS --- 86 87The important contribution of this code is the framework to experiment 88with different scheduling algorithms. 'Anticipatory scheduling' 89is a very powerful technique based on the following reasoning: 90 91 The disk throughput is much better if it serves sequential requests. 92 If we have a mix of sequential and random requests, and we see a 93 non-sequential request, do not serve it immediately but instead wait 94 a little bit (2..5ms) to see if there is another one coming that 95 the disk can serve more efficiently. 96 97There are many details that should be added to make sure that the 98mechanism is effective with different workloads and systems, to 99gain a few extra percent in performance, to improve fairness, 100insulation among processes etc. A discussion of the vast literature 101on the subject is beyond the purpose of this short note. 102 103-------------------------------------------------------------------------- 104 105TRANSPARENT INSERT/DELETE 106 107geom_sched is an ordinary geom module, however it is convenient 108to plug it transparently into the geom graph, so that one can 109enable or disable scheduling on a mounted filesystem, and the 110names in /etc/fstab do not depend on the presence of the scheduler. 111 112To understand how this works in practice, remember that in GEOM 113we have "providers" and "geom" objects. 114Say that we want to hook a scheduler on provider "ad0", 115accessible through pointer 'pp'. Originally, pp is attached to 116geom "ad0" (same name, different object) accessible through pointer old_gp 117 118 BEFORE ---> [ pp --> old_gp ...] 119 120A normal "geom sched create ad0" call would create a new geom node 121on top of provider ad0/pp, and export a newly created provider 122("ad0.sched." accessible through pointer newpp). 123 124 AFTER create ---> [ newpp --> gp --> cp ] ---> [ pp --> old_gp ... ] 125 126On top of newpp, a whole tree will be created automatically, and we 127can e.g. mount partitions on /dev/ad0.sched.s1d, and those requests 128will go through the scheduler, whereas any partition mounted on 129the pre-existing device entries will not go through the scheduler. 130 131With the transparent insert mechanism, the original provider "ad0"/pp 132is hooked to the newly created geom, as follows: 133 134 AFTER insert ---> [ pp --> gp --> cp ] ---> [ newpp --> old_gp ... ] 135 136so anything that was previously using provider pp will now have 137the requests routed through the scheduler node. 138 139A removal ("geom sched destroy ad0.sched.") will restore the original 140configuration. 141 142# $FreeBSD$ 143