README.BOOTP
1# $FreeBSD: src/share/examples/diskless/README.BOOTP,v 1.2.4.1 2002/02/12 17:43:11 luigi Exp $
2# Notes on diskless boot.
3
4IMPORTANT NOTE:
5
6For quite some time
7the /etc/rc.d/{initdiskless,diskless} scripts support a slightly different
8diskless boot process than the one documented in the rest of this file.
9
10I am not deleting the information below because it contains some
11useful background information on diskless operation, but for the
12actual details you should look at diskless(8), /etc/rc.d/initdiskless,
13/etc/rc.d/diskless,
14and the /usr/share/examples/diskless/clone_root script which can
15be useful to set up clients and server for diskless boot.
16
17-----------------------------------------------------------------------
18
19 BOOTP configuration mechanism
20
21 Matthew Dillon
22 dillon@backplane.com
23
24 BOOTP kernels automatically configure the machine's IP address, netmask,
25 optional NFS based swap, and NFS based root mount. The NFS server will
26 typically export a shared read-only /, /usr, and /var to any number of
27 workstations. The shared read-only root is typically either the server's
28 own root or, if you are more security conscious, a contrived root.
29
30 The key issue with starting up a BOOTP kernel is that you typically want
31 to export read-only NFS partitions from the server, yet still be able to
32 customize each workstation ( or not ).
33
34 /etc/rc.diskless1 is responsible for doing core mounts and for retargeting
35 /conf/ME ( part of the read-only root NFS mount ) to /conf/$IP_OF_CLIENT.
36 /etc/rc.conf.local and /etc/rc.local, along with other machine-specific
37 configuration files, are typically softlinks to /conf/ME/<filename>.
38
39 In the BOOTP workstation /conf/$IP/rc.conf.local, you must typically
40 turn *OFF* most of the system option defaults in /etc/rc.conf as well
41 as do additional custom configuration of your environment
42
43 The /usr/src/share/examples/diskless directory contains a typical
44 X session / sshd based workstation configuration. The directories
45 involved are HT.DISKLESS/ and 192.157.86.12/.
46
47 Essentially, the $IP/ directory ( which rc.diskless looks for in
48 /conf/$IP/ ) contains all the junk. The HT.DISKLESS directory exists
49 to hold common elements of your custom configuration so you do not have
50 to repeat those elements for each workstation. The example /conf
51 structure included here shows how to create a working sshd setup ( so
52 you can sshd into the diskless workstation ), retarget xdm's pid and error
53 files to R+W directories if /usr is mounted read-only, and retarget
54 syslogd and other programs. This example is not designed to run out of
55 the box and some modifications are required.
56
57 >> NOTE << HT.DISKLESS/ttys contains the typical configuration required
58 to bring X up at boot time. Essentially, it runs xdm in the foreground
59 with the appropriate arguments rather then a getty on ttyv0. You must
60 run xdm on ttyv0 in order to prevent xdm racing with getty on a virtual
61 terminal. Such a race can cause your keyboard to be directed away from
62 the X session, essentially making the session unusable.
63
64 Typically you should start with a clean slate by tar-copying this example
65 directory to /conf and then hack on it in /conf rather then in
66 /usr/share/examples/diskless.
67
68 BOOTP CLIENT SETUP
69
70 Here is a typical kernel configuration. If you have only one ethernet
71 interface you do not need to wire BOOTP to a specific interface name.
72 BOOTP requires NFS and NFS_ROOT, and our boot scripts require MFS. If
73 your /tmp is *not* a softlink to /var/tmp, the scripts also require NULLFS
74
75# BootP
76#
77options BOOTP # Use BOOTP to obtain IP address/hostname
78options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info
79options BOOTP_COMPAT # Workaround for broken bootp daemons.
80#options "BOOTP_WIRED_TO=de0"
81
82options MFS # Memory File System
83options NFS # Network Filesystem
84options NFS_ROOT # Nfs can be root
85options NULLFS # nullfs to map /var/tmp to /tmp
86
87 BOOTP SERVER SETUP
88
89 The BOOTP server must be running on the same logical LAN as the
90 BOOTP client(s). You need to setup two things:
91
92 (1) You need to NFS-export /, /usr, and /var.
93
94 (2) You need to run a BOOTP server. DHCPD can do this.
95
96
97 NFS Export:
98
99 Here is an example "/etc/exports" file.
100
101/ -ro -maproot=root: -network 192.157.86.0 -mask 255.255.255.192
102/usr -ro -maproot=root: -network 192.157.86.0 -mask 255.255.255.192
103/var -ro -maproot=root: -network 192.157.86.0 -mask 255.255.255.192
104
105 In order to be an NFS server, the server must run portmap, mountd,
106 nfsd, and rpc.statd. The standard NFS server options in /etc/rc.conf
107 will work ( you should put your overrides in /etc/rc.conf.local on the
108 server and not edit the distribution /etc/rc.conf, though ).
109
110 BOOTP Server:
111
112 This configuration file "/etc/dhcpd.conf" example is for
113 the '/usr/ports/net/isc-dhcp' dhcpd port.
114
115 subnet 192.157.86.0 netmask 255.255.255.192 {
116 # range if you want to run the core dhcpd service of
117 # dynamic IP assignment, but it is not used with BOOTP
118 # workstations
119 range 192.157.86.32 192.157.86.62;
120
121 # misc configuration.
122 #
123 option routers 192.157.86.2;
124 option domain-name-servers 192.157.86.2;
125
126 server-name "apollo.fubar.com";
127 option subnet-mask 255.255.255.192;
128 option domain-name-servers 192.157.86.2;
129 option domain-name "fubar.com";
130 option broadcast-address 192.157.86.63;
131 option routers 192.157.86.2;
132 }
133
134 host test1 {
135 hardware ethernet 00:a0:c9:d3:38:25;
136 fixed-address 192.157.86.11;
137 option root-path "192.157.86.2:/";
138 option option-128 "192.157.86.2:/images/swap";
139 }
140
141 host test2 {
142 # hardware ethernet 00:e0:29:1d:16:09;
143 hardware ethernet 00:10:5a:a8:94:0e;
144 fixed-address 192.157.86.12;
145 option root-path "192.157.86.2:/";
146 option option-128 "192.157.86.2:/images/swap";
147 }
148
149 SWAP. This example includes options to automatically BOOTP configure
150 NFS swap on each workstation. In order to use this capabilities you
151 need to NFS-export a swap directory READ+WRITE to the workstations.
152
153 You must then create a swap directory for each workstation you wish to
154 assign swap to. In this example I created a dummy user 'lander' and
155 did an NFS export of /images/swap enforcing a UID of 'lander' for
156 all accesses.
157
158 apollo:/usr/ports/net# ls -la /images/swap
159 total 491786
160 drwxr-xr-x 2 root wheel 512 Dec 28 07:00 .
161 drwxr-xr-x 8 root wheel 512 Jan 20 10:54 ..
162 -rw-r--r-- 1 lander wheel 33554432 Dec 23 14:35 swap.192.157.86.11
163 -rw-r--r-- 1 lander wheel 335544320 Jan 24 16:55 swap.192.157.86.12
164 -rw-r--r-- 1 lander wheel 134217728 Jan 21 17:19 swap.192.157.86.6
165
166 A swap file is best created with dd:
167
168 # create a 32MB swap file for a BOOTP workstation
169 dd if=/dev/zero of=swap.IPADDRESS bs=1m count=32
170
171 It is generally a good idea to give your workstations some swap space,
172 but not a requirement if they have a lot of memory.
173
174
README.TEMPLATING
1# $FreeBSD: src/share/examples/diskless/README.TEMPLATING,v 1.2.4.1 2002/02/12 17:49:13 luigi Exp $
2
3IMPORTANT NOTE:
4
5For quite some time
6the /etc/rc.d/{initdiskless,diskless} scripts support a slightly different
7diskless boot process than the one documented in the rest of this file.
8
9I am not deleting the information below because it contains some
10useful background information on diskless operation, but for the
11actual details you should look at diskless(8), /etc/rc.d/initdiskless,
12/etc/rc.d/diskless,
13and the /usr/share/examples/diskless/clone_root script which can
14be useful to set up clients and server for diskless boot.
15
16-----------------------------------------------------------------------
17
18 TEMPLATING machine configurations
19
20 Matthew Dillon
21 dillon@backplane.com
22
23 This document describes a general mechanism by which you can template
24 / and /usr. That is, to keep a 'master template' of / and /usr on a
25 separate machine which is then used to update the rest of your machines.
26
27 Generally speaking, you can't simply mirror /. You might be able to
28 get away with mirroring /usr. There are two main problems involved with
29 templating:
30
31 (1) Avoiding overwriting run-time generated files
32
33 By default, the system maintains a number of files in the root
34 partition. For example, sendmail will dbm /etc/aliases into
35 /etc/aliases.db. vipw or chpass or other password related routines
36 will regenerate the password dbm's /etc/spwd.db, /etc/pwd.db, and
37 passwd. /etc/namedb/s might contain generated secondaries. And
38 so forth.
39
40 The templating mechanism must avoid copying over such files.
41
42 (2) Customizing machines.
43
44 Customizing machines is actually considerably simpler. You create
45 a configuration hierarchy and convert the configuration files that
46 have to be customized into softlinks that run through a special
47 softlink in the configuration directory. This will work for every
48 configuration file except possibly /etc/master.passwd
49
50 For example, /etc/resolv.conf would be turned into a softlink to
51 /conf/ME/resolv.conf, and /conf/ME itself would be a softlink to
52 /conf/<HOSTNAME>. The actual resolv.conf configuration file
53 would reside in /conf/<HOSTNAME>.
54
55 If you have a lot of hosts, some configuration files may be commonly
56 classified. For example, all your shell machines might have the
57 same /etc/resolv.conf. The solution is to make
58 /conf/<HOSTNAME>/resolv.conf a softlink to a common directory, say
59 /conf/HT.SHELL/resolv.conf. It may sound a little messy, but this
60 sort of categorization actually makes the sysadmins job much, much
61 easier.
62
63 The /conf/ directory hierarchy is stored on the template and
64 distributed to all the machines along with the rest of the root
65 partition.
66
67 This type of customization is taken from my direct experience
68 instituting such a system at BEST. At the time, BEST had over 45
69 machines managed from a single template.
70
71 RUN-TIME GENERATED OR MODIFIED FILES IN / or /USR
72
73 /etc/aliases.db
74 /etc/master.passwd
75 /etc/spwd.db
76 /etc/pwd.db
77 /etc/passwd
78 /etc/namedb/s
79 /root/.history
80 /root/.ssh/identity
81 /root/.ssh/identity.pub
82 /root/.ssh/random_seed
83 /root/.ssh/known_hosts
84 /conf/ME
85 /kernel* ( note 2 )
86 /dev ( note 3 )
87 /var ( note 4 )
88 /home ( note 4 )
89 /lost+found
90
91 /usr/lost+found
92 /usr/home ( note 4 )
93 /usr/crash ( note 5 )
94 /usr/obj ( note 5 )
95 /usr/ports ( note 5 )
96 /usr/src ( note 5 )
97 /usr/local/crack ( note 5 )
98 /usr/X11R6/lib/X11/xdm/xdm-errors ( note 6 )
99 /usr/X11R6/lib/X11/xdm/xdm-pid ( note 6 )
100 /usr/local/etc/ssh_host_key ( note 6 )
101 /usr/local/etc/ssh_host_key.pub ( note 6 )
102 /usr/local/etc/ssh_random_seed ( note 6 )
103
104 /conf/ME ( note 7 )
105
106 note 2: You typically want to update kernels manually and *NOT*
107 template them as a safety measure. This also allows you to run
108 different kernels on different machines or.
109
110 note 3: /dev must be updated manually. Some devices, such as tty's and
111 pty's, use the access and/or modify time and/or user/group
112 operationally and regenerating the devices on the fly would be
113 bad.
114
115 note 4: /var and /home are usually separately mounted partitions and
116 thus would not fall under the template, but as a safety measure
117 the template copier refuse to copy directories named 'home'.
118
119 note 5: These are directories that are as often created directly on
120 /usr as they are separately-mounted partitions. You typically
121 do not want to template such directories.
122
123 note 6: Note that you can solve the problem of xdm and sshd creating
124 files in /usr. With xdm, edit /usr/X11R6/lib/xdm/xdm-config
125 and change the errorLogFile and pidFile config lines.
126
127 With sshd, add 'HostKey' and 'RandomSeed' directives to specify
128 /var/db for the location of the host key and run-time sshd
129 random seed:
130
131 HostKey /var/db/ssh_host_key
132 RandomSeed /var/db/ssh_random_seed
133
134 note 7: In this example, /conf/ME is the machine customizer and must
135 be pointed to the /conf/<full-host-name>/ directory, which is
136 different for each machine. Thus, the /conf/ME softlink
137 should never be overwritten by the templating copy.
138
139
140 TYPICAL CUSTOMIZED CONFIGRATION SOFTLINKS
141
142 The following files typically need to be turned into softlinks
143 to /conf/ME/<filename>:
144
145 /etc/ccd.conf -> /conf/ME/ccd.conf
146 /etc/ipfw.conf ...
147 /etc/fstab
148 /etc/motd
149 /etc/resolv.conf
150 /etc/aliases
151 /etc/sendmail.cw
152 /etc/organization
153 /etc/named.conf
154 /etc/rc.conf.local
155 /etc/printcap
156 /etc/inetd.conf
157 /etc/login.conf
158 /etc/gettytab
159 /etc/ntp.conf
160 /etc/exports
161 /root/.k5login -> /conf/ME/root/.k5login
162
163 And, of course, /conf/ME is usually a softlink to the appropriate
164 /conf/<full-host-name>/. Depending on your system configuration,
165 there may be other files not listed above that you have to worry about.
166
167 In many cases, /conf/ME/filename is itself a softlink to
168 "../HT.xxxx/filename", where HT.xxxx is something like HT.STD ... this
169 added complexity actually makes it easier to manage multiple
170 classifications of machines.
171
172 DELETION OF FILES
173
174 Any file found on the template destination that does not exist in the
175 source and is not listed as an exception by the source should be deleted.
176 However, deletion can be dangerous and cpdup will ask for confirmation
177 by default. Once you know you aren't going to blow things up, you can
178 turn this feature off and update your systems automatically from cron.
179
180 By formalizing the delete operation, you can be 100% sure that it is
181 possible to recreate / and /usr on any machine with only the original
182 template and a backup of the ( relatively few ) explicitly-excepted
183 files. The most common mistake a sysop makes is to make a change to a
184 file in / or /usr on a target machine instead of the template machine.
185 If the target machine is updated once a night from cron, the sysop
186 quickly learns not to do this ( because his changes get overwritten
187 overnight ). With a manual update, these sorts of mistakes can propagate
188 for weeks or months before they are caught.
189
190 TEMPLATE COPYING AND SAFETY
191 THE CPDUP PROGRAM
192
193 The 'cpdup' program is a program which efficiently duplicates a directory
194 tree. The program copies source to destination, duplicating devices,
195 softlinks, hardlinks, files, modification times, uid, gid, flags, perms,
196 and so forth. The program incorporates several major features:
197
198 * The program refuses, absolutely, to cross partition boundaries.
199 i.e. if you were copying the template /usr from an NFS mount to
200 your /usr, and you had a mount point called /usr/home, the
201 template copying program would *NOT* descend into /usr/home on
202 the destination.
203
204 This is a safety.
205
206 * The program accesses a file called .cpignore in each directory
207 it descending into on the source to obtain a list of exceptions
208 for that directory -- that is, files not to copy or mess with.
209
210 This is a templating function.
211
212 * The program refuses to delete a directory on the destination
213 being replaced by a softlink or file on the source.
214
215 This is a safety mechanism
216
217 * The program is capable of maintaing MD5 check cache files and
218 doing an MD5 check between source and destination during the
219 scan.
220
221 * The program is capable of deleting files/directories on the
222 destination that do not exist on the source, but asks for
223 confirmation by default.
224
225 This is a templating and a safety mechanism.
226
227 * The program uses a copy-to-tmp-and-rename methodology allowing
228 it to be used to update live filesystems.
229
230 This is a templating mechanism.
231
232 * The program, by default, tries to determine if a copy is required
233 by checking modify times, file size, perms, and other stat
234 elements. If the elements match, it does not bother to copy
235 ( unless an MD5 check is being made, in which case it must read
236 the destination file ).
237
238 You typically run cpdup on the target machine. The target machine
239 temporarily mounts the template machine's / and /usr via NFS, read-only,
240 and runs cpdup to update / and /usr. If you use this methodology note
241 that THERE ARE SECURITY CONSIDERATIONS! See 'SECURITY CONSIDERATIONS WITH
242 NFS' below.
243
244 Whatever script you use that does the NFS mounts should ensure that the
245 mount succeeded before continuing with the cpdup.
246
247 You should create .cpignore files in the appropriate directories on the
248 template machine's / and /usr partitions so as not to overwrite active
249 files on the target. The most critical .cpignore files should be
250 protected with 'chflags schg .cpignore'. Specifically, the ones in /
251 and /etc, but possibly others as well. For example, the .cpignore
252 hierarchy for protect /root is:
253
254 # /root/.cpignore contains
255 .history
256
257 # /root/.ssh/.cpignore contains
258 random_seed
259 known_hosts
260 authorized_keys
261 identity
262 identity.pub
263
264 WHEN INITIALLY CONVERTING A TARGET MACHINE TO USE TEMPLATING, ALWAYS
265 MAKE A FULL BACKUP OF THE TARGET MACHINE FIRST! You may accidentally delete
266 files on the target during the conversion due to forgetting to enter
267 items into appropriate .cpignore files on the source.
268
269 SECURITY CONSIDERATIONS WITH NFS ROOT EXPORT FROM TEMPLATE MACHINE
270 SECURITY CONSIDERATIONS WITH NFS USR EXPORT FROM TEMPLATE MACHINE
271
272 There are some serious security considerations that must be taken into
273 account when exporting / and /usr on the template machine.
274
275 * only export read-only
276
277 * the password file ( aka vipw ) may not contain any crypted passwords
278 at all. You MUST use ssh or kerberos to access the template machine.
279
280 You can get away with giving only root a crypted password, but only
281 if you disallow network root logins and only allow direct root
282 logins on the console.
283
284 * The machine's private ssh_host_key usually resides in /usr/local/etc.
285 You must move this key to /var/db. You can softlink link so no
286 modification of sshd_config is required.
287
288 * The machine's private ~root/.ssh/identity file is also exposed by
289 the NFS export, you should move this file to /var/db as well and
290 put a softlink in ~root/.ssh.
291
292 * DON'T EXPORT /var ! Either that, or don't put the private keys
293 in /var/db ... put them somewhere else.
294
295 * You may want to redirect the location of the random_seed file, which
296 can be done by editing ~root/.ssh/sshd_config and
297 /usr/local/etc/sshd_config so it is not exposed either.
298
299 -Matt
300 Matthew Dillon
301 dillon@backplane.com
302
303